¿Cuándo parar de testear?

¿Cuántas veces nos habremos preguntado hasta qué punto debemos continuar haciendo testing? En un mundo ideal la respuesta a esta pregunta podría ser tan fácil como seguir realizando pruebas hasta que estemos completamente convencidos de que el sistema que estamos probando cumple todas las características de calidad que hayamos establecido con anterioridad.

Read more


Desde que trabajo en el mundo del testing me he encontrado alguna vez con compañeros que me han pedido que les explicase en qué consistía exactamente el smoke test ya que no lo habían utilizado o que simplemente no lo conocían. Cierto es que no es tan famoso como sus otros hermanos regression test, acceptance test, exploratory test… pero creo que puede ser una parte muy importante y útil dentro de nuestros proyectos.

Read more

The (mis)estimation of the bewilderment factor in test automation projects

gearsWhat happens in a test automation project when input test case designs to be automated and their associated requirements are not trustworthy or they change, evolve or become inconsistent, even when the project is already started? One of the main consequences of such kind of situations is bewilderment. This answer poses a challenge: how to deal with the bewilderment factor in projects?

The bewilderment factor is the extra effort produced as a result of one or more of the following circumstances:

  • Getting over contradictory or unclear indications, requirements, specifications or test case designs.
  • Ambiguity or missing information regarding the input elements which serve as input for the project.
  • Unclear objectives to be achieved.
  • Working in a non-clearly defined environment.
  • Unexpected/non-communicated changes in the application.

Such circumstances are especially risky in test automation projects, since automated test cases perform a non-ambiguous sequence of actions on the system under test. If such specifications are not clear, then they need to be clarified to become unambiguous. Sometimes, this effort also creates communication issues, responsibility conflicts and misalignments between people expectations. The consequence is, again, an increased bewilderment.

We promote to include the bewilderment factor in estimation techniques. It is a subjective factor, but it usually raises the effort to be done in many projects. This factor may aggregate many of the social effects that are not usually considered when estimating a project, although their consequences become tangible in terms of time, money and results. The bewilderment is especially critical when the kickoff of the project does not ensure that work environment, objectives and necessary inputs are clearly stated. Sometimes, this ambiguity cannot be totally resolved, but if we know the risk, why not taking it into account?

As a social factor, we need to evaluate bewilderment by using empirical techniques based on theconsequences experienced in previous projects, evaluated on a common set of project environment variables. Particularly, in test automation projects, it is essential to evaluate if:

  • The automation scope is well-specified.
  • Input manual test cases to be automated are well-specified with an adequate level of detail. Otherwise, test cases need to be (re)designed first according to the scope and the testing criteria (coverage, risks, key functionalities, etc.)
  • The execution criteria and the reporting formats are decided. It includes aspects such as the testing environment that will be used to execute the test cases, how the results will be generated and managed, etc.
  • There exists a test environment that fits data requirements and controlled changes.
  • A framework for reducing maintenance effort and code comprehensibility is defined and explicitly specified.

Social cross-wise abilities and attitudes of the project team, such as experience in programming, autonomous learning, communication abilities and management skills may contribute to mitigate this factor.

Many times, in software engineering or automation testing projects, estimations may be overtaken by reality when the bewilderment factor is not taken into account. Bewilderment is a non-countable factor at first sight, and therefore it is more difficult to be estimated and conceived as real effort from the customer point of view. However, not considering it increases the risk of failing effort estimations, since it may be a frequent cause of extra effort in many contexts.

In summary, we should take into account that people matter, and that dealing with social effects focused on people (like bewilderment) in projects planning and management may count (or discount) in the results.

Albert Tort | Especialista de Software Control & Testing en Sogeti España

‘Empathy’ puts a Tester in the users’ shoes

In-users-shoes-300x237At a recent  follow-up meeting, our client leader told my team that, in addition to being convinced of our excellent work in designing and executing test cases to assure quality for the applications, he also trusts blindly our ability to think as users and greatly values our skills in writing user manuals for the applications that we test. Read more

Performance is key in I.T. systems

How can we tackle performance, during design, build and run phases of a system to handle issues before they happen?

We all have been confronted with critical systems not performing as they should. This subject is fascinating as it is crossing nearly all the fields of I.T. systems design, build and run.

Read more