The shakedown test! (What?)

Last year, when I was in Australia, I visited a test team and one of the testers talked about the shakedown test. I hadn’t heard that one before (have you?). The other testers seemed to think it was an ordinary term. They explained to me that it is a test to see if a newly installed version of a system that is to be tested, is good enough to start the actual test execution.

‘Aha, an intake- / pre- / smoke test’, I said. Fortunately, we again understood each other...

Is this shakedown test a test level or a test type? Who knows? Looking at the definitions, it could be both. Does it matter? I don’t think so. The point is that your test has a specific goal and that you reach that goal.

The shakedown test’s goal is to determine whether the test object has a pre-defined quality level. Only then do we start the execution of the actual test, the test that’s designed to measure the quality and risks related to the test object. So, this example has two separate tests with two different goals.

Recently I visited an Agile team, a truly cross-functional team that was jointly responsible for the product to be delivered. I was interested in their work and asked what kind of tests they did. They looked at me indignantly: ‘We work in an Agile process, so we don’t distinguish different kinds of testing like test levels!’ The implied causality surprised me. Furthermore, I asked about “kinds of tests”, and not “test levels”. But I realized that there is no hierarchy in Agile, and that the term “test level” sound very hierarchical indeed. That’s because test levels are usually presented with the aid of the V-model, which we don’t want to associate with Agile.

However, the fact that you don’t use separate test levels shouldn’t imply -as it did with this team- that you only perform functional unit tests. There are many different aspects and you will have to cover them to a greater or lesser extent (depending on the risk level). So an Agile team also must have different goals for different tests. On the one hand you want to know if programs operate individually, while on the other hand you need to know whether the system works as desigend. And of course, whether the business processes are supported properly so that the whole can be accepted.

A good Agile team ensures that each team member (including the product owner) puts testing activities on the scrum board, using their own perspective. In this fashion, they bring variety to testing. A software programmer will naturally focus on unit testing. A designer will act more from the system perspective. And the product owner is interested in the overall quality from a business perspective. The tester in the team ensures that also the less obvious testing tasks are put on the scrum board, for example with regard to usability, security, performance, maintainability, and so on. The “definition-of-done” is used to define the various criteria for the quality level as desired.

To make an Agile team (and indeed any other team) aware of the necessity of variety in testing, in TMap HD we introduce the term “test variety”. Does that mean that we say farewell to the terms “test level” and “test type”? Well, in some cases. If you are working in a high structure and high risk situation, then you will certainly continue to appreciate separate test levels and test types. But if you're in a modern, cross-functional team, that division doesn’t add value; what’s left is the need to cover the important aspects in a varied manner. So test varieties are here to help you divide and spread your efforts and attention over all important aspects of quality and risk.

Conclusion:

No matter how you structure your IT-lifecycle, and the testing within it, make sure to have variety in your test approach so that tests of various aspects and components complement each other. That leads to the best evaluation of the important quality characteristics and risks of the test object.