Test Levels? Test Types? Test Varieties!

When organizing testing, the test manager adhering to the traditional view on testing had to structure the testing activities in a hierarchical way, based on quality characteristics. But a test manager often distinguished various stages too. Defined terms such as Test Level, Test Type, Test Phase and Test Stage were often used.

In today’s view on testing, the people involved in testing are hesitant to use the word ‘Test Level’ since it seems to imply that various groups, based on various hierarchical responsibilities, will perform various testing tasks without any interaction between these test levels. Moreover, many testers have often struggled to distinguish between Test Levels and Test Types. And a Test Stage – is that identical to a Test Level or not? What should be our focus when organizing testing?

All testing activities must collectively cover all important areas and aspects of the system under test – that is the main objective. To cope with the confusion around how to distinguish testing tasks, we introduce the term Test Variety.

The term Test Variety aims at making all stakeholders aware that there will always be different needs for testing, and therefore different test varieties will have to be organized. Whether these are organized separately or combined depends on the situation. There may be many reasons for having different test varieties. For example, there are different stakeholders who ought to be involved; programmers have a different focus in their testing than business representatives do. This is often related to responsibility and accountability for testing activities. The quality characteristics that have to be addressed form another reason for distinguishing test varieties. Maintainability for example, demands totally different testing activities than usability does.

Traditionally, different aspects were separately approached as a group of testing activities that had been brought together in a test level. Many people know the ‘functional acceptance test’,

a name that already indicates that testing was not complete because it obviously didn’t focus on non-functional aspects. In the new view, functional and non-functional testing can be seen as test varieties. Depending on the circumstances, such as the application lifecycle model that is used, these test varieties are organized either together or separately. The main concern is that all relevant test varieties are carried out one way or another.

So let’s use the term Test Variety, to make everybody involved aware of the fact that there are different points of view towards testing activities, and we can make sure that the interests of all stakeholders will be covered by addressing these in a well-considered way.