Guiding Development with the Agile Testing Quadrants

Most IT-organizations nowadays are working in an Agile or DevOps way. Or busy transitioning towards Agile and DevOps. Often they struggle how to organize testing in their new organization. In this blog, I’ll give you an overview of how the well-known model of Agile Testing Quadrants can be used.

In 2003 Brian Marick introduced the Agile Testing Quadrants. A very useful model to pair different testing varieties with various perspectives of the IT-activities in an Agile development model. (read his original article here)

The picture below shows the original model of Brian Marick. It consists of four quadrants:

• Quadrant 1 (left-below) is all about testing the components that an IT-system consists of.
• Quadrant 2 (left-upper) is about testing the system as a whole and its connection with business processes and other systems.
• Quadrant 3 (right-upper) is about the quality in use, can users actually reach their goal.
• Quadrant 4 (right-below) is about the technical perspectives that most can only be tested when the system is ready (e.g. performance).

Figure 1: Testing Quadrants by Brian Marick

Basically all test varieties involved have been known for a long time, in that sense, this model doesn’t seem very innovative to people that are experienced in IT in general, and testing in particular. And this was also my thought.

But recently I got a revelation. During the Agile Testing Days conference in Germany, I had the privilege to attend a tutorial of Janet Gregory and Lisa Crispin, the authors of the book “Agile Testing”. And they showed me their version of the Agile Testing Quadrants. See the photo below.

Figure 2: Testing Quadrants adepted by Lisa Crispin and Janet Gregory

The revelation is in the slight difference in wording. Brian Marick puts “Supporting Programming” on the left. Janet and Lisa have replaced that by “Guiding the team”.

Now, you may wonder, what’s the big difference?

It’s in the feeling that the terms given. “Supporting” feels like developers build something and while doing so are supported by testing. “Guiding” the team emphasizes a “test – first approach” where at first the team members think about how they will test a user story, automate that test and then develop the software and use the test to verify it.

I don’t mean to imply that Brian Marick didn’t intend this, actually in his original post he speaks of EDD – Example Driven Development. But by their wording Lisa Crispin and Janet Gregory made it more obvious that this is the way to go.

Figure 3: Objective of testing positioned in testing quadrants

Fran O’Hara (a test expert from Ireland whom I met at various conferences this year) added the perspective that the testing activities on the left mainly have a preventive objective whereas the testing activities on the right are mainly focused on detecting defects.

Next to Brian Marick’s “EDD” nowadays there are various other test-first approaches, for example, TDD (Test Driven Development) which is focused at creating unit tests and nicely fits in Q1. But also BDD (Behavior Driven Development) and ATDD (Acceptance Test Driven Development) that are focused on creating the business requirements and specifications, on which also the acceptance tests are based. These approaches fit well in Q2. In the same group of approaches as BDD and ATDD we of course also see Specification by Example as described by Gojko Adzic. I would like to add a note to Specification by Example: by its name, people might have the idea that creating some examples will replace a formal specification. That’s not the case. In his book, Gojko actually describes that the approach should be to create specifications that are supported by examples. To most people this is a very natural way of working, specification defines and examples contribute to the understanding.

Another interesting perspective to the testing quadrants was presented by Kristian Karl (test manager at Spotify) at the EuroSTAR conference. He showed the perspective of what you test for on the left and right side, his picture looked like this (I recreated the picture based on my photo):

Figure 4: Focus of testing in the testing quadrants

In this version of the testing quadrants, the emphasis is on the difference between “checking” and “testing”. On the left the main focus is on verifying whether the system works as designed. At the right, the main focus is on validating if the business process will be fit for purpose.

Summarizing, we have seen that the testing quadrants are very useful.  In preventive testing, they guide the team to verify the expected behavior. In detective testing they ensure critiquing the product to validate if it is fit for purpose. And in general, for testers and other IT-people the testing quadrants structure and organize the testing activities such that testing actually guides the team as a whole towards the fast delivery of business value.