TMap® HD was introduced to meet the testing challenges that modern agile organization face. As discussed in a previous post of mine, agile has caused many organizations to phase out the test manager role, reasoning that testing activities should be organized and managed within the agile teams.
This is causing unforeseen challenges for these organizations. In this post I will look at a number of these challenges and how a test manager can be positioned to resolve these.
Challenge: the business feels it has no control over quality
With quality of work the responsibility of the team and part of the definition of done, business may feel a loss of control over quality. As a reaction we’ve seen an acceptance testing phase added after the delivery of increments. However this negates many of the benefits received from agile software development in the first place. Delivering working software every 4 weeks will not help your organization when that software is stuck on an acceptance environment for months.
A solution is to put the test manager in a supporting position where he can act on behalf of the business, not unlike that of an architect. This could possibly be combined with the role of product manager. Aldert Boersma, co-author of “Neil’s quest for quality”, described this as one of the innovations of TMap® HD. The test manager could not only attend the sprint review taking into account the perspective of testing and acceptance, but it would also allow him to get direct feedback when gathering acceptance criteria, organizing product risk analyses and creating a test strategy. He would also be in a position to add acceptance criteria or important testing or testability requirements to user stories on the product backlog or the definition of done. In this way the test manager creates transparency on how quality is built into the product, giving the business the necessary confidence in the product delivered by the team.
One downside of this position however is that the test manager is distanced from the development team that will have to execute his test strategy. One could argue though that that would have been a test coordinator’s role anyway. Which leads us to the next challenge.
Challenge: the team struggles with the concept of built in quality
If a team does not contain any certified test specialists, it may lack the knowledge of those best practices that help to build in quality or validate that quality before the sprint review. Those teams may end up unable to prove that their increment meets the definition of done. In this case, the test manager should be in a position where he can have more direct contact with the development team.
The solution could be to add a test master to the team. He is the master of the test framework and enables the team to optimally use that framework to deliver high quality increments. The test master could operate in a role similar to or combined with that of scrum master. The test master can also coach the team on all matters pertaining to testing, from explaining which test tools could be of use to providing resources on test varieties that the team may not have experience with. The test master may also suggest additional test varieties or test activities to be added to the sprint backlog if this is necessary to validate the definition of done. While a test master does not necessarily act directly on behalf of the business, the team does so through the product owner. Therefore this position may also be explored within the context of TMap HD.
One risk of this position is that the team loses the initiative to build in quality. They may end up looking to the test master to provide all the knowledge with regards to quality and testing. This is obviously not very agile, as the team would then depend on the test master to meet the definition of done. The test master should take the initiative to share his knowledge and help developers explore possibilities to build in quality, not to determine in detail the entire test approach.
Challenge: inconsistent quality of products across teams
Another challenge that we’ve seen agile organizations struggle with is inconsistency in the quality of products delivered by different teams. Some organizations try to resolve this by adding people from high-quality teams in teams struggling with quality. However this approach disrupts the flow of multiple teams and gives no guarantee of success.
The solution would be to have a test manager operating as a test coach, similar to an agile coach. In this position he will operate on a higher abstraction level. The test coach advises the organization on how to perform structured risk-based testing within the context of agile software development. He helps product owners identify the necessary acceptance criteria to include in user stories, helps teams create consistent, usable and testable definitions of done and gives developers guidance to integrate test activities within their work. He can advise teams on best practices to use for building quality into their products. He can also address frequently occurring problems and provide standardized solutions for those to help industrialize test activities. He may also create a quality policy to document and communicate his knowledge across the organization.
Challenge: products do not function correctly within the chain of systems.
The final challenge that I’d like to discuss here is that of mitigating end-to-end (E2E) risks. We’ve seen agile organizations struggle to integrate products delivered by different teams into a functioning chain of systems. This challenge is actually not limited to agile organizations. In fact we at Sogeti have addressed this quite frequently over the past decade and created a methodology for this as an extension of TMap NEXT®.
To overcome this challenge the test manager should receive the position of E2E test director. As described in the book “End-to-end testing with TMap NEXT®”, the E2E test director identifies and describes processes and systems across the chain, analyses the E2E product risks, determines the necessity for E2E testing and creates an E2E testing strategy. While this may not sound very agile, the trick is to enable the agile teams to build in the quality of the chain themselves. The E2E test director needs to make sure each team knows what behavior is expected from their product within the chain of systems. The E2E test director may also provide information necessary to create stubs and drivers to test interfaces. Ideally this will eliminate the need for a separate E2E test phase, since that would make the delivery process a lot less agile.
The E2E test director also proposes requirements to test environments with regards to E2E testing. Ideally such an environment allows teams to continuously integrate their increments into their products, which continuously integrate into the chain of systems. The E2E test director can also provide the necessary input for creating automated E2E regression test sets.
Challenge: there’s not enough budget to hire this many test managers
Fortunately you probably won’t need to. These challenges won’t require one full-time test manager each per product being developed. The positions described here may be combined with each other, or with other positions as described above or may be fulfilled by one test manager for several products. However if you face multiple of these challenges and you wish to add one test manager to the team, make sure to carefully select the position that will help resolve your most critical challenge. You could also look to provide one of your team members with a TMap® Suite test master / test manager training.
Published: 18 August 2017
Author: Niek Fraanje
