QAM

Improvement of the DevOps QA & Testing activities

QA and testing activities are integrated and executed throughout all DevOps activities. The QA and testing activities are not executed only by a person with a testing role, but by the whole team, known as the whole-team approach. In this situation, let's take the DevOps activities as the starting point for improvement.

This DevOps activities-oriented approach requires an effort from each person involved to give substance – per DevOps activity – to six pre-defined Quality key areas [Sogeti 2020]: QA Awareness, QA & Testing, Governance, Transparency, Automation and Infrastructure. We call this approach Quality to Activity Mapping (QAM).

• QA Awareness
  Are all people involved concerned with quality and do they all have a quality mindset?
• QA & Testing
  Which test topics are embraced by the team and how?
• Governance
  Is the team supported in carrying out their activities and is the team in line with the policy of the organization?
• Transparency
  Is the team transparent about insight into progress and quality at any given moment?
• Automation
  Which activities are automated and how?
• Infrastructure
  Does the team have the required infrastructure (including tools) to carry out the activities?

The above is just a brief summary of the key areas.

How does QAM work?

QAM is a team effort and demands participation of all the people involved. Gather these people and request a quality architect (preferably together with a DevOps connoisseur) to be the moderator of the QAM workshop as not all participants will be Quality and DevOps activities experts. First, establish which activities are carried out by the team. Then try to find out together how substance is given to the quality key areas for each of the defined activities.

What does a QAM table look like?

A QAM table maps the quality key areas with the established DevOps activities. At the intersections, the participants will mention the quality measures that are implemented in this activity. Therefore, per activity, all six key areas are discussed one by one. The result is a table that you can use to provide clarity (transparency) to everyone involved about how each activity will contribute to the quality of software development.

If everyone agrees with the current status, suggestions for improvement can be implemented. You can think of (the mentioned suggestions are only meant as examples):

• QA Awareness
  • Create QA & Testing awareness for all people involved.
  • Start using risk analysis approaches at the beginning of a project to clarify who is going to cover which risks in which test level with which test coverage.
• QA & testing
  • Apply risk analysis/poker, give story points for testing in planning poker/sprint planning.
  • Apply test design techniques. (spent time, money and resources wisely in covering the risk)
  • Apply Behavior Driven Development (BDD) (or Acceptance Test Driven Development or Specification by Example) and Test Driven Development (TDD).
• Governance
  • Establish central QA & testing support.
  • Provide risk analysis/poker and test design training for all people involved.
  • Provide test training for developers.
  • Support shift-left (start as early as possible with QA & testing activities).
• Transparency
  • Define Quality indicators per team and/or application.
  • Implement cognitive QA dashboard for critical applications and flows and predictive dashboards. (using AI technology)
• Automation
  • Automate all unit test cases in the CI/CD pipeline (e.g. Jenkins, JUnit test framework)
  • Use BDD to automate tests (e.g. with Cucumber/JBehave/Behat)
• Infrastructure
  • Create a production-like environment as closely as you can for performance testing. (e.g. virtualization)
  • Use SonarQube not only for measuring code quality, but also for statement coverage (e.g. in Unit Testing, System Testing or User Acceptance Testing.
  • Think about the data that is used.

To summarize, identify quality measures per key area, measure the effects of applying these measures and define actions to improve – the application of – these quality measures. If the team regularly repeats this QAM approach, a mechanism is created whereby the Quality & Testing key areas per DevOps activity are continuously improved.