Quality Gates in SAP testing: what’s in it for me?

In previous blogs, we have been talking about different aspects of SAP testing. All of these aspects help us work towards improving the quality of an SAP project. Quality Gates help project members to take control of their SAP project, adjust when required and improve the quality of the final product. It also provides confidence and experience for the pursued business value.

What are quality gates?
A quality gate is a milestone in an IT project that requires predefined criteria to be met before the project can proceed to a next phase. For example, when an SAP project moves from the ‘Functional Testing’ phase to the ‘User Acceptance Test (UAT)’ phase. It is important that the demand party (the business) and supply party (the system integrator) who are involved with these different phases are aligned on the expected quality levels. Who is responsible for creating and validating these quality levels should be transparent and clear in advance.

A quality gate can represent the start or end of a testing phase and can be visualized by a checklist. This checklist is used during and at the end of a testing phase in an SAP project. Meaning that in a testing context, it requires predefined criteria to be met before the project can proceed to the next testing phase. In other words, the checklist contains entry and exit criteria.

How to use quality gates
The most common tool for a quality gate is a checklist and this checklist should be created for a specific transition from one phase to the next. In a scrum or devops model the quality gates can be used for the transition to a new sprint. In our example, we will use the start of the UAT phase in a waterfall approach. A couple of weeks before the start of the UAT phase, the demand party and the supply party set up a meeting where the demand party hands over their requirements (entry criteria) for the start of the UAT to the supply party, and the supply party agrees to deliver the required IT system to the demand party.

An example of a checklist with entry criteria for the start of a UAT phase in an SAP project:

• Are SAP business testers selected and are their schedules blocked for the UAT period?
• Are SAP business roles and SAP logons mapped for the attending SAP business testers?
• Are the required interfaces for the SAP application available?
• Is the test environment set up and available for the SAP business testers?
• Is the test environment filled with correct test data?
• Is availability and correctness of SAP master data validated?
• Is the in-/outbound communication verified for all interfaces?
• Are test cases for UAT available?
• Is there a physical room booked for the UAT?
• Is all the required hardware available?

The test manager in an SAP project usually takes the lead in setting up and monitoring the status of the quality gate. A quality gate has a deadline that matches the end of the previous phase and the start of the next phase. If the quality gate is not completed at the deadline, it is important that the test manager takes the following actions:

1. Discuss with the involved stakeholders which actions need to be completed at least to start the next testing phase or go-live.
2. Set new actions and deadlines together with stakeholders and make sure the actions are assigned.
3. Make sure that all required resources are available to complete the open actions to meet the quality gate.
4. Inform all stakeholders about the potential delay in the next testing phase or end of the project and discuss and share how to move forward. This can impact the Go/No-go decision for the go-live of the project.

Benefits of quality gates for your SAP project
Using the method of quality gates with entry criteria before the start of a test phase helps the SAP project in the following way:

1. The quality gates are created, defined, agreed, and signed off by all stakeholders before the start of a phase so it is clear what to deliver, what to expect and what to check.
2. The test manager can be sure that the business testers start the UAT phase with a proper working test environment. It would be bad for the support of the SAP project>if the business testers cleared their schedule for the UAT and they are not able to execute tests on the test environment.
3. The demand party is forced to think in advance about what they require to be able to start the test phase and the supply party knows what is expected and when.
4. The test manager can set up regular meeting leading up to the upcoming test phase to check the progress of the quality gate and create actions to meet the deadlines.