The execution of tests is one of the most important parts of dynamic testing. Various people with the role of tester will execute test cases in the SAP test environment. A test case consists of preconditions, steps with input and actions, and expected results.
Before the test execution starts the tester performs some pre-activities, such as test environment setup, test case design, and test data preparation.
During test execution the tester uses the test scenario to follow the steps of each test case and determines if the test is passed or failed by comparing the expected result (of all steps) with the actual result. When the tester observes an anomaly during the test execution, the anomaly will be logged according to the anomaly management process.
After the test execution has ended the tester investigates and assesses the outcome of the testing. If any anomalies are observed these are reported, so that the anomaly management process can be followed. And as a closing activity there are “test cleanup” activities such as making sure the user is logged out of the system and the database is cleaned up when needed.
The test execution may differ for different test varieties, for example because the scope is different, different people are involved and different tools are used. Based on the nature of the SAP project some specific test varieties may be applicable, separately or combined. As per the SAP Activate methodology the following test varieties are often relevant:
- Unit Test
- System Integration Test
- User Acceptance Test
- Regression Test
Business users, operations – and maintenance people are mainly involved in User Acceptance Testing and sometimes in System Integration Testing. They often may be involved in Regression Testing. During test execution, the testers will assess actual outcome versus expected outcome. Progress and any observations or anomalies will be logged. The following sections briefly describes these relevant test varieties.
System Integration Test (SIT)
System Integration Testing is performed to verify proper execution of the entire application including interfaces to external applications (See end-to-end testing). This aims to ensure that the integrated components are functioning properly according to the requirements and specifications. Mainly focused on in- and outbound communication to check if connectivity is established between systems in scope. It is a test carried out by operations- and maintenance people, and possibly involving future users, with the aim of demonstrating that (sub)system interface agreements have been met, correctly interpreted and correctly implemented.
User Acceptance Test (UAT)
UAT is a test variety carried out by (or on behalf of) the future user(s) in an optimally simulated production environment, with the aim of demonstrating that the system supports the operational process of the users. Activities included for this phase are:
- Prepare a User Acceptance Test plan,
- Prepare and document User Acceptance Test Cases,
- Execute User Acceptance Test Cases,
- Handle anomalies according to the anomaly management process agreed within the organization,
- Obtain User Acceptance Test Sign Off to formalize the acceptance.
Regression Test (RT)
The implementation of a new system or changes to an existing system may impact existing IT systems and SAP business processes. Therefore, after each change a regression test should be executed to confirm that no unforeseen impact exists in the parts that should not have been changed. Because regression tests usually have to be executed very often the execution is preferably automated using a test execution tool.
A regression test may be a separate test variety, but often it is combined with a SIT or UAT.
Other Test varieties
It is also recommended to assess the need for other test varieties (such as non-functional testing). Examples are Performance Testing, Usability Testing and Security Testing.