Preparation of the PRA

Preparing the product risk analysis

To ensure that the product risk analysis is off to a flying start, the test manager derives the elements relevant to the product risk analysis from existing information, such as the requirements, designs, or similar documents.

The elements relevant to the execution of a product risk analysis are:

1. Processes: (business) processes and subprocesses supported by the test object.
    
2. Product requirements: the client’s requirements for the test object. Damage is caused if the test object does not (no longer) support these requirements. The organisation determines how and at what detail level the product requirements are formulated. ICT driven organisations often go with user requirements. Organisation with high user involvement generally decide to formulate critical success factors.
    
3. Characteristics: Characteristics on the basis of which the product can be evaluated by means of testing, e.g. functionality, performance, user friendliness, etc. By preference, these are specified in the form of quality characteristics because it is relatively easy to establish a concrete test approach on this basis. If the concept of a quality characteristic is difficult to understand for participants, the test manager can also use other terms that are more in line with the participants' perceptions.
    
4. Object parts: An object part consists of one or more logically interrelated parts of the test object viewed from the perspective of the characteristic to be tested. The chance of failure can be estimated on the basis of an object part.
    

Inventorying damage elements

In order to gain insight into the damage elements, the test manager creates an overview, for each characteristic, of the processes and subprocesses supported by the test object. This results in a structure recognisable to the participants, to which the product requirements can be linked. The subdivision into processes and subprocesses continues down to the level at which all inventoried product requirements can be linked, for each characteristic and in the right place, to a process or subprocess. An indication of the damage (high, moderate, low) can be established for each product requirement.

Inventorying chance of failure elements.

The test manager creates an overview of object parts per characteristic to determine the chance of failure. Together, the object parts represent the total test object by means of their interrelationships. This results in a structure recognisable to the participants, allowing the chance of failure to be determined for each combination of characteristic and object part. The subdivision into object parts continues down to the level at which an indication of the chance of failure (high, moderate, low) can be determined.

The division into object parts makes it possible to realise more refinement later in selecting the test coverage. An important point requiring attention here is that the division into object parts depends on the characteristic. For instance, the characteristic ‘functionality’ is divided into object parts in a different way than ‘performance’ or ‘security’. To improve recognisability, we recommend aligning the division with that used in the design unless there are good reasons for deviating from this rule.

It is difficult to provide a clear guideline for the number of object parts into which a test object should be split up. The detail level used for the division into object parts depends heavily on the level at which the product requirements are formulated. The greater the detail level, the greater the number of object parts. Generally speaking, a maximum of seven object parts is manageable, but we see situations with dozens of object parts in actual practice. In this situation, each function in the system is treated as an object part. The advantage of this division is the possibility of a highly detailed product risk analysis. Later on, hours can be estimated and techniques allocated quickly for each function.

The division into characteristic/object part combinations also allows one to distinguish strongly deviating risk parts early on in the test process. Such a part may be defined as a separate object part, for instance. This enables a diverging approach for testing different components.

Combining the damage and chance of failure elements

The inventoried damage and chance of failure elements are collected and interrelated in the table below. This makes it easier to link the results of ‘Determining the damage’ to the results of ‘Determining the chance of failure’.

^{Collecting table of elements inventoried.}

Sometimes, one finds that processes and product requirements cannot be linked to object parts. This would mean that they will not be supported by the system. In this case, the test manager must check with the vendor whether the test object does not support those processes or product requirements. If such a process or product requirement is not supported by the test object, it counts as a defect. At a later stage, the stakeholders may decide to modify the requirements or expand the system.

The result of the preparation for the product risk analysis is an understanding of which of the damage and chance of failure elements of the test object should be incorporated into the product risk analysis.