Preparing the product risk analysis
To ensure that the product risk analysis is off to a flying start, the test manager derives the elements relevant to the product risk analysis from existing information, such as the requirements, designs, or similar documents.
The elements relevant to the execution of a product risk analysis are:
- Processes: (business) processes and subprocesses supported by the test object.
- Product requirements: the client’s requirements for the test object. Damage is caused if the test object does not (no longer) support these requirements. The organisation determines how and at what detail level the product requirements are formulated. ICT driven organisations often go with user requirements. Organisation with high user involvement generally decide to formulate critical success factors.
- Characteristics: Characteristics on the basis of which the product can be evaluated by means of testing, e.g. functionality, performance, user friendliness, etc. By preference, these are specified in the form of quality characteristics because it is relatively easy to establish a concrete test approach on this basis. If the concept of a quality characteristic is difficult to understand for participants, the test manager can also use other terms that are more in line with the participants' perceptions.
- Object parts: An object part consists of one or more logically interrelated parts of the test object viewed from the perspective of the characteristic to be tested. The chance of failure can be estimated on the basis of an object part.
Inventorying damage elements
In order to gain insight into the damage elements, the test manager creates an overview, for each characteristic, of the processes and subprocesses supported by the test object. This results in a structure recognisable to the participants, to which the product requirements can be linked. The subdivision into processes and subprocesses continues down to the level at which all inventoried product requirements can be linked, for each characteristic and in the right place, to a process or subprocess. An indication of the damage (high, moderate, low) can be established for each product requirement.
Inventorying chance of failure elements.
The test manager creates an overview of object parts per characteristic to determine the chance of failure. Together, the object parts represent the total test object by means of their interrelationships. This results in a structure recognisable to the participants, allowing the chance of failure to be determined for each combination of characteristic and object part. The subdivision into object parts continues down to the level at which an indication of the chance of failure (high, moderate, low) can be determined.
In more detail
One point requiring attention when determining object parts is that the setup of a process or subprocess may have an impact on the chance of failure. In this case, it is wise to incorporate the relevant (sub)process descriptions as an object part. This means that in addition to the system, the process is also included in the test – e.g. an acceptance test.
The division into object parts makes it possible to realise more refinement later in selecting the test coverage. An important point requiring attention here is that the division into object parts depends on the characteristic. For instance, the characteristic ‘functionality’ is divided into object parts in a different way than ‘performance’ or ‘security’. To improve recognisability, we recommend aligning the division with that used in the design unless there are good reasons for deviating from this rule.
Object parts per characteristic
From the functional point of view, the system can be split up into 2 big subsystems that can be tested separately. An integral test of the total system is necessary as well. This means a division into three object parts. The characteristic ‘Performance’ distinguishes between the online and batch performance, i.e. what is the direct response to an online transaction 2 and what is the time required to execute batch process 1.
It is difficult to provide a clear guideline for the number of object parts into which a test object should be split up. The detail level used for the division into object parts depends heavily on the level at which the product requirements are formulated. The greater the detail level, the greater the number of object parts. Generally speaking, a maximum of seven object parts is manageable, but we see situations with dozens of object parts in actual practice. In this situation, each function in the system is treated as an object part. The advantage of this division is the possibility of a highly detailed product risk analysis. Later on, hours can be estimated and techniques allocated quickly for each function.
The division into characteristic/object part combinations also allows one to distinguish strongly deviating risk parts early on in the test process. Such a part may be defined as a separate object part, for instance. This enables a diverging approach for testing different components.
Combining the damage and chance of failure elements
The inventoried damage and chance of failure elements are collected and interrelated in the table below. This makes it easier to link the results of ‘Determining the damage’ to the results of ‘Determining the chance of failure’.
|Characteristic||Process||Product requirement||Object part|
|Functionality||Sales||Compliance with the functional requirements||System 1|
|Sales/Advice||With an eye to the legal duty of care, the advice given and how the client decides to deviate from the advice must be recorded||Subsystem 1|
|Sales/Advice||The system must comply with the requirements for socially responsible business||System 1|
Compliance with the security policy
|Sales||Compliance with the security policy||Web access X|
Collecting table of elements inventoried.
Sometimes, one finds that processes and product requirements cannot be linked to object parts. This would mean that they will not be supported by the system. In this case, the test manager must check with the vendor whether the test object does not support those processes or product requirements. If such a process or product requirement is not supported by the test object, it counts as a defect. At a later stage, the stakeholders may decide to modify the requirements or expand the system.
The result of the preparation for the product risk analysis is an understanding of which of the damage and chance of failure elements of the test object should be incorporated into the product risk analysis.
Product Risk Analysis - Execution
- Executing the product risk analysis (PRA)
- Alternative PRA
- Risk poker
- BDTM Product Risk Analysis
- Dealing with incomplete information
- Product risk management