Semantic Test (SEM)

Description

The semantic test, together with the syntactic test, belongs among the validation tests, with which the validity of the data input is tested. In practice, the semantic test is often executed in combination with the syntactic test (see "Syntactic Test (SYN)").

Test approach: Coverage-based – condition oriented 
Test variety: Validation test, Functional testing, Security testing
Test basis:

The test basis consists of the semantic rules that specify what a data item should comply with in order to be accepted by the system as valid input. Semantic rules are connected with the relationships between data. These relationships may be between the data within a screen, between data on various screens and between input data and existing data in the database. Semantic rules may be established in various documents, but are usually described in:

  • Functional specifications of the relevant function or input screen
  • The business rules that apply to the functions overall
Coverage type: Modified condition decision coverage (MCDC) and oother coverage types possible, see below.
Tip

If the semantic rules describe the conditions for meeting security requirements, the SEM can also be applied to the "Security test" test type.

With the semantic test, user-friendliness aspects can also be tested, by assessing the messages that occur in invalid situations thus:

  • Are they understandable and unambiguous?
  • Do they offer clear indications of how the invalid situation can be resolved?

Modified condition/decision coverage. Since the semantic rules can be specified as decision points that consist of compound conditions, one of the coverage types for the semantic test is selected from the area of decision points. The default choice for the semantic test is:

Variants can be realized simply by replacing this with:

Points of focus in the steps

In principle, for the SEM, too, the generic steps (see "Introduction") are carried out. However, the formulation of a semantic test is very simple: each semantic rule is tested separately. Each rule leads to one or more test situations and each test situation generally leads to one test case.

For that reason, this section is restricted to explaining the first step "identifying test situations". This will be explained and expanded on through an example.

1 - Identifying test situations

A semantic rule that describes the conditions of validity can generally be set out as follows:

IF (semantic rule) THEN valid input or processing
  ELSE error message

In the event that the semantic rule describes the invalid situations in which an error message should occur, this becomes:

IF (semantic rule) THEN error message
  ELSE valid input or processing

The semantic rule is a decision point that consists of one or more conditions connected by AND and OR. A single condition has only two test situations, one for valid input and one for invalid input. For compound conditions, the test situations are derived by applying modified condition/decision coverage (MCDC), as explained in "Decision Points". 

 

Example

Suppose that the following semantic check is specified:


"IF customer lives in the Netherlands AND (postcode does not comply with Netherlands format OR country code is different from 31) THEN this results in an error message." The following occurs after applying MCDC:

D1
A AND (B OR C)
1
Error message
0
Valid input
A: Customer in NL 1 1 0   (1) 0 1 0   (3)
B: Postcode not in NL 1 1 0 1 0 0   (4)
C: Country code ≠ 31 1 0 1   (2) 1 0 0

 

In more detail

In practice, semantic rules are sometimes described in the form:
"IF item X meets condition A, THEN condition ... should also be met"

The pitfall here is that it appears as though the semantic rule only consists of the condition "IF item X meets condition A". However, that is not the case. Everything that comes after the "THEN" also describes the conditions that should be met. In fact, this way of writing the semantic rule is an example of the "imply operator" in Boolean algebra. The truth table for this operator, which is shown by the symbol "→", is:

A B A → B
1 1 1
1 0 0
0 1 1
0 0 1

Now, a condition that is described by the imply operator can be converted simply into a compound condition with the same truth table:
"A → B" is equivalent to "(NOT A) OR B"

 The basic technique of modified condition/decision coverage can be applied to the resulting compound condition – that contains only the operators AND, OR and NOT – without difficulty. The example below explains this further.

 Suppose that the following semantic rule is specified:
"When code_contribution = V THEN code_employment must be = F AND Age ≥ 55"

 An imply operator has been applied here, whereby the rule actually looks like this:

"code_contribution = V → (code_employment = F AND Age ≥ 55)"

 This can be converted into the following compound condition:

"(NOT code_contribution = V) OR (code_employment = F AND Age ≥ 55)"
or
"code_contribution ≠ V OR (code_employment = F AND Age ≥ 55)"

 Application of the basic MCDC technique delivers the following four test situations:

D1
A OR (B AND C)
1
Valid input
0
Error message
A: code_contribution ≠ V 1 1 0   (1) 0 1 0   (3)
B: code_employment = F 0 1 1   (2) 0 0 1   (4)
C: age ≥ 55 0 1 1 0 1 0

2 - Creating logical test cases

The test situations from step 1 at once form the logical test cases.

Example

The formulation of the four test situations from our example immediately gives us the four logical test cases:

Test cases/Test situations D1-1 D1-2 D1-3 D1-4
Customer in NL in NL not in NL in NL
Postcode not in NL in NL not in NL in NL
Country code 31 ≠ 31 31 31
Expected result Error message Error message OK OK

3 - Creating physical test cases

Sometimes when making physical test cases the test data to be used can be determined by applying an additional test design technique like Boundary Value Analysis (BVA).

If for example with the Semantic Test design technique four logical test cases have been determined:

- two test cases are about a person whose age is less than 33
- two test cases are about a person whose age is 33 years or more

then the physical test cases can be for people of whom the age is 31, 32, 33 and 34 years old (so 4-value BVA is applied). This way the testing with Semantic Testing is further enhanced.

4 - Establishing the starting point

No remarks.

 

An overview of all featured Test Design Techniques can be found here.

In Quality for DevOps teams, and the syllabus for TMAP: High-Performance Quality Engineering the coverage type MCDC and the test design techniques Semantic testing and Elementary Comparison Testing are combined.

Read here how these techniques are to be used in a combined way.