ISO/IEC/IEEE FDIS 29119-4

Software and systems engineering -- Software testing

Ingénierie du logiciel et des systèmes -- Essais du logiciel

General Information

• Status: Published
• ICS: 35.080 - Software
• Technical Committee: ISO/IEC JTC 1/SC 7 - Software and systems engineering
• Drafting Committee: ISO/IEC JTC 1/SC 7/WG 26 - Software testing
• Current Stage: 5060 - Close of voting Proof returned by Secretariat
• Start Date: 06-Aug-2021
• Completion Date: 05-Aug-2021

RELATIONS

• Revised: ISO/IEC/IEEE 29119-4:2015 - Software and systems engineering -- Software testing
• Effective Date: 23-Apr-2020

Standards Content (sample)

FINAL
INTERNATIONAL ISO/IEC/
DRAFT
STANDARD IEEE/FDIS
29119-4
ISO/IEC JTC 1/SC 7
Software and systems engineering —
Secretariat: BIS
Software testing —
Voting begins on:
2021-06-10
Part 4:
Voting terminates on:
Test techniques
2021-08-05
Ingénierie du logiciel et des systèmes — Essais du logiciel —
Partie 4: Techniques d'essai
RECIPIENTS OF THIS DRAFT ARE INVITED TO
SUBMIT, WITH THEIR COMMENTS, NOTIFICATION
OF ANY RELEVANT PATENT RIGHTS OF WHICH
THEY ARE AWARE AND TO PROVIDE SUPPOR TING
DOCUMENTATION.
IN ADDITION TO THEIR EVALUATION AS
Reference number
BEING ACCEPTABLE FOR INDUSTRIAL, TECHNO-
ISO/IEC/IEEE/FDIS 29119-4:2021(E)
LOGICAL, COMMERCIAL AND USER PURPOSES,
DRAFT INTERNATIONAL STANDARDS MAY ON
OCCASION HAVE TO BE CONSIDERED IN THE
LIGHT OF THEIR POTENTIAL TO BECOME STAN-
ISO/IEC 2021
DARDS TO WHICH REFERENCE MAY BE MADE IN
NATIONAL REGULATIONS. IEEE 2021
---------------------- Page: 1 ----------------------
ISO/IEC/IEEE/FDIS 29119-4:2021(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2021
© IEEE 2021

All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting

on the internet or an intranet, without prior written permission. Permission can be requested from either ISO or IEEE at the

respective address below or ISO’s member body in the country of the requester.
ISO copyright office Institute of Electrical and Electronics Engineers, Inc
CP 401 • Ch. de Blandonnet 8 3 Park Avenue, New York
CH-1214 Vernier, Geneva NY 10016-5997, USA
Phone: [phone]
Email: [e-mail] Email: [e-mail]
Website: www.iso.org Website: www.ieee.org
Published in Switzerland
© ISO/IEC 2021 – All rights reserved
ii © IEEE 2021 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC/IEEE/FDIS 29119-4:2021(E)
Contents Page

Foreword ..........................................................................................................................................................................................................................................v

Introduction ..............................................................................................................................................................................................................................vii

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ...................................................................................................................................................................................... 1

3 Terms and definitions ..................................................................................................................................................................................... 1

4 Conformance ............................................................................................................................................................................................................. 7

4.1 Intended usage ........................................................................................................................................................................................ 7

4.2 Full conformance................................................................................................................................................................................... 7

4.3 Tailored conformance ....................................................................................................................................................................... 7

5 Test design techniques ................................................................................................................................................................................... 8

5.1 Overview ...................................................................................................................................................................................................... 8

5.2 Specification-based test design techniques ................................................................................................................10

5.2.1 Equivalence partitioning ........................................................................................................................................10

5.2.2 Classification tree method ....................................................................................................................................12

5.2.3 Boundary value analysis ........................................................................................................................................12

5.2.4 Syntax testing...................................................................................................................................................................14

5.2.5 Combinatorial test design techniques ........................................................................................................15

5.2.6 Decision table testing ........................................................................................................................................... .....18

5.2.7 Cause-effect graphing ...............................................................................................................................................18

5.2.8 State transition testing ............................................................................................................................................19

5.2.9 Scenario testing .............................................................................................................................................................20

5.2.10 Random testing ..............................................................................................................................................................21

5.2.11 Metamorphic testing .................................................................................................................................................21

5.2.12 Requirements-based testing ...............................................................................................................................22

5.3 Structure-based test design techniques .........................................................................................................................23

5.3.1 Statement testing..........................................................................................................................................................23

5.3.2 Branch testing .................................................................................................................................................................23

5.3.3 Decision testing ..............................................................................................................................................................24

5.3.4 Branch condition testing ........................................................................................................................................25

5.3.5 Branch condition combination testing ......................................................................................................25

5.3.6 Modified condition/decision coverage (MCDC) testing ..............................................................26

5.3.7 Data flow testing ...........................................................................................................................................................27

5.4 Experience-based test design techniques ....................................................................................................................29

5.4.1 Error guessing .................................................................................................................................................................29

6 Test coverage measurement ..................................................................................................................................................................30

6.1 Overview ...................................................................................................................................................................................................30

6.2 Test measurement for specification-based test design techniques ........................................................30

6.2.1 Equivalence partition coverage ........................................................................................................................30

6.2.2 Classification tree method coverage ............................................................................................................30

6.2.3 Boundary value analysis coverage .................. ...............................................................................................31

6.2.4 Syntax testing coverage ...........................................................................................................................................31

6.2.5 Combinatorial test design techniques coverage ................................................................................31

6.2.6 Decision table testing coverage ........................................................................................................................32

6.2.7 Cause-effect graphing coverage .......................................................................................................................32

6.2.8 State transition testing coverage.....................................................................................................................32

6.2.9 Scenario testing coverage .................. ....................................................................................................................33

6.2.10 Random testing coverage ......................................................................................................................................33

6.2.11 Metamorphic testing coverage .........................................................................................................................33

6.2.12 Requirements-based testing coverage .......................................................................................................33

6.3 Test measurement for structure-based test design techniques ................................................................33

6.3.1 Statement testing coverage ..................................................................................................................................33

6.3.2 Branch testing coverage .........................................................................................................................................33

© ISO/IEC 2021 – All rights reserved
© IEEE 2021 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC/IEEE/FDIS 29119-4:2021(E)

6.3.3 Decision testing coverage ......................................................................................................................................34

6.3.4 Branch condition testing coverage ................................................................................................................34

6.3.5 Branch condition combination testing coverage ..............................................................................34

6.3.6 Modified condition/decision coverage (MCDC) testing coverage ......................................34

6.3.7 Data flow testing coverage ...................................................................................................................................35

6.4 Test measurement for experience-based testing design techniques — Error

guessing coverage .................. ............................................................................................................................................................35

Annex A (informative) Testing quality characteristics ...................................................................................................................36

Annex B (informative) Guidelines and examples for the application of specification-based

test design techniques .................................................................................................................................................................................49

Annex C (informative) Guidelines and examples for the application of structure-based test

design techniques .........................................................................................................................................................................................103

Annex D (informative) Guidelines and examples for the application of experience-based

test design techniques .............................................................................................................................................................................123

Annex E (informative) Guidelines and examples for the application of grey-box test design

techniques ............................................................................................................................................................................................................126

Annex F (informative) Test design technique effectiveness ..................................................................................................129

Annex G (informative) ISO/IEC/IEEE 29119-4 and BS 7925-2 test design technique alignment ...132

Annex H (informative) Test models ................................................................................................................................................................134

Bibliography .........................................................................................................................................................................................................................135

IEEE Notices and Abstract .......................................................................................................................................................................................137

© ISO/IEC 2021 – All rights reserved
iv © IEEE 2021 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC/IEEE/FDIS 29119-4:2021(E)
Foreword

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical

Commission) form the specialized system for worldwide standardization. National bodies that are

members of ISO or IEC participate in the development of International Standards through technical

committees established by the respective organization to deal with particular fields of technical

activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international

organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the

work.

The procedures used to develop this document and those intended for its further maintenance are

described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the

different types of ISO/IEC documents should be noted. This document was drafted in accordance with

the rules given in the ISO/IEC Directives, Part 2 (see www .iso .org/ directives or www .iec .ch/ members

_experts/ refdocs).

IEEE Standards documents are developed within the IEEE Societies and the Standards Coordinating

Committees of the IEEE Standards Association (IEEE-SA) Standards Board. The IEEE develops its

standards through a consensus development process, approved by the American National Standards

Institute, which brings together volunteers representing varied viewpoints and interests to achieve the

final product. Volunteers are not necessarily members of the Institute and serve without compensation.

While the IEEE administers the process and establishes rules to promote fairness in the consensus

development process, the IEEE does not independently evaluate, test, or verify the accuracy of any of

the information contained in its standards.

Attention is drawn to the possibility that some of the elements of this document may be the subject

of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent

rights. Details of any patent rights identified during the development of the document will be in the

Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC

list of patent declarations received (see https:// patents .iec .c).

Any trade name used in this document is information given for the convenience of users and does not

constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and

expressions related to conformity assessment, as well as information about ISO's adherence to the

World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www .iso .org/

iso/ foreword .html. In the IEC, see www .iec .ch/ understanding -standards.

ISO/IEC/IEEE 29119-4 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information

Technology, Subcommittee SC 7, Software and systems engineering, in cooperation with the Systems and

Software Engineering Standards Committee of the IEEE Computer Society, under the Partner Standards

Development Organization cooperation agreement between ISO and IEEE.

This second edition cancels and replaces the first edition (ISO/IEC/IEEE 29119-4:2015), which has been

technically revised.
The main changes compared to the previous edition are as follows:

— The test techniques in this document are defined using a new form of the test design and

implementation process from ISO/IEC/IEEE 29119-2. In the first version, this process was based

on the use of test conditions. Feedback on use of the previous edition highlighted a problem with

users’ understanding of test conditions and their use for deriving test cases. This second edition has

replaced the use of test conditions with test models. Annex H provides more detail on this change

and the Introduction describes the new process.

A list of all parts in the ISO/IEC/IEEE 29119 series can be found on the ISO and IEC websites.

© ISO/IEC 2021 – All rights reserved
© IEEE 2021 – All rights reserved v
---------------------- Page: 5 ----------------------
ISO/IEC/IEEE/FDIS 29119-4:2021(E)

Any feedback or questions on this document should be directed to the user’s national standards body. A

complete listing of these bodies can be found at www .iso .org/ members .html and www .iec .ch/ national

-committees.
© ISO/IEC 2021 – All rights reserved
vi © IEEE 2021 – All rights reserved
---------------------- Page: 6 ----------------------
ISO/IEC/IEEE/FDIS 29119-4:2021(E)
Introduction

The purpose of this document is to provide an International Standard that defines software test design

techniques (also known as test case design techniques or test methods) that can be used within the test

design and implementation process that is defined in ISO/IEC/IEEE 29119-2. This document does not

describe a process for test design and implementation. The intent is to describe a series of techniques

that have wide acceptance in the software testing industry. This document is originally based on the

British standard, BS 7925-2. Annex G provides a mapping from the requirements of BS 7925-2 to the

clauses and subclauses of this document.

The test design techniques presented in this document can be used to derive test cases that, when

executed, generate evidence that test item requirements have been met or that defects are present in

a test item (i.e. that requirements have not been met). Risk-based testing can be used to determine

the set of techniques that are applicable in specific situations (risk-based testing is covered in

ISO/IEC/IEEE 29119-1 and ISO/IEC/IEEE 29119-2).

NOTE A “test item” is a work product that is being tested (see ISO/IEC/IEEE 29119-1).

EXAMPLE 1 “Test items” include systems, software items, objects, classes, requirements documents, design

specifications, and user guides.

Each technique follows the test design and implementation process that is defined in

ISO/IEC/IEEE 29119-2 and shown in Figure 1.

Of the activities in this process, this document provides guidance on how to implement the following

activities in detail for each technique that is described:
— create test model (TD1);
— identify test coverage items (TD2);
— derive test cases (TD3).

A test model represents testable aspects of a test item, such as a function, transaction, feature, quality

attribute, or structural element identified as a basis for testing. The test model reflects the required

test completion criterion in the test strategy.

EXAMPLE 2 If a test completion criterion for state transition testing was identified that required coverage of

all states then the test model would show the states the test item can be in.

Test coverage items are attributes of the test model that can be covered during testing. A single test

model will typically be the basis for several test coverage items.

A test case is a set of preconditions, inputs (including actions, where applicable), and expected results,

developed to determine whether or not the covered part of the test item has been implemented

correctly.

Specific (normative) guidance on how to implement the create test procedures activity (TD4) in the test

design and implementation process of ISO/IEC/IEEE 29119-2 is not included in Clauses 5 or 6 because

the process is the same for all techniques.
© ISO/IEC 2021 – All rights reserved
© IEEE 2021 – All rights reserved vii
---------------------- Page: 7 ----------------------
ISO/IEC/IEEE/FDIS 29119-4:2021(E)
Figure 1 — ISO/IEC/IEEE 29119-2 test design and implementation process

ISO/IEC 25010 defines eight quality characteristics (including functional suitability) that can be used

to identify types of testing that may be applicable for testing a specific test item. Annex A provides

example mappings of test design techniques that apply to testing quality characteristics defined in

ISO/IEC 25010.

Experience-based testing practices like exploratory testing and other test practices such as model-

based testing are not defined in this document because this document only describes techniques for

designing test cases. Test practices such as exploratory testing, which can use the test techniques

defined in this document, are described in ISO/IEC/IEEE 29119-1.

Templates and examples of test documentation that are produced during the testing process are defined

in ISO/IEC/IEEE 29119-3. The test techniques in this document do not describe how to document

test cases (e.g. they do not include information or guidance on assigning unique identifiers, test case

descriptions, priorities, traceability or pre-conditions to test cases). Information on how to document

test cases can be found in ISO/IEC/IEEE 29119-3.

This document aims to provide stakeholders with the ability to design test cases for software testing in

any organization.
© ISO/IEC 2021 – All rights reserved
viii © IEEE 2021 – All rights reserved
---------------------- Page: 8 ----------------------
FINAL DRAFT INTERNATIONAL STANDARD ISO/IEC/IEEE/FDIS 29119-4:2021(E)
Software and systems engineering — Software testing —
Part 4:
Test techniques
1 Scope

This document defines test design techniques that can be used during the test design and implementation

process that is defined in ISO/IEC/IEEE 29119-2.

Each technique follows the test design and implementation process that is defined in

ISO/IEC/IEEE 29119-2 and shown in Figure 1. This document is intended for, but not limited to, testers,

test managers, and developers, particularly those responsible for managing and implementing software

testing.
2 Normative references

The following documents are referred to in the text in such a way that some or all of their content

constitutes requirements of this document. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any amendments) applies.

ISO/IEC/IEEE 29119-2, Software and systems engineering — Software testing — Part 2: Test processes

3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/

NOTE For additional terms and definitions in the field of systems and software engineering, see

ISO/IEC/IEEE 24765, which is published periodically as a “snapshot” of the SEVOCAB (Systems and

software Engineering Vocabulary) database and is publicly accessible at www .computer .org/ sevocab.

ISO/IEC/IEEE 29119-1 includes all terms that are used in ISO/IEC/IEEE 29119 (all parts).

3.1
Backus-Naur Form

formal meta-language used for defining the syntax of a language in a textual format

3.2
base choice
base value

input parameter value used in ‘base choice testing’ that is normally selected based on being a

representative or typical value for the parameter
3.3
boundary value analysis

specification-based test case (3.49) design technique based on exercising the boundaries of equivalence

partitions (3.28)
© ISO/IEC 2021 – All rights reserved
© IEEE 2021 – All rights reserved 1
---------------------- Page: 9 ----------------------
ISO/IEC/IEEE/FDIS 29119-4:2021(E)
3.4
branch condition combination testing

structure-based test case (3.49) design technique based on exercising combinations of Boolean values

of conditions (3.13) within a decision
3.5
branch condition testing

structure-based test case (3.49) design technique based on exercising Boolean values of the conditions

(3.13) within decisions and the decision outcomes (3.21)
3.6
branch testing

structure-based test case (3.49) design technique based on exercising branches in the control flow

(3.14) of the test item (3.52)
3.7
c-use
computation data use
use of the value of a variable in any type of statement
3.8
cause-effect graph

graphical representation of decision rules (3.22) between causes (inputs described as Boolean

conditions) and effects (outputs described as Boolean expressions)
3.9
cause-effect graphing

specification-based test case (3.49) design technique based on exercising decision rules (3.22) in a cause-

effect graph (3.8)
3.10
classification tree

hierarchical tree model of the input data to a program in which the inputs are represented by distinct

classifications (relevant test aspects) and classes (input values)
3.11
classification tree method

specification-based test case (3.49) design technique based on exercising classes in a classification tree

(3.10)
3.12
combinatorial test design techniques

class of specification-based test case (3.49) design techniques based on exercising combinations of P-V

pairs (3.38)

EXAMPLE All combinations testing (3.57), pair-wise testing, each choice testing, base choice testing

3.13
condition
Boolean expression containing no Boolean operators

EXAMPLE “A < B” is a condition but “A and B” is not. See http:// www .testingstandards .co .uk/ bs _7925 -1

_online .htm #Condition for the definition of "condition".
3.14
control flow

sequence in which operations are performed during the execution of a test item (3.52)

3.15
control flow sub-path
sequence of executable statements (3.30) within a test item (3.52)
© ISO/IEC 2021 – All rights reserved
2 © IEEE 2021 – All rights reserved
---------------------- Page: 10 ----------------------
ISO/IEC/IEEE/FDIS 29119-4:2021(E)
3.16
data definition
variable definition
statement where a variable is assigned a value
3.17
data definition c-use pair

data definition (3.16) and subsequent computation data use (3.7), where the data use (3.20) uses the

value defined in the data definition
3.18
data definition p-use pair

data definition (3.16) and subsequent predicate data use (3.37), where the data use (3.20) uses the value

defined in the data definition
3.19
data flow testing

class of structure-based test case (3.49) design techniques based on exercising definition-use pairs (3.26)

EXAMPLE All-definitions testing (3.57), all-c-uses testing, all-p-uses testing, all-uses testing, all-du-paths

testing.
3.20
data use
executable statement (3.30) where the value of a variable is accessed
3.21
decision outcome
result of a decision that determines the branch to be executed

Note 1 to entry: See http:// www .testingstandards .co .uk/ bs _7925 -1 _online .htm #Decision for the definition of

"decision".
3.22
decision rule

combination of conditions (3.13) (also known as causes) and actions (also known as effects) that produce

a specific outcome in decision table testing (3.24) and cause-effect graphing (3.9)

3.23
decision table

tabular representation of decision rules (3.22) between causes (inputs described as Boolean conditions

(3.13)) and effects (outputs described as Boolean expressions)
3.24
decision table testing

specification-based test case (3.49) design technique based on exercising decision rules (3.22) in a

decision table (3.23)
3.25
decision testing

structure-based test case (3.49) design technique based on exercising decision outcomes (3.21) in the

control flow (3.14) of
...