ISO/IEC/IEEE FDIS 29119-2

Software and systems engineering -- Software testing

Ingénierie du logiciel et des systèmes -- Essais du logiciel

General Information

• Status: Published
• ICS: 35.080 - Software
• Technical Committee: ISO/IEC JTC 1/SC 7 - Software and systems engineering
• Drafting Committee: ISO/IEC JTC 1/SC 7/WG 26 - Software testing
• Current Stage: 5060 - Close of voting Proof returned by Secretariat
• Start Date: 06-Aug-2021
• Completion Date: 05-Aug-2021

RELATIONS

• Revised: ISO/IEC/IEEE 29119-2:2013 - Software and systems engineering -- Software testing
• Effective Date: 23-Apr-2020

Standards Content (sample)

FINAL
INTERNATIONAL ISO/IEC/
DRAFT
STANDARD IEEE/FDIS
29119-2
ISO/IEC JTC 1/SC 7
Software and systems engineering —
Secretariat: BIS
Software testing —
Voting begins on:
2021-06-10
Part 2:
Voting terminates on:
Test processes
2021-08-05
Ingénierie du logiciel et des systèmes — Essais du logiciel —
Partie 2: Processus des essais
RECIPIENTS OF THIS DRAFT ARE INVITED TO
SUBMIT, WITH THEIR COMMENTS, NOTIFICATION
OF ANY RELEVANT PATENT RIGHTS OF WHICH
THEY ARE AWARE AND TO PROVIDE SUPPOR TING
DOCUMENTATION.
IN ADDITION TO THEIR EVALUATION AS
Reference number
BEING ACCEPTABLE FOR INDUSTRIAL, TECHNO-
ISO/IEC/IEEE/FDIS 29119-2:2021(E)
LOGICAL, COMMERCIAL AND USER PURPOSES,
DRAFT INTERNATIONAL STANDARDS MAY ON
OCCASION HAVE TO BE CONSIDERED IN THE
LIGHT OF THEIR POTENTIAL TO BECOME STAN-
ISO/IEC 2021
DARDS TO WHICH REFERENCE MAY BE MADE IN
NATIONAL REGULATIONS. IEEE 2021
---------------------- Page: 1 ----------------------
ISO/IEC/IEEE/FDIS 29119-2:2021(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2021
© IEEE 2021

All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting

on the internet or an intranet, without prior written permission. Permission can be requested from either ISO or IEEE at the

respective address below or ISO’s member body in the country of the requester.
ISO copyright office Institute of Electrical and Electronics Engineers, Inc
CP 401 • Ch. de Blandonnet 8 3 Park Avenue, New York
CH-1214 Vernier, Geneva NY 10016-5997, USA
Phone: [phone]
Email: [e-mail] Email: [e-mail]
Website: www.iso.org Website: www.ieee.org
Published in Switzerland
© ISO/IEC 2021 – All rights reserved
ii © IEEE 2021 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC/IEEE/FDIS 29119-2:2021(E)
Contents Page

Foreword ..........................................................................................................................................................................................................................................v

Introduction ................................................................................................................................................................................................................................vi

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ...................................................................................................................................................................................... 1

3 Terms and definitions ..................................................................................................................................................................................... 1

4 Conformance ............................................................................................................................................................................................................. 8

4.1 Intended usage ........................................................................................................................................................................................ 8

4.1.1 General...................................................................................................................................................................................... 8

4.1.2 Full conformance ............................................................................................................................................................. 9

4.1.3 Tailored conformance ...............................................................................................................................................10

5 Multi-layer test process model............................................................................................................................................................10

6 Organizational test process ....................................................................................................................................................................12

6.1 General ........................................................................................................................................................................................................12

6.2 Organizational test process .......................................................................................................................................................13

6.2.1 Overview ..............................................................................................................................................................................13

6.2.2 Purpose .................................................................................................................................................................................14

6.2.3 Outcomes .............................................................................................................................................................................14

6.2.4 Activities and tasks .....................................................................................................................................................14

6.2.5 Information items ........................................................................................................................................................15

7 Test management processes .................................................................................................................................................................16

7.1 General ........................................................................................................................................................................................................16

7.2 Test strategy and planning process ....................................................................................................................................17

7.2.1 Overview ..............................................................................................................................................................................17

7.2.2 Purpose .................................................................................................................................................................................18

7.2.3 Outcomes .............................................................................................................................................................................18

7.2.4 Activities and tasks .....................................................................................................................................................19

7.2.5 Information items ........................................................................................................................................................23

7.3 Test monitoring and control process ................................................................................................................................23

7.3.1 Overview ..............................................................................................................................................................................23

7.3.2 Purpose .................................................................................................................................................................................24

7.3.3 Outcomes .............................................................................................................................................................................24

7.3.4 Activities and tasks .....................................................................................................................................................25

7.3.5 Information items ........................................................................................................................................................26

7.4 Test completion process ..............................................................................................................................................................27

7.4.1 Overview ..............................................................................................................................................................................27

7.4.2 Purpose .................................................................................................................................................................................27

7.4.3 Outcomes .............................................................................................................................................................................27

7.4.4 Activities and tasks .....................................................................................................................................................28

7.4.5 Information items ........................................................................................................................................................29

8 Dynamic test processes ..............................................................................................................................................................................29

8.1 General ........................................................................................................................................................................................................29

8.2 Test design and implementation process .....................................................................................................................31

8.2.1 Overview ..............................................................................................................................................................................31

8.2.2 Purpose .................................................................................................................................................................................32

8.2.3 Outcomes .............................................................................................................................................................................32

8.2.4 Activities and tasks .....................................................................................................................................................32

8.2.5 Information items ........................................................................................................................................................35

8.3 Test environment and data management process ................................................................................................35

8.3.1 Overview ..............................................................................................................................................................................35

8.3.2 Purpose .................................................................................................................................................................................36

8.3.3 Outcomes .............................................................................................................................................................................36

© ISO/IEC 2021 – All rights reserved
© IEEE 2021 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC/IEEE/FDIS 29119-2:2021(E)

8.3.4 Activities and tasks .....................................................................................................................................................36

8.3.5 Information items ........................................................................................................................................................38

8.4 Test execution process ..................................................................................................................................................................38

8.4.1 Overview ..............................................................................................................................................................................38

8.4.2 Purpose .................................................................................................................................................................................39

8.4.3 Outcomes .............................................................................................................................................................................39

8.4.4 Activities and tasks .....................................................................................................................................................39

8.4.5 Information items ........................................................................................................................................................40

8.5 Test incident reporting process .............................................................................................................................................40

8.5.1 Overview ..............................................................................................................................................................................40

8.5.2 Purpose .................................................................................................................................................................................41

8.5.3 Outcomes .............................................................................................................................................................................41

8.5.4 Activities and tasks .....................................................................................................................................................41

8.5.5 Information items ........................................................................................................................................................42

Annex A (informative) Example application of the test design and implementation process ..............43

Annex B (informative) ISO/IEC/IEEE 29119-2 and ISO/IEC/IEEE 12207:2017 process alignment 47

Annex C (informative) ISO/IEC/IEEE 29119-2 and ISO/IEC 17025:2017 process alignment ................51

Annex D (informative) ISO/IEC/IEEE 29119-2 and BS 7925-2:1998 process alignment ............................52

Annex E (informative) Test models .....................................................................................................................................................................53

Bibliography .............................................................................................................................................................................................................................54

IEEE Notices and Abstract ...........................................................................................................................................................................................55

© ISO/IEC 2021 – All rights reserved
iv © IEEE 2021 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC/IEEE/FDIS 29119-2:2021(E)
Foreword

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical

Commission) form the specialized system for worldwide standardization. National bodies that are

members of ISO or IEC participate in the development of International Standards through technical

committees established by the respective organization to deal with particular fields of technical

activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international

organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the

work.

The procedures used to develop this document and those intended for its further maintenance are

described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the

different types of ISO/IEC documents should be noted. This document was drafted in accordance with

the rules given in the ISO/IEC Directives, Part 2 (see www .iso .org/ directives or www .iec .ch/ members

_experts/ refdocs).

IEEE Standards documents are developed within the IEEE Societies and the Standards Coordinating

Committees of the IEEE Standards Association (IEEE-SA) Standards Board. The IEEE develops its

standards through a consensus development process, approved by the American National Standards

Institute, which brings together volunteers representing varied viewpoints and interests to achieve the

final product. Volunteers are not necessarily members of the Institute and serve without compensation.

While the IEEE administers the process and establishes rules to promote fairness in the consensus

development process, the IEEE does not independently evaluate, test, or verify the accuracy of any of

the information contained in its standards.

Attention is drawn to the possibility that some of the elements of this document may be the subject

of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent

rights. Details of any patent rights identified during the development of the document will be in the

Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC

list of patent declarations received (see https:// patents .iec .ch).

Any trade name used in this document is information given for the convenience of users and does not

constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and

expressions related to conformity assessment, as well as information about ISO's adherence to the

World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www .iso .org/

iso/ foreword .html. In the IEC, see www .iec .ch/ understanding -standards.

ISO/IEC/IEEE 29119-2 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information

technology, Subcommittee SC 7, Software and systems engineering, in cooperation with the Systems and

Software Engineering Standards Committee of the IEEE Computer Society, under the Partner Standards

Development Organization cooperation agreement between ISO and IEEE.

This second edition cancels and replaces the first edition (ISO/IEC/IEEE 29119-2:2013), which has been

technically revised.
The main changes compared to the previous edition are as follows:

— The definition of the test design and implementation process (8.2) has been updated. In the first

edition, this process was based on the use of test conditions. Feedback on use of the standard

highlighted a problem with users’ understanding of ‘test conditions’ and their use for deriving

test cases. This second edition has replaced the use of ‘test conditions’ with ‘test models’. Annex E

provides more detail on this change.

A list of all parts in the ISO/IEC/IEEE 29119 series can be found on the ISO and IEC websites.

Any feedback or questions on this document should be directed to the user’s national standards body. A

complete listing of these bodies can be found at www .iso .org/ members .html and www .iec .ch/ national

-committees.
© ISO/IEC 2021 – All rights reserved
© IEEE 2021 – All rights reserved v
---------------------- Page: 5 ----------------------
ISO/IEC/IEEE/FDIS 29119-2:2021(E)
Introduction

The purpose of this document is to define a generic process model for software testing that can be

used by any organization when performing any form of software testing. It comprises test process

descriptions that define the software testing processes at the organizational level, test management

level and dynamic test levels. Supporting informative diagrams describing the processes are also

provided. ISO/IEC/IEEE 29119 (all parts) supports dynamic testing, functional and non-functional

testing, manual and automated testing, and scripted and unscripted testing. The processes defined in

this document can be used in conjunction with any software development lifecycle model. Each process

is defined using the generic process template that is provided in ISO/IEC TR 24774, and covers the

purpose, outcomes, activities, tasks and information items of each test process.

Testing is a key approach to risk treatment in software development. This document follows a risk-

based approach to testing. Risk-based testing is a best-practice approach to strategizing and managing

testing, as it allows testing to be prioritized and focused on the most important features and quality

attributes.

This document uses the traditional concept of organizations and projects, but some organizations,

especially those using an agile approach, do not organize their development in terms of projects; instead,

they run product development based on more long-lasting product teams. Users of this document can

substitute the term ‘product’ for ‘project’ where appropriate.

The concepts and definitions that support ISO/IEC/IEEE 29119 (all parts) are defined in

ISO/IEC/IEEE 29119-1. Templates and examples of test documentation that are produced during the

testing process are defined in ISO/IEC/IEEE 29119-3. Software test design techniques that can be used

during testing are defined in ISO/IEC/IEEE 29119-4.

ISO/IEC/IEEE 29119 (all parts) aims to provide those responsible for software testing with the

information required to manage and perform software testing in any organization.

Users of ISO/IEC/IEEE 12207 perform several activities and tasks which are related to software testing.

Annex B provides a mapping for such users between the clauses and subclauses of ISO/IEC/IEEE 12207

and the clauses and subclauses of this document.

Users of ISO/IEC 17025 perform several activities and tasks which are related to software testing.

Annex C provides a mapping for such users between the clauses and subclauses of ISO/IEC 17025 and

the clauses and subclauses of this document.

Users of BS 7925-2 perform several activities and tasks which are related to software component

testing. Annex D provides a mapping for such users between the clauses and subclauses of BS 7925-2

and the clauses and subclauses of this document.
© ISO/IEC 2021 – All rights reserved
vi © IEEE 2021 – All rights reserved
---------------------- Page: 6 ----------------------
FINAL DRAFT INTERNATIONAL STANDARD ISO/IEC/IEEE/FDIS 29119-2:2021(E)
Software and systems engineering — Software testing —
Part 2:
Test processes
1 Scope

This document specifies test processes that can be used to govern, manage and implement software

testing for any organization, project or testing activity. It comprises generic test process descriptions

that define the software testing processes. Supporting informative diagrams describing the processes

are also provided.

This document is applicable to testing in all software development lifecycle models.

This document is intended for, but not limited to, testers, test managers, developers and project

managers, particularly those responsible for governing, managing and implementing software testing.

2 Normative references

The following documents are referred to in the text in such a way that some or all of their content

constitutes requirements of this document. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any amendments) applies.

ISO/IEC/IEEE 12207, Systems and software engineering — Software life cycle processes

3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.

ISO, IEC and IEEE maintain terminological databases for use in standardization at the following

addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/

— IEEE Standards Dictionary Online: available at https:// ieeexplore .ieee .org/ xpls/ dictionary .jsp

NOTE For additional terms and definitions in the field of systems and software engineering, see

ISO/IEC/IEEE 24765, which is published periodically as a “snapshot” of the SEVOCAB (Systems and

software Engineering Vocabulary) database and is publicly accessible at www .computer .org/ sevocab.

ISO/IEC/IEEE 29119-1 includes all terms that are used in ISO/IEC/IEEE 29119 (all parts).

3.1
actual results

set of behaviours or conditions of a test item (3.42), or set of conditions of associated data or the test

environment (3.34), observed as a result of test execution (3.38)

EXAMPLE Outputs to screen, outputs to hardware, changes to data, reports and communication messages

sent.
3.2
completion criteria
conditions under which the testing (3.61) activities are considered complete
© ISO/IEC 2021 – All rights reserved
© IEEE 2021 – All rights reserved 1
---------------------- Page: 7 ----------------------
ISO/IEC/IEEE/FDIS 29119-2:2021(E)
3.3
dynamic testing
testing (3.61) in which a test item (3.42) is evaluated by executing it
3.4
expected results

observable predicted behaviour of the test item (3.42) under specified conditions based on its

specification or another source
3.5
exploratory testing

type of unscripted experience-based testing (3.61) in which the tester spontaneously designs and

executes tests based on the tester's existing relevant knowledge, prior exploration of the test item (3.42)

(including the results of previous tests), and heuristic "rules of thumb" regarding common software

behaviours and types of failure
3.6
incident

anomalous or unexpected event, set of events, condition, or situation at any time during the life cycle of

a project, product, service, or system
3.7
incident report
documentation of the occurrence, nature, and status of an incident (3.6)

Note 1 to entry: Incident reports are also known as anomaly reports, bug reports, defect reports, error reports,

issues, problem reports and trouble reports, amongst other terms.
3.8
organizational test practices

documentation that expresses the recommended approaches or methods for the testing (3.61) to be

performed within an organization, providing detail on how the testing is to be performed

Note 1 to entry: The organizational test practices is aligned with the organizational test policy (3.52).

Note 2 to entry: An organization can have more than one organizational test practices document to cover

markedly different contexts, such one for mobile apps and one for safety critical systems.

Note 3 to entry: The organizational test practices can incorporate the context of the test policy where no separate

test policy is available.
3.9
organizational test process

test process (3.55) for developing and managing organizational test specifications (3.10)

3.10
organizational test specification

document that provides information about testing (3.61) for an organization, i.e. information that is not

project-specific

EXAMPLE The most common examples of organizational test specifications are the organizational test policy

(3.52) and the organizational test practices (3.8).
3.11
performance testing

type of testing (3.61) conducted to evaluate the degree to which a test item (3.42) accomplishes its

designated functions within given constraints of time and other resources
3.12
product risk

risk that a product can be defective in some specific aspect of its function, quality, or structure

© ISO/IEC 2021 – All rights reserved
2 © IEEE 2021 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/IEC/IEEE/FDIS 29119-2:2021(E)
3.13
project risk
risk related to the management of a project
EXAMPLE Lack of staffing, strict deadlines, changing requirements.
3.14
regression testing

testing (3.61) to performed following modifications a test item (3.42) or to its operational environment,

to identify whether failures in unmodified parts of the test item occur

Note 1 to entry: Regression testing differs from retesting (3.15) in that it does not test that the modification

works correctly, but that other parts of the system have not been accidentally affected by the change.

Note 2 to entry: The adequacy of a set of regression test cases (3.23) depends on the item under test (3.21) and on

the modifications to that item or its operational environment.
3.15
retesting
confirmation testing

testing (3.61) performed to check that modifications made to correct a fault have successfully removed

the fault

Note 1 to entry: When retesting is performed it is often complemented by regression testing (3.14), to ensure that

other unmodified parts of the test item (3.42) have not been accidentally adversely affected by the modifications.

3.16
risk-based testing

testing (3.61) in which the management, selection, prioritisation, and use of testing activities and

resources are consciously based on corresponding types and levels of analysed risk

3.17
risk treatment
process to eliminate risk or reduce it to a tolerable level

Note 1 to entry: Risk treatment measures can include avoiding, optimizing, mitigating, transferring, or retaining

risk.

[SOURCE: ISO/IEC/IEEE 15026-1:2019, 3.3.11, modified — Note 1 to entry has been added.]

3.18
scripted testing
testing (3.61) performed based on a documented test script (3.54)
Note 1
...