Software and systems engineering -- Software testing

Ingénierie du logiciel et des systèmes -- Essais du logiciel

General Information

Status

Published

ICS

35.080 - Software

Technical Committee

ISO/IEC JTC 1/SC 7 - Software and systems engineering

Drafting Committee

ISO/IEC JTC 1/SC 7/WG 26 - Software testing

Current Stage

4060 - Close of voting

Start Date

03-Oct-2020

Completion Date

02-Oct-2020

Ref Project

RELATIONS

Revised

ISO/IEC/IEEE 29119-2:2013 - Software and systems engineering -- Software testing

Effective Date

23-Apr-2020

Buy Standard

Draft

ISO/IEC/IEEE DIS 29119-2 - Software and systems engineering -- Software testing

English language

50 pages

sale 15% off

Preview

sale 15% off

Preview

Draft

ISO/IEC/IEEE DIS 29119-2 - Software and systems engineering -- Software testing

English language

50 pages

sale 15% off

Preview

sale 15% off

Preview

Standards Content (sample)

DRAFT INTERNATIONAL STANDARD
ISO/IEC/IEEE/DIS 29119-2
ISO/IEC JTC 1/SC 7 Secretariat: BIS
Voting begins on: Voting terminates on:
2020-07-10 2020-10-02
Software and systems engineering — Software testing —
Part 2:
Test processes
Ingénierie du logiciel et des systèmes — Essais du logiciel —
Partie 2: Processus des essais
ICS: 35.080
THIS DOCUMENT IS A DRAFT CIRCULATED
FOR COMMENT AND APPROVAL. IT IS
THEREFORE SUBJECT TO CHANGE AND MAY
NOT BE REFERRED TO AS AN INTERNATIONAL
STANDARD UNTIL PUBLISHED AS SUCH.
IN ADDITION TO THEIR EVALUATION AS
BEING ACCEPTABLE FOR INDUSTRIAL,
This document is circulated as received from the committee secretariat.
TECHNOLOGICAL, COMMERCIAL AND
USER PURPOSES, DRAFT INTERNATIONAL
STANDARDS MAY ON OCCASION HAVE TO
BE CONSIDERED IN THE LIGHT OF THEIR
POTENTIAL TO BECOME STANDARDS TO
WHICH REFERENCE MAY BE MADE IN
Reference number
NATIONAL REGULATIONS.
ISO/IEC/IEEE/DIS 29119-2:2020(E)
RECIPIENTS OF THIS DRAFT ARE INVITED
TO SUBMIT, WITH THEIR COMMENTS,
NOTIFICATION OF ANY RELEVANT PATENT
ISO/IEC 2020
RIGHTS OF WHICH THEY ARE AWARE AND TO
PROVIDE SUPPORTING DOCUMENTATION. IEEE 2020
---------------------- Page: 1 ----------------------
ISO/IEC/IEEE/DIS 29119-2:2020(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2020
© IEEE 2020

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting

on the internet or an intranet, without prior written permission. Permission can be requested from either ISO or IEEE at the

respective address below or ISO’s member body in the country of the requester.
ISO copyright office Institute of Electrical and Electronics Engineers, Inc
CP 401 • Ch. de Blandonnet 8 3 Park Avenue, New York
CH-1214 Vernier, Geneva NY 10016-5997, USA
Phone: +41 22 749 01 11
Email: copyright@iso.org Email: stds.ipr@ieee.org
Website: www.iso.org Website: www.ieee.org
Published in Switzerland
© ISO/IEC 2020 – All rights reserved
ii © IEEE 2020 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC/IEEE/DIS 29119-2:2020(E)
Contents Page

Foreword ..........................................................................................................................................................................................................................................v

Introduction ..............................................................................................................................................................................................................................vii

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ...................................................................................................................................................................................... 1

3 Terms and definitions ..................................................................................................................................................................................... 1

4 Conformance ............................................................................................................................................................................................................. 8

4.1 Intended usage ........................................................................................................................................................................................ 8

4.1.1 Full conformance ............................................................................................................................................................. 9

4.1.2 Tailored conformance .................................................................................................................................................. 9

5 Multi-Layer Test Process Model ............................................................................................................................................................ 9

6 Organizational Test Process ..................................................................................................................................................................11

6.1 Introduction ...........................................................................................................................................................................................11

6.2 Organizational Test Process......................................................................................................................................................12

6.2.1 Overview ..............................................................................................................................................................................12

6.2.2 Purpose .................................................................................................................................................................................13

6.2.3 Outcomes .............................................................................................................................................................................13

6.2.4 Activities and tasks .....................................................................................................................................................13

6.2.5 Information items ........................................................................................................................................................14

7 Test Management Processes ..................................................................................................................................................................15

7.1 Introduction ...........................................................................................................................................................................................15

7.2 Test Planning Process ....................................................................................................................................................................16

7.2.1 Overview ..............................................................................................................................................................................16

7.2.2 Purpose .................................................................................................................................................................................17

7.2.3 Outcomes .............................................................................................................................................................................17

7.2.4 Activities and tasks .....................................................................................................................................................17

7.2.5 Information items ........................................................................................................................................................21

7.3 Test Monitoring and Control Process ...............................................................................................................................21

7.3.1 Overview ..............................................................................................................................................................................21

7.3.2 Purpose .................................................................................................................................................................................22

7.3.3 Outcomes .............................................................................................................................................................................22

7.3.4 Activities and tasks .....................................................................................................................................................23

7.3.5 Information Items ........................................................................................................................................................24

7.4 Test Completion Process .............................................................................................................................................................25

7.4.1 Overview ..............................................................................................................................................................................25

7.4.2 Purpose .................................................................................................................................................................................25

7.4.3 Outcomes .............................................................................................................................................................................25

7.4.4 Activities and tasks .....................................................................................................................................................26

7.4.5 Information Items ........................................................................................................................................................27

8 Dynamic Test Processes .............................................................................................................................................................................27

8.1 Introduction ...........................................................................................................................................................................................27

8.2 Test Design & Implementation Process ..........................................................................................................................29

8.2.1 Overview ..............................................................................................................................................................................29

8.2.2 Purpose .................................................................................................................................................................................30

8.2.3 Outcomes .............................................................................................................................................................................30

8.2.4 Activities and tasks .....................................................................................................................................................30

8.2.5 Information Items ........................................................................................................................................................32

8.3 Test Environment & Data Management Process .....................................................................................................32

8.3.1 Overview ..............................................................................................................................................................................32

8.3.2 Purpose .................................................................................................................................................................................33

8.3.3 Outcomes .............................................................................................................................................................................33

8.3.4 Activities and tasks .....................................................................................................................................................33

© ISO/IEC 2020 – All rights reserved
© IEEE 2020 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC/IEEE/DIS 29119-2:2020(E)

8.3.5 Information Items ........................................................................................................................................................35

8.4 Test Execution Process ..................................................................................................................................................................35

8.4.1 Overview ..............................................................................................................................................................................35

8.4.2 Purpose .................................................................................................................................................................................36

8.4.3 Outcomes .............................................................................................................................................................................36

8.4.4 Activities and tasks .....................................................................................................................................................36

8.4.5 Information Items ........................................................................................................................................................37

8.5 Test Incident Reporting Process ...........................................................................................................................................38

8.5.1 Overview ..............................................................................................................................................................................38

8.5.2 Purpose .................................................................................................................................................................................38

8.5.3 Outcomes .............................................................................................................................................................................38

8.5.4 Activities and tasks .....................................................................................................................................................38

8.5.5 Information Items ........................................................................................................................................................39

Annex A (informative) Example Application of the Test Design & Implementation Process .................40

Annex B (normative) ISO/IEC/IEEE 29119-2 and ISO/IEC 12207:2017 Process Alignment ..................43

Annex C (informative) ISO/IEC/IEEE 29119-2 and ISO/IEC 17025:2017 process alignment ................47

Annex D (informative) ISO/IEC/IEEE 29119-2 and BS 7925-2:1998 process alignment ............................48

Annex E (informative) Test Models .....................................................................................................................................................................49

Bibliography .............................................................................................................................................................................................................................50

© ISO/IEC 2020 – All rights reserved
iv © IEEE 2020 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC/IEEE/DIS 29119-2:2020(E)
Foreword

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical

Commission) form the specialized system for worldwide standardization. National bodies that are

members of ISO or IEC participate in the development of International Standards through technical

committees established by the respective organization to deal with particular fields of technical

activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international

organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the

work. In the field of information technology, ISO and IEC have established a joint technical committee,

ISO/IEC JTC 1.

IEEE Standards documents are developed within the IEEE Societies and the Standards Coordinating

Committees of the IEEE Standards Association (IEEE-SA) Standards Board. The IEEE develops its

standards through a consensus development process, approved by the American National Standards

Institute, which brings together volunteers representing varied viewpoints and interests to achieve the

final product. Volunteers are not necessarily members of the Institute and serve without compensation.

While the IEEE administers the process and establishes rules to promote fairness in the consensus

development process, the IEEE does not independently evaluate, test, or verify the accuracy of any of

the information contained in its standards.

International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.

The main task of ISO/IEC JTC 1 is to prepare International Standards. Draft International Standards

adopted by the joint technical committee are circulated to national bodies for voting. Publication as an

International Standard requires approval by at least 75 % of the national bodies casting a vote.

Attention is called to the possibility that implementation of this standard may require the use of subject

matter covered by patent rights. By publication of this standard, no position is taken with respect

to the existence or validity of any patent rights in connection therewith. ISO/IEEE is not responsible

for identifying essential patents or patent claims for which a license may be required, for conducting

inquiries into the legal validity or scope of patents or patent claims or determining whether any

licensing terms or conditions provided in connection with submission of a Letter of Assurance or a

Patent Statement and Licensing Declaration Form, if any, or in any licensing agreements are reasonable

or non-discriminatory. Users of this standard are expressly advised that determination of the validity

of any patent rights, and the risk of infringement of such rights, is entirely their own responsibility.

Further information may be obtained from ISO or the IEEE Standards Association.

ISO/IEC/IEEE 29119-2 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information

technology, Subcommittee SC 7, Software and systems engineering, in cooperation with the Software &

Systems Engineering Standards Committee of the IEEE Computer Society, under the Partner Standards

Development Organization cooperation agreement between ISO and IEEE.

ISO/IEC 29119 consists of the following standards, under the general title Software and systems

engineering — Software testing:
— Part 1: Concepts and definitions
— Part 2: Test processes
— Part 3: Test documentation
— Part 4: Test techniques
— Part 5: Keyword-driven testing

This is the second version of this standard. The main difference between the first version and the

second version is in the definition of the Test Design and Implementation Process (clause 8.2). In the

first version, this process was based on the use of test conditions. Feedback on use of the standard

highlighted a problem with users’ understanding of ‘test conditions’ and their use for deriving test

© ISO/IEC 2020 – All rights reserved
© IEEE 2020 – All rights reserved v
---------------------- Page: 5 ----------------------
ISO/IEC/IEEE/DIS 29119-2:2020(E)

cases. This second version has replaced the use of ‘test conditions’ with ‘test models’. Annex E provides

more detail on this change.
© ISO/IEC 2020 – All rights reserved
vi © IEEE 2020 – All rights reserved
---------------------- Page: 6 ----------------------
ISO/IEC/IEEE/DIS 29119-2:2020(E)
Introduction

The purpose of this standard is to define a generic process model for software testing that can be used by

any organization when performing any form of software testing. It comprises test process descriptions

that define the software testing processes at the organizational level, test management level and

dynamic test levels. Supporting informative diagrams describing the processes are also provided.

ISO/IEC/IEEE 29119 supports dynamic testing, functional and non-functional testing, manual and

automated testing, and scripted and unscripted testing. The processes defined in this document can be

used in conjunction with any software development lifecycle model. Each process is defined using the

generic process template that is provided in ISO/IEC TR 24774 Guidelines for Process Description, and

covers the purpose, outcomes, activities, tasks and information items of each test process.

Testing is a key approach to risk mitigation in software development. This document follows a risk-

based approach to testing. Risk-based testing is a best-practice approach to strategizing and managing

testing, as it allows testing to be prioritized and focused on the most important features and quality

attributes.

The concepts and vocabulary that support this series of international standards are defined in

ISO/IEC/IEEE 29119-1 Concepts and definitions. Templates and examples of test documentation that

are produced during the testing process are defined in ISO/IEC/IEEE 29119-3 Test documentation.

Software test design techniques that can be used during testing are defined in ISO/IEC/IEEE 29119-4

Test techniques.

This series of international standards aims to provide those responsible for software testing with the

information required to manage and perform software testing in any organization.
© ISO/IEC 2020 – All rights reserved
© IEEE 2020 – All rights reserved vii
---------------------- Page: 7 ----------------------
DRAFT INTERNATIONAL STANDARD ISO/IEC/IEEE/DIS 29119-2:2020(E)
Software and systems engineering — Software testing —
Part 2:
Test processes
1 Scope

This document specifies test processes that can be used to govern, manage and implement software

testing for any organization, project or testing activity. It comprises generic test process descriptions

that define the software testing processes. Supporting informative diagrams describing the processes

are also provided.

This document is applicable to testing in all software development lifecycle models.

This document is intended for, but not limited to, testers, test managers, developers and project

managers, particularly those responsible for governing, managing and implementing software testing.

2 Normative references

The following document, in whole or in part, is normatively referenced in this document and is

indispensable for its application. The latest edition of the referenced document (including any

amendments) applies.

ISO/IEC/IEEE 12207, Systems and software engineering — Software life cycle processes

NOTE Other International Standards useful for the implementation and interpretation of this document are

listed in the bibliography.
3 Terms and definitions

For the purposes of this document, the terms and definitions given in ISO/IEC/IEEE 24765 and the

following apply.

Note 1 to entry Use of the terminology in this document is for ease of reference and is not mandatory

for conformance with this document. The following terms and definitions are provided to assist with the

understanding and readability of this document. Only terms critical to the understanding of this document

are included. This clause is not intended to provide a complete list of testing terms. The Systems and Software

Engineering Vocabulary ISO/IEC/IEEE 24765 can be referenced for terms not defined in this clause. This source

is available at the following web site: http:// www .computer .org/ sevocab. All terms defined in this clause are also

intentionally included in ISO/IEC/IEEE 29119-1, as that international standard includes all terms that are used in

parts 1 to 5 of ISO/IEC/IEEE 29119.
3.1
actual results

set of behaviours or conditions of a test item, or set of conditions of associated data or the test

environment, observed as a result of test execution

EXAMPLE Outputs to screen, outputs to hardware, changes to data, reports and communication

messages sent.
3.2
completion criteria
conditions under which the testing activities are considered complete
© ISO/IEC 2020 – All rights reserved
© IEEE 2020 – All rights reserved 1
---------------------- Page: 8 ----------------------
ISO/IEC/IEEE/DIS 29119-2:2020(E)
3.3
dynamic testing
testing in which a test item is evaluated by executing it
3.4
expected results

observable predicted behaviour of the test item under specified conditions based on its specification or

another source
3.5
exploratory testing

type of unscripted experience-based testing in which the tester spontaneously designs and executes

tests based on the tester's existing relevant knowledge, prior exploration of the test item (including the

results of previous tests), and heuristic "rules of thumb" regarding common software behaviours and

types of failure
3.6
incident

anomalous or unexpected event, set of events, condition, or situation at any time during the life cycle of

a project, product, service, or system
3.7
Incident Report
documentation of the occurrence, nature, and status of an incident

Note 1 to entry: Incident reports are also known as anomaly reports, bug reports, defect reports, error reports,

issues, problem reports and trouble reports, amongst other terms.
3.8
Organizational Test Practices

documentation that expresses the generic requirements for the testing to be performed within an

organization, providing detail on how the testing is to be performed

Note 1 to entry: The Organizational Test Practices is aligned with the Organizational Test Policy.

Note 2 to entry: An organization could have more than one Organizational Test Practices document to cover

markedly different contexts, such one for mobile apps and one for safety critical systems.

Note 3 to entry: The Organizational Test Practices could incorporate the context of the Test Policy where no

separate Test Policy is available.
3.9
Organizational Test Process
test process for developing and managing organizational test specifications
3.10
Organizational Test Specification

document that provides information about testing for an organization, i.e. information that is not

project-specific

EXAMPLE The most common examples of organizational test specifications are the Organizational Test

Policy and the Organizational Test Practices.
3.11
performance testing

type of testing conducted to evaluate the degree to which a test item accomplishes its designated

functions within given constraints of time and other resources
3.12
product risk

risk that a product may be defective in some specific aspect of its function, quality, or structure

© ISO/IEC 2020 – All rights reserved
2 © IEEE 2020 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/IEC/IEEE/DIS 29119-2:2020(E)
3.13
project risk
risk related to the management of a project
EXAMPLE Lack of staffing, strict deadlines, changing requirements.
3.14
regression testing

testing performed following modifications to a test item or to its operational environment, to identify

whether failures in unmodified parts of the test item occur

Note 1 to entry: Regression testing differs from retesting in that it does not test that the modification works

correctly, but that other parts of the system have not been accidentally affected by the change.

Note 2 to entry: The adequacy of a set of regression test cases depends on the item under test and on the

modifications to that item or its operational environment.
3.15
retesting
confirmation testing

testing performed to check that modifications made to correct a fault have successfully removed the fault

Note 1 to entry: When retesting is performed it is often complemented by regression testing, which ensures that

other unmodified parts of the test item have not been accidentally adversely affected by the modifications.

3.16
risk-based testing

testing in which the management, selection, prioritisation, and use of testing activities and resources

are consciously based on corresponding types and levels of analysed risk
3.17
scripted testing
testing performed based on a documented test script

Note 1 to entry: This term normally applies to manually executed testing, rather than the execution of an

automated script.
3.18
security testing

type of testing conducted to evaluate the degree to which a test item, and associated data and

information, are protected so that unauthorized persons or systems cannot use, read, or modify them,

and authorized persons or systems are not denied access to them
3.19
static testing

testing in which a test item is examined against a set of quality or other criteria without the test item

being executed
EXAMPLE Reviews, static analysis.
3.20
test

activity in which a system or component is executed under specified conditions, the results are observed

or recorded, and an evaluation is made of some aspect of the system or component
3.21
test basis
information used as the basis for test planning and test design

Note 1 to entry: The test basis may take the form of documentation, such as a requirements specification,

design specification, or module specification, but may also be an undocumented understanding of the required

behaviour.
© ISO/IEC 2020 – All rights reserved
© IEEE 2020 – All rights reserved 3
---------------------- Page: 10 ----------------------
ISO/IEC/IEEE/DIS 29119-2:2020(E)
3.22
test case

set of test case preconditions, inputs (including actions, where applicable) and expected results,

develop
...

DRAFT INTERNATIONAL STANDARD
ISO/IEC/IEEE/DIS 29119-2
ISO/IEC JTC 1/SC 7 Secretariat: BIS
Voting begins on: Voting terminates on:
2020-07-10 2020-09-10
Software and systems engineering — Software testing —
Part 2:
Test processes
Ingénierie du logiciel et des systèmes — Essais du logiciel —
Partie 2: Processus des essais
ICS: 35.080
THIS DOCUMENT IS A DRAFT CIRCULATED
FOR COMMENT AND APPROVAL. IT IS
THEREFORE SUBJECT TO CHANGE AND MAY
NOT BE REFERRED TO AS AN INTERNATIONAL
STANDARD UNTIL PUBLISHED AS SUCH.
IN ADDITION TO THEIR EVALUATION AS
BEING ACCEPTABLE FOR INDUSTRIAL,
This document is circulated as received from the committee secretariat.
TECHNOLOGICAL, COMMERCIAL AND
USER PURPOSES, DRAFT INTERNATIONAL
STANDARDS MAY ON OCCASION HAVE TO
BE CONSIDERED IN THE LIGHT OF THEIR
POTENTIAL TO BECOME STANDARDS TO
WHICH REFERENCE MAY BE MADE IN
Reference number
NATIONAL REGULATIONS.
ISO/IEC/IEEE/DIS 29119-2:2020(E)
RECIPIENTS OF THIS DRAFT ARE INVITED
TO SUBMIT, WITH THEIR COMMENTS,
NOTIFICATION OF ANY RELEVANT PATENT
ISO/IEC 2020
RIGHTS OF WHICH THEY ARE AWARE AND TO
PROVIDE SUPPORTING DOCUMENTATION. IEEE 2020
---------------------- Page: 1 ----------------------
ISO/IEC/IEEE/DIS 29119-2:2020(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2020
© IEEE 2020

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting

on the internet or an intranet, without prior written permission. Permission can be requested from either ISO or IEEE at the

respective address below or ISO’s member body in the country of the requester.
ISO copyright office Institute of Electrical and Electronics Engineers, Inc
CP 401 • Ch. de Blandonnet 8 3 Park Avenue, New York
CH-1214 Vernier, Geneva NY 10016-5997, USA
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org Email: stds.ipr@ieee.org
Website: www.iso.org Website: www.ieee.org
Published in Switzerland
© ISO/IEC 2020 – All rights reserved
ii © IEEE 2020 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC/IEEE/DIS 29119-2:2020(E)
Contents Page