ISO/IEC 20246:2017

Software and systems engineering - Work product reviews

ISO/IEC 20246:2017 establishes a generic framework for work product reviews that can be referenced and used by all organizations involved in the management, development, test and maintenance of systems and software. It contains a generic process, activities, tasks, review techniques and documentation templates that are applied during the review of a work product. A work product is any artefact produced by a process. This document defines work product reviews that can be used during any phase of the life cycle of any work product. This document is intended for, but not limited to, project managers, development managers, quality managers, test managers, business analysts, developers, testers, customers and all those involved in the development, testing and maintenance of systems and software.

Ingénierie du logiciel et des systèmes — Examens de produits de travail

General Information

• Status: Published
• Publication Date: 26-Feb-2017
• ICS: 35.080 - Software
• Technical Committee: ISO/IEC JTC 1/SC 7 - Software and systems engineering
• Drafting Committee: ISO/IEC JTC 1/SC 7/WG 26 - Software testing
• Current Stage: 9093 - International Standard confirmed
• Start Date: 21-Dec-2022
• Completion Date: 30-Oct-2025

Overview

ISO/IEC 20246:2017 - Software and systems engineering - Work product reviews defines a generic framework for work product reviews applicable across software and systems life cycles. The standard describes a configurable review process, review types (inspections, walkthroughs, technical reviews, milestone reviews), review techniques (checklist-based, ad hoc, perspective-based reading, scenario-based, role-based), and documentation templates. It is intended for project managers, development and quality managers, test managers, business analysts, developers, testers and customers involved in development, testing and maintenance of systems and software.

Key technical topics and requirements

• Generic review process (Clause 6): A structured process with activities and tasks including planning, initiate review, individual review, issue communication & analysis, and fixing & reporting.
• Review attributes and types (Clause 5): Definitions and characteristics of formal vs informal reviews, inspections, walkthroughs, technical reviews and milestone reviews.
• Review techniques (Clause 7): Individual and group techniques such as ad hoc reviewing, checklist-based, scenario-based, perspective-based reading (PBR), role-based reviewing, pair review and page-by-page.
• Documentation & templates (Annex A, B): Normative documentation items and informative examples for capturing review outcomes, issues and decisions. Clause 6 and Annex A contain the normative requirements.
• Conformance options: Organizations may claim full or tailored conformance depending on selected techniques and scope.
• Support material (informative annexes): Mapping to IEEE 1028-2008 (Annex E), lifecycle mapping (Annex G), measurement and improvement (Annex H), and tool support (Annex I).

Practical applications and users

ISO/IEC 20246:2017 is used to:

• Improve work product quality and reduce rework by detecting defects early through structured reviews.
• Standardize review practices across teams and projects for consistent quality assurance.
• Configure review activities for different work products (requirements, design docs, source code, test plans, project plans, etc.). Typical users:
• Project managers, development managers, quality managers, test managers
• Developers, testers, business analysts
• Customers and stakeholders who need objective review evidence for milestone acceptance

Related standards

• ISO/IEC/IEEE 24765 - Systems and software engineering vocabulary (referenced normative vocabulary).
• IEEE 1028-2008 - Formal review processes (mapping provided in Annex E).

ISO/IEC 20246:2017 is a practical, lifecycle-aware standard for implementing repeatable, measurable work product reviews in software and systems engineering projects.

Standards Content (Sample)

INTERNATIONAL ISO/IEC
STANDARD 20246
First edition
2017-02
Software and systems engineering —
Work product reviews
Ingénierie du logiciel et des systèmes — Revue des produits de travail
Reference number
©
ISO/IEC 2017
© ISO/IEC 2017, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. [phone]
Fax [phone]
[e-mail]
www.iso.org
ii © ISO/IEC 2017 – All rights reserved

Contents Page
Foreword .v
Introduction .vi
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Conformance . 3
4.1 Intended usage . 3
4.2 Full conformance. 3
4.3 Tailored conformance . 3
5 Work product reviews. 4
5.1 Overview . 4
5.2 Review attributes . 4
5.3 Review types . 4
6 Work product review process . 5
6.1 Overview . 5
6.2 Purpose . 5
6.3 Outcomes . 5
6.4 Activities and tasks . 6
6.4.1 Planning . 6
6.4.2 Initiate review . 6
6.4.3 Individual review . . 7
6.4.4 Issue communication and analysis . 7
6.4.5 Fixing and reporting . 8
6.5 Information items . 8
7 Review techniques . 8
7.1 Overview . 8
7.2 Individual reviewing techniques . 9
7.2.1 Overview . 9
7.2.2 Ad hoc reviewing . 9
7.2.3 Checklist-based reviewing . 9
7.2.4 Scenario-based reviewing . 9
7.2.5 Perspective-based reading (PBR).10
7.2.6 Role-based reviewing .11
7.3 Issue analysis techniques .11
7.3.1 Overview .11
7.3.2 Individual analysis .11
7.3.3 Review meeting techniques .11
7.3.4 Group decision making .12
Annex A (normative) Review documentation .13
Annex B (informative) Review documentation examples .21
Annex C (informative) Review attributes .26
Annex D (informative) Review types .30
Annex E (informative) Mapping to IEEE 1028-2008 .34
Annex F (informative) Review selection based on work product .35
Annex G (informative) Reviews — Life cycle mapping .37
Annex H (informative) Review measurement and improvement .39
Annex I (informative) Tool support .41
© ISO/IEC 2017 – All rights reserved iii

Bibliography .42
iv © ISO/IEC 2017 – All rights reserved

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the meaning of ISO specific terms and expressions related to conformity assessment,
as well as information about ISO’s adherence to the World Trade Organization (WTO) principles in the
Technical Barriers to Trade (TBT) see the following URL: www . i so .org/ iso/ foreword .html.
This document was prepared by Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 7, Software and systems engineering.
© ISO/IEC 2017 – All rights reserved v

Introduction
The purpose of this document is to provide an International Standard that defines work product
reviews, such as inspections, reviews and walkthroughs that can be used at any stage of the software
and systems life cycle. It can be used to review any system or software work product. This document
defines a generic process for work product reviews that can be configured based on the purpose of the
review and the constraints of the reviewing organization. The intent is to describe a generic process
that can be applied both efficiently and effectively by any organization to any work product.
The main objectives of reviews are to detect issues, to evaluate alternatives, to improve organizational
and personal processes, and to improve work products. When applied early in the life cycle, reviews are
typically shown to reduce the amount of unnecessary rework on a project. The work product review
techniques presented in this document can be used at various stages of the generic review process to
identify defects and evaluate the quality of the work product.
Review documents that are produced during work product reviews are defined in Annex A.
vi © ISO/IEC 2017 – All rights reserved

INTERNATIONAL STANDARD ISO/IEC 20246:2017(E)
Software and systems engineering — Work product reviews
1 Scope
This document establishes a generic framework for work product reviews that can be referenced and
used by all organizations involved in the management, development, test and maintenance of systems
and software. It contains a generic process, activities, tasks, review techniques and documentation
templates that are applied during the review of a work product. A work product is any artefact
produced by a process. This document defines work product reviews that can be used during any phase
of the life cycle of any work product. This document is intended for, but not limited to, project managers,
development managers, quality managers, test managers, business analysts, developers, testers,
customers and all those involved in the development, testing and maintenance of systems and software.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC/IEEE 24765, Systems and software engineering — Vocabulary
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC/IEEE 24765 and the
following apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— IEC Electropedia: available at http:// www .electropedia .org/
— ISO Online browsing platform: available at http:// www .iso .org/ obp
3.1
ad hoc reviewing
unstructured independent review technique
3.2
author check
informal review performed by the author of the work product
3.3
buddy check
informal review performed independently by a colleague of the author
3.4
checklist-based reviewing
review technique guided by a list of questions or required attributes
3.5
formal review
form of review that follows a defined process with formal documented output
3.6
informal review
form of review that does not follow a defined process and has no formal documented output
© ISO/IEC 2017 – All rights reserved 1

3.7
informal group review
informal review performed by three or more persons
3.8
inspection
formal review of a work product to identify issues, which uses defined team roles and measurement to
improve the review process
[7]
EXAMPLE Fagan Inspections are a specific type of inspection and code inspections are used to review
program source code.
3.9
issue
observation that deviates from expectations
EXAMPLE Potential defect, improvement or point needing clarification.
3.10
milestone review
formal review of a work product and supporting evidence used to determine its acceptability for use in
the next stage of development or for delivery
Note 1 to entry: The requirement for this form of review is normally specified in the project plan.
3.11
page-by-page reviewing
technique where reviewers review a work product in a sequential order
3.12
pair review
informal review of a work product performed by two suitably qualified people other than the author
working together
3.13
peer desk check
informal review where the author and a colleague walk through a work product
3.14
peer review
review of work products performed by others qualified to do the same work
3.15
perspective-based reading
form of role-based reviewing that uses checklists and involves the creation of prototype deliverables to
check the completeness and other quality characteristics of the work product
3.16
role-based reviewing
technique where reviewers review a work product from the perspective of different stakeholder roles
EXAMPLE Typical stakeholder roles include specific user types, such as work product maintainer, tester and
developer.
3.17
scenario-based reviewing
technique where the review is guided by determining the ability of the work product to address specific
scenarios
2 © ISO/IEC 2017 – All rights reserved

3.18
technical review
formal peer review of a work product by a team of technically-qualified personnel that examines the
suitability of the work product for its intended use and identifies discrepancies from specifications and
standards
Note 1 to entry: Technical review may also provide recommendations of alternatives and examination of various
alternatives.
3.19
walkthrough
formal review in which an author leads members of the review through a work product, and the
participants ask questions and make comments about possible issues
3.20
work product
artefact produced by a process
EXAMPLE Project plan, requirements specification, design documentation, source code, test plan, test
meeting minutes, schedules, budgets, and incident reports.
Note 1 to entry: A subset of the work products can be baselined to be used as the basis of further work and some
will form the set of project deliverables.
4 Conformance
4.1 Intended usage
The normative requirements in this document are contained in Clause 6 and Annex A. It is recognized
that particular projects or organizations may not need to use all of the techniques defined by this
document. Therefore, implementation of this document typically involves selecting a set of techniques
suitable for the project or organization. There are two ways that an organization or individual can claim
conformance to the provisions of this document. The organization or individual shall assert whether
full or tailored conformance to this document is claimed.
4.2 Full conformance
Full conformance is achieved by demonstrating that all of the requirements (i.e. “shall” statements)
of the work product review process defined in Clause 6 and the review documentation annex of this
document have been satisfied.
4.3 Tailored conformance
When this document is used as a basis for establishing a review process that does not qualify for full
conformance, the subset of activities for which tailored conformance is claimed, is recorded. Tailored
conformance is achieved by demonstrating that all of the requirements (i.e. “shall” statements) for the
recorded subset of activities have been satisfied.
Where tailoring occurs, justification shall be provided (either directly or by reference), whenever an
activity defined in Clause 6 of this document is not followed. All tailoring decisions shall be recorded
with their rationale, including the consideration of any applicable risks. Tailoring decisions shall be
agreed by the relevant stakeholders.
© ISO/IEC 2017 – All rights reserved 3

5 Work product reviews
5.1 Overview
Work product reviews are performed on many projects, typically as a means of contributing to the
early detection of defects, so that these defects can be removed as early as possible thus reducing
unnecessary rework. In practice, reviews are performed for a variety of purposes in addition to defect
detection (examples are listed in C.1.2.1).
Reviews can be classified in a number of ways. In this document, reviews are classified as either formal
or informal. Many review techniques can be used over the course of a review, such as role-based
reviewing for individual review and checklist-based reviewing during a review meeting.
The generic process for conducting work product reviews (defined in Clause 6) includes a number of
selectable attributes (including review techniques). This allows users to configure their specific review
type according to their unique situation. These attributes are described in detail in Annex C. This
configuration of the generic process allows users to define reviews that suit their purpose while still
conforming to their constraints in the most effective and efficient manner, rather than forcing them to
choose a specific named review type that they cannot practically use in full.
Historically in the literature a number of distinct review types have been defined but some differ
only in the extent to which a particular attribute is emphasized (these types are listed in 5.3 and the
mapping between the characteristics and review types is provided in Annex D). For example, some
believe the difference between inspection and technical review simply to be that inspection requires
process improvement.
5.2 Review attributes
The following is a list of review attributes that can be used to define the review to be performed.
Annex C provides more detail on each of the attributes.
— Purpose (see C.1.2.1);
— Roles (see C.1.2.2);
— Individual review techniques (see C.1.2.3);
— Optional activities (see C.1.2.4);
— Number of reviewers (see C.1.2.5);
— Planned number of reviews (see C.1.2.6);
— Formal reporting (see C.1.2.7);
— Training required (see C.1.2.8);
— Review improvement (see C.1.2.9);
— Entry and exit criteria (see C.1.2.10).
Annex F provides guidelines on the selection of review attributes for different work product types and
work product formats.
5.3 Review types
[13]
The following is a list of review types commonly referenced in the literature and found in IEEE 1028.
Annex E describes the alignment of the activities defined in this document with the procedures of
4 © ISO/IEC 2017 – All rights reserved

IEEE 1028-2008. Annex D provides more detail on each of the types and maps the relevant attributes
from 5.2 to the different review types.
— Author check;
— Buddy check;
— Informal group review;
— Inspection;
— Milestone review;
— Pair review;
— Peer desk check;
— Technical review;
— Walkthrough.
Annex G provides examples of how each review type can be used within specific software/systems
development life cycle models. Users of this document are not restricted to using the above review
types. They can also use hybrid types based on selected attributes applied to the generic review process
according to their needs.
6 Work product review process
6.1 Overview
The Work Product Review Process comprises activities for the review of work products (see Figure 1).
The process shown in Figure 1 is not always performed on “complete” work products, but can be
performed on parts of work products, and in this situation these activities will typically be invoked
a number of times to complete the review for a complete work product. Thus, the process shown in
Figure 1 can be applied more than once on a single work product.
Figure 1 — Work Product Review Process
6.2 Purpose
The purpose of the Work Product Review Process is to provide a structured but flexible framework
from which review processes (both formal and informal) may be tailored for specific contexts and
purposes.
6.3 Outcomes
As a result of the successful implementation of the Work Product Review Process:
a) defects/issues in the work product are identified;
b) quality characteristics of the work product are evaluated;
NOTE A list of quality characteristics can be found in the ISO/IEC 25000 series of standards.
© ISO/IEC 2017 – All rights reserved 5

c) reviewers have gained knowledge about the work product;
d) consensus on decisions made has been reached;
e) new ideas have been generated;
f) updates to the work product are made;
g) participants have identified potential improvements in their working practices.
6.4 Activities and tasks
The person(s) responsible for the work product review shall implement the following activities and
tasks in accordance with applicable organization policies and procedures with respect to the Work
Product Review Process.
6.4.1 Planning
This activity consists of the following tasks:
a) The scope of the review, which comprises the purpose, the work product to be reviewed, quality
characteristics to be evaluated, areas to focus on, exit criteria, supporting information such as
standards, effort and the timeframes for the review, shall be defined.
NOTE 1 The work product to be reviewed can be part of a larger work product.
EXAMPLE 1 Areas to focus on can be specific features, non-functional attributes or selected pages.
b) The review characteristics shall be identified and agreed.
EXAMPLE 2 Review characteristics can include review activities, roles, effort, individual review
techniques and checklists.
NOTE 2 The responsibility for identifying and agreeing the review characteristics usually involves roles
such as the review leader, management and reviews coordinator as defined in C.1.2.2.
c) The review participants, along with their expected roles, shall be identified and agreed.
6.4.2 Initiate review
This activity consists of the following tasks:
a) Required review materials shall be distributed to review participants.
EXAMPLE Review materials can include, but are not limited to, the work product, checklists, review
guidelines and the baseline specification.
b) The review leader shall communicate the scope and characteristics of the review to the review
participants.
c) The review leader shall communicate the roles, responsibilities and focus to each review
participant.
d) The author (or a suitably qualified person) may describe the work product under review.
NOTE 1 Tasks b), c), and d) can be performed at an overview meeting.
NOTE 2 The decision to hold an overview meeting typically depends on factors such as whether the
reviewers:
— have previously participated in or been trained in formal reviews;
— know and understand the review process to be used;
6 © ISO/IEC 2017 – All rights reserved

— understand its objectives (e.g. documenting issues versus proposing resolutions);
— are familiar with the concept of assigned roles, the requirements of their specific roles, the classification
system for issues and the forms and tools (see Annex I) to be used in the process;
— require additional background information about the work product or its context.
e) Training for reviewers may be arranged.
6.4.3 Individual review
This activity consists of the following tasks:
a) Each reviewer shall perform a review to identify issues with the work product.
NOTE 1 Issues will typically be documented in an Issue Log (as described in A.2) and will be classified in
terms of severity.
NOTE 2 Issues can be supported by proposed changes.
6.4.4 Issue communication and analysis
This activity consists of the following tasks:
a) Identified issues shall be communicated.
NOTE 1 If a review meeting is held, then issues can be voiced at the meeting or can be sent for collation
and prioritization prior to the meeting.
NOTE 2 If a review meeting is not held, then issues are typically sent to the individual performing the
analysis.
b) Previously identified issues, and any new issues identified during this activity, shall be analysed to
assign them a status based on the subsequent action to be taken on them.
EXAMPLE 1 Typical examples of issue status are “rejected”, “issue to be noted but no action” and
“issue to be addressed”.
c) Issues shall be assigned to an appropriate individual or team based on their status.
NOTE 3 In an informal review the assignment and status of an issue do not need to be documented.
EXAMPLE 2 This can include the assignment of issues to work product authors or individuals (or
teams) external to the review (where an issue relates to supporting documentation, such as an organization-
wide standard).
d) The quality characteristics of the work product under review shall be evaluated and, along with
other relevant criteria, used to make the review decision.
EXAMPLE 3 Relevant criteria used to make the review decision can include the time or budget available.
EXAMPLE 4 Review decision outcomes typically include whether the reviewed work product will be “used as
is”, “updated based on the identified issues and used”, “reworked and re-reviewed” or “discarded”. In the event
that the review decision is to discard the work product, then the status of all issues would need to be suitably
updated.
NOTE 4 Tasks in this activity can be performed by an individual (such as the author of the work product under
review), a reviewer, a number of reviewers, or they can be performed as part of a review meeting.
© ISO/IEC 2017 – All rights reserved 7

6.4.5 Fixing and reporting
This activity consists of the following tasks:
a) Incident reports for those issues that require changes to artefacts other than the work product
shall be created and communicated to the assigned person or team.
EXAMPLE 1 If the work product under review is a design specification, then incident reports on
supporting documentation, such as the requirements specification and organizational design standards, can
be raised.
b) Issues with a status of requiring a change to the work product shall be actioned.
NOTE 1 This task is normally performed by the author of the work product.
EXAMPLE 2 This task can include further analysis of an issue, implementation of a solution, or a
decision not to change the work product.
c) The completion of review actions on the work product shall be confirmed; otherwise their status
shall be updated.
NOTE 2 In an informal review the change of status does not need to be documented.
NOTE 3 Review actions or changes to status can require agreement of the comment originator.
d) The reviewed work product shall be accepted when the review decision outcome has been satisfied.
NOTE 4 When the review decision outcome has not been satisfied, steps b) and c) will normally be
repeated.
NOTE 5 Depending on the level of risk, a meeting of relevant stakeholders might be held to determine the
outcome of the review.
e) The results of the review shall be reported.
6.5 Information items
As a result of carrying out this process, the following information items shall be produced:
a) Issue Log (see A.2);
b) Incident Report (see A.3);
c) Review Report (see A.4).
NOTE 1 Formal documentation (Issue Log, Incident Reports and Review Report) is not always required and
verbal reports can be produced in some situations. Formal documentation is rarely required for the following
review types: informal group reviews, author check, buddy check, pair review and peer desk check.
NOTE 2 Issue descriptions or a reference to the Issue Log are typically included in the Review Report.
7 Review techniques
7.1 Overview
This document defines a number of review techniques associated with the various activities that were
defined in the generic review process that was described in Clause 6, including techniques for individual
reviewing (7.2) and issue analysis (7.3).
8 © ISO/IEC 2017 – All rights reserved

7.2 Individual reviewing techniques
7.2.1 Overview
The techniques associated with the “Individual Reviewing” activity in 6.4.3 are used to identify issues
(which might be defects) in the work product under review.
7.2.2 Ad hoc reviewing
Ad hoc reviewing is a very common approach to issue detection by reviewers; it is completely
unstructured. Each reviewer is expected to find as many defects as possible of any type, but are
provided with little or no guidance on how this task should be performed. Reviewers often read the
work product sequentially, on a page-by-page basis, identifying and documenting issues, as they
encounter them in the work product. This approach is highly dependent on reviewer skills and often
leads to the same issues being identified by different reviewers.
7.2.3 Checklist-based reviewing
Checklist-based reviewing is a systematic approach to identifying issues that is based on checklists.
If different reviewers are assigned different checklists, then this provides wider coverage overall and
helps prevent the duplication inherent in the ad hoc approach. One disadvantage of using checklists
is that some reviewers limit themselves to only considering the checklist entries and ignore other
potential issues with the work product under review. Care should be taken to make reviewers aware
that they have a wider responsibility than simply following the checklist.
Typically, review checklists take the form of a set of questions based on potential defects, which may
be derived from experience within the project, the organization or across the industry as a whole.
Checklists should be specific to the type of work product under review. A checklist for a requirements
document will be different to one for a design document or a test plan, and may be specific to the
methodology used to develop the work product (e.g. there may be different checklist questions for
requirements in the form of plain text to those in the form of use cases or user stories). Checklists
may also be specific to the application domain of the work product (e.g. a checklist for a banking work
product may be based on banking regulations while a checklist for an avionics work product would be
based on avionics standards).
Typical problems with checklists are that they are too long and never change. The ideal checklist should
be constrained to about 10 entries and regularly updated; as entries become stale and find fewer issues
(hopefully because the authors have learned and improved) then they should be replaced with newer
entries reflecting issues missed in the recent past. It is possible to enhance the checklist-based approach
by using risk information to ensure that those defects that have the highest impact on the business and
have the highest probability of occurring are included in the checklists and so are explicitly looked for
during the reviews.
7.2.4 Scenario-based reviewing
With scenario-based reviewing, reviewers are provided with structured guidelines on how to read
through the work product under review. Where requirements, designs or tests are documented in a
suitable format (e.g. use cases) then a scenario-based approach supports reviewers in performing “dry
runs” on the work product based on expected usage of the work product. Another form of scenario-
based reviewing is based on detecting specific defect types (as with checklist-based reviewing), this
is also known as defect-based reading. When used to identify specific defect types, these scenarios
provide reviewers with structured reading guidelines on how to identify different fault types, which
are more detailed than simple checklist entries.
There is a danger that if this form of review is used in isolation, and thus is constrained to the documented
scenarios, other defects that are not specifically targeted by the scenarios will be missed, such as defects
of omission, where required functionality is not included in the work product under review.
© ISO/IEC 2017 – All rights reserved 9

As with the checklist-based approach, it is possible to enhance the scenario-based approach with risk
information to ensure that the most important scenarios to the business and the most frequently used
scenarios are reviewed in more depth.
7.2.5 Perspective-based reading (PBR)
[17]
According to available research , the most generally effective and efficient form of defect detection
for formal reviews is perspective-based reading. In perspective-based reading, reviewers take on
different stakeholder viewpoints and review the work product from that stakeholder’s viewpoint. The
idea is that if all stakeholders are happy with a work product and believe it can be used from their
viewpoint then it should be of high quality. Using separate stakeholder viewpoints means that each
reviewer can check the work product from their stakeholder’s view in more depth with less duplication
of effort between reviewers.
Typical stakeholder viewpoints used in PBR are:
— business analyst;
— business owner;
— designer;
— maintainer;
— marketing;
— operations;
— programmer;
— regulator;
— tester;
— user.
It is important that the correct balance of viewpoints is included in the review. For instance, if
reviewing a requirements document, then the user, designer and tester viewpoints would normally be
the most important to cover. If a system is being built within a highly regulated area, then the regulator
viewpoint should be included. If the system is to be long-lived, then the maintainer viewpoint becomes
more important.
Not all reviewers can easily take on a new role and so PBR scenarios are used to make this approach
easier to use. These scenarios comprise three parts:
— The first part describes the stakeholder view that the reviewer should take during the review.
— The second part describes the high-level product that the stakeholder would be expected to develop
from the work product under review (e.g. a tester may be expected to develop an acceptance test
plan based on the requirements specification). In PBR the reviewer is often expected to create a
first draft of this product to “test” whether it is possible to create it from the information provided
in the work product under review (these first drafts can form the basis of subsequent development
and testing).
— The third part of the PBR scenario typically comprises a checklist of questions specific to the high-
level product developed in the second part.
PBR scenarios are specific to the type of work product under review (e.g. a designer PBR scenario for
a requirements specification), but once created should be updated as appropriate to keep them useful
(e.g. updating questions in the third part) and reused as needed.
10 © ISO/IEC 2017 – All rights reserved

7.2.6 Role-based reviewing
Role-based reviewing is a review technique in which the reviewer evaluates the work product from the
perspective of various stakeholder roles, which may be different from their day-to-day role. The roles
will be similar to the stakeholder viewpoints used in perspective-based reading (see 7.2.5) and thus the
same principles apply.
7.3 Issue analysis techniques
7.3.1 Overview
The techniques associated with the “Issue Communication and Analysis” activity in 6.4.4 are used to
analyse raised issues either by an individual or in the context of a review meeting.
7.3.2 Individual analysis
Individual analysis is usually performed on the outputs generated during the “Individual Review”
activity in 6.4.3. Initially the issues are gathered from the independent reviewers, and collated and
prioritized based on their severity. The issues are analysed to determine appropriate actions to be
performed to address them. The result of this analysis is communicated back to the individual who
raised the issue.
NOTE 1 This approach is especially useful for distributed teams as an alternative to review meetings.
NOTE 2 If there are sufficient issues of high enough severity, a review meeting can be called.
NOTE 3 This technique is normally performed by the author.
7.3.3 Review meeting techniques
7.3.3.1 Overview
The techniques associated with Review Meetings (which can take place as part of the “Issue
communication and analysis” activity in 6.4.4) describe various ways for a group of reviewers to
consider previously-raised issues, identify new issues, analyse issues and sometimes decide on how to
address these issues depending on the review type.
7.3.3.2 Page-by-page reviewing
Page-by-page reviewing is a review technique in which the participants review the work product in
a sequential order, considering previously-identified issues and identifying additional issues, as they
encounter them in the work product.
7.3.3.3 Checklist-based reviewing
Checklist-based reviewing is a review technique in which the participants review the work product
using a checklist to focus the review on specific aspects of the work product.
7.3.3.4 Role-based reviewing
Role-based reviewing is a review technique in which the participants review the work product from the
perspective of various stakeholder roles, which may be different from their day-to-day role.
NOTE Reviewers will often have also used a role-based approach taking on the same stakeholder role to
identify issues during individual reviewing.
© ISO/IEC 2017 – All rights reserved 11

7.3.3.5 Scenario-based reviewing
Where work products (e.g. requirements, designs or tests) are documented in a suitable format
(e.g. use cases), then a scenario-based approach to defect detection may be the most appropriate. With
this approach, the reviewers perform “dry runs” on the work product to check whether the correct
functionality is described and typical error conditions are handled suitably. There is a danger that
scenario-based reviewing is constrained to documented scenarios and so misses defects of omission,
where required functionality is not included in the work product under review.
As with the checklist-based approach, it is possible to enhance the scenario-based approach with risk
information to ensure that the most important scenarios to the business and the most frequently-used
scenarios are reviewed in more depth.
The walkthrough is an example of a review type that is based on the scenario-based reviewing
technique, where the author leads the reviewers through the work product content. The attributes of
walkthroughs are defined in Annex D.
7.3.4 Group decision making
When the group needs to make decisions, if the best-qualified group members can be identified, then
getting them to make group decisions will normally be the most effective approach (giving equal weight
to all reviewers’ input will normally result in worse decisions).
NOTE When identifying the best-qualified members, different criteria will apply depending on the work
product type and purpose of the review.
12 © ISO/IEC 2017 – All rights reserved

Annex A
(normative)
Review documentation
A.1 Overview
This annex describes the information that shall be included in the Issue Log, Incident Report and
Review Report, but use of the nomenclature provided here for the specific records is not mandatory
and other terms may be used. In practice, these do not have to be published as single documents, but
may be made available in electronic form or may be divided into separate documents or combined with
other documents.
Whe
...