Systems and software engineering -- Capabilities of review tools

This document specifies the capabilities of a tool to support review work. The evaluation and selection of the review tools are performed in accordance with ISO/IEC 20741 which defines the general evaluation selection process and evaluation characteristics. This document defines capabilities specific to review tools in the process. By using these two standards together, it is possible to derive objective and reasonable results of the evaluation and selection of review tools. The review work is based on the process, activities, and tasks defined in ISO/IEC 20246. It is also assumed that the review targets are defined in ISO/IEC 20246. The review work in this document is assumed not to be performed by a 3rd party, but within a project. The review tool capabilities specified in this document harmonize with the review process defined in ISO/IEC 20246. This document does not include automated process, activities, or tasks for conducting reviews such as automated source code checkers defined in ISO/IEC 30130. Issues which are identified in the review are recorded and managed by the tool; but defects found in tests and issues found in general except for reviews are out of the scope of this document.

Ingénierie des systèmes et du logiciel -- Capacités des outils d'analyse

General Information

Status

Published

Publication Date

07-Jul-2020

ICS

35.080 - Software

Technical Committee

ISO/IEC JTC 1/SC 7 - Software and systems engineering

Drafting Committee

ISO/IEC JTC 1/SC 7/WG 4 - Tools and environment

Current Stage

5060 - Close of voting Proof returned by Secretariat

Start Date

02-Jun-2020

Completion Date

01-Jun-2020

Ref Project

Buy Standard

ISO/IEC 23396:2020 - Systems and software engineering -- Capabilities of review tools

Standard

ISO/IEC 23396:2020 - Systems and software engineering -- Capabilities of review tools

English language

31 pages

sale 15% off

Preview

sale 15% off

Preview

ISO/IEC FDIS 23396 - Systems and software engineering -- Capabilities of review tools

Draft

ISO/IEC FDIS 23396 - Systems and software engineering -- Capabilities of review tools

English language

31 pages

sale 15% off

Preview

sale 15% off

Preview

Standards Content (sample)

INTERNATIONAL ISO/IEC
STANDARD 23396
First edition
2020-07
Systems and software engineering —
Capabilities of review tools
Ingénierie des systèmes et du logiciel — Capacités des outils d'analyse
Reference number
ISO/IEC 23396:2020(E)
ISO/IEC 2020
---------------------- Page: 1 ----------------------
ISO/IEC 23396:2020(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2020

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting

on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address

below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2020 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC 23396:2020(E)
Contents Page

Foreword ..........................................................................................................................................................................................................................................v

Introduction ................................................................................................................................................................................................................................vi

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ...................................................................................................................................................................................... 1

3 Terms and definitions ..................................................................................................................................................................................... 1

4 Object model for review tools ................................................................................................................................................................. 2

4.1 Overview of the object model .................................................................................................................................................... 2

4.2 Object model of review entities ................................................................................................................................................ 2

4.3 Category of capabilities for review tools .......................................................................................................................... 3

5 Entities of review tools ................................................................................................................................................................................... 5

5.1 Overview ...................................................................................................................................................................................................... 5

5.2 ReviewFolder ............................................................................................................................................................................................ 5

5.3 Review ............................................................................................................................................................................................................ 6

5.4 Member ......................................................................................................................................................................................................... 6

5.5 WorkProduct ............................................................................................................................................................................................. 7

5.6 IssueByReview ........................................................................................................................................................................................ 7

5.7 PlaceOfIssueByReview ..................................................................................................................................................................... 7

6 Capabilities of review tools ....................................................................................................................................................................... 8

6.1 Overview ...................................................................................................................................................................................................... 8

6.2 Administration ........................................................................................................................................................................................ 8

6.2.1 Overview ................................................................................................................................................................................. 8

6.2.2 Administration of review folder ......................................................................................................................... 8

6.2.3 Review definition ............................................................................................................................................................ 9

6.2.4 Control of review implementation ................................................................................................................... 9

6.2.5 Dashboard of review progress ..........................................................................................................................10

6.3 IssueByReview management ...................................................................................................................................................10

6.3.1 Creation, deletion, reference, and update of issues ........................................................................10

6.3.2 Highlighting place of issue ....................................................................................................................................11

6.3.3 Transition of issue status .......................................................................................................................................11

6.3.4 Issue confirmation support .................................................................................................................................11

6.3.5 Similar issue detection support .......................................................................................................................11

6.3.6 Issue entry support ....................................................................................................................................................12

6.4 Review metrics collection and report output ............................................................................................................12

6.4.1 Review metrics collection .....................................................................................................................................12

6.4.2 Report output ..................................................................................................................................................................12

6.5 Human communication support ...........................................................................................................................................13

6.5.1 Overview ..............................................................................................................................................................................13

6.5.2 Notification of events and schedule information of the review ...........................................13

6.5.3 Online chat between review participants ...............................................................................................13

6.5.4 Messaging with specific review participants .......................................................................................13

6.5.5 Sharing issues among review participants ............................................................................................13

6.5.6 Sharing work product view on a screen among review participants ..............................13

6.6 Linkage with other tools ..............................................................................................................................................................13

6.6.1 Overview ..............................................................................................................................................................................13

6.6.2 Importing and exporting member definition .......................................................................................14

6.6.3 Importing and exporting classification definition ...........................................................................14

6.6.4 Importing and exporting viewpoints of review .................................................................................14

6.6.5 Importing work products to be reviewed ...............................................................................................14

6.6.6 Exporting identified issues ..................................................................................................................................14

6.6.7 Exporting review findings ....................................................................................................................................14

Annex A (informative) Review tool entities and attributes .......................................................................................................15

Annex B (informative) Review tool entities CRUD matrix ...........................................................................................................19

Annex C (informative) Review tool capabilities and review activities ...........................................................................20

Annex D (informative) Scope of review tools ............................................................................................................................................22

Annex E (informative) How to use this document with ISO/IEC 20741 ........................................................................23

Annex F (informative) Use case of a review tool ....................................................................................................................................24

Bibliography .............................................................................................................................................................................................................................31

iv © ISO/IEC 2020 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC 23396:2020(E)
Foreword

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical

Commission) form the specialized system for worldwide standardization. National bodies that

are members of ISO or IEC participate in the development of International Standards through

technical committees established by the respective organization to deal with particular fields of

technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other

international organizations, governmental and non-governmental, in liaison with ISO and IEC, also

take part in the work.

The procedures used to develop this document and those intended for its further maintenance are

described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for

the different types of document should be noted. This document was drafted in accordance with the

editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).

Attention is drawn to the possibility that some of the elements of this document may be the subject

of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent

rights. Details of any patent rights identified during the development of the document will be in the

Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC

list of patent declarations received (see http:// patents .iec .ch).

Any trade name used in this document is information given for the convenience of users and does not

constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and

expressions related to conformity assessment, as well as information about ISO's adherence to the

World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www .iso .org/

iso/ foreword .html.

This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,

Subcommittee SC 7, Software and systems engineering.

Any feedback or questions on this document should be directed to the user’s national standards body. A

complete listing of these bodies can be found at www .iso .org/ members .html.
© ISO/IEC 2020 – All rights reserved v
---------------------- Page: 5 ----------------------
ISO/IEC 23396:2020(E)
Introduction

It is widely recognized that almost all software development organizations need to improve the quality

of software at an appropriate cost and review is one important means. Applying reviews early in the

lifecycle is known to reduce the amount of unnecessary rework in regular projects. This is because

problem detection in upstream requires less cost than detection of problems by performing tests in

downstream processes. By conducting the review in this way, it is possible to detect problems early,

evaluate alternatives, improve organizational and personal processes, and improve work outcomes.

ISO/IEC 20246 specifies a standard process for work product reviews.

Review support tools provide capabilities to improve review work and improve review quality, for

example support for collaboration and communication between reviewers, logging and highlighting

comments made on a work product, and support for review report generation.

Support tools are indispensable in the large-scale development project where the number of items

pointed out in the review exceeds one thousand. Support tools are also actively used in small and

medium-sized projects to improve review quality and efficiency. Indeed, various kinds of “review tools”

are provided. For example, there is a relatively simple capability that allows you to add comments as a

tag using the macro function of document creation. On the other hand, there are tools to support the

progress management of the entire project in the large-scale development in which dozens of reviewers

point out issues and the project manager monitors the status of the review implementation status.

In large organizations, it is very important to select appropriate tools from among various kinds of

review tools. It is essential that the selected tool has a high degree of fairness and is evaluated according

to public standards. For this purpose, ISO/IEC 20741 on the process of evaluation and selection of

software engineering tools was published in 2017. However, ISO/IEC 20741 does not prescribe standard

capabilities specific to reviews because it is generalized without depending on a specific tool field.

This document aims to define the capabilities of review support tools and to select the appropriate

tool in combination with ISO/IEC 20741 for tool evaluation and selection (see Annex E). The review

support tool assumed in this document supports the entire process specified in ISO/IEC 20246. For

example, capabilities which support personal activities such as viewing and pointing out deliverables

are necessary, and capabilities which support group activities such as reporting of situations are

necessary (see Annex D). It is assumed that the check work itself such as the source code check defined

in the ISO/IEC 30130 test tool is not included, and it is assumed that humans are checking.

vi © ISO/IEC 2020 – All rights reserved
---------------------- Page: 6 ----------------------
INTERNATIONAL STANDARD ISO/IEC 23396:2020(E)
Systems and software engineering — Capabilities of
review tools
1 Scope
This document specifies the capabilities of a tool to support review work.

The evaluation and selection of the review tools are performed in accordance with ISO/IEC 20741 which

defines the general evaluation selection process and evaluation characteristics. This document defines

capabilities specific to review tools in the process. By using these two standards together, it is possible

to derive objective and reasonable results of the evaluation and selection of review tools.

The review work is based on the process, activities, and tasks defined in ISO/IEC 20246. It is also

assumed that the review targets are defined in ISO/IEC 20246. The review work in this document is

assumed not to be performed by a 3rd party, but within a project.

The review tool capabilities specified in this document harmonize with the review process defined in

ISO/IEC 20246. This document does not include automated process, activities, or tasks for conducting

reviews such as automated source code checkers defined in ISO/IEC 30130.

Issues which are identified in the review are recorded and managed by the tool; but defects found in

tests and issues found in general except for reviews are out of the scope of this document.

2 Normative references

The following documents are referred to in the text in such a way that some or all of their content

constitutes requirements of this document. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any amendments) applies.

ISO/IEC 20246, Software and systems engineering — Work product reviews
3 Terms and definitions

For the purposes of this document, the terms and definitions given in ISO/IEC 20246 and the

following apply.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

— ISO Online browsing platform: available at http:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
3.1
entity
data concept that may have attributes and relationships to other entities
[SOURCE: ISO/TR 25100:2012, 2.1.3, modified — NOTE has been removed.]
3.2
review folder

entity (3.1) for binding one or more related reviews, including a list of the reviews and information

common to the reviews

Note 1 to entry: The information common to the reviews can include information on members who can participate

in or organize the reviews, and information on the classification given to the issues identified during the reviews.

© ISO/IEC 2020 – All rights reserved 1
---------------------- Page: 7 ----------------------
ISO/IEC 23396:2020(E)
4 Object model for review tools
4.1 Overview of the object model

The overall structure of the object model of review consists of the following elements:

a) Review Process: a set of processes defined by ISO/IEC 20246,

b) Review Entity: a set of elements that represents identifiable information which appears in Review

Process and is described as a class in the object model, and

c) Review Tool: a tool which supports to create, refer to, update, and delete Review Entity.

Figure 1 — Overall structure of object model of review

The object model diagrams, Figure 1 to Figure 3, are described using UML 2 (Unified Modelling

Language 2) (see ISO/IEC 19505-2).

An object model for review tools can be identified to define tool capabilities by its input, process,

and output.
4.2 Object model of review entities

ReviewFolder in this object model (hereinafter referred as ReviewFolder) stands for a review folder.

Review in this object model (hereinafter referred as Review) is an entity representing a single review

in which all information, such as a list of work products to be reviewed, a list of participants, and a list

of issues identified during the review, is recorded.

WorkProduct in this object model (hereinafter referred as WorkProduct) is an entity representing a

work product which is defined in ISO/IEC 20246.

Member in this object model (hereinafter referred as Member) is an entity representing a member who

participates in or organizes the review. It represents all roles defined in ISO/IEC 20246 such as author,

review leader, and reviewer.

A single review is usually carried out on one or more WorkProduct by participants selected from Member.

2 © ISO/IEC 2020 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/IEC 23396:2020(E)

IssueByReview in this object model (hereinafter referred as IssueByReview) is an entity representing

an issue which is identified in the review.
NOTE 1 Issue is defined in ISO/IEC 20246.

PlaceOfIssueByReview in this object model (hereinafter referred as PlaceOfIssueByReview) is an entity

representing a place where the issue is identified.

NOTE 2 There are two reasons why detection places of issues are set as independent entities on the model.

The first reason is that the multiplicity of the relationship between WorkProduct and IssueByReview is many

to many. That is, the same IssueByReview may be given to more than one PlaceOfIssueByReview, and multiple

IssueByReview may be given to the same PlaceOfIssueByReview. The second reason is that the capability to

support the discovery of the issue detection place is decisively important on the tool.

Figure 2 — Entities of review

The relationship between the entities described here and the activities defined in ISO/IEC 20246 is

shown in Annex B.
4.3 Category of capabilities for review tools

On the object model of the review entity, capabilities are categorized into groups according to the group

of entities to be handled (see Figure 3).
© ISO/IEC 2020 – All rights reserved 3
---------------------- Page: 9 ----------------------
ISO/IEC 23396:2020(E)
Key
(1) A group of entities for capabilities of category "IssueByReview management"

(2) A group of entities for capabilities of category" Review metrics collection and report output

(3) A group of entities for capabilities of category "Human communication report"

(4) A group of entities for capabilities of category "Linkage with other schools"

(5) A group of entities for capabilities of category "Administration"
Linking destination tools.
Figure 3 — Category of capabilities
1) IssueByReview management

It is a set of capabilities of managing individual IssueByReview identified during the review of a

WorkProduct. It generates and updates IssueByReview. At the same time, accompanying pointing

PlaceOfIssueByReview is also generated and updated.

This category is required for individual review and issue communication and analysis, such

as supporting the easy detection of issues, recording of issues, recording of the status of issues,

communication of issues, and recording of solutions. Furthermore, it is important that the records

can be used for monitoring the status of each issue and analysing the progress of review as a whole.

2) Review metrics collection and report output

It is a set of capabilities to collect and compile information related to review metrics for a single

review. It creates a review report. Review rate, reviewer progress rate, and reviewer completion

criteria are managed by this group of entities.
4 © ISO/IEC 2020 – All rights reserved
---------------------- Page: 10 ----------------------
ISO/IEC 23396:2020(E)
3) Human communication support

It is a set of capabilities to support communication among the review organizer and review

participants, such as messaging, chat, and mail distribution of review holding information.

It is important to support communication between each reviewer who usually performs an

individual review at different places. These capabilities can avoid duplication of issues and promote

new discoveries.
4) Linkage with other tools
It is a set of capabilities to import/export the information of each entity.

In the review, cooperation with multiple external tools is required, such as checking out the target

work product from the configuration management tool, registering a problem that cannot be solved

in the review process to the issue management tool.
5) Administration

It is a set of capabilities used by the administrator of the review, such as defining review folders

and reviews and controlling the implementation of reviews. It generates and updates ReviewFolder

and Review as well as related Member and WorkProduct.
5 Entities of review tools
5.1 Overview

The following six entities to be handled by a review tool presented in 4.2 are defined in this clause:

— ReviewFolder;
— Review;
— Member;
— WorkProduct;
— IssueByReview; and
— PlaceOfIssueByReview.

Detailed examples of each entity and the attributes contained therein are shown in Annex A.

5.2 ReviewFolder
The following attributes are normally recorded:
— a name for identifying the review folder;

— a list of members who may participate in or organize reviews bound by this entity;

— names and values of classifications which are standard vocabularies to be uniformly used for each

issue identified during reviews bound by this entity; and
— a list of reviews.

EXAMPLE The names of the classifications can include “severity” and “priority”. The values of the

classifications can include “high”, “medium”, and “low”.
© ISO/IEC 2020 – All rights reserved 5
---------------------- Page: 11 ----------------------
ISO/IEC 23396:2020(E)
5.3 Review
This is information of each review bound by the review folder.
The following attributes are normally recorded:
— a review ID;
— a review name;
— a review type;
— a list of work products to be reviewed;
— a list of members planned to participate in the review;
— an organizer of the review;
— a list of participants in the review;
— a list of issues identified during the review;
— date and time when review started;
— date and time when review ended;
— time spent in the review;
— work product appraisal; and
— review decision.
In addition, the following attributes may be recorded:
— purpose of the review;
— scheduled start date and time of the review;
— scheduled end date and time of the review;
— checklists used in the review;
— scenarios used in the review;
— support information such as standards;
— the review man-hours; and
— viewpoints of the review.
5.4 Member

This is information of members who are involved in the review. These members can be the organizer or

participant of the review.
The following attributes are normally recorded:
— a member ID;
— a member name;
— an affiliation; and
— contact information.
6 © ISO/IEC 2020 – All rights reserved
---------------------- Page: 12 ----------------------
ISO/IEC 23396:2020(E)
5.5 WorkProduct
This is information of work products to be reviewed.
The following attributes are normally recorded:
— an identifier of a work product;
— a file path;
— an author;
— number of pages or lines of code; and
— a scope to be reviewed.

NOTE If the changed parts are specified by "a scope to be reviewed", reviewers can concentrate on only the

changed parts.
In addition, the following attributes may be recorded:

— the formats of files of work products such as documents, spreadsheets, presentations, or source

codes; and
— status of review implementation for each page or for each line of code.
5.6 IssueByReview
This is information of issues identified during the review.
The following attributes are normally recorded:
— an issue ID;
— the date and time when the issue was identified;
— originator who identified the issue;
— place on the work product where the issue was found (see 5.7);
— description of the issue;
— classifications of the issue;
— member assigned responsibility for addressing the issue; and
— status of the issue (e.g. “corrected” or “confirmed”).
The following attribute may be also recorded:
— a resolution of the issue.
5.7 PlaceOfIssueByReview
This is information of places on work product where the issues were identified.
The following attributes are normally recorded:
— an identifier of the work product; and

— a physical position in the page in the case of a document or the column position and the row in the

case of source code.
© ISO/IEC 2020 – All rights reserved 7
---------------------- Page: 13 ----------------------
ISO/IEC 23396:2020(E)
The following attribute may also be recorded:
— a logical place such as chapter and clause.
6 Capabilities of review tools
6.1 Overview

This clause defines capabilities of review tools which are divided into five categories in 4.3. Figure 4

shows overall categories of capabilities of review tools. These capabilities are used as tool specific

capabilities in ISO/IEC 20741 evaluation and selection process (see Annex E). The following capabilities

are used in the activities of the review process defined in ISO/IEC 20246 (see Annex C). Furthermore,

how each capability is applied in the use case is shown in Annex F.
Figure 4 — Overall categories of capabilities of review tools
NOTE See Annex A for attributes included i
...

FINAL
INTERNATIONAL ISO/IEC
DRAFT
STANDARD FDIS
23396
ISO/IEC JTC 1/SC 7
Systems and software engineering —
Secretariat: BIS
Capabilities of review tools
Voting begins on:
20200406
Voting terminates on:
202006-01
RECIPIENTS OF THIS DRAFT ARE INVITED TO
SUBMIT, WITH THEIR COMMENTS, NOTIFICATION
OF ANY RELEVANT PATENT RIGHTS OF WHICH
THEY ARE AWARE AND TO PROVIDE SUPPOR TING
DOCUMENTATION.
IN ADDITION TO THEIR EVALUATION AS
Reference number
BEING ACCEPTABLE FOR INDUSTRIAL, TECHNO
ISO/IEC FDIS 23396:2020(E)
LOGICAL, COMMERCIAL AND USER PURPOSES,
DRAFT INTERNATIONAL STANDARDS MAY ON
OCCASION HAVE TO BE CONSIDERED IN THE
LIGHT OF THEIR POTENTIAL TO BECOME STAN
DARDS TO WHICH REFERENCE MAY BE MADE IN
NATIONAL REGULATIONS. ISO/IEC 2020
---------------------- Page: 1 ----------------------
ISO/IEC FDIS 23396:2020(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2020

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting

on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address

below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2020 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC FDIS 23396:2020(E)
Contents Page