ISO/IEC 29155-3:2015

Systems and software engineering — Information technology project performance benchmarking framework — Part 3: Guidance for reporting

ISO/IEC 29155-3:2015 provides general requirements and guidance for reporting processes and contents of typical reports within benchmarking activities of "the information technology (IT) project performance benchmarking framework" by prescribing: - requirements and guidance for the reporting processes within the benchmarking framework; ? requirements and guidance for the contents of reports. ISO/IEC 29155-3:2015 focuses on three major activities, i.e. "conduct benchmarking", "maintain repository", and "issue benchmarks" activities. NOTE 1 These activities are selected, not only because the importance of the outcomes of these activities, but also the outcomes of these activities are the direct input for benchmarking users who execute "core benchmarking" activities. In addition, benchmarking users are not usually so deeply involved to these activities even though they need in-deep information to understand the benchmarking results or to select appropriate data (i.e. benchmarking repository and benchmarks). ISO/IEC 29155-3:2015 also focuses on two types of reports in the benchmarking framework: a) the benchmarking report, that describes the results of an instance of benchmarking; b) the explanatory report, that provides complementary information about the released benchmarking repository or benchmark(s). ISO/IEC 29155-3:2015 is intended for use by stakeholder(s) of IT project performance benchmarking (e.g. benchmarking user, benchmark provider, and benchmarking service provider). NOTE 2 The following are examples of how this part of ISO/IEC 29155 can be used: - by a systems and software acquirer (or a third-party agent), to define, order, obtain and evaluate an acceptable and creditable benchmarking report; - by a benchmarking service provider, to produce a high-quality benchmarking report; - by a benchmark provider, to provide complementary information about the released benchmarking repository or issued benchmarks. It is out of the scope of this part of ISO/IEC 29115 to prescribe the particular names, formats, or explicit contents of the reports of the benchmarking activities.

Ingénierie des systèmes et du logiciel — Cadre de conduite de tests de performance de projet de technologies de l'information — Partie 3: Directives de rapport

General Information

• Status: Published
• Publication Date: 15-Nov-2015
• ICS: 35.080 - Software
• Technical Committee: ISO/IEC JTC 1/SC 7 - Software and systems engineering
• Drafting Committee: ISO/IEC JTC 1/SC 7 - Software and systems engineering
• Current Stage: 9093 - International Standard confirmed
• Completion Date: 24-May-2022

Standards Content (Sample)

INTERNATIONAL ISO/IEC
STANDARD 29155-3
First edition
2015-11-15
Systems and software engineering —
Information technology project
performance benchmarking
framework —
Part 3:
Guidance for reporting
Ingénierie des systèmes et du logiciel — Cadre de conduite de tests de
performance de projet de technologies de l’information —
Partie 3: Directives de rapport
Reference number
ISO/IEC 29155-3:2015(E)
©
ISO/IEC 2015

---------------------- Page: 1 ----------------------
ISO/IEC 29155-3:2015(E)

COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2015, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. [phone]
Fax [phone]
[e-mail]
www.iso.org
ii © ISO/IEC 2015 – All rights reserved

---------------------- Page: 2 ----------------------
ISO/IEC 29155-3:2015(E)

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 2
4 Abbreviated terms . 2
5 Roles of reports in the IT project performance benchmarking framework .2
5.1 General quality requirements for the benchmarking reports . 2
5.2 Identification of reports within benchmarking framework. 3
5.2.1 Benchmarking outcomes and types of reports . 3
5.2.2 Reporting related benchmarking activities . 3
5.3 Relationships between benchmarking activities and reports . 4
6 General requirements and guidance for reporting process . 5
6.1 Planning phase . 5
6.2 Executing phase . 6
6.3 Supporting phase . 6
6.3.1 Support for reports and deliverables . 6
6.3.2 Support for benchmarking experience base . 6
7 Requirements and guidance for contents of reports within benchmarking framework .6
7.1 General requirements for reports . 6
7.1.1 Requirements for structure and wording . 6
7.1.2 Requirements for administrative information of reports . 7
7.2 Activity-specific requirements and guidance for reports . 7
7.2.1 Reports of “Conduct benchmarking” activity . 7
7.2.2 Reports of “Maintain repository” activity . 8
7.2.3 Reports of “Issue benchmarks” activity . 8
Annex A (informative) Framework model of IT project performance benchmarking in ISO/
IEC 29155-1 .10
Bibliography .14
© ISO/IEC 2015 – All rights reserved iii

---------------------- Page: 3 ----------------------
ISO/IEC 29155-3:2015(E)

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the meaning of ISO specific terms and expressions related to conformity
assessment, as well as information about ISO’s adherence to the WTO principles in the Technical
Barriers to Trade (TBT) see the following URL: Foreword - Supplementary information
The committee responsible for this document is ISO/IEC JTC1, Information technology, Subcommittee
SC 7, Software and systems engineering.
ISO/IEC 29155 consists of the following parts, under the general title Systems and software engineering —
Information technology project performance benchmarking framework:
— Part 1: Concepts and definitions
— Part 2: Requirements for benchmarking
— Part 3: Guidance for reporting
— Part 4: Guidance for data collection and maintenance
Further parts might follow.
Annex A of this part of ISO/IEC 29155 is for information only.
iv © ISO/IEC 2015 – All rights reserved

---------------------- Page: 4 ----------------------
ISO/IEC 29155-3:2015(E)

Introduction
Benchmarking is an activity of comparing objects of interest to each other or against a benchmark
to evaluate characteristic(s). In the context of the ISO/IEC 29155 series, the “object of interest” is the
performance of information technology (IT) project, and the characteristic is a particular aspect of an
IT project such as productivity.
The benchmarking is one of the fastest-growing techniques in the area of IT project management.
Instances of IT project performance benchmarking are initiated and conducted for various reasons.
Among the most common reasons are
a) the need to compare project productivity between similar industries,
b) the need to compare productivity between different project types and technologies,
c) the need to find the most effective targets for IT development process improvement,
d) the need to compare productivity between different suppliers,
e) the need to improve project management maturity,
f) the need to improve project estimation capability.
Much has been written regarding the trials of establishing IT project performance benchmarking,
and statistics bear witness to the high failure rate of measurement and benchmarking programs. The
most probable causes for failure have been disappointment in the benchmarking outcomes due to a
lack of alignment between the selected measurements and business goals, and the misunderstanding
of project level measurements in relation to program and portfolio management levels. When there is
no alignment between executed measurements and provided outcomes, unnecessary effort is required
from the IT project teams collecting the project data. The result is decreased motivation to continue
and institutionalize benchmarking.
As shown in Figure 1, ISO/IEC 29155 series consists of multiple parts:
— Part 1 provides the overall framework model for IT project performance benchmarking. It consists
of activities and components that are necessary to successfully identify, define, select, apply, and
improve benchmarking;
— Part 2 prescribes the required tasks in individual benchmarking activities that are necessary to
execute various activities to conduct and/or support successful benchmarking in an organization;
— Part 3 prescribes the guidance for reporting processes and contents of typical reports;
— Part 4 provides guidance for the activities to collect IT project data to be entered into and maintained
in a benchmarking repository.
Further parts might follow.
This part of ISO/IEC 29155 is intended to provide guidance for reporting process and contents of
typical reports in IT project performance benchmarking to produce high-quality deliverables (e.g. the
benchmarking report of an instance of benchmarking, the explanatory report for issued benchmarks,
and the explanatory report for released benchmarking repository) which include sufficient information
to avoid misunderstanding or inappropriate usage.
© ISO/IEC 2015 – All rights reserved v

---------------------- Page: 5 ----------------------
ISO/IEC 29155-3:2015(E)

RESOURCES
Part 1:
Concepts and
deinitions
INPUT OUTPUT
Part 4:
Part 3:
PROCESS
Guidance for
Guidance for
data collection
Part 2:
reporting
and maintenance
Requirements for
benchmarking
Data characteristics Data analysis
CONTROLS
Benchmarking
domains
Data exchange
Figure 1 — IT project performance benchmarking standards overview
vi © ISO/IEC 2015 – All rights reserved

---------------------- Page: 6 ----------------------
INTERNATIONAL STANDARD ISO/IEC 29155-3:2015(E)
Systems and software engineering — Information
technology project performance benchmarking
framework —
Part 3:
Guidance for reporting
1 Scope
This part of ISO/IEC 29155 provides general requirements and guidance for reporting processes and
contents of typical reports within benchmarking activities of “the information technology (IT) project
performance benchmarking framework” by prescribing:
— requirements and guidance for the reporting processes within the benchmarking framework;
— requirements and guidance for the contents of reports.
This part of ISO/IEC 29155 focuses on three major activities, i.e. “conduct benchmarking”, “maintain
repository”, and “issue benchmarks” activities.
NOTE 1 These activities are selected, not only because the importance of the outcomes of these activities,
but also the outcomes of these activities are the direct input for benchmarking users who execute “core
benchmarking” activities. In addition, benchmarking users are not usually so deeply involved to these activities
even though they need in-deep information to understand the benchmarking results or to select appropriate
data (i.e. benchmarking repository and benchmarks).
This part of ISO/IEC 29155 also focuses on two types of reports in the benchmarking framework:
a) the benchmarking report, that describes the results of an instance of benchmarking;
b) the explanatory report, that provides complementary information about the released benchmarking
repository or benchmark(s).
This part of ISO/IEC 29155 is intended for use by stakeholder(s) of IT project performance benchmarking
(e.g. benchmarking user, benchmark provider, and benchmarking service provider).
NOTE 2 The following are examples of how this part of ISO/IEC 29155 can be used:
— by a systems and software acquirer (or a third-party agent), to define, order, obtain and evaluate an acceptable
and creditable benchmarking report;
— by a benchmarking service provider, to produce a high-quality benchmarking report;
— by a benchmark provider, to provide complementary information about the released benchmarking repository
or issued benchmarks.
It is out of the scope of this part of ISO/IEC 29115 to prescribe the particular names, formats, or explicit
contents of the reports of the benchmarking activities.
2 Normative references
The following documents, in whole or in part, are normatively referenced in this document and are
indispensable for its application. For dated references, only the edition cited applies. For undated
references, the latest edition of the referenced document (including any amendments) applies.
© ISO/IEC 2015 – All rights reserved 1

---------------------- Page: 7 ----------------------
ISO/IEC 29155-3:2015(E)

ISO/IEC 29155-1, Systems and software engineering — Information technology project performance
benchmarking framework — Part 1: Concepts and definitions
ISO/IEC 29155-2, Systems and software engineering — Information technology project performance
benchmarking framework — Part 2: Requirements for benchmarking
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC 29155-1, ISO/IEC 29155-
2, and the following apply.
3.1
benchmarking analyst
person or organization that executes “conduct benchmarking” activity
3.2
benchmarking report
document of the results of an instance of benchmarking
Note 1 to entry: Document usually consists of various formats (e.g. textual descriptions, numeric values,
statistical charts and tables), and is exchanged via various media (e.g. electronic documents, electronic data set,
printed documents, and embedded data within specific computer software).
3.3
core report
document for providing descriptions of the process and outcomes of the benchmarking activity
Note 1 to entry: Two kinds of core reports (i.e. executive summary and detailed report) are often produced for
reporting results of an instance of benchmarking activity.
3.4
explanatory report
document attached to a product for providing complementary information in order to assist
understanding and to avoid inappropriate usage of the product
Note 1 to entry: Examples of an explanatory report are data element definitions, data demographics, data source
information which are attached to benchmarking repositories or benchmarks.
Note 2 to entry: Examples of the product are benchmarking repository, benchmark(s), or software tools to
support benchmarking activities.
4 Abbreviated terms
IT Information Technology
5 Roles of reports in the IT project performance benchmarking framework
5.1 General quality requirements for the benchmarking reports
As shown in Figure A.1, IT project performance benchmarking framework in ISO/IEC 29155 series
consists of two categories of benchmarking activities:
— core benchmarking activities (i.e. “conduct benchmarking” and “utilize benchmarking results”);
— supporting activities (i.e. “maintain repository”, “issue benchmarks”, “measure IT project”, “submit
data”, and “provide instruments”).
NOTE 1 See ISO/IEC 29155–1 and ISO/IEC 29155–2 for more information about activities, processes and tasks
in the benchmarking framework.
2 © ISO/IEC 2015 – All rights reserved

---------------------- Page: 8 ----------------------
ISO/IEC 29155-3:2015(E)

Since individual benchmarking activities require different expertise, these activities are usually role-
shared by multiple stakeholders (e.g. benchmarking user, benchmark provider, benchmarking service
provider, IT project team). In addition, same activity is often assigned to multiple persons or divisions,
and those are executed in parallel within and/or across organization(s).
In these circumstances, various data and documents are exchanged as the outputs and inputs of
activities. Therefore, high-quality reports are pre-requisite (necessary) to ensure and facilitate
effective communication between stakeholders to drive whole activities successfully.
To minimize misunderstanding, a benchmarking report should be specific and consistent with
terminology and definitions.
NOTE 2 When comparing a user data element with a benchmark, misunderstanding might occur, if
— the names are similar to each other, but the definitions are significantly different (e.g. scale,
measurement unit, measurement timing),
— the names and the definitions are similar to each other, but the nature of the user software is
significantly different (e.g. business domain, technical complexity, degree of quality requirements),
— the names and the definitions are similar to each other, but the nature of the user project is
significantly different (e.g. skills and experience of IT project teams, scale of IT projects, resource
constraints of IT projects).
5.2 Identification of reports within benchmarking framework
5.2.1 Benchmarking outcomes and types of reports
The outcomes of a benchmarking activity result in a set of deliverables which usually include:
a) core reports, which provide descriptions of the process and outcomes of the benchmarking activity;
b) data products, which provide the data resulting from the benchmarking activity;
c) explanatory reports, which provide complementary information for assisting the audience to
understand the outcomes of the benchmarking activity or to avoid inappropriate use of deliverables.
Core reports and explanatory reports represent the two types of benchmarking reports.
NOTE 1 Two kinds of core reports (i.e. executive summary and detailed report) are often produced for
reporting results of an instance of benchmarking activity.
NOTE 2 Core reports and explanatory reports are usually presented in the form of textual descriptions
together with statistics (e.g. numeric values, charts, and tables).
5.2.2 Reporting related benchmarking activities
This part of ISO/IEC 29155 focuses on three major benchmarking activities, i.e. “con
...