ISO/IEC 29155-1:2017

Systems and software engineering — Information technology project performance benchmarking framework — Part 1: Concepts and definitions

ISO/IEC 29155-1:2017 identifies a framework for information technology (IT) project performance benchmarking (e.g. development or maintenance productivity) and related aspects (e.g. data collection and software classification). The framework consists of activities and components that are necessary to successfully identify, define, select, apply, and improve benchmarking for IT project performance. It also provides definitions for IT project performance benchmarking terms, which are also applicable to other parts of the ISO/IEC 29155 series. The target audience of this document are stakeholders of IT project performance benchmarking. NOTE The following are examples of how this document can be used: - by a benchmarking service provider who wants to align their benchmarking process to be consistent with this document; - by a benchmarking user (or third-party agents) for evaluating the performance of an IT project; - by an organization internally to answer specific information needs. ISO/IEC 29155-1:2017 does not cover how to organize benchmarking. It is out of the scope of this document to prescribe the name, format, or explicit content of the documentation to be produced from the benchmarking process.

Ingénierie des systèmes et du logiciel — Cadre de conduite de tests de performance de projet de technologies de l'information — Partie 1: Concepts et définitions

General Information

• Status: Published
• Publication Date: 05-Dec-2017
• ICS: 35.080 - Software
• Technical Committee: ISO/IEC JTC 1/SC 7 - Software and systems engineering
• Drafting Committee: ISO/IEC JTC 1/SC 7 - Software and systems engineering
• Current Stage: 9093 - International Standard confirmed
• Start Date: 10-May-2024
• Completion Date: 12-Feb-2026

Relations

• Revises: ISO/IEC 29155-1:2011 - Systems and software engineering — Information technology project performance benchmarking framework — Part 1: Concepts and definitions
• Effective Date: 01-Jul-2017

Overview

ISO/IEC 29155-1:2017 - "Systems and software engineering - Information technology project performance benchmarking framework - Part 1: Concepts and definitions" defines a conceptual framework and common vocabulary for IT project performance benchmarking. It establishes the activities, components and terminology needed to identify, select, apply and improve benchmarking of IT project performance (for example, development or maintenance productivity). Part 1 focuses on concepts and definitions and is the foundation for subsequent parts that cover tasks, reporting and data collection.

Key topics and technical requirements

ISO/IEC 29155-1:2017 emphasizes a structured benchmarking framework rather than prescriptive methods. Key technical topics and requirements include:

• Framework components: core benchmarking activities, supporting activities, benchmarking information base, instruments, external reference base, and experience base.
• Data structure: definitions for data elements, data records, and how records map to IT projects for consistent comparison.
• Roles & stakeholders: definitions for benchmarking analyst, data submitter, benchmarking user and other stakeholders involved in benchmarking programs.
• Benchmarking outputs: requirements for producing benchmarking reports and core reports (executive summary and detailed reports), and for attaching explanatory reports (data definitions, demographics, sources).
• Data collection & maintenance: guidance to ensure data quality, persistent storage in a benchmarking repository, and ongoing maintenance of benchmarking data.
• Benchmarking methods & types: classification of benchmarking approaches and considerations for method selection (comparative steps, attributes, scales).
• Scope limits: the standard explicitly does not prescribe how to organize benchmarking or mandate formats/content of documentation produced.

Applications

ISO/IEC 29155-1 is practical for:

• Aligning a benchmarking service provider’s process with an international benchmarking framework.
• Enabling benchmarking users or third-party agents to evaluate IT project performance consistently.
• Helping organizations set up internal benchmarking to answer information needs such as productivity comparison, supplier performance, estimation improvement, or process improvement.

Practical uses include establishing a benchmarking repository, defining data element taxonomies, standardizing reporting templates, and improving repeatability and comparability of productivity metrics across projects and suppliers.

Who should use this standard

• Benchmarking service providers and consultants
• IT project managers and PMO teams
• Software engineering measurement specialists
• Quality assurance and process improvement professionals
• Third-party evaluators and benchmarking repository operators

Related standards

ISO/IEC 29155-1:2017 is Part 1 of the ISO/IEC 29155 series. It references and aligns with other standards used for implementation, including:

• ISO/IEC 29155 Parts 2–4 (tasks, reporting, data collection)
• ISO/IEC/IEEE 15939 (measurement processes)
• ISO/IEC 12207, ISO/IEC/IEEE 15288 (life cycle processes)
• ISO/IEC 14143 (functional size measurement) and ISO/IEC 25000 (quality)
• ISO/IEC 15504 / ISO/IEC 33000 (process assessment)

Keywords: ISO/IEC 29155-1:2017, IT project performance benchmarking, benchmarking framework, benchmarking repository, data collection, benchmarking report, software productivity.

Standards Content (Sample)

INTERNATIONAL ISO/IEC
STANDARD 29155-1
Second edition
2017-12
Systems and software engineering —
Information technology project
performance benchmarking
framework —
Part 1:
Concepts and definitions
Ingénierie des systèmes et du logiciel — Cadre de conduite de tests de
performance de projet de technologies de l'information —
Partie 1: Concepts et définitions
Reference number
©
ISO/IEC 2017
© ISO/IEC 2017, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. [phone]
Fax [phone]
[e-mail]
www.iso.org
ii © ISO/IEC 2017 – All rights reserved

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 4
5 Overview of the framework . 4
5.1 General . 4
5.2 Concepts of IT project performance benchmarking . 4
5.2.1 Overall framework . 4
5.2.2 Core benchmarking activities . 6
5.2.3 Supporting activities. 6
5.2.4 Benchmarking information base . 7
5.2.5 Benchmarking instruments . 8
5.2.6 External reference base . 8
5.2.7 Benchmarking experience base . 8
5.3 Stakeholders . 8
5.4 Types of benchmarking methods .10
5.5 IT project categories.11
6 Requirements for benchmarking .11
7 Benchmarking reporting .12
8 Data collection and maintenance .13
9 Additional standardization areas of benchmarking framework .14
9.1 General .14
9.2 Benchmarking domains .14
9.3 Data characteristics .15
9.4 Data analysis .15
9.5 Data exchange .15
Bibliography .16
© ISO/IEC 2017 – All rights reserved iii

Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see the following
URL: www.iso.org/iso/foreword.html.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 7, Software and systems engineering.
This second edition cancels and replaces the first edition (ISO/IEC 29155-1:2011), of which it constitutes
a minor revision. The changes compared to the previous edition are as follows:
— the introduction was restructured and updated to add descriptions for ISO/IEC 29155-2, -3, and -4,
which were published after the publication of the first edition;
— Clause 2 (Normative references) was added following the new requirements of ISO/IEC Directives,
Part 2 (Seventh edition);
— definitions were added to include major terms and definitions of other parts of ISO/IEC 29155;
— abbreviations of names of benchmarking activities were added;
— minor editorial improvements were made to increase understandability and readability throughout
the document; and
— Clause 5 of the first edition was restructured to Clauses 5 to 9.
iv © ISO/IEC 2017 – All rights reserved

Introduction
Benchmarking consists of comparing “objects of interest” to each other, or against a benchmark, to
evaluate characteristic(s). In the context of the ISO/IEC 29155 series, the “object of interest” is the
performance of information technology (IT) projects, and the characteristic is a particular aspect of an
IT project such as productivity.
Benchmarking is one of the fastest-growing techniques in the area of IT project management. Instances
of IT project performance benchmarking are initiated and conducted for various reasons. Among the
most common reasons are:
— the need to compare project productivity between similar industries;
— the need to compare productivity between different project types and technologies;
— the need to find the most effective targets for IT development process improvement;
— the need to compare productivity between different suppliers;
— the need to improve project management maturity;
— the need to improve project estimation capability.
Much has been written regarding the trials of establishing IT project performance benchmarking,
and statistics bear witness to the high failure rate of measurement and benchmarking programs. The
most likely causes for failure have been disappointment in the benchmarking outcomes due to a lack
of alignment between the selected measurements and business goals, and the misunderstanding of
project level measurements in relation to program and portfolio management levels. When there is no
alignment between executed measurements and provided outcomes, unnecessary effort is required
from the IT project teams collecting the project data. This results in decreased motivation to continue
and institutionalize benchmarking.
As is shown in Figure 1, the ISO/IEC 29155 series contains multiple parts:
— Part 1 provides the overall framework model for IT project performance benchmarking. It consists
of activities and components that are necessary to successfully identify, define, select, apply, and
improve benchmarking. It also provides definitions for IT project performance benchmarking terms;
— Part 2 describes the required tasks in individual benchmarking activities that are necessary to
execute various activities to conduct and/or support successful benchmarking in an organization;
— Part 3 provides general requirements and guidance for reporting processes and contents of typical
reports;
— Part 4 provides general requirements and guidance for the activities to collect data of IT project to
be entered into and maintained in a benchmarking repository.
It is possible that further parts follow in the future.
This document is intended to provide a framework about issues and considerations for data selection
and comparison in IT project performance benchmarking.
The starting point for this document and the ISO/IEC 29155 series was the concept outlined by the draft
ISBSG (International Software Benchmarking Standards Group) benchmarking standard. IT project
performance benchmarking is a combination of several different advanced technologies and practices
in the area of quantitative analysis and management. Thus, the framework introduced in this document
can be built on the basis of various standardized key technologies such as:
— project management (e.g. PMBOK Guide and ISO 10006);
— systems and software measurements (e.g. ISO/IEC/IEEE 15939);
© ISO/IEC 2017 – All rights reserved v

— software life cycle process (e.g. ISO/IEC 12207);
— system life cycle process (e.g. ISO/IEC/IEEE 15288);
— functional size measurement (e.g. ISO/IEC 14143 series and related methods);
— systems and software quality evaluations (e.g. the ISO/IEC 25000 family and ISO/IEC 9126 series).
This document is designed to conform to the concepts of ISO/IEC 12207 (Software life cycle processes),
ISO/IEC/IEEE 15288 (Systems life cycle processes), the ISO/IEC 14143 series (Functional size
measurement), the ISO/IEC 15504 series and the ISO/IEC 33000 family (Process assessment), ISO/IEC/
TR 12182 (Categorization of systems and software products), or ISO/IEC 14764 (Software life cycle
processes — Maintenance).
Figure 1 — IT project performance benchmarking framework overview
vi © ISO/IEC 2017 – All rights reserved

INTERNATIONAL STANDARD ISO/IEC 29155-1:2017(E)
Systems and software engineering — Information
technology project performance benchmarking
framework —
Part 1:
Concepts and definitions
1 Scope
This document identifies a framework for information technology (IT) project performance
benchmarking (e.g. development or maintenance productivity) and related aspects (e.g. data collection
and software classification).
The framework consists of activities and components that are necessary to successfully identify,
define, select, apply, and improve benchmarking for IT project performance. It also provides definitions
for IT project performance benchmarking terms, which are also applicable to other parts of the
ISO/IEC 29155 series.
The target audience of this document are stakeholders of IT project performance benchmarking.
NOTE The following are examples of how this document can be used:
— by a benchmarking service provider who wants to align their benchmarking process to be consistent with
this document;
— by a benchmarking user (or third-party agents) for evaluating the performance of an IT project;
— by an organization internally to answer specific information needs.
This document does not cover how to organize benchmarking. It is out of the scope of this document
to prescribe the name, format, or explicit content of the documentation to be produced from the
benchmarking process.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https://www.iso.org/obp
— IEC Electropedia: available at http://www.electropedia.org/
3.1
attribute
property or characteristic of an entity that can be distinguished quantitatively or qualitatively by
human or automated means
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.2]
© ISO/IEC 2017 – All rights reserved 1

3.2
benchmark
reference point against which comparisons can be made
Note 1 to entry: In the context of the ISO/IEC 29155 (all parts), IT project performance(s) is the object of
comparison.
3.3
benchmarking
activity of comparing objects of interest to each other or against a benchmark (3.2) to evaluate
characteristic(s)
Note 1 to entry: In the context of the ISO/IEC 29155 (all parts), the object of interest is IT project performance,
and the characteristic is a particular aspect of an IT project such as productivity.
3.4
benchmarking analyst
person or organization that conducts benchmarking (3.3) activities
3.5
benchmarking experience base
information store that contains the evaluation of the information products and the benchmarking (3.3)
activity as well as any lessons learned during benchmarking and analysis
Note 1 to entry: ISO/IEC/IEEE 15939:2007 defines “information product” as “one or more indicators and their
associated interpretations that address an information need”. For example, information product can include
templates, graphs, statistical algorithms, and interpretation guidelines.
3.6
benchmarking method
logical sequence of general steps to describe the process of comparing one or more attributes (3.1)
against a reference attribute with respect to a specified scale
3.7
benchmarking report
document showing the results of an instance of benchmarking (3.15)
Note 1 to entry: Document usually consists of various formats (e.g. textual descriptions, numeric values,
statistical charts and tables), and is exchanged via various media (e.g. electronic documents, electronic dataset,
printed documents, and embedded data within specific computer software).
3.8
benchmarking repository
organized and persistent data storage which is designated for benchmarking (3.3)
3.9
benchmarking user
person or organization that utilizes the outcome of benchmarking (3.3)
3.10
core report
document for providing descriptions of the process and outcomes of the benchmarking (3.3) activity
Note 1 to entry: Two kinds of core reports (executive summary and detailed report) are often produced for
reporting results of an instance of benchmarking activity.
3.11
data element
smallest unit of data of an IT project
Note 1 to entry: Data element is usually implemented to be a data cell in a benchmarking repository and/or an “IT
project data”,
2 © ISO/IEC 2017 – All rights reserved

Note 2 to entry: Data element is defined by a responsible person who plans data collection or benchmarking.
3.12
data record
defined group of related data elements, in which all the necessary data elements are included to
represent attributes (3.1) of interest
Note 1 to entry: One data record usually corresponds to a specific IT project with in IT project performance
benchmarking framework.
3.13
data submitter
person or organization that provides IT project (3.16) data to be included into a benchmarking
repository (3.8)
3.14
explanatory report
document attached to a product and which provides complementary information in order to assist
understanding and avoid inappropriate usage of the product
Note 1 to entry: Examples of an explanatory report are data element definitions, data demographics, data source
information which are attached to benchmarking repositories or benchmarks.
Note 2 to entry: Examples of the product are benchmarking repository, benchmark(s), or software tools to
support benchmarking activities.
3.15
instance of benchmarking
set of operations, described specifically, used in the execution of a particular benchmarking (3.3)
according to a given method
3.16
IT project
information technology project
temporary endeavor undertaken to create or change a unique information technology product, system,
or service
Note 1 to entry: The PMBOK Guide defines “project” as “a temporary endeavor undertaken to create a unique
product, service or result”. “IT project” is a specified subset of projects.
3.17
IT project dataset
classified group of data records, into which collected data records are selected by pre-defined criteria
Note 1 to entry: Classification criteria for an IT project dataset can be based on the information needs of the
owner and/or users of data.
3.18
project performance
derived measure that gives an indication of some attribute (3.1) associated with how well, how quickly,
how effectively or how efficiently a project is carried out
3.19
repository owner
person or organization that owns and maintains a benchmarking repository
Note 1 to entry: Also called repository administrator.
© ISO/IEC 2017 – All rights reserved 3

3.20
task
smallest unit of work subject to management accountability; a well-defined work assignment for one or
more project members
Note 1 to entry: Related tasks are usually grouped to form activities.
[SOURCE: IEEE 829-2008 IEEE Standard for Software and System Test Documentation, 3.1.38]
4 Abbreviated terms
CBa “conduct benchmarking” activity
IBa “issue benchmarks” activity
IT information technology
MBLa “manage benchmarking business level” activity
MPa “measure IT project” activity
MPLa “manage benchmarking program level” activity
MRa “maintain repository” activity
PIa “provide instruments” activity
SDa “submit data” activity
URa “utilize benchmarking results” activity
5 Overview of the framework
5.1 General
This clause presents an overview of the IT project performance benchmarking framework. The
objective is to orient the users of this document so that they can apply benchmarking properly within
context.
5.2 Concepts of IT project performance benchmarking
5.2.1 Overall framework
This clause outlines the overall framework of IT project performance benchmarking.
As shown in Figure 2, the framework consists of the following categories of components:
— [C1] “Core benchmarking activities”, which collectively illustrate an instance of benchmarking;
— [C2] “Supporting activities”, which provides the base and instruments for benchmarking information
for use in an instance of benchmarking;
— [C3] “Benchmarking instruments”, used by stakeholders to conduct an instance of benchmarking;
— [C4] “Benchmarking information base”, which contains data for use during an instance of
benchmarking;
— [C5] “External reference base”, which provides alternative or additional external dat
...