ISO/IEC 29155-1:2011
(Main)Systems and software engineering — Information technology project performance benchmarking framework — Part 1: Concepts and definitions
Systems and software engineering — Information technology project performance benchmarking framework — Part 1: Concepts and definitions
ISO/IEC 29155-1:2011 identifies a framework for information technology (IT) project performance benchmarking (e.g. development or maintenance productivity) and related aspects (e.g. data collection and software classification). The framework consists of activities and components that are necessary to successfully identify, define, select, apply, and improve benchmarking for IT project performance. It also provides definitions for IT project performance benchmarking terms. The target audience of ISO/IEC 29155-1:2011 are the stakeholders of IT project performance benchmarking. ISO/IEC 29155-1:2011 does not prescribe how to organize benchmarking. It is out of the scope of ISO/IEC 29155-1:2011 to prescribe the name, format, or explicit content of the documentation to be produced from the benchmarking process.
Ingénierie des systèmes et du logiciel — Cadre de conduite de tests de performance de projet de technologies de l'information — Partie 1: Concepts et définitions
General Information
Relations
Standards Content (Sample)
INTERNATIONAL ISO/IEC
STANDARD 29155-1
First edition
2011-12-01
Systems and software engineering —
Information technology project
performance benchmarking framework —
Part 1:
Concepts and definitions
Ingénierie des systèmes et du logiciel — Cadre de conduite de tests de
performance de projet de technologies de l'information —
Partie 1: Concepts et définitions
Reference number
ISO/IEC 29155-1:2011(E)
©
ISO/IEC 2011
---------------------- Page: 1 ----------------------
ISO/IEC 29155-1:2011(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2011
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56 CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO/IEC 2011 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC 29155-1:2011(E)
Contents Page
Foreword . iv
Introduction . v
1 Scope . 1
2 Terms and definitions . 1
3 Abbreviated terms . 2
4 Overview of the framework . 3
4.1 General . 3
4.2 Concepts of IT project performance benchmarking . 3
4.2.1 Overall framework . 3
4.2.2 Core benchmarking activities . 4
4.2.3 Supporting activities . 5
4.2.4 Benchmarking information base . 5
4.2.5 Benchmarking instruments . 6
4.2.6 External reference base . 7
4.2.7 Benchmarking experience base . 7
4.3 Stakeholders . 7
4.4 Types of benchmarking methods . 9
4.5 IT project categories . 9
5 Additional standardization areas of benchmarking framework . 9
5.1 General . 9
5.2 Requirements for benchmarking . 10
5.3 Benchmarking domains . 11
5.4 Data collection and retention . 11
5.5 Data characteristics . 11
5.6 Data analysis . 11
5.7 Benchmarking reporting . 11
Bibliography . 12
© ISO/IEC 2011 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC 29155-1:2011(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees
established by the respective organization to deal with particular fields of technical activity. ISO and IEC
technical committees collaborate in fields of mutual interest. Other international organizations, governmental
and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as
an International Standard requires approval by at least 75 % of the national bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC 29155-1 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 7, Software and systems engineering.
ISO/IEC 29155 consists of the following parts, under the general title Systems and software engineering —
Information technology project performance benchmarking framework:
Part 1: Concepts and definitions
The following part is under preparation:
Part 2: Requirements for benchmarking
iv © ISO/IEC 2011 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC 29155-1:2011(E)
Introduction
This part of ISO/IEC 29155 identifies a framework which consists of activities and components that are
necessary to successfully identify, define, select, apply, and improve benchmarking for information technology
(IT) project performance. It also provides definitions for IT project performance benchmarking terms.
This part of ISO/IEC 29155 is intended to provide a framework about issues and considerations for data
selection and comparison in information technology (IT) project performance benchmarking.
The starting point for this part of ISO/IEC 29155 and the ISO/IEC 29155 series was the concept outlined by
the draft ISBSG (International Software Benchmarking Standards Group) benchmarking standard. IT project
performance benchmarking is a combination of several different advanced technologies and practices in the
area of quantitative analysis and management. Thus the framework introduced in this part of ISO/IEC 29155
can be built on the basis of various standardized key technologies such as:
project management (e.g. PMBOK Guide and ISO 10006),
systems and software measurements (e.g. ISO/IEC 15939),
software life cycle processes (e.g. ISO/IEC 12207),
systems life cycle processes (e.g. ISO/IEC 15288),
functional size measurement (e.g. ISO/IEC 14143 series and related methods),
systems and software quality evaluations (e.g. ISO/IEC 25000 family and ISO/IEC 9126 series).
This part of ISO/IEC 29155 is designed to conform to the concepts within ISO/IEC 12207 (software life cycle
processes), ISO/IEC 15288 (systems life cycle processes), the ISO/IEC 14143 series (functional size
measurement), the ISO/IEC 15504 series (process assessment), ISO/IEC TR 12182 (categorization of
software), or ISO/IEC 14764 (maintenance of software life cycle processes).
IT project performance benchmarking instances are initiated and conducted for various reasons. Among the
most common reasons are:
a) need to improve project management maturity,
b) need to improve project estimation capability,
c) need to compare productivity between different project types and technologies,
d) need to compare project productivity between similar industries,
e) need to find the most effective targets for IT development process improvement.
Much has been written regarding the trials of establishing an IT project benchmarking framework, and
statistics bear witness to the high failure rate of measurement programs. The most probable causes for failure
have been disappointment in the benchmarking outcome due to a lack of alignment between the selected
measurements and business goals, and the misunderstanding of project level measurements in relation to
program and portfolio management levels.
This part of ISO/IEC 29155 is developed as the first of multiple parts which will complete the IT project
performance benchmarking framework as is shown in Figure 1. This part of ISO/IEC 29155 is the concept
standard; it contains no mandatory requirements. The requirements for the benchmarking process will be
© ISO/IEC 2011 – All rights reserved v
---------------------- Page: 5 ----------------------
ISO/IEC 29155-1:2011(E)
specified subsequent to the standardization of the concepts and definitions. Then, the guidelines for the
quality and quantity of different inputs and benchmarking outputs will be drafted, together with the domain-
specific control needs.
RESOURCES
Concepts and
definitions
INPUT OUTPUT
PROCESS
Data collection Benchmarking
and retention reporting
Requirements
for benchmarking
Data Data
characteristics analysis
CONTROLS
Benchmarking
domains
Figure 1 — IT project performance benchmarking framework overview
vi © ISO/IEC 2011 – All rights reserved
---------------------- Page: 6 ----------------------
INTERNATIONAL STANDARD ISO/IEC 29155-1:2011(E)
Systems and software engineering — Information technology
project performance benchmarking framework —
Part 1:
Concepts and definitions
1 Scope
This part of ISO/IEC 29155 identifies a framework for information technology (IT) project performance
benchmarking (e.g. development or maintenance productivity) and related aspects (e.g. data collection and
software classification).
The framework consists of activities and components that are necessary to successfully identify, define, select,
apply, and improve benchmarking for IT project performance. It also provides definitions for IT project
performance benchmarking terms.
The target audience of this part of ISO/IEC 29155 are stakeholders of IT project performance benchmarking.
NOTE The following are examples of how this part of ISO/IEC 29155 can be used:
by a Benchmarking service provider who wants to align their benchmarking process to be consistent with this part of
ISO/IEC 29155;
by a Benchmarking user (or third-party agents) for evaluating the performance of an IT project;
by an organization internally to answer specific information needs.
This part of ISO/IEC 29155 does not prescribe how to organize benchmarking. It is out of the scope of this
part of ISO/IEC 29155 to prescribe the name, format, or explicit content of the documentation to be produced
from the benchmarking process.
2 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
2.1
benchmark
reference point against which comparisons can be made
NOTE In the context of the ISO/IEC 29155 series, IT project performance(s) is the object of comparison.
2.2
benchmarking
activity of comparing objects of interest to each other or against a benchmark to evaluate characteristic(s)
NOTE In the context of the ISO/IEC 29155 series, the object of interest is IT project performance, and the
characteristic is a particular aspect of an IT project such as productivity.
© ISO/IEC 2011 – All rights reserved 1
---------------------- Page: 7 ----------------------
ISO/IEC 29155-1:2011(E)
2.3
Benchmarking experience base
information store that contains the evaluation of the information products and the benchmarking activity as
well as any lessons learned during benchmarking and analysis
NOTE ISO/IEC 15939:2007 defines “information product” as “one or more indicators and their associated
interpretations that address an information need”. For example, information product could include templates, graphs,
statistical algorithms, and interpretation guidelines.
2.4
benchmarking method
logical sequence of general steps to describe the process of comparing one or more attributes against a
reference attribute with respect to a specified scale
2.5
Benchmarking user
person or organization that utilizes the outcome of benchmarking
2.6
instance of benchmarking
set of operations, described specifically, used in the execution of a particular benchmarking according to a
given method
2.7
IT project
information technology project
temporary endeavor undertaken to create or change a unique information technology product, system, or
service
NOTE The PMBOK Guide, Fourth Edition defines “project” as “a temporary endeavor undertaken to create a unique
product, service or result”. “IT project” is a specified subset of projects.
2.8
project performance
derived measure that gives an indication of some attribute associated with how well, how quickly, how
effectively or how efficiently a project is carried out
2.9
repository
organized and persistent data storage that allows data retrieval
NOTE Benchmarking repository is a repository which is designated for use as the source of comparative measures
for the purpose of benchmarking.
2.10
repository administrator
person or organization that maintains and administrates data in a repository
3 Abbreviated terms
IT Information Technology
2 © ISO/IEC 2011 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/IEC 29155-1:2011(E)
4 Overview of the framework
4.1 General
This clause presents an overview of the IT project performance benchmarking framework. The objective is to
orient the users of this part of ISO/IEC 29155 so that they can apply benchmarking properly within context.
4.2 Concepts of IT project performance benchmarking
4.2.1 Overall framework
This clause outlines the overall framework of IT project performance benchmarking.
As shown in Figure 2, the framework consists of the following categories of components:
Core benchmarking activities category, which illustrates an instance of benchmarking;
Supporting activities category, which provides Benchmarking information base and instruments for use
in an instance of benchmarking;
Benchmarking information base category, which contains data for use in an instance of benchmarking;
Benchmarking instruments category, which assists stakeholders to conduct an instance of
benchmarking;
External reference base category, which provides alternative or additional external data (i.e. repository
and/or benchmarks) for an instance of benchmarking;
Benchmarking experience base category, which contains knowledge and lessons learned during
benchmarking.
© ISO/IEC 2011 – All rights reserved 3
---------------------- Page: 9 ----------------------
ISO/IEC 29155-1:2011(E)
External
External
benchmarks
repository
Import data External reference base
Maintain Submit Measure
repository data IT project
Submit data
Retain data Retain/
refer
Extract
data
data
Benchmarking
Refer
repository
benchmark
Refer data
IT project
Extract
benchmark data
Extract data
Internal
Issue
benchmarks
benchmarks
Benchmarking
Produce
information base
Refer benchmark
Tools
Use/refer
Conduct Utilize
benchmarking benchmarking
Methods
results
Provide
Use/
Report
instruments
refer
Guides results
Supply
Supporting Benchmarking
Core benchmarking activities
activities instruments
Retain/refer
Benchmarking experience base
Organized
Data flow
Activity Tool Method
data store
Legends
Generic Component
Utility flow
Benchmark Guide
data store category
Figure 2 — IT project performance benchmarking framework
4.2.2 Core benchmarking activities
The Core benchmarking activities mainly address information needs of the Benchmarking users. Each activity
has one or more processes to initiate, plan, execute, review and improve the activity. The activities can be
grouped as follows:
a) Conduct benchmarking. This activity initiates and executes an instance of benchmarking.
b) Utilize benchmarking results. This activity indicates use of benchmarking results for various business
purposes.
4 © ISO/IEC 2011 – All rights reserved
---------------------- Page: 10 ----------------------
ISO/IEC 29155-1:2011(E)
There are two approaches for conducting benchmarking:
Extract benchmark from Benchmarking repository and compare it to specific IT project data.
Refer a benchmark (i.e. Internal benchmarks or External benchmarks) and compare it to specific IT
project data.
Examples of the usage are estimating, comparing, evaluating and managing performances of a specific IT
project.
4.2.3 Supporting activities
The supporting activities provide the framework to assist Benchmarking users. Each activity has one or more
processes to initiate, plan, execute, review and improve the activity. The activities can be grouped as follows:
a) Measure IT project. This activity measures IT project and retains data.
b) Submit data. This activity selects and provides IT project data to be included into a Benchmarking
repository.
c) Maintain repository. This activity accepts, verifies, and then stores IT project
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.