ISO/IEC TR 15504-4:1998
(Main)Information technology — Software process assessment — Part 4: Guide to performing assessments
Information technology — Software process assessment — Part 4: Guide to performing assessments
Technologies de l'information — Évaluation des procédés du logiciel — Partie 4: Guide pour la réalisation d'évaluations
General Information
Relations
Standards Content (Sample)
TECHNICAL ISO/IEC
REPORT TR 15504-4
First edition
1998-08-15
Information technology — Software process
assessment —
Part 4:
Guide to performing assessments
Technologies de l’information — Évaluation des procédés du logiciel —
Partie 4: Guide pour l’exécution des évaluations
Reference number
B C
ISO/IEC TR 15504-4:1998(E)
---------------------- Page: 1 ----------------------
ISO/IEC TR 15504-4:1998(E)
Contents
1 Scope .1
2 Normative reference .1
3 Terms and definitions.1
4 Overview of process assessment .2
4.1 Process assessment.2
4.2 Process rating scheme.3
4.3 Assessment approaches.3
4.3.1 Self-assessment.3
4.3.2 Independent assessment .3
4.4 Assessment process .3
4.4.1 Documented Assessment Process .3
4.5 Compatible model.4
4.6 Supporting instruments and tools .4
4.7 Success factors for process assessment .4
4.7.1 Commitment .4
4.7.2 Motivation .5
4.7.3 Confidentiality .5
4.7.4 Relevance .5
4.7.5 Credibility.5
5 Selection and use of a compatible model .5
5.1 Compatibility with the reference model.5
5.1.1 Model Purpose .6
5.1.2 Model Scope.6
© ISO/IEC 1998
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or
utilized in any form or by any means, electronic or mechanical, including photocopying and micro-
film, without permission in writing from the publisher.
ISO/IEC Copyright Office • Case postale 56 • CH-1211 Genève 20 • Switzerland
Printed in Switzerland
ii
---------------------- Page: 2 ----------------------
© ISO/IEC
ISO/IEC TR 15504-4:1998(E)
5.1.3 Model elements and indicators . 6
5.1.4 Mapping . 7
5.1.5 Translation. 7
5.2 Criteria for selecting a model . 7
5.3 Using a model in an assessment . 8
6 Selection and use of a documented assessment process. 8
6.1 Compatibility with the requirements. 8
6.2 Assessment Input. 9
6.3 The assessment process . 10
6.3.1 Planning. 10
6.3.2 Data collection . 11
6.3.3 Data validation . 11
6.3.4 Process rating . 11
6.3.5 Reporting . 12
6.4 Selecting a documented assessment process . 12
6.5 Role of the competent assessor. 13
7 Selection of instruments and tools. 13
7.1 The purpose and use of instruments and tools within an assessment . 13
7.2 Selecting instruments and tools . 14
Annex A (informative) Guidance on indicators . 16
iii
---------------------- Page: 3 ----------------------
© ISO/IEC
ISO/IEC TR 15504-4:1998(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical Commission)
form the specialized system for worldwide standardization. National bodies that are members of ISO or IEC
participate in the development of International Standards through technical committees established by the
respective organization to deal with particular fields of technical activity. ISO and IEC technical committees
collaborate in fields of mutual interest. Other international organizations, governmental and non-governmental, in
liaison with ISO and IEC, also take part in the work.
In the field of information technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
The main task of technical committees is to prepare International Standards, but in exceptional circumstances a
technical committee may propose the publication of a Technical Report of one of the following types:
— type 1, when the required support cannot be obtained for the publication of an International Standard, despite
repeated efforts;
— type 2, when the subject is still under technical development or where for any other reason there is the future
but not immediate possibility of an agreement on an International Standard;
— type 3, when a technical committee has collected data of a different kind from that which is normally published
as an International Standard (“state of the art”, for example).
Technical Reports of types 1 and 2 are subject to review within three years of publication, to decide whether they
can be transformed into International Standards. Technical Reports of type 3 do not necessarily have to be
reviewed until the data they provide are considered to be no longer valid or useful.
ISO/IEC TR 15504-4, which is a Technical Report of type 2, was prepared by Joint Technical Committee ISO/IEC
JTC 1, Information technology, Subcommittee SC 7, Software engineering.
ISO/IEC TR 15504 consists of the following parts, under the general title Information technology — Software
process assessment :
Part 1: Concepts and introductory guide
Part 2: A reference model for processes and process capability
Part 3: Performing an assessment
Part 4: Guide to performing assessments
Part 5: An assessment model and indicator guidance
Part 6: Guide to competency of assessors
Part 7: Guide for use in process improvement
Part 8: Guide for use in determining supplier process capability
Part 9: Vocabulary
Annex A of this part of ISO/IEC TR 15504 is for information only.
iv
---------------------- Page: 4 ----------------------
© ISO/IEC
ISO/IEC TR 15504-4:1998(E)
Introduction
Process assessment is a means of capturing information describing the current capability of an organization's
processes and is initiated as a result of a desire to determine and/or improve the capability of these processes.
This part of ISO/IEC TR 15504 provides guidance on interpreting the requirements set out in ISO/IEC TR 15504-3.
As an aid to understanding, the requirements are embedded verbatim in italics at appropriate points within the text
of this part of ISO/IEC TR 15504.
The guidance in this part of ISO/IEC TR 15504 is primarily aimed at the competent assessor who has the
responsibility for the selection and use of models, documented assessment process and tools for the assessment.
The guidance may also be of use to the developers of assessment models, documented assessment processes
and tools as an aid to understanding the requirements.
The assessors and other participants in an assessment may use the guidance to gain an understanding of process
assessment.
v
---------------------- Page: 5 ----------------------
TECHNICAL REPORT © ISO/IEC ISO/IEC TR 15504-4:1998(E)
Information technology — Software process assessment —
Part 4:
Guide to performing assessments
1 Scope
This part of ISO/IEC TR 15504 provides guidance on meeting the requirements for performing an assessment
contained in ISO/IEC TR 15504-3.
It provides an overview of process assessment and interprets the requirements through the provision of guidance
on the selection and use of compatible models, documented assessment processes, and instruments or tools for
assessment.
Process assessment is applicable in the following circumstances:
a) by or on behalf of an organization with the objective of understanding the state of its own processes for process
improvement;
b) by or on behalf of an organization with the objective of determining the suitability of its own processes for a
particular requirement or class of requirements;
c) by or on behalf of one organization with the objective of determining the suitability of another organization's
processes for a particular contract or class of contracts.
2 Normative reference
The following normative documents contain provisions which, through reference in this text, constitute provisions of
this part of ISO/IEC TR 15504. For dated references, subsequent amendments to, or revisions of, any of these
publications do not apply. However, parties to agreements based on this part of ISO/IEC TR 15504 are encouraged
to investigate the possibility of applying the most recent editions of the normative documents indicated below. For
undated references, the latest edition of the normative document referred to applies. Members of ISO and IEC
maintain registers of currently valid International Standards.
ISO/IEC TR 15504-9:1998, Information technology — Software process assessment — Part 9: Vocabulary.
3 Terms and definitions
For the purposes of this part of ISO/IEC TR 15504, the terms and definitions given in ISO/IEC TR 15504-9 apply.
1
---------------------- Page: 6 ----------------------
© ISO/IEC
ISO/IEC TR 15504-4:1998(E)
4 Overview of process assessment
4.1 Process assessment
Process assessment is undertaken to understand the capability of an organizational unit's current processes.
Process assessment deals with all the software related processes (e.g. management, development, maintenance,
support) used by an organization. This is accomplished by assessing the organizational unit's processes against a
model(s) compatible with the reference model described in ISO/IEC TR 15504-2.
The reference model defines the set of universal software engineering processes that are fundamental to good
software engineering and a set of process attributes, applicable to any process, that characterize the capability of
an implemented process.
Processes in the reference model are grouped according to the type of activity they address. Each process has a
defined purpose describing the high-level objectives that the process should achieve. The purpose statements
describe what to do, but do not prescribe how the process should achieve its objectives.
Each process attribute in the reference model represents measurable characteristics of process management
providing the capability of the process to effectively achieve its purpose and contribute to meeting the business
goals of the organization. The process attributes are grouped into capability levels that define an ordinal scale of
process capability and provide a rational route for improvement of each individual process.
The assessment and rating framework is based upon assessing processes. The fundamental assessment output,
therefore, consists of up to nine process attribute ratings for each process assessed.
Although the reference model contained in ISO/IEC TR 15504-2 covers a range of processes applicable to the
software process, in many cases a subset of these processes may be selected for assessment. For instance the
sponsor may wish to focus attention on one or more critical processes or on processes which are candidates for
improvement actions. In process capability determination mode, an acquirer may wish to evaluate the capabilities
of suppliers only for the processes related to the tender or contract requirements.
The sophistication and complexity of the implemented process will be dependent upon the context of that process
within the organizational unit. For instance, the planning required for a five person project team will be much less
than for a fifty person team. This process context, recorded in the assessment input, influences how a competent
assessor should judge and rate the process attributes for an implemented process. The process context also
influences the degree of comparability between process attribute and/or process capability level ratings.
In some circumstances it may be desirable to compare the outputs of the assessment of two or more organizational
units, or for the same organizational unit at different times. A number of factors should be taken into account when
comparing assessment results. These include but are not limited to;
the sample size used to generate the ratings which will influence the precision with which results may be
compared;
the purposes of the assessments that generated the assessment outputs - it may not be meaningful, for
example, to compare an assessment whose purpose was to identify best (or worst) practice with one whose
purpose was to identify representative practice;
the documented assessment process or model(s) used;
the competency of the assessors;
the candour of the participants;
the time spent on the assessment;
the motivation of the assessor (i.e., internal assessor with incentives on the line based on the assessment
results or a consultant with a long-term relationship with the organization);
the motivation of the assessment participants to be frank and forthcoming.
2
---------------------- Page: 7 ----------------------
© ISO/IEC
ISO/IEC TR 15504-4:1998(E)
4.2 Process rating scheme
The process assessment rating framework is based on assessing processes. The guidance in clause 6 on
information collection will help to increase the level of repeatability by different assessors.
Each process has a set of process attribute ratings that constitute the process profile. Process attribute ratings are
expressed using the process attribute scale as defined in ISO/IEC TR 15504-2.
The process capability level model defines a six point ordinal scale of increasing process capability ranging from a
process which is not capable of achieving its purpose (process capability level zero) to a process which optimizes
its performance (process capability level 5). The process capability level model is described in terms of the process
attribute ratings that must be achieved in order to achieve a particular level.
When more than one instance of a process is assessed, the assessor will be required to use the recorded
assessment information collected on all of the instances to make a judgment on the rating of each of the process
attributes assessed for that process.
4.3 Assessment approaches
4.3.1 Self-assessment
A self-assessment is carried out by an organization to assess the capability of its own software process. The
sponsor of a self-assessment is normally internal to the organization.
4.3.2 Independent assessment
An independent assessment is an assessment conducted by assessors that are independent of the organizational
unit being assessed. An independent assessment may be conducted, for example, by an organization on its own
behalf as independent verification that its assessment program is functioning properly; the assessment sponsor will
belong to the same organization but not necessarily the organizational unit being assessed.
The sponsor of an independent assessment may be external to the organizational unit being assessed, such as an
acquirer who wishes to have an independently derived assessment output. The degree of independence, however,
may vary according to the purpose and circumstances of the assessment.
4.4 Assessment process
The assessment shall be conducted according to a documented process that is capable of meeting the
assessment purpose.
[ISO/IEC TR 15504-3, 4.4.1]
Irrespective of the type of assessment or the approach adopted, an assessment should be conducted according to
a documented process. Some of the key elements of a documented assessment process are briefly described
below, and described in more detail in clauses 5, 6 and 7. Note, however, that the guidance provided does not
constitute a complete, documented process. Its role is to provide help in interpreting the requirements in
ISO/IEC TR 15504-2 and ISO/IEC TR 15504-3, and to provide a starting point for selecting or creating a
documented process.
The key elements of the assessment process that are described are the documented assessment process, the
compatible assessment model(s), and supporting instruments and tools.
4.4.1 Documented Assessment Process
The documented assessment process is the set of instructions and procedure for conducting the assessment.
Depending upon the approach, a documented assessment process should provide guidance on the following topics:
roles and responsibilities;
use of tools and techniques;
3
---------------------- Page: 8 ----------------------
© ISO/IEC
ISO/IEC TR 15504-4:1998(E)
required resources;
sequenced activities and procedures that fall under the following categories:
planning
data collection
data validation
process rating
4.5 Compatible model
A compatible model is one that meets the requirements specified in ISO/IEC TR 15504-2. In summary, a
compatible model is one:
that is suitable for the purpose of process assessment;
whose fundamental elements can be and are mapped to the process and capability dimensions of the
reference model in ISO/IEC TR 15504-2;
that is equipped with sets of indicators for use during an assessment to gather the information about processes
and process attributes;
that has a formal mechanism for translating the information gathered using the model into process attribute
ratings as defined in ISO/IEC TR 15504-2.
Clause 5 provides guidance on the selection and use of a compatible model. The model in ISO/IEC TR 15504-5 is
an exemplar of a compatible model.
4.6 Supporting instruments and tools
In any assessment, information will need to be collected, recorded, stored, collated, processed, analysed, retrieved
and presented. In general, a documented assessment process will be supported by various instruments and tools
for information gathering, processing and presentation. For some assessments, the support tools and instruments
may be manual and paper-based (forms, questionnaires, checklists, etc.). In some cases the volume and
complexity of the assessment information is considerable resulting in the need for computer-based support tools.
Regardless of the form of the supporting instruments and tools, their objectives should be to help an assessor
perform an assessment in a consistent and reliable manner, reducing assessor subjectivity and helping to ensure
the validity, useability and comparability of assessment results. In order to achieve these objectives, the
instruments and tools need to make the assessment model and its indicators accessible to the assessors.
4.7 Success factors for process assessment
The following factors are essential to a successful process assessment.
4.7.1 Commitment
The sponsor should commit himself to the objectives established for an assessment to provide the authority to
undertake the assessment within an organization. This commitment requires that the necessary resources, time
and personnel are available to undertake the assessment. The commitment of the sponsor and the assessors is
fundamentally important to ensuring that the objectives are met.
4
---------------------- Page: 9 ----------------------
© ISO/IEC
ISO/IEC TR 15504-4:1998(E)
4.7.2 Motivation
The attitude of the organization's management, and the documented assessment process by which the information
is collected, has a significant influence on the outcome of an assessment. The organization's management,
therefore, needs to motivate participants to be open and constructive. Process assessments focus on the process,
not on the performance of organizational unit members implementing the process. The intent is to make the
processes more effective to support the defined business goals, not to allocate blame to individuals.
Providing feedback and maintaining an atmosphere that encourages open discussion about preliminary findings
during the assessment helps to ensure that the assessment output is meaningful to the organizational unit. The
organization needs to recognize that the participants are a principal source of knowledge and experience about the
process and that they are in a good position to identify potential weaknesses.
4.7.3 Confidentiality
Respect for the confidentiality of the sources of information and documentation gathered during assessment is
essential in order to secure that information. If discussion techniques are utilized, consideration should be given to
ensuring that participants do not feel threatened or have any concerns regarding confidentiality. Some of the
information provided might be proprietary to the organization. It is therefore important that adequate controls are in
place to handle such information.
4.7.4 Relevance
The organizational unit members should believe that the assessment will result in some benefits that will accrue to
them directly or indirectly.
4.7.5 Credibility
The sponsor and the management and staff of the organizational unit must all believe that the assessment will
deliver a result which is objective and is representative of the assessment scope. It is important that all parties can
be confident that the assessors have adequate experience of assessment, are sufficiently impartial and have an
adequate understanding of the organizational unit and its business to conduct the assessment.
5 Selection and use of a compatible model
This clause provides guidance on the selection and use of a compatible model as the basis for performing a
software process assessment. The guidance is intended for the use of assessors and sponsors of assessments. It
is not directed specifically at the developers of compatible models, though it may be of use to them.
In performing a process assessment, the practices observed in the organization unit being assessed are compared
against those defined in a base model of good practice, to determine the extent to which the performance of the
practices results in achievement of the attributes of capability. In order to achieve this, the model must contain
descriptions of the practices to be observed, and indicators of the performance of these practices, so that the
judgments of capability may be made reliably and consistently.
5.1 Compatibility with the reference model
The identity of the model(s) used within the assessment, which shall be a compatible model(s) of good
software engineering practice that meet the requirements defined in ISO/IEC TR 15504-2.
[ISO/IEC TR 15504-3, 4.2.2 e]
The first criterion for the selection of a model(s) is that it achieves compatibility with the reference model.
Compatibility is essential in order to provide a degree of comparability between the results of different assessments
by maximizing the reliability of different approaches and achieving a greater degree of uniformity in the reporting of
results.
5
---------------------- Page: 10 ----------------------
© ISO/IEC
ISO/IEC TR 15504-4:1998(E)
5.1.1 Model Purpose
A model shall be based on good software engineering and process management principles and be suitable
for the purpose of assessing software process capability.
[ISO/IEC TR 15504-2, 7.2]
There are many different types of modelling techniques available for describing, specifying and enacting processes.
Models that have not been specifically developed for the purpose of process assessment may not yield reliable
results, and their suitability for purpose should be validated before selection.
5.1.2 Model Scope
A model shall encompass all, or a non-empty subset, of the set of processes in the process dimension of the
reference model.
A model shall address all or a continuous subset of the levels (starting at level 1) of the capability dimension
of the reference model for all of the processes within its scope.
[ISO/IEC TR 15504-2, 7.3]
The reference model defines a set of universal software engineering processes that are fundamental to good
software engineering and that cover best practice activities. Any model, to be compatible with the reference model,
must contain at least a part of this scope. The model may be a sub-set of the reference model. It may be a super-
set of the reference model, covering all of the defined processes together with additional process descriptions
outside the standard scope. A compatible model may also include processes outside the reference model providing
it encompasses at least one process from it. Finally, the scope of the model may be directly equivalent to the
reference model.
For the capability dimension, a model must cover a complete set of capabilities for all of the processes in its scope.
The set of capabilities must encompass the whole of the capability level scale in the reference model or a subset
starting at level 1. It is permissible, therefore, for a model to claim coverage of levels 1 to 3 only, but not of only
levels 3 to 5.
In selecting a model, the assessors should ensure that the scope of the model covers the intended area of interest
for the assessment.
5.1.3 Model elements and indicators
A model shall be based on a set of elements that explicitly address the purposes, as defined in the reference
model, of all the processes within the scope of the model, and that demonstrate the achievement of the
process attributes within the capability level scope of the model.
[ISO/IEC TR 15504-2, 7.4]
In order for a model to be compatible with the reference model, it must address the purposes of the processes as
defined in the reference model, and the achievement of the process attributes that constitute the capability
dimension. In order to meet the requirements of the other components of ISO/IEC TR 15504, it must also document
a set of indicators of process performance and process capability that enable judgements of process capability to
be soundly based on objective evidence.
There is a clear expectation that the indicators will fall into two categories: factors that indicate the performance of
the process, and factors that indicate its capability. In selecting a model, clear attention should be paid to the use of
indicators in the model, the comprehensiveness of the indicator set, and the applicability of the indicator set.
ISO/IEC TR 15504-5 comprises a mode
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.