ISO/IEC TR 15504-8:1998

Information technology — Software process assessment — Part 8: Guide for use in determining supplier process capability

Technologies de l'information — Évaluation des procédés du logiciel — Partie 8: Guide pour l'utilisation dans la détermination d'aptitude de processus de fournisseur

General Information

• Status: Withdrawn
• Publication Date: 26-Aug-1998
• Withdrawal Date: 26-Aug-1998
• ICS: 35.080 - Software
• Technical Committee: ISO/IEC JTC 1/SC 7 - Software and systems engineering
• Drafting Committee: ISO/IEC JTC 1/SC 7/WG 10 - Process assessment
• Current Stage: 9599 - Withdrawal of International Standard
• Completion Date: 02-Jul-2004

Relations

• Revised: ISO/IEC 15504-4:2004 - Information technology — Process assessment — Part 4: Guidance on use for process improvement and process capability determination
• Effective Date: 15-Apr-2008

Standards Content (Sample)

TECHNICAL ISO/IEC
REPORT TR 15504-8
First edition
1998-08-15
Information technology — Software process
assessment —
Part 8:
Guide for use in determining supplier process
capability
Technologies de l’information — Évaluation des procédés du logiciel —
Partie 8: Guide pour l’utilisation dans la détermination de la capacité du
procédé du fournisseur
Reference number
B C
ISO/IEC TR 15504-8:1998(E)

---------------------- Page: 1 ----------------------
ISO/IEC TR 15504-8:1998(E)
Contents
1 Scope .1
2 Normative reference .1
3 Terms and definitions.2
4 Introduction to process capability determination .2
4.1 Overview .2
4.1.1 Purpose.2
4.1.2 Core and extended process capability determination .2
4.1.3 Compatible assessment methods and models.2
4.1.4 Basis of process capability determination.3
4.1.5 Assessment approaches.3
4.1.6 Process-oriented risk .3
4.1.7 Key processes.4
4.1.8 Process-oriented risk analysis .4
4.1.9 Output .4
4.2 Target capability.4
4.2.1 Initial key processes.5
4.2.2 Default process attribute achievement targets.6
4.2.3 Reviewing and adjusting process attribute achievement targets.6
4.2.4 Adding further processes .6
4.3 Process-oriented risk analysis .6
4.3.1 Assessed capability profile.6
4.3.2 Target capability statement .7
© ISO/IEC 1998
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or
utilized in any form or by any means, electronic or mechanical, including photocopying and micro-
film, without permission in writing from the publisher.
ISO/IEC Copyright Office • Case postale 56 • CH-1211 Genève 20 • Switzerland
Printed in Switzerland
ii

---------------------- Page: 2 ----------------------
© ISO/IEC
ISO/IEC TR 15504-8:1998(E)
4.3.3 Probability . 7
4.3.4 Impact . 9
4.3.5 Overall risk. 9
4.4 The process capability report. 10
5 Conducting a process capability determination. 11
5.1 Core process capability determination. 11
5.1.1 The target definition stage. 12
5.1.2 The response stage . 12
5.1.3 The verification and risk analysis stage. 13
5.2 Extended process capability determination .14
5.2.1 The response stage . 14
5.2.2 The verification and risk analysis stage. 17
iii

---------------------- Page: 3 ----------------------
© ISO/IEC
ISO/IEC TR 15504-8:1998(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical Commission)
form the specialized system for worldwide standardization. National bodies that are members of ISO or IEC
participate in the development of International Standards through technical committees established by the
respective organization to deal with particular fields of technical activity. ISO and IEC technical committees
collaborate in fields of mutual interest. Other international organizations, governmental and non-governmental, in
liaison with ISO and IEC, also take part in the work.
In the field of information technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
The main task of technical committees is to prepare International Standards, but in exceptional circumstances a
technical committee may propose the publication of a Technical Report of one of the following types:
— type 1, when the required support cannot be obtained for the publication of an International Standard, despite
repeated efforts;
— type 2, when the subject is still under technical development or where for any other reason there is the future
but not immediate possibility of an agreement on an International Standard;
— type 3, when a technical committee has collected data of a different kind from that which is normally published
as an International Standard (“state of the art”, for example).
Technical Reports of types 1 and 2 are subject to review within three years of publication, to decide whether they
can be transformed into International Standards. Technical Reports of type 3 do not necessarily have to be
reviewed until the data they provide are considered to be no longer valid or useful.
ISO/IEC TR 15504-8, which is a Technical Report of type 2, was prepared by Joint Technical Committee ISO/IEC
JTC 1, Information technology, Subcommittee SC 7, Software engineering.
ISO/IEC TR 15504 consists of the following parts, under the general title Information technology — Software
process assessment :
 Part 1: Concepts and introductory guide
 Part 2: A reference model for processes and process capability
 Part 3: Performing an assessment
 Part 4: Guide to performing assessments
 Part 5: An assessment model and indicator guidance
 Part 6: Guide to competency of assessors
 Part 7: Guide for use in process improvement
 Part 8: Guide for use in determining supplier process capability
 Part 9: Vocabulary
iv

---------------------- Page: 4 ----------------------
TECHNICAL REPORT  © ISO/IEC ISO/IEC TR 15504-8:1998(E)
Information technology — Software process assessment —
Part 8:
Guide for use in determining supplier process capability
1 Scope
This part of ISO/IEC TR 15504 provides guidance on utilizing process assessment for the purposes of process
capability determination. This part of ISO/IEC TR 15504 is informative and is intended to provide guidance on how
to apply the requirements.
A process capability determination is a systematic assessment and analysis of selected software processes within
an organization, carried out with the aim of identifying the strengths, weaknesses and risks associated with
deploying the processes to meet a particular specified requirement.
The specified requirement may involve a project, product or a service, a new or an existing task, a contract or an
internal undertaking, or any other requirement which is to be met by deploying an organization's software
processes.
This guidance is intended to be applicable across all software application domains, over all software organizational
structures, within any software customer-supplier relationship, and to any organization wishing to determine the
process capability of its own software processes.
This part of ISO/IEC TR 15504 is primarily aimed at:
 the sponsor who initiates the process capability determination;
 the organization whose process capability is to be determined;
 the assessment team;
 method developers.
ISO/IEC TR 15504 is not intended to be used in any scheme for the certification/registration of the process
capability of an organization.
2 Normative reference
The following normative documents contain provisions which, through reference in this text, constitute provisions of
this part of ISO/IEC TR 15504. For dated references, subsequent amendments to, or revisions of, any of these
publications do not apply. However, parties to agreements based on this part of ISO/IEC TR 15504 are encouraged
to investigate the possibility of applying the most recent editions of the normative documents indicated below. For
undated references, the latest edition of the normative document referred to applies. Members of ISO and IEC
maintain registers of currently valid International Standards.
ISO/IEC TR 15504-9:1998, Information technology — Software process assessment — Part 9: Vocabulary.
1

---------------------- Page: 5 ----------------------
© ISO/IEC
ISO/IEC TR 15504-8:1998(E)
3 Terms and definitions
For the purposes of this part of ISO/IEC TR 15504, the terms and definitions given in ISO/IEC TR 15504-9 apply.
4 Introduction to process capability determination
4.1 Overview
4.1.1 Purpose
A process capability determination is a systematic assessment and analysis of selected software processes within
an organization, carried out with the aim of identifying the strengths, weaknesses and risks associated with
deploying the processes to meet a particular specified requirement.
One of the main reasons for carrying out a process capability determination is to obtain information upon which to
base a procurement-related decision. A procurer may initiate a process capability determination to assess the risk
of entering into a contract with a particular supplier. The procurer may carry out process capability determinations
on a number of competing suppliers during a pre-contract supplier selection activity; software process capability is
of course only one of the factors taken into account during supplier selection. Conversely, suppliers may wish to
carry out a process capability determination on their own processes before deciding whether to bid for a contract, as
part of their own assessment of the business risks involved. A process capability determination may also be initiated
for a number of other reasons; for example, by a supplier during the course of a project to establish the risks
involved in completing the work.
Process capability determination may be applied to a variety of situations: the specified requirement may involve a
new or an existing task, a contract or an internal undertaking, a product or a service, or any other requirement which
is to be met by deploying an organization's software processes.
4.1.2 Core and extended process capability determination
This part of ISO/IEC TR 15504 presents two alternative approaches to process capability determination described
below.
Core process capability determination is a minimum, streamlined set of activities applicable whenever a single
organization proposes to meet a specified requirement by deploying its current process capability, without any
partners or sub-contractors being involved.
Extended process capability determination is applicable when an enhanced capability is proposed, or when
consortia or sub-contractors are involved.
In either case the conduct of process capability determination is described in three separate stages, as set out in
clause 5.
4.1.3 Compatible assessment methods and models
ISO/IEC TR 15504-3 sets out the minimum requirements for performing an assessment in order to ensure
consistency and repeatability of the process assessment ratings. The requirements help to ensure that the
assessment output is internally self-consistent, and provides evidence to substantiate the ratings and to verify
compliance with the requirements. ISO/IEC TR 15504-2 sets out compatibility requirements which enable outputs
from assessments conducted with different, compatible assessment models to be compared. They include
requirements for mapping from the fundamental elements of the compatible model to the processes and process
attributes of the reference model. The guidance contained in this part of ISO/IEC TR 15504 is intended to apply to
outputs from assessments performed with compatible models after they have been mapped onto the reference
model.
2

---------------------- Page: 6 ----------------------
© ISO/IEC
ISO/IEC TR 15504-8:1998(E)
4.1.4 Basis of process capability determination
The output of a process assessment which has been mapped to the reference model is a set of process profiles.
These profiles represent the capability of the organization's implementation of the processes in a particular
assessment context and are reusable for both process capability determination and process improvement in that
particular context or a similar context.
4.1.5 Assessment approaches
Either self-assessment or independent assessment approaches may be used during a process capability
determination. In a two-party contractual situation, a procurer may wish to invite potential suppliers to provide a self-
assessment profile - produced from an assessment using compatible models and mapped to the reference model -
when submitting a proposal for a contract. Such an approach offers the benefit of sharing both the cost and the
benefit of the process assessment, since suppliers may also use the assessment results within their own process
improvement programmes.
The procurer may choose to:
 initiate and rely entirely upon a full independent assessment and make this a condition of contract award;
 accept a self-assessment at face value;
 initiate an independent sample assessment to verify that the self-assessment is a true representation of the
supplier's process capability.
ISO/IEC TR 15504 thus offers the benefit of reducing disruption to suppliers' business activities caused by multiple
process assessments, since the same assessment results may be offered to many procurers. It also provides
procurers with a rigorous and defensible approach to supplier process capability determination, and the potential to
reduce assessment costs through the reuse of results and the utilization of self-assessments.
4.1.6 Process-oriented risk
During a process capability determination, a selection of an organization's software processes are assessed and
the results analysed to identify strengths, weaknesses and risks. Process capability determination does not address
all aspects of risk, which may include strategic, organizational, financial, personnel and many other factors. The
output from a process capability determination feeds into this wider risk analysis, but confines itself to process-
oriented risk.
The process architecture of ISO/IEC TR 15504 rests on the reference model. This model sets out 40 processes and
defines the purpose and outcomes of each, as well as a set of nine process attributes which apply to all processes.
The process attributes are concerned with process management and are grouped into ordered capability levels,
which progressively describe major enhancements to process capability. The single process attribute in the
Performed capability level measures the extent to which the execution of a process uses a set of practices that
transform identifiable input work products into identifiable output work products and satisfy the defined process
purpose. Additional, user-defined processes can also be added if required.
During a compatible process assessment, individual process attributes are rated by competent assessors against
either a percentage scale representing the extent of achievement of the attribute, or a 4-point ordinal scale whereby
process attributes are rated as fully, largely, partially or not achieved. ISO/IEC TR 15504-2 describes the
relationship between the two scales. The guidance presented within this part of ISO/IEC TR 15504 uses the 4-point
representation exclusively. Ratings are made utilising an appropriate set of indicators of process performance and
an appropriate set of indicators of process capability.
The nine process attribute ratings for an assessed process form its process profile. Process attribute ratings for
several process may then be collected into a process capability profile that indicates, for each process assessed,
which process attributes are being achieved. Process ratings are described in ISO/IEC TR 15504-2.
The key to process-oriented risk lies in the reference model, the good process management practices it reflects
through the process attributes, and the benefits that arise from deploying them. Process-oriented risk arises from
inappropriate process management - i.e. not deploying appropriate management practices, or from deploying them
in a way which is assessed in the particular context as not achieving the required process attributes.
3

---------------------- Page: 7 ----------------------
© ISO/IEC
ISO/IEC TR 15504-8:1998(E)
4.1.7 Key processes
Within this part of ISO/IEC TR 15504, the capability of a process is expressed in terms of the achievement of its
process attributes.
The sponsor of the process capability determination may be a procurer initiating a process capability determination
to determine whether a potential supplier’s processes are suitable for a particular requirement, or an organization
initiating a process capability determination to determine whether its own processes are suitable.
The sponsor determines which of the 40 process in the reference model will be most important to meeting the
specified requirement. These processes are termed the key processes for the process capability determination. The
sponsor lists the key processes within a target capability statement, and states - for each key process - which
process attributes are required, and - for each attribute - what achievement rating is deemed necessary.
The target capability is chosen to be that capability which the sponsor judges will represent a minimal process risk
to the successful implementation of the specified requirement.
4.1.8 Process-oriented risk analysis
Within this part of ISO/IEC TR 15504, process-oriented risk is assessed firstly from the probability of a particular
problem occurring, and secondly from its potential impact, should it occur.
Suppose that a sponsor indicates in a target capability statement that a particular process attribute should be fully
achieved for a particular process. The assessed achievement of the process attribute is less than fully achieved.
There is therefore a gap between target and assessed attributes which increases the probability that the process
will not contribute satisfactorily towards meeting the specified requirement. If the sponsor believes that, for a
particular process, all of the process attributes up to and including the Managed capability level should be fully
achieved, and if the assessed process profile shows that the process attribute at the Performed capability level is
not fully achieved, then a major gap exists and there is a high probability of a problem occurring.
The potential impact of the problem depends upon the capability level within which it occurs. For example, if a key
process is assessed less than fully performed, as reflected by the rating for the Process Performance attribute at
the Performed capability level, then the process is incomplete and this may lead to missing work products, or
unacceptable product quality, or both.
4.1.9 Output
The output of a process capability determination is the process capability report. It summarizes, for each key
process included within the target capability statement, strengths and weaknesses expressed in terms of process
attribute gaps, and the risks associated with each.
4.2 Target capability
Sponsors may wish to develop or purchase an appropriate method for defining target capability. A number of
approaches are possible, but most will be based on the following principles.
The target capability is chosen to be that capability which the sponsor judges will represent a minimal process risk
to the successful implementation of the specified requirement.
Target capability is expressed within a target capability statement, which lists processes key to meeting the
specified requirements and states, for each key process, the required achievement of each process attribute.
Only process attribute achievement targets of fully, or largely, or not required should be set.
For each key process, sponsors should identify which process attributes are required, and set the degree of
achievement for each. Process attribute achievement may be set in several ways. For example, the same degree of
achievement may be allocated to:
a) all of the process attributes up to a certain capability level;
b) individually selected process attributes.
Table 1 illustrates a target capability statement.
4

---------------------- Page: 8 ----------------------
© ISO/IEC
ISO/IEC TR 15504-8:1998(E)
Table 1 — Example target capability statement
Key Process Process Attributes Process attribute
ratings required
CUS.3 Requirements elicitation PA1.1, PA2.1, PA2.2 Fully Achieved
(i.e. all up to and including
the Managed capability
level)
CUS.4.2 Customer support PA1.1, PA2.1, PA2.2, PA3.1, Fully Achieved
PA3.2
(i.e. all up to and including
the Established capability
level)
ENG.1.3 Software design PA1.1, PA2.1, PA2.2, PA3.1, Fully Achieved
PA3.2
ENG.1.4 Software construction PA1.1, PA2.1, PA2.2, PA3.1, Fully Achieved
PA3.2
PA4.1, PA4.2 Largely Achieved
ENG.1.6 Software testing PA1.1, PA2.1, PA2.2, PA3.1, Fully Achieved
PA3.2
PA4.1, PA4.2 Largely Achieved
MAN.2 Project management PA1.1, PA2.1, PA2.2 Fully Achieved
PA3.1, PA3.2 Largely Achieved
MAN.3 Quality management PA1.1, PA2.1, PA2.2 Fully Achieved
PA3.1, PA3.2 Largely Achieved
SUP.2 Configuration PA1.1, PA2.1, PA2.2 Fully Achieved
management
PA3.1, PA3.2 Largely Achieved
A number of approaches to setting target capability are possible. One approach is to:
a) identify a set of initial key processes;
b) set default process attribute achievement targets for the set of initial key processes;
c) review and adjust the default process attribute achievement targets;
d) add further processes, and set achievement targets for the further processes.
These steps are described in the following paragraphs.
4.2.1 Initial key processes
The processes in the reference model which contribute most directly to the delivery of products and services are
those within the Customer-Supplier and Engineering process categories. Processes from the Management, Support
and Organization process categories provide a more indirect contribution.
5

---------------------- Page: 9 ----------------------
© ISO/IEC
ISO/IEC TR 15504-8:1998(E)
Key processes are identified, starting with the processes in the Customer-Supplier and Engineering process
categories. Any processes in these categories which are not relevant to the specified requirement should be
eliminated, and the remainder designated as the initial set of key processes.
4.2.2 Default process attribute achievement targets
A good starting position is to state, for each key process, that all of the process attributes in the first three capability
levels - Performed, Managed, and Established - should be rated as fully achieved, with the other process attributes
not being specified.
This approach ensures firstly that processes are complete and fully performed; secondly that management
practices are in place to reduce unpredictability, missed deadlines, budget overspend and reduced output quality;
and thirdly that processes are deployed following organization-wide standard process definitions, thus providing
confidence that future performance will be consistent with past accomplishments.
4.2.3 Reviewing and adjusting process attribute achievement targets
Requiring that process attributes in the Predictable capability level should also be fully or largely achieved for a
given process may reduce performance risks. For instance, a particular specified requirement may demand that
some processes be controlled quantitatively. Process attributes within the Optimizing capability level may
occasionally also be needed, but for many organizations, this degree of process management may not yet be
practical. Alternatively, sponsors may feel that for a particular key process, only process attributes within the first
two capability levels are appropriate.
4.2.4 Adding further processes
Many process attributes are related to processes within the Management, Support and Organization process
categories.
For example, if the Performance Management attribute (PA2.1) has been included for a process within the
Engineering process category, then the Project Management process within the Management process category
should also be included as a key process.
The target capability for processes in the Management, Support and Organization process categories is determined
by the extent to which they support process attributes applying to the initial set of key processes. Other processes
from the Support, Management and Organization process categories may also be included in the target capability
statement where they are relevant to the specified requirement.
Note that the specified requirement may be for an organizational capability, rather than a product or service. The
specified requirement may be to establish a strong configuration management process as an end in itself, and the
key process set would then include just this single process. This class of specified requirement would arise from an
organization's business goals and priorities.
4.3 Process-oriented risk analysis
A number of approaches to analysing process-oriented risk are possible. One approach is to infer process-oriented
risk from the existence of gaps between target capability and assessed capability. If the target capability statement
indicates that a particular process attribute should be fully achieved, while the assessed process attribute rating is
less than fully achieved, then a gap is said to exist.
Within this approach, process-oriented risk is assessed firstly from the probability of a particular problem occurring,
and secondly from the nature of its impact. The probability is derived from the extent of any gaps between an
assessed capability profile and a target capability statement. The nature of the impact depends upon the capability
level within which the gap occurs.
4.3.1 Assessed capability profile
The assessed capability profile will be in the form of an output from a process assessment which has been mapped
to the reference model. This profile will contain process attribute ratings as defined in ISO/IEC TR 15504-2,
paragraph 6.7.4. For each process assessed and for each process attribute, the process attribute rating profile
6

---------------------- Page: 10 ----------------------
© ISO/IEC
ISO/IEC TR 15504-8:1998(E)
Table 3 — Capability level gaps
Number of process attribute Capability level
gaps within capability level gap
No major or minor gaps None
Minor gaps only Slight
A single major gap at Levels 2 - Significant
5
A single major gap at Level 1, or Substantial
more than one major gap at
Levels 2 - 5
4.3.4 Impact
The previous section showed how the probability of problems occurring is inferred from the extent of a gap at a
capability level.
The potential impact of a particular problem depends upon the capability level in which the gap occurs:
 A gap at the Optimising level may lead to reduced cost/time optimisation and reduced ability to cope with
changes in technology;
 A gap at the Predictable level may also result in an inability to predict performance or timely detect problems;
 A gap at the Established level may lead, in addition to the above problems, to reduced cost effectiveness, plus
reduced spatial and temporal uniformity of performance;
 A gap at the Managed level may further l
...