Information technology -- Software process assessment

Technologies de l'information -- Évaluation des procédés du logiciel

General Information

Status
Withdrawn
Publication Date
26-Aug-1998
Withdrawal Date
26-Aug-1998
Current Stage
6060 - International Standard published
Start Date
24-Jun-1998
Completion Date
27-Aug-1998
Ref Project

RELATIONS

Buy Standard

Technical report
ISO/IEC TR 15504-8:1998 - Information technology -- Software process assessment
English language
17 pages
sale 15% off
Preview
sale 15% off
Preview

Standards Content (sample)

TECHNICAL ISO/IEC
REPORT TR 15504-8
First edition
1998-08-15
Information technology — Software process
assessment —
Part 8:
Guide for use in determining supplier process
capability
Technologies de l’information — Évaluation des procédés du logiciel —
Partie 8: Guide pour l’utilisation dans la détermination de la capacité du
procédé du fournisseur
Reference number
B C
ISO/IEC TR 15504-8:1998(E)
---------------------- Page: 1 ----------------------
ISO/IEC TR 15504-8:1998(E)
Contents

1 Scope .....................................................................................................................................................1

2 Normative reference .............................................................................................................................1

3 Terms and definitions...........................................................................................................................2

4 Introduction to process capability determination .............................................................................2

4.1 Overview ................................................................................................................................................2

4.1.1 Purpose..................................................................................................................................................2

4.1.2 Core and extended process capability determination ......................................................................2

4.1.3 Compatible assessment methods and models..................................................................................2

4.1.4 Basis of process capability determination.........................................................................................3

4.1.5 Assessment approaches......................................................................................................................3

4.1.6 Process-oriented risk ...........................................................................................................................3

4.1.7 Key processes.......................................................................................................................................4

4.1.8 Process-oriented risk analysis ............................................................................................................4

4.1.9 Output ....................................................................................................................................................4

4.2 Target capability....................................................................................................................................4

4.2.1 Initial key processes.............................................................................................................................5

4.2.2 Default process attribute achievement targets..................................................................................6

4.2.3 Reviewing and adjusting process attribute achievement targets....................................................6

4.2.4 Adding further processes ....................................................................................................................6

4.3 Process-oriented risk analysis ............................................................................................................6

4.3.1 Assessed capability profile..................................................................................................................6

4.3.2 Target capability statement .................................................................................................................7

© ISO/IEC 1998

All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or

utilized in any form or by any means, electronic or mechanical, including photocopying and micro-

film, without permission in writing from the publisher.
ISO/IEC Copyright Office • Case postale 56 • CH-1211 Genève 20 • Switzerland
Printed in Switzerland
---------------------- Page: 2 ----------------------
© ISO/IEC
ISO/IEC TR 15504-8:1998(E)

4.3.3 Probability ............................................................................................................................................. 7

4.3.4 Impact .................................................................................................................................................... 9

4.3.5 Overall risk............................................................................................................................................. 9

4.4 The process capability report............................................................................................................ 10

5 Conducting a process capability determination.............................................................................. 11

5.1 Core process capability determination............................................................................................. 11

5.1.1 The target definition stage................................................................................................................. 12

5.1.2 The response stage ............................................................................................................................ 12

5.1.3 The verification and risk analysis stage........................................................................................... 13

5.2 Extended process capability determination ....................................................................................14

5.2.1 The response stage ............................................................................................................................ 14

5.2.2 The verification and risk analysis stage........................................................................................... 17

iii
---------------------- Page: 3 ----------------------
© ISO/IEC
ISO/IEC TR 15504-8:1998(E)
Foreword

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical Commission)

form the specialized system for worldwide standardization. National bodies that are members of ISO or IEC

participate in the development of International Standards through technical committees established by the

respective organization to deal with particular fields of technical activity. ISO and IEC technical committees

collaborate in fields of mutual interest. Other international organizations, governmental and non-governmental, in

liaison with ISO and IEC, also take part in the work.

In the field of information technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.

The main task of technical committees is to prepare International Standards, but in exceptional circumstances a

technical committee may propose the publication of a Technical Report of one of the following types:

— type 1, when the required support cannot be obtained for the publication of an International Standard, despite

repeated efforts;

— type 2, when the subject is still under technical development or where for any other reason there is the future

but not immediate possibility of an agreement on an International Standard;

— type 3, when a technical committee has collected data of a different kind from that which is normally published

as an International Standard (“state of the art”, for example).

Technical Reports of types 1 and 2 are subject to review within three years of publication, to decide whether they

can be transformed into International Standards. Technical Reports of type 3 do not necessarily have to be

reviewed until the data they provide are considered to be no longer valid or useful.

ISO/IEC TR 15504-8, which is a Technical Report of type 2, was prepared by Joint Technical Committee ISO/IEC

JTC 1, Information technology, Subcommittee SC 7, Software engineering.

ISO/IEC TR 15504 consists of the following parts, under the general title Information technology — Software

process assessment :
 Part 1: Concepts and introductory guide
 Part 2: A reference model for processes and process capability
 Part 3: Performing an assessment
 Part 4: Guide to performing assessments
 Part 5: An assessment model and indicator guidance
 Part 6: Guide to competency of assessors
 Part 7: Guide for use in process improvement
 Part 8: Guide for use in determining supplier process capability
 Part 9: Vocabulary
---------------------- Page: 4 ----------------------
TECHNICAL REPORT © ISO/IEC ISO/IEC TR 15504-8:1998(E)
Information technology — Software process assessment —
Part 8:
Guide for use in determining supplier process capability
1 Scope

This part of ISO/IEC TR 15504 provides guidance on utilizing process assessment for the purposes of process

capability determination. This part of ISO/IEC TR 15504 is informative and is intended to provide guidance on how

to apply the requirements.

A process capability determination is a systematic assessment and analysis of selected software processes within

an organization, carried out with the aim of identifying the strengths, weaknesses and risks associated with

deploying the processes to meet a particular specified requirement.

The specified requirement may involve a project, product or a service, a new or an existing task, a contract or an

internal undertaking, or any other requirement which is to be met by deploying an organization's software

processes.

This guidance is intended to be applicable across all software application domains, over all software organizational

structures, within any software customer-supplier relationship, and to any organization wishing to determine the

process capability of its own software processes.
This part of ISO/IEC TR 15504 is primarily aimed at:
 the sponsor who initiates the process capability determination;
 the organization whose process capability is to be determined;
 the assessment team;
 method developers.

ISO/IEC TR 15504 is not intended to be used in any scheme for the certification/registration of the process

capability of an organization.
2 Normative reference

The following normative documents contain provisions which, through reference in this text, constitute provisions of

this part of ISO/IEC TR 15504. For dated references, subsequent amendments to, or revisions of, any of these

publications do not apply. However, parties to agreements based on this part of ISO/IEC TR 15504 are encouraged

to investigate the possibility of applying the most recent editions of the normative documents indicated below. For

undated references, the latest edition of the normative document referred to applies. Members of ISO and IEC

maintain registers of currently valid International Standards.

ISO/IEC TR 15504-9:1998, Information technology — Software process assessment — Part 9: Vocabulary.

---------------------- Page: 5 ----------------------
© ISO/IEC
ISO/IEC TR 15504-8:1998(E)
3 Terms and definitions

For the purposes of this part of ISO/IEC TR 15504, the terms and definitions given in ISO/IEC TR 15504-9 apply.

4 Introduction to process capability determination
4.1 Overview
4.1.1 Purpose

A process capability determination is a systematic assessment and analysis of selected software processes within

an organization, carried out with the aim of identifying the strengths, weaknesses and risks associated with

deploying the processes to meet a particular specified requirement.

One of the main reasons for carrying out a process capability determination is to obtain information upon which to

base a procurement-related decision. A procurer may initiate a process capability determination to assess the risk

of entering into a contract with a particular supplier. The procurer may carry out process capability determinations

on a number of competing suppliers during a pre-contract supplier selection activity; software process capability is

of course only one of the factors taken into account during supplier selection. Conversely, suppliers may wish to

carry out a process capability determination on their own processes before deciding whether to bid for a contract, as

part of their own assessment of the business risks involved. A process capability determination may also be initiated

for a number of other reasons; for example, by a supplier during the course of a project to establish the risks

involved in completing the work.

Process capability determination may be applied to a variety of situations: the specified requirement may involve a

new or an existing task, a contract or an internal undertaking, a product or a service, or any other requirement which

is to be met by deploying an organization's software processes.
4.1.2 Core and extended process capability determination

This part of ISO/IEC TR 15504 presents two alternative approaches to process capability determination described

below.

Core process capability determination is a minimum, streamlined set of activities applicable whenever a single

organization proposes to meet a specified requirement by deploying its current process capability, without any

partners or sub-contractors being involved.

Extended process capability determination is applicable when an enhanced capability is proposed, or when

consortia or sub-contractors are involved.

In either case the conduct of process capability determination is described in three separate stages, as set out in

clause 5.
4.1.3 Compatible assessment methods and models

ISO/IEC TR 15504-3 sets out the minimum requirements for performing an assessment in order to ensure

consistency and repeatability of the process assessment ratings. The requirements help to ensure that the

assessment output is internally self-consistent, and provides evidence to substantiate the ratings and to verify

compliance with the requirements. ISO/IEC TR 15504-2 sets out compatibility requirements which enable outputs

from assessments conducted with different, compatible assessment models to be compared. They include

requirements for mapping from the fundamental elements of the compatible model to the processes and process

attributes of the reference model. The guidance contained in this part of ISO/IEC TR 15504 is intended to apply to

outputs from assessments performed with compatible models after they have been mapped onto the reference

model.
---------------------- Page: 6 ----------------------
© ISO/IEC
ISO/IEC TR 15504-8:1998(E)
4.1.4 Basis of process capability determination

The output of a process assessment which has been mapped to the reference model is a set of process profiles.

These profiles represent the capability of the organization's implementation of the processes in a particular

assessment context and are reusable for both process capability determination and process improvement in that

particular context or a similar context.
4.1.5 Assessment approaches

Either self-assessment or independent assessment approaches may be used during a process capability

determination. In a two-party contractual situation, a procurer may wish to invite potential suppliers to provide a self-

assessment profile - produced from an assessment using compatible models and mapped to the reference model -

when submitting a proposal for a contract. Such an approach offers the benefit of sharing both the cost and the

benefit of the process assessment, since suppliers may also use the assessment results within their own process

improvement programmes.
The procurer may choose to:

 initiate and rely entirely upon a full independent assessment and make this a condition of contract award;

 accept a self-assessment at face value;

 initiate an independent sample assessment to verify that the self-assessment is a true representation of the

supplier's process capability.

ISO/IEC TR 15504 thus offers the benefit of reducing disruption to suppliers' business activities caused by multiple

process assessments, since the same assessment results may be offered to many procurers. It also provides

procurers with a rigorous and defensible approach to supplier process capability determination, and the potential to

reduce assessment costs through the reuse of results and the utilization of self-assessments.

4.1.6 Process-oriented risk

During a process capability determination, a selection of an organization's software processes are assessed and

the results analysed to identify strengths, weaknesses and risks. Process capability determination does not address

all aspects of risk, which may include strategic, organizational, financial, personnel and many other factors. The

output from a process capability determination feeds into this wider risk analysis, but confines itself to process-

oriented risk.

The process architecture of ISO/IEC TR 15504 rests on the reference model. This model sets out 40 processes and

defines the purpose and outcomes of each, as well as a set of nine process attributes which apply to all processes.

The process attributes are concerned with process management and are grouped into ordered capability levels,

which progressively describe major enhancements to process capability. The single process attribute in the

Performed capability level measures the extent to which the execution of a process uses a set of practices that

transform identifiable input work products into identifiable output work products and satisfy the defined process

purpose. Additional, user-defined processes can also be added if required.

During a compatible process assessment, individual process attributes are rated by competent assessors against

either a percentage scale representing the extent of achievement of the attribute, or a 4-point ordinal scale whereby

process attributes are rated as fully, largely, partially or not achieved. ISO/IEC TR 15504-2 describes the

relationship between the two scales. The guidance presented within this part of ISO/IEC TR 15504 uses the 4-point

representation exclusively. Ratings are made utilising an appropriate set of indicators of process performance and

an appropriate set of indicators of process capability.

The nine process attribute ratings for an assessed process form its process profile. Process attribute ratings for

several process may then be collected into a process capability profile that indicates, for each process assessed,

which process attributes are being achieved. Process ratings are described in ISO/IEC TR 15504-2.

The key to process-oriented risk lies in the reference model, the good process management practices it reflects

through the process attributes, and the benefits that arise from deploying them. Process-oriented risk arises from

inappropriate process management - i.e. not deploying appropriate management practices, or from deploying them

in a way which is assessed in the particular context as not achieving the required process attributes.

---------------------- Page: 7 ----------------------
© ISO/IEC
ISO/IEC TR 15504-8:1998(E)
4.1.7 Key processes

Within this part of ISO/IEC TR 15504, the capability of a process is expressed in terms of the achievement of its

process attributes.

The sponsor of the process capability determination may be a procurer initiating a process capability determination

to determine whether a potential supplier’s processes are suitable for a particular requirement, or an organization

initiating a process capability determination to determine whether its own processes are suitable.

The sponsor determines which of the 40 process in the reference model will be most important to meeting the

specified requirement. These processes are termed the key processes for the process capability determination. The

sponsor lists the key processes within a target capability statement, and states - for each key process - which

process attributes are required, and - for each attribute - what achievement rating is deemed necessary.

The target capability is chosen to be that capability which the sponsor judges will represent a minimal process risk

to the successful implementation of the specified requirement.
4.1.8 Process-oriented risk analysis

Within this part of ISO/IEC TR 15504, process-oriented risk is assessed firstly from the probability of a particular

problem occurring, and secondly from its potential impact, should it occur.

Suppose that a sponsor indicates in a target capability statement that a particular process attribute should be fully

achieved for a particular process. The assessed achievement of the process attribute is less than fully achieved.

There is therefore a gap between target and assessed attributes which increases the probability that the process

will not contribute satisfactorily towards meeting the specified requirement. If the sponsor believes that, for a

particular process, all of the process attributes up to and including the Managed capability level should be fully

achieved, and if the assessed process profile shows that the process attribute at the Performed capability level is

not fully achieved, then a major gap exists and there is a high probability of a problem occurring.

The potential impact of the problem depends upon the capability level within which it occurs. For example, if a key

process is assessed less than fully performed, as reflected by the rating for the Process Performance attribute at

the Performed capability level, then the process is incomplete and this may lead to missing work products, or

unacceptable product quality, or both.
4.1.9 Output

The output of a process capability determination is the process capability report. It summarizes, for each key

process included within the target capability statement, strengths and weaknesses expressed in terms of process

attribute gaps, and the risks associated with each.
4.2 Target capability

Sponsors may wish to develop or purchase an appropriate method for defining target capability. A number of

approaches are possible, but most will be based on the following principles.

The target capability is chosen to be that capability which the sponsor judges will represent a minimal process risk

to the successful implementation of the specified requirement.

Target capability is expressed within a target capability statement, which lists processes key to meeting the

specified requirements and states, for each key process, the required achievement of each process attribute.

Only process attribute achievement targets of fully, or largely, or not required should be set.

For each key process, sponsors should identify which process attributes are required, and set the degree of

achievement for each. Process attribute achievement may be set in several ways. For example, the same degree of

achievement may be allocated to:
a) all of the process attributes up to a certain capability level;
b) individually selected process attributes.
Table 1 illustrates a target capability statement.
---------------------- Page: 8 ----------------------
© ISO/IEC
ISO/IEC TR 15504-8:1998(E)
Table 1 — Example target capability statement
Key Process Process Attributes Process attribute
ratings required
CUS.3 Requirements elicitation PA1.1, PA2.1, PA2.2 Fully Achieved
(i.e. all up to and including
the Managed capability
level)
CUS.4.2 Customer support PA1.1, PA2.1, PA2.2, PA3.1, Fully Achieved
PA3.2
(i.e. all up to and including
the Established capability
level)
ENG.1.3 Software design PA1.1, PA2.1, PA2.2, PA3.1, Fully Achieved
PA3.2
ENG.1.4 Software construction PA1.1, PA2.1, PA2.2, PA3.1, Fully Achieved
PA3.2
PA4.1, PA4.2 Largely Achieved
ENG.1.6 Software testing PA1.1, PA2.1, PA2.2, PA3.1, Fully Achieved
PA3.2
PA4.1, PA4.2 Largely Achieved
MAN.2 Project management PA1.1, PA2.1, PA2.2 Fully Achieved
PA3.1, PA3.2 Largely Achieved
MAN.3 Quality management PA1.1, PA2.1, PA2.2 Fully Achieved
PA3.1, PA3.2 Largely Achieved
SUP.2 Configuration PA1.1, PA2.1, PA2.2 Fully Achieved
management
PA3.1, PA3.2 Largely Achieved

A number of approaches to setting target capability are possible. One approach is to:

a) identify a set of initial key processes;

b) set default process attribute achievement targets for the set of initial key processes;

c) review and adjust the default process attribute achievement targets;
d) add further processes, and set achievement targets for the further processes.
These steps are described in the following paragraphs.
4.2.1 Initial key processes

The processes in the reference model which contribute most directly to the delivery of products and services are

those within the Customer-Supplier and Engineering process categories. Processes from the Management, Support

and Organization process categories provide a more indirect contribution.
---------------------- Page: 9 ----------------------
© ISO/IEC
ISO/IEC TR 15504-8:1998(E)

Key processes are identified, starting with the processes in the Customer-Supplier and Engineering process

categories. Any processes in these categories which are not relevant to the specified requirement should be

eliminated, and the remainder designated as the initial set of key processes.
4.2.2 Default process attribute achievement targets

A good starting position is to state, for each key process, that all of the process attributes in the first three capability

levels - Performed, Managed, and Established - should be rated as fully achieved, with the other process attributes

not being specified.

This approach ensures firstly that processes are complete and fully performed; secondly that management

practices are in place to reduce unpredictability, missed deadlines, budget overspend and reduced output quality;

and thirdly that processes are deployed following organization-wide standard process definitions, thus providing

confidence that future performance will be consistent with past accomplishments.
4.2.3 Reviewing and adjusting process attribute achievement targets

Requiring that process attributes in the Predictable capability level should also be fully or largely achieved for a

given process may reduce performance risks. For instance, a particular specified requirement may demand that

some processes be controlled quantitatively. Process attributes within the Optimizing capability level may

occasionally also be needed, but for many organizations, this degree of process management may not yet be

practical. Alternatively, sponsors may feel that for a particular key process, only process attributes within the first

two capability levels are appropriate.
4.2.4 Adding further processes

Many process attributes are related to processes within the Management, Support and Organization process

categories.

For example, if the Performance Management attribute (PA2.1) has been included for a process within the

Engineering process category, then the Project Management process within the Management process category

should also be included as a key process.

The target capability for processes in the Management, Support and Organization process categories is determined

by the extent to which they support process attributes applying to the initial set of key processes. Other processes

from the Support, Management and Organization process categories may also be included in the target capability

statement where they are relevant to the specified requirement.

Note that the specified requirement may be for an organizational capability, rather than a product or service. The

specified requirement may be to establish a strong configuration management process as an end in itself, and the

key process set would then include just this single process. This class of specified requirement would arise from an

organization's business goals and priorities.
4.3 Process-oriented risk analysis

A number of approaches to analysing process-oriented risk are possible. One approach is to infer process-oriented

risk from the existence of gaps between target capability and assessed capability. If the target capability statement

indicates that a particular process attribute should be fully achieved, while the assessed process attribute rating is

less than fully achieved, then a gap is said to exist.

Within this approach, process-oriented risk is assessed firstly from the probability of a particular problem occurring,

and secondly from the nature of its impact. The probability is derived from the extent of any gaps between an

assessed capability profile and a target capability statement. The nature of the impact depends upon the capability

level within which the gap occurs.
4.3.1 Assessed capability profile

The assessed capability profile will be in the form of an output from a process assessment which has been mapped

to the reference model. This profile will contain process attribute ratings as defined in ISO/IEC TR 15504-2,

paragraph 6.7.4. For each process assessed and for each process attribute, the process attribute rating profile

---------------------- Page: 10 ----------------------
© ISO/IEC
ISO/IEC TR 15504-8:1998(E)
Table 3 — Capability level gaps
Number of process attribute Capability level
gaps within capability level gap
No major or minor gaps None
Minor gaps only Slight
A single major gap at Levels 2 - Significant
A single major gap at Level 1, or Substantial
more than one major gap at
Levels 2 - 5
4.3.4 Impact

The previous section showed how the probability of problems occurring is inferred from the extent of a gap at a

capability level.

The potential impact of a particular problem depends upon the capability level in which the gap occurs:

 A gap at the Optimising level may lead to reduced cost/time optimisation and reduced ability to cope with

changes in technology;

 A gap at the Predictable level may also result in an inability to predict performance or timely detect problems;

 A gap at the Established level may lead, in addition to the above problems, to reduced cost effectiveness, plus

reduced spatial and temporal uniformity of performance;
 A gap at the Managed level may further l
...

Questions, Comments and Discussion

Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.