ISO 8000-63:2019

Data quality — Part 63: Data quality management: Process measurement

This document specifies a structure for process measurement stacks that organizations can instantiate to measure the characteristics of processes for data quality management. This structure consists of goal, sub goal, question, indicator and metric. The instantiated stack consists of content that is determined by a chosen model for assessing the maturity of the processes under consideration. The following are within the scope of this document: — the fundamentals of process measurement; — the inspection plan and inspection order by which to perform measurement; — the structure of the measurement stack; — the role of the chosen maturity model in instantiating the stack. The following are outside the scope of this document: — methods for inspecting implemented processes; — methods for generating measured values from the inspection of processes; — how to choose a suitable maturity model. This document can be used in conjunction with, or independently of, quality management systems standards. NOTE ISO 8000‑8 specifies concepts and methods for measuring information and data quality.

Qualité des données — Partie 63: Gestion de la qualité des données: Évaluation du processus

General Information

• Status: Published
• Publication Date: 17-Dec-2019
• ICS: 25.040.40 - Industrial process measurement and control
• Technical Committee: ISO/TC 184/SC 4 - Industrial data
• Drafting Committee: ISO/TC 184/SC 4/WG 13 - Industrial Data Quality
• Current Stage: 6060 - International Standard published
• Start Date: 18-Dec-2019
• Completion Date: 18-Dec-2019

Standards Content (Sample)

INTERNATIONAL ISO
STANDARD 8000-63
First edition
2019-12
Data quality —
Part 63:
Data quality management: Process
measurement
Qualité des données —
Partie 63: Gestion de la qualité des données: Évaluation du processus
Reference number
ISO 8000-63:2019(E)
©
ISO 2019

---------------------- Page: 1 ----------------------
ISO 8000-63:2019(E)

COPYRIGHT PROTECTED DOCUMENT
© ISO 2019
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: [phone]
Fax: [phone]
Email: [e-mail]
Website: www.iso.org
Published in Switzerland
ii © ISO 2019 – All rights reserved

---------------------- Page: 2 ----------------------
ISO 8000-63:2019(E)

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 2
5 Fundamentals of process measurement . 2
6 Process measurement and maturity models . 2
7 Inspection plans . 4
7.1 Overview of inspection plans . 4
7.2 The measurement stack structure . 4
7.3 Goals . 5
7.4 Sub goals . 5
7.5 Questions . 6
7.6 Indicators . 6
7.7 Metrics . 6
7.8 Logical coherence of the measurement stack . 6
7.9 Role of value assessment rules in process measurement . 7
7.10 Exploiting the measurement stack with specific maturity models . 8
7.11 Performing process measurement. 8
Annex A (informative) Document identification .13
Annex B (informative) Reference tables for particular process assessment models .14
Annex C (informative) Example measurement stacks for data quality management .16
Annex D (informative) Example of process assessment based on ISO/IEC 33020 .19
Annex E (informative) Examples of how to define characteristics to measure for data
quality processes .23
Bibliography .25
© ISO 2019 – All rights reserved iii

---------------------- Page: 3 ----------------------
ISO 8000-63:2019(E)

Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www .iso .org/ patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www .iso .org/
iso/ foreword .html.
This document was prepared by Technical Committee ISO/TC 184, Automation systems and integration,
Subcommittee SC 4, Industrial data.
A list of all parts in the ISO 8000 series can be found on the ISO website.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/ members .html.
iv © ISO 2019 – All rights reserved

---------------------- Page: 4 ----------------------
ISO 8000-63:2019(E)

Introduction
The ability to create, collect, store, maintain, transfer, process and present information and data to
support business processes in a timely and cost-effective manner requires both an understanding of
the characteristics of the information and data that determine its quality, and an ability to measure,
manage and report on information and data quality.
ISO 8000 defines characteristics of information and data that determine its quality, and provides
methods to manage, measure and improve the quality of information and data.
When assessing the quality of information and data, it is useful to perform the assessment in accordance
with documented methods. It is also important to document the tailoring of standardized methods with
respect to the expectation and requirements pertinent to the business case at hand.
ISO 8000 includes parts applicable to all types of data and parts applicable to specific types of data.
ISO 8000 can be used independently or in conjunction with quality management systems.
There is a limit to data quality improvement when only the nonconformity in data is corrected, since the
nonconformity can recur. However, when the root causes of the data nonconformity and their related
data are traced and corrected through data quality processes, recurrence of the same type of data
nonconformity can be prevented. Therefore, a framework for process-centric data quality management
is required to improve data quality more effectively and efficiently. Furthermore, data quality can be
improved through assessing processes and improving under-performing processes identified by the
assessment.
This document specifies a process measurement approach that is appropriate for use when assessing
process maturity. This approach can serve when an organization is looking to improve the maturity of
data quality management.
This document can be used on its own or in conjunction with other parts of ISO 8000.
This document is intended for use by those actors that have a vested interest in information or data
quality, have a focus on one or more information systems and have a concern for both inter- and intra-
organization views, throughout all phases of the data life cycle.
Annex A contains an identifier that unambiguously identifies this document in an open information
system.
© ISO 2019 – All rights reserved v

---------------------- Page: 5 ----------------------
INTERNATIONAL STANDARD ISO 8000-63:2019(E)
Data quality —
Part 63:
Data quality management: Process measurement
1 Scope
This document specifies a structure for process measurement stacks that organizations can instantiate
to measure the characteristics of processes for data quality management. This structure consists
of goal, sub goal, question, indicator and metric. The instantiated stack consists of content that is
determined by a chosen model for assessing the maturity of the processes under consideration.
The following are within the scope of this document:
— the fundamentals of process measurement;
— the inspection plan and inspection order by which to perform measurement;
— the structure of the measurement stack;
— the role of the chosen maturity model in instantiating the stack.
The following are outside the scope of this document:
— methods for inspecting implemented processes;
— methods for generating measured values from the inspection of processes;
— how to choose a suitable maturity model.
This document can be used in conjunction with, or independently of, quality management systems
standards.
NOTE ISO 8000-8 specifies concepts and methods for measuring information and data quality.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO 8000-2, Data quality — Part 2: Vocabulary
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO 8000-2 apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
© ISO 2019 – All rights reserved 1

---------------------- Page: 6 ----------------------
ISO 8000-63:2019(E)

4 Abbreviated terms
CMMI Capability Maturity Model Integration
EFQM European Foundation for Quality Management
TPI Test Process Improvement
5 Fundamentals of process measurement
This document specifies an approach for measuring the characteristics of data quality management
processes. These processes are specified by ISO 8000-61 (see Figure 1), which describes each process
with a purpose, one or more outcomes and one or more activities.
Figure 1 — Detailed structure of the data quality management processes in ISO 8000-61
This document specifies a structure that enables an organization to define a measurement stack based
on the purpose, outcomes and activities of the process that the organization is intending to measure.
6 Process measurement and maturity models
The purpose of process measurement is to provide inputs to maturity assessment of an implemented
process (the “target process”). This assessment is performed using a chosen maturity model (e.g. CMMI,
TPI, EFQM). Such assessment involves analysing characteristics that are specified by the maturity
2 © ISO 2019 – All rights reserved

---------------------- Page: 7 ----------------------
ISO 8000-63:2019(E)

model. These characteristics are represented by either qualitative or quantitative values. The model
specifies how these values indicate the maturity of the target process.
This document specifies an approach that involves identifying how to generate appropriate measured
values and then using rules to convert those values to the characteristics specified by the maturity
model. This approach can be used with all types of maturity model.
In order to perform assessments, an organization requires an inspection plan that includes measurement
stacks to identify the measured values that are appropriate for the characteristics specified by the
chosen maturity model (see Figure 2).
Figure 2 — Relationships between inspection plan, maturity model and process
Each inspection plan has a detailed internal structure (see Clause 7). This structure enables the plan to
support assessment of the target process using the chosen maturity model (see Figure 3).
© ISO 2019 – All rights reserved 3

---------------------- Page: 8 ----------------------
ISO 8000-63:2019(E)

NOTE In this figure, the references A1 to A4 are to activities in the model represented by Figures 7 to 11.
Figure 3 — The relationships between inspection plan, maturity model and process
An inspection plan is specific to one or more identified processes. Each process provides a purpose,
outcomes and activities that are the basis for the measurement stacks within the plan.
The plan identifies the measurements that are necessary to support the intended assessment of
maturity.
When deciding to assess an implemented process, an organization creates an inspection order from
the applicable inspection plan. This inspection order takes account of previous inspection history in
identifying the timing and the resources necessary to perform the inspection. The order provides a
checklist of the total set of measurements to support the overall assessment of the process maturity.
7 Inspection plans
7.1 Overview of inspection plans
An inspection plan consists of:
— one or more measurement stacks (see 7.2);
— the rules by which to convert measured indicator values into converted indicator values (see 7.9).
The chosen maturity model describes how to represent the converted indicator values. It establishes a
common and comparable scale for the assessment of maturity.
7.2 The measurement stack structure
The measurement stack structure (see Figure 4) is applicable to any type of process and any type of
maturity model.
4 © ISO 2019 – All rights reserved

---------------------- Page: 9 ----------------------
ISO 8000-63:2019(E)

To perform a process measurement, an organization uses the structure to create an instance of the
measurement stack. Each instance has specific content that depends on the details of:
— the process to be measured (including the purpose, outcomes and activities of the process);
— the chosen maturity model.
NOTE See ISO/IEC 19505-1 for details on the notation in this diagram.
Figure 4 — The measurement stack structure
Each instance of the measurement stack consists of the following elements:
— a goal (see 7.3);
— zero or more sub goals (see 7.4);
— one or more questions (see 7.5);
— one or more indicators (see 7.6);
— one or more metrics (see 7.7).
7.3 Goals
A goal defines the primary subject of interest for process measurement (i.e. if an implemented process
is not achieving the identified goals then this process is a prime candidate for improvement). Each goal
relates to the purpose of the target process and captures aspects of the performance of that process
and the requirements of those who benefit from execution of the process (e.g. customers).
7.4 Sub goals
A goal can be represented by a set of one or more sub goals. Each sub goal is similar to a goal but is
typically more specific, addressing one individual outcome, for example, of the target process.
© ISO 2019 – All rights reserved 5

---------------------- Page: 10 ----------------------
ISO 8000-63:2019(E)

7.5 Questions
A question identifies what needs to be measured in order to understand whether and to what degree
the target process fulfils the applicable goal. Each question asks how the goal will be achieved. Each
question corresponds to one or more indicators.
7.6 Indicators
An indicator is specified by the chosen maturity model as a characteristic that provides evidence of the
maturity of a process.
Process measurement initially generates a measured value for the state of an indicator at the time of
the measurement. This value is the measured indicator value.
The inspection plan specifies a value assessment rule for each indicator. This rule is the mathematical
function that converts the measured indicator value into a converted indicator value, i.e. each rule has
the form of the following mathematical formula:
C = f (M)
where
C is the converted indicator value;
f is the conversion function;
M is the measured indicator value.
The inspection plan can express the function f either as an algorithm or as a textual description.
7.7 Metrics
A metric relates to one indicator and is the scale by which the chosen maturity model represents
that indicator. The metric is the scale for the converted indicator value that is generated as a result of
performing process measurement (see Table B.4 for examples of indicators and metrics).
7.8 Logical coherence of the measurement stack
Each instantiated measurement stack is the basis for one measurement within the corresponding
inspection plan.
Elements of one instantiated measurement stack can also be a part of one or more other measurement
stacks (see Figure 4).
EXAMPLE An organization creates an inspection plan that addresses a set of processes. It includes more
than one measurement stack, making multiple use of individual indicators and the corresponding metrics
(see Figure 5).
6 © ISO 2019 – All rights reserved

---------------------- Page: 11 ----------------------
ISO 8000-63:2019(E)

Figure 5 — Example measurement stacks
7.9 Role of value assessment rules in process measurement
Each value assessment rule (see 7.6) is the fundamental link between collecting evidence about an
implemented process and determining the maturity of that process (see Figure 6). This link enables an
organization to ensure maturity assessment avoids subjective judgement and is repeatable.
© ISO 2019 – All rights reserved 7

---------------------- Page: 12 ----------------------
ISO 8000-63:2019(E)

NOTE In this figure, the references A4.1 to A4.3 are to activities in the model represented by Figures 7 to 11.
Figure 6 — Calculating the converted indicator value from the measured indicator value
7.10 Exploiting the measurement stack with specific maturity models
Annexes B, C and D show how different maturity models can be used with this document.
Annex E shows examples of how to define characteristics to measure for data quality processes.
7.11 Performing process measurement
Process measurement (see Figure 7) consists of a hierarchy of activities, as follows:
— perform process measurement (see Figure 8);
— define measurement stack structure (see Figure 9);
— define inspection plan (see Figure 10);
— prepare inspection order (see Figure 11);
— apply inspection order (see Figure 12).
NOTE Using the notation of ISO/IEC/IEEE 31320-1, Figures 7 and 12 represent the content of Figures 3 and 6.
8 © ISO 2019 – All rights reserved

---------------------- Page: 13 ----------------------
ISO 8000-63:2019(E)

NOTE 1 See ISO/IEC/IEEE 31320-1 for details on the notation in this diagram.
NOTE 2 The dashed arrow represents the instance of a data quality management process and is the subject of
the process measurement. All other arrows in the activity model are solid lines and represent information.
Figure 7 — Perform process measurement (activity model diagram A-0)
© ISO 2019 – All rights reserved 9

---------------------- Page: 14 ----------------------
ISO 8000-63:2019(E)

NOTE See ISO/IEC/IEEE 31320-1 for details on the notation in this diagram.
Figure 8 — Perform process measurement (activity model diagram A0)
NOTE See ISO/IEC/IEEE 31320-1 for details on the notation in this diagram.
Figure 9 — Define measurement stack structure (activity model diagram A1)
10 © ISO 2019 – All rights reserved

---------------------- Page: 15 ----------------------
ISO 8000-63:2019(E)

NOTE See ISO/IEC/IEEE 31320-1 for details on the notation in this diagram.
Figure 10 — Define inspection plan (activity model diagram A2)
NOTE See ISO/IEC/IEEE 31320-1 for details on the notation in this diagram.
Figure 11 — Prepare inspection order (activity model diagram A3)
© ISO 2019 – All rights reserved 11

---------------------- Page: 16 ----------------------
ISO 8000-63:2019(E)

NOTE See ISO/IEC/IEEE 31320-1 for details on the notation in this diagram.
Figure 12 — Apply inspection order (activity model diagram A4)
12 © ISO 2019 – All rights reserved

---------------------- Page: 17 ----------------------
ISO 8000-63:2019(E)

Annex A
(informative)

Document identification
To provide for unambiguous identification of an information object in an open system, the object
identifier
{ iso standard 8000 part(63) version(1) }
is assigned to this document. The meaning of this value is defined in ISO/IEC 8824-1 and is described in
ISO 10303-1.
© ISO 2019 – All rights reserved 13

---------------------- Page: 18 ----------------------
ISO 8000-63:2019(E)

Annex B
(informative)

Reference tables for particular process assessment models
[14]
This annex provides examples of how to define questions for different process assessment models .
For each of these models, a table below lists the process capability levels in the model and, for each
level, the type of questions that are necessary to determine if the assessed process is achieving that
level. The models are:
— the process assessment model in ISO/IEC 33020 and related standards (see Table B.1), where the
process outcomes of the assessed process and process attributes results from ISO/IEC 33020 are
the basis for questions;
— the process institutionalization model of CMMI for Development (see Table B.2), where the generic
practices are the basis for questions;
— from applying the Test Process Improvement method (see Table B.3), where the outcomes and
activities are the basis for questions.
In addition, special indicator types can be used as input for different models (see Table B.4).
Table B.1 — Capability based on ISO/IEC 33020
Process capability level Source for the questions to determine achievement of the capability level
1. Performed process Outcomes of the assessed process
2. Managed process Performance management results
Work product management results
3. Established process Process definition results
Process deployment results
4. Predictable process Process measurement results
Process control results
5. Innovating process Process innovation results
Process innovation implementation results
NOTE  The questions address the process attributes specified by ISO/IEC 33020.
Table B.2 — Capability based on CMMI (Capability Maturity Model Integration)
Process capability level Subject of the questions to determine achievement of the capability level
1. Performed process Performing specific practices
NOTE  The questions address the generic practices specified by CMMI for Development.
14 © ISO 2019 – All rights reserved

---------------------- Page: 19 ----------------------
ISO 8000-63:2019(E)

Table B.2 (continued)
Process capability level Subject of the questions to determine achievement of the capability level
2. Managed process Establishing an organizational policy
Planning the process
Providing resources
Assigning responsibility
Training people
Controlling work products
Identifying and involving relevant stakeholders
Monitoring and controlling the process
Objectively evaluating adherence
Reviewing status with higher level management
3. Defined process Establishing a defined process
Collecting process-related experiences
NOTE  The questions address the generic practices specified by CMMI for Development.
Table B.3 — Capability based on the TPI method
Process capability level Subject of the questions to determine achievement of the capability level
1. Controlled process Control of process and environment
2. Efficient process Achieving efficiency and better integration
3. Optimized process Optimization
NOTE  In line with the Test Process Improvement method, the questions address the outcomes and activities of the process.
Table B.4 — Example indicators for the different process assessment models
Example ISO/IEC 33020 CMMI for ISO 9004 EFQM TPI method
Development
Indicator Process Status of Name of Mean error Name of
performance commitment author existing probability of author existing
commitment
Metric “N” | “P” | “L” | “F” 0: not achieved Yes/No Qualitative: A: 0 to ≤15 %
insignificant/
where 1: partially B: >15 % to ≤50 %
noticeable/
   achieved
“N” = not achieved critical/ C: >50 % to ≤85 %
2: largely catastrophic
“P” = partially D: >85 % to ≤100 %
   achieved
Quantitative:
     achieved
Depend on the
0 to 100 %
3: fully
“L” = largely requirements
   achieved
     achieved
“F” = fully achieved
Measured 65 % 10 % Yes 35 % 30 %
indicator
value
Converted “L” “1” Yes Qualitative: “B”
indicator noticeable
value
Quantitative: 35 %
© ISO 2019 – All rights reserved 15

---------------------- Page: 20 ----------------------
ISO 8000-63:2019(E)

Annex C
(informative)

Example measurement stacks for data quality management
This annex shows example measurement stacks for the following processes from ISO 8000-61:
— Data Quality Strategy Management;
— Data Quality Policy/Standards/Procedures Management.
For each stack, the outcomes of the corresponding process determine appropriate goals and sub goals.
The annex also shows how the following different maturity models determine the appropriate metric to
define in the measurement stack:
— EFQM, requiring the use of percentage as the metric (see Table C.1);
— ISO 9004, requiring the use of “yes” and “no” as the metric (see Table C.2).
Table C.1 — Example measurement stacks for use with EFQM
Process: Data Quality Strategy Management (see ISO 8000-61:2016, 6.2.3)
Purpose: The purpose of Data Quality Strategy Management is to establish the basis on which subse-
quently to develop policies, standards, procedures and implementation plans that apply to
data quality management across the organization and that align with strategic intentions
for data quality.
Goal: G1 Top management is committed to the improvement of data quality to agreed
levels at the organizational level.
Sub goal: S1.1 Clear definition of the commitment.
Question Indicator Metric
Q1.1.1 Who defines the commitment? I1.1.1 Name of author existing M1.1.1 %
(0 . 100 %)
Q1.1.2 Where is the commitment I1.1.2 Location of commitment
defined? known (0 . 100 %)
Sub goal: S1.2 Clear definitions of responsibilities.
Question Indicator Metric
Q1.2.1 Are there any responsibilities? I1.2.1 Degree of responsibility M1.2.1 %
Goal: G2 A data quality strategy is created, describing the vision, long-term goals, an im-
plementation roadmap and short-term objectives, which are defined in terms of
quantitative outcomes.
Sub goal: S2.1 Clear definition of the data quality strategy.
Question Indicator Metric
Q2.1.1 What is meant under data I2.1.1 Degree of qualification (edu- M2.1.1 %
quality? cated user)
Q2.1.2 What should a data quality I2.1.2 Degree of fulfilment (man-
...