ISO/IEC 25020:2019

Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - Quality measurement framework

This document provides a framework for developing quality measurement. The contents of this document are as follows: - quality measurement reference model; - relationships among different types of quality measures; - guidelines for selecting quality measures; - guidelines for constructing quality measures; - guidelines for planning and performing measurements; - guidelines for the application of measurement results. It includes considerations for selecting quality measures and quality measure elements (Annex A), assessing the reliability of measurement and the validity of quality measures (Annex B), elements for documenting quality measures (Annex C), normalized measurement function for quality measures (Annex D) and the measurement information model in ISO/IEC/IEEE 15939 (Annex E). This document can be applied for designing, identifying, evaluating and executing the measurement model of system and software product quality, quality in use, data quality and IT service quality. This reference model can be used by developers, acquirers, quality assurance staff and independent evaluators-essentially by people responsible for specifying and evaluating the quality of information and communication technology (ICT) systems and services.

Ingénierie des systèmes et du logiciel — Exigences de qualité du produit logiciel et évaluation (SQuaRE) — Modèle de référence de mesure et guide

General Information

• Status: Published
• Publication Date: 11-Jul-2019
• ICS: 35.080 - Software
• Technical Committee: ISO/IEC JTC 1/SC 7 - Software and systems engineering
• Drafting Committee: ISO/IEC JTC 1/SC 7/WG 6 - Software Product and System Quality
• Current Stage: 9093 - International Standard confirmed
• Start Date: 03-Jan-2025
• Completion Date: 30-Oct-2025

Relations

• Revises: ISO/IEC 25020:2007 - Software engineering - Software product Quality Requirements and Evaluation (SQuaRE) - Measurement reference model and guide
• Effective Date: 06-Aug-2016

Overview

ISO/IEC 25020:2019 - part of the SQuaRE (Systems and software Quality Requirements and Evaluation) series - defines a quality measurement framework for systems and software engineering. The standard provides a structured reference model and practical guidance for developing, selecting, constructing, planning and applying quality measures (QMs) used to assess product quality, quality in use, data quality and IT service quality. It is the 2019 second edition and is aligned with ISO/IEC 25000 and ISO/IEC/IEEE 15939.

Key topics and requirements

• Quality measurement reference model - a formal structure for defining how measures relate to attributes, quality characteristics and information needs.
• Types of measures - clarifies base measures, derived measures and indicators and their relationships.
• Selecting quality measures (QMs) - criteria and considerations for choosing measures that meet information needs and objectives.
• Constructing quality measures - guidance to build QMs from quality measure elements (QMEs), including measurement functions and formulas.
• Measurement planning and execution - instructions for planning measurements, choosing measurement methods (objective vs subjective), and performing reliable measurements.
• Applying measurement results - guidance on interpreting results, reporting, and using outcomes for decision making, quality assurance and improvement.
• Assurance of measurement - annexed guidance on assessing reliability of measurement and validity of quality measures.
• Documentation and models - templates and elements for documenting QMs (Annex C), a normalized measurement function (Annex D) and a measurement information model mapped to ISO/IEC/IEEE 15939 (Annex E).

Practical applications

ISO/IEC 25020:2019 is useful for organizations that need repeatable, auditable quality measurement practices:

• Designing measurement programs for software product quality and quality in use.
• Defining and validating data quality metrics and IT service quality measures.
• Constructing composite indicators from QMEs (e.g., defect density, availability indicators).
• Planning measurement activities for procurement, acceptance testing and independent evaluations.
• Ensuring measurement results are reliable and suitable for managerial decisions or compliance reporting.

Who should use it

• Developers and engineering teams creating measurable quality requirements.
• Acquirers and procurement teams specifying acceptance criteria.
• Quality assurance staff implementing measurement processes.
• Independent evaluators and auditors validating quality claims of ICT systems and services.

Related standards

• ISO/IEC 25000 (SQuaRE guide), ISO/IEC 25021, ISO/IEC 25022, ISO/IEC 25023, ISO/IEC 25024, ISO/IEC TS 25025, and ISO/IEC/IEEE 15939 (measurement process).

Keywords: ISO/IEC 25020:2019, SQuaRE, quality measurement framework, quality measures, QM, QME, measurement function, software quality, data quality, IT service quality.

Standards Content (Sample)

INTERNATIONAL ISO/IEC
STANDARD 25020
Second edition
2019-07
Systems and software engineering —
Systems and software Quality
Requirements and Evaluation
(SQuaRE) — Quality measurement
framework
Ingénierie des systèmes et du logiciel — Exigences de qualité du
produit logiciel et évaluation (SQuaRE) — Modèle de référence de
mesure et guide
Reference number
©
ISO/IEC 2019
© ISO/IEC 2019
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: [phone]
Fax: [phone]
Email: [e-mail]
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2019 – All rights reserved

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 5
5 Conformance . 5
6 Quality measurement . 5
6.1 Quality measurement reference model . 5
6.2 Different QMs and their relationships . 7
6.3 Selecting QMs .10
6.4 Constructing QMs .10
6.4.1 Identify QMs needed to be constructed .10
6.4.2 Description of the QM .11
6.4.3 Definitions of the QMEs .11
6.5 Plan and perform measurement .12
6.6 Application of the measurement results .13
Annex A (informative) Considerations for selecting QMs and QMEs .14
Annex B (informative) Assessing the reliability of measurement and the validity of QMs .16
Annex C (informative) Elements for documenting QMs .18
Annex D (informative) Normalized measurement function for QMs .21
Annex E (informative) Measurement information model in ISO/IEC/IEEE 15939 .24
Bibliography .27
© ISO/IEC 2019 – All rights reserved iii

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www .iso .org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the meaning of ISO specific terms and expressions related to conformity
assessment, as well as information about ISO's adherence to the World Trade Organization (WTO)
principles in the Technical Barriers to Trade (TBT) see www .iso .org/iso/foreword .html.
This document was prepared by Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 7, Software and systems engineering.
This second edition cancels and replaces the first edition (ISO/IEC 25020:2007), which has been
technically revised.
The main changes compared to the previous edition are as follows:
— relationships among different types of quality measures have been added;
— application of measurement results and description of quality measure have been added;
— elements for documenting quality measures in Annex C have been supplemented and categorized;
— Annex D has been added showing a normalized measurement function for QMs;
— Annex E has been added showing the measurement information model in ISO/IEC/IEEE 15939;
— harmonized with ISO/IEC 25000:2014, ISO/IEC 25022:2016, ISO/IEC 25023:2016, ISO/IEC 25024:2015
and ISO/IEC/IEEE 15939:2017.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at https: //www .iso .org/members .html.
iv © ISO/IEC 2019 – All rights reserved

Introduction
0.1  General
This document is a part of the SQuaRE series of International Standards. It provides a framework
for measuring the quality characteristics and sub-characteristics (defined in ISO/IEC 2501n). This
document serves as a guideline for developing and selecting quality measures for quality in use
(in conjunction with ISO/IEC 25022), system and software product quality (in conjunction with
ISO/IEC 25023), data quality (in conjunction with ISO/IEC 25024) and IT service quality (in conjunction
1)
with ISO/IEC TS 25025 ).
0.2  Quality measurement division
This document is a part of ISO/IEC 2502n Quality Measurement Division of the SQuaRE series that
consists of the following International Standards:
— ISO/IEC 25020 — Quality measurement framework: provides a framework for developing quality
measurement;
— ISO/IEC 25021 — Quality measure elements: provides a format for specifying QMEs (Quality Measure
Elements) and a few examples of QMEs that can be used to construct software quality measures;
— ISO/IEC 25022 — Measurement of quality in use: provides measures, including associated
measurement functions for the quality characteristics in the quality in use model;
— ISO/IEC 25023 — Measurement of system and software product quality: provides measures,
including associated measurement functions and QMEs for the quality characteristics in the
product quality model;
— ISO/IEC 25024 — Measurement of data quality: provides measures, including associated measurement
functions and QMEs for the quality characteristics in the data quality model;
— ISO/IEC TS 25025 — Measurement of IT service quality: provides measures for the IT service
quality model.
Figure 1 shows the relationship between this document and other standards in the ISO/IEC 2502n
division.
Figure 1 — Structure of the Quality Measurement Division
1) To be developed.
© ISO/IEC 2019 – All rights reserved v

0.3  Outline and organization of the SQuaRE series
The SQuaRE series consists of five main divisions and one extension division. The outlines of each
division within the SQuaRE series are as follows.
— ISO/IEC 2500n - Quality Management Division. The standards comprising this division define
all common models, terms and definitions referred further by all other standards in the SQuaRE
series. The division also provides requirements and guidance for the planning and management of
a project.
— ISO/IEC 2501n - Quality Model Division. The standards comprising this division present quality
models for system/software products, quality in use and data. The IT service quality model is
published as a Technical Specification.
— ISO/IEC 2502n - Quality Measurement Division. The standards comprising this division include
a system/software product quality measurement reference model, definitions of quality measures,
and practical guidance for their application. This division presents QMs on internal and external
property of a system and software product, QMs for quality in use, QMs for data quality and QMs for
IT service. Quality measure elements forming the foundations of the quality measures are defined
and presented.
— ISO/IEC 2503n - Quality Requirements Division. The standards comprising this division help
specify quality requirements. These quality requirements can be used in the process of quality
requirements elicitation for a system/software product to be developed, designing a process for
achieving necessary quality, or as inputs for an evaluation process.
— ISO/IEC 2504n - Quality Evaluation Division. The standards comprising this division provide
requirements, recommendations and guidelines for system/software product evaluation, whether
performed by independent evaluators, acquirers or developers. The support for documenting a
quality measure as an Evaluation Module is presented as well.
— ISO/IEC 25050-25099 - SQuaRE Extension Division. These standards are reserved for SQuaRE
extension International Standards, which currently include ISO/IEC 25051 and ISO/IEC TR 25060
to ISO/IEC 25069.
vi © ISO/IEC 2019 – All rights reserved

INTERNATIONAL STANDARD ISO/IEC 25020:2019(E)
Systems and software engineering — Systems and software
Quality Requirements and Evaluation (SQuaRE) — Quality
measurement framework
1 Scope
This document provides a framework for developing quality measurement.
The contents of this document are as follows:
— quality measurement reference model;
— relationships among different types of quality measures;
— guidelines for selecting quality measures;
— guidelines for constructing quality measures;
— guidelines for planning and performing measurements;
— guidelines for the application of measurement results.
It includes considerations for selecting quality measures and quality measure elements (Annex A),
assessing the reliability of measurement and the validity of quality measures (Annex B), elements for
documenting quality measures (Annex C), normalized measurement function for quality measures
(Annex D) and the measurement information model in ISO/IEC/IEEE 15939 (Annex E).
This document can be applied for designing, identifying, evaluating and executing the measurement
model of system and software product quality, quality in use, data quality and IT service quality.
This reference model can be used by developers, acquirers, quality assurance staff and independent
evaluators—essentially by people responsible for specifying and evaluating the quality of information
and communication technology (ICT) systems and services.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC/IEEE 15939, Systems and software engineering — Measurement process
ISO/IEC 25000, Systems and software engineering — Systems and software Quality Requirements and
Evaluation (SQuaRE) — Guide to SQuaRE
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC 25000 and
ISO/IEC/IEEE 15939 and the following apply.
ISO and IEC maintain terminological databases for use in standardisation at the following addresses:
— ISO Online browsing platform: available at https: //www .iso .org/obp
— IEC Electropedia: available at http: //www .electropedia .org/
© ISO/IEC 2019 – All rights reserved 1

3.1
attribute
inherent property or characteristic of an entity that can be distinguished quantitatively or qualitatively
by human or automated means
Note 1 to entry: ISO 9000 distinguishes two types of attributes: a permanent characteristic existing inherently in
something; and an assigned characteristic of a product, process or system (e.g. the price of a product, the owner of
a product). The assigned characteristic is not an inherent quality characteristic of that product, process or system.
[SOURCE: ISO/IEC 25000:2014, 4.1, modified — Note 1 to entry has been removed; Note 2 to entry has
become Note 1 to entry.]
3.2
base measure
measure (3.6) defined in terms of an attribute (3.1) and the method for quantifying it
Note 1 to entry: A base measure is functionally independent of other measures.
Note 2 to entry: Based on the definition of “base quantity” in the International Vocabulary of Metrology – Basic
and General Concepts and Associated Terms, 2012.
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.3]
3.3
derived measure
measure (3.6) defined as a function of two or more values of base measures (3.2)
Note 1 to entry: Adapted from the definition of “derived quantity” in the International Vocabulary of Metrology –
Basic and General Concepts and Associated Terms, 2012.
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.8]
3.4
indicator
measure (3.6) that provides an estimate or evaluation of specified attributes (3.1) derived from a model
with respect to defined information needs (3.5)
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.10]
3.5
information need
insight necessary to manage objectives, goals, risks and problems
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.12]
3.6
measure, noun
variable to which a value is assigned as the result of measurement (3.8)
Note 1 to entry: The plural form “measures” is used to refer collectively to base measures (3.2), derived measures
(3.3) and indicators (3.4).
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.15]
3.7
measure, verb
make a measurement (3.8)
[SOURCE: ISO/IEC 25000:2014, 4.19]
2 © ISO/IEC 2019 – All rights reserved

3.8
measurement
set of operations having the objective of determining a value of a measure (3.6)
Note 1 to entry: Measurement can include assigning a qualitative category such as the language of a source
program (ADA, C, JAVA, etc.).
[SOURCE: ISO/IEC 25000:2014, 4.20]
3.9
measurement function
algorithm or calculation performed to combine two or more base measures (3.2)
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.20]
3.10
measurement method
logical sequence of operations, described generically, used in quantifying an attribute (3.1) with respect
to a specified scale
Note 1 to entry: The type of measurement method depends on the nature of the operations used to quantify an
attribute. Two types can be distinguished:
— subjective: quantification involving human judgment;
— objective: quantification based on numerical rules.
Note 2 to entry: Based on the definition of “method of measurement” in the International Vocabulary of Metrology
– Basic and General Concepts and Associated Terms, 2012.
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.21]
3.11
property to quantify
property of a target entity that is related to a quality measure element (3.14) and which can be quantified
by a measurement method (3.10)
Note 1 to entry: A software artifact is an example of a target entity.
[SOURCE: ISO/IEC 25023:2016, 4.7]
3.12
quality in use
degree to which a product or system can be used by specific users to meet their needs to achieve specific
goals with effectiveness, efficiency, freedom from risk and satisfaction in specific contexts of use
Note 1 to entry: Before the product is released, quality in use can be specified and measured (3.7) in a test
environment designed and used exclusively by the intended users for their goals and contexts of use, e.g. User
Acceptance Testing Environment.
[SOURCE: ISO/IEC 25000:2014, 4.24]
3.13
quality measure
QM
derived measure (3.3) that is defined as a measurement function (3.9) of two or more values of quality
measure elements (3.14)
[SOURCE: ISO/IEC 25021:2012, 4.13, modified — The abbreviated term “QM” has been added.]
© ISO/IEC 2019 – All rights reserved 3

3.14
quality measure element
QME
measure (3.6) defined in terms of a property and the measurement method (3.10) for quantifying it,
including optionally the transformation by a mathematical function
Note 1 to entry: The system or software quality characteristic or subcharacteristic of the entity is derived
afterwards by calculating a software quality measure (3.13).
[SOURCE: ISO/IEC 25000:2014, 4.26, modified — The abbreviated term “QME” has been added.]
3.15
quality measure on external property
QM on external property
measure (3.6) of the degree to which a system or software product enables its behaviour to satisfy
stated and implied needs for the system including the software to be used under specified conditions
Note 1 to entry: Attributes (3.1) of the behaviour can be measured (3.7), verified and/or validated by executing
the system or software product during testing and operation.
EXAMPLE The failure density against test cases found during testing is a quality measure on external
property related to the number of faults present in the computer system. The two measures are not necessarily
identical since testing may not find all faults, and a fault may give rise to apparently different failures in different
circumstances.
[SOURCE: ISO/IEC 25000:2014, 4.11, modified — The term has been changed from “external measure of
system or software quality" to "quality measure on external property”; "QM on external property" has
been added as an alternative; in Note 1 to entry, the word "measured" has been added; in EXAMPLE,
"number of failures" has been changed to "failure density against test cases".]
3.16
quality measure on internal property
QM on internal property
measure (3.6) of the degree to which a set of static attributes (3.1) of a software product satisfies stated
and implied needs for the software product to be used under specified conditions
Note 1 to entry: Static attributes include those that relate to the software architecture, structure and its
components, data structure and its formats, structure and appearance of graphical display on screen and menus
for users or recipients of service.
Note 2 to entry: Static attributes can be verified by review, inspection, simulation and/or automated tools.
Note 3 to entry: Quality measures on internal property are typically associated with quality requirements on
static properties and attributes that can be specified in or derived from requirements.
EXAMPLE Complexity measures and the number, severity, and failure frequency of faults found in a walk
through are typical quality measures on internal property made on the product itself.
[SOURCE: ISO/IEC 25000:2014, 4.16, modified — The term has been changed from “internal measure of
software quality" to "quality measure on internal property"; "QM on internal property" has been added
as an alternative; in Note 1 to entry, more information on static attributes has been added; Note 3 to
entry has been added.]
3.17
system and software product quality
product quality
capability of a system and/or software to satisfy stated and implied needs when used under specified
conditions
Note 1 to entry: Product quality model refers to the system and software product quality model defined in
ISO/IEC 25010.
4 © ISO/IEC 2019 – All rights reserved

4 Abbreviated terms
ICT Information and Communication Technology
QM-RM Quality Measurement Reference Model
QM Quality Measure
QME Quality Measure Element
5 Conformance
Any measurement process for system and software product quality and quality in use, data quality and
IT service quality that conforms to this document shall fulfil the requirements of Clause 6.
6 Quality measurement
6.1 Quality measurement reference model
The Quality Measurement Reference Model (QM-RM) describes the relationship between a quality
model and the construction of QMs from QMEs, as shown in Figure 2. The relationship constitutes
the reference model for the measurement of system and software product quality, quality in use, data
quality and IT service quality. The measurement information model presented in Annex E describes the
relationship between attributes and measurement.
© ISO/IEC 2019 – All rights reserved 5

NOTE Target entity can be a system, a software product, data or IT service.
Figure 2 — Quality Measurement Reference Model (QM-RM)
The quality of a system, software product, data or IT service is the degree to which it satisfies the stated
and implied needs of various stakeholders, and thus provides value. User needs for quality include
requirements for system quality in specific contexts of use. These stated and implied needs are represented
in the SQuaRE series of standards by quality models that categorise quality into characteristics, which
are further subdivided into sub-characteristics. Quality properties are measured using a measurement
method. A measurement method is a logical sequence of operations used to quantify a property against a
specified scale. The result of applying a measurement method is called a QME.
QMs are constructed by applying a measurement function to a set of QMEs. A measurement function is
an algorithm used to combine QMEs. The result of applying a measurement function is called a QM. In
this way, QMs serve as quantifications of quality characteristics (and sub-characteristics). More than
one QM may be used for measuring a quality characteristic (and sub-characteristics).
In the special case where the QME serves as a QM as well, the measurement function applied would
be the identity function. QMEs may either be base or derived measures. Annex B provides assessment
information for the validation and verification of the measure. QMEs are constructed based on the
guidance provided in ISO/IEC/IEEE 15939. Refer to ISO/IEC 25030 for guidance on selecting quality
characteristics and sub-characteristics of interest in conjunction with the specification of quality
requirements and ISO/IEC 25040 for guidance on using software QMs for software product evaluation.
6 © ISO/IEC 2019 – All rights reserved

6.2 Different QMs and their relationships
There are four quality models in the SQuaRE series:
a) the quality in use model in ISO/IEC 25010 that can be applied to software products, systems and IT
services;
b) the product quality model in ISO/IEC 25010 that can be applied to systems and software products;
c) the data quality model in ISO/IEC 25012 that can be applied to data within a computer system and
used by humans and systems;
d) the IT service quality model defined in ISO/IEC/TS 25011 that can be applied to IT services that
support the needs of an individual user or a business.
These models provide a set of quality characteristics and sub-characteristics, as well as their definitions.
The relationships among various QMs for different quality models from the SQuaRE series are shown in
Figure 3.
Figure 3 — Relationships among different QMs
© ISO/IEC 2019 – All rights reserved 7

QMs for quality in use are defined or selected to specify stakeholder requirements in a specific context
of use, when quality requirements are derived from user needs by analysing the concept of operation.
QMs for quality in use are to measure the extent to which a product meets the needs of specific
users with respect to their specific personal or business goals by means of quantifying outcomes of
interaction between a user and a system or of effects to stakeholders, including indirect users as well as
direct users. These measures can only be prepared in a realistic and operational system environment.
QMs on external and internal property of product are for the user (including executing testing
engineer) and the developer, respectively. There is no distinction between the two, even at the level of
characteristics and sub-characteristics. However, when users apply the QMs depending on the purpose
and stage of the software product life cycle, the QME and QM selected should be related and suitable
to either the user or the developer. QMs on external property are used to measure the quality of the
system and software product based on the behaviour of the system. QMs on external property are used
in the testing and operational stages of the product life cycle. QMs on internal property allow users to
measure the quality of intermediate deliverables or work products. Additionally, these measures may
be used with an analysis model to predict the quality of the final system and software product. This
allows users to detect system and software product quality issues and take corrective and preventive
actions during the early stages of the development life cycle.
Data quality measures can be transformed from quality in use, system and software product quality
requirements and measures. Then, these measures representing the targeted data quality requirements
are used to evaluate the data quality of system and software product, to verify, validate and improve
data and product phase-by-phase during design, implementation, testing or in use. QMs for data quality
are to measure data in the system and software product from two viewpoints, namely, “Inherent” and
“System-dependent”, to detect potential quality problems related to the data and database. These QMs
can be applied during the development, testing and operation stages. Data quality has a big influence on
quality in use, in particular for effectiveness, usefulness and risks management.
QMs for IT service quality quantify the degree to which the properties of an IT service can satisfy the
stated and implied needs of the IT service when used under specified conditions. IT service has its own
provision system. QMs for IT service quality typically measure interactions between the system and
service recipients.
Process quality (the quality of any life cycle process defined in ISO/IEC/IEEE 12207) contributes to the
improvement in system and software product quality, data quality and IT service quality. Evaluating
whether software products can satisfy users’ quality needs is one of the processes in the software
development life cycle. Software product, IT service and data in different contexts influence quality
in use. Therefore, assessing and improving a process is a means for improving system and software
product quality, and evaluating and improving system and software product quality is one means
of improving quality in use. Similarly, evaluating quality in use can provide feedback for improving
a software product, and evaluating a software product can provide feedback to improve a process.
System and software product quality can be evaluated using QMs on internal and external property.
System and software product quality influences IT service quality and data quality. IT service quality
can be evaluated using IT service quality measures. IT service quality depends on system and software
product quality and data quality. In the specific context of use (when system, software product and IT
service are in real or simulated use), IT service quality in use depends on system and software product
quality in use.
Figure 4 illustrates the quality life cycle as a set of coordinated QMs which can be used to specify
quality requirements in detail and evaluate quality by means of measuring the degree of achievement
of the required quality for verification and validation, through the life cycle, covering development,
operation and maintenance of the system and software product, data and IT service. From user and/
or stakeholder’s point of view, the quality life cycle consists of 3 layers: user layer, runtime layer and
implementation layer. Quality requirements and target entity validate and/or verify each other in
different layers. Users and/or stakeholder’s quality needs for any of various target entities including
system, software product, data and IT service can be elicited and transformed into quality in use
requirements, and then into quality requirements using external property (i.e. behaviours) and into
quality requirements using internal property (i.e. static attributes). Correspondingly, target entity can
8 © ISO/IEC 2019 – All rights reserved

be implemented from the requirements. Conducting and iterating the quality life cycle leads to evolving
and improving the quality.
QMs include QMs for quality in use, QMs on external property and QMs on internal property.
Stakeholders' impact and influence in a context of use can be measured by QMs for quality in use. QMs
on external property are measures of behavioural attributes. QMs on internal property are used to
measure technical/structural attributes of software and/or system. Quality property of target entity
includes external quality property and internal quality property. Internal quality property influences
external quality property when the software and/or system are in the status of runtime, while outcome
or consequence of software and/or system in a certain context of use is influenced by external quality
property.
a
Measures are constructed using measurement functions applied.
Figure 4 — QMs in the quality life cycle
Quality in use requirements are based on the expected outcome/consequence of system and/or software
product (e.g. time spent for user to complete specific intended tasks), considering effectiveness,
efficiency, satisfaction, freedom from risk and context coverage. Quality requirements using external
property (e.g. throughput, response time, etc.) can be derived from quality in use requirements. Quality
requirements using external property should be stated quantitatively in the quality requirements
specification by using QMs on external property that are used when a target entity is evaluated. Quality
requirements using internal property (e.g. complexity of program structure, etc.) can be derived from
quality requirements using external property. Quality requirements using internal property reflect the
technical/structural property. They may be used to specify properties of deliverable, non-executable
software products such as documentation and manuals. They can also be used as target entity for
verification, and to define the criteria for verification at various stages of development.
© ISO/IEC 2019 – All rights reserved 9

Throughout the quality life cycle, measuring the degree of achievement to the required quality for
verification and validation can be done in different layers. In use layer, the context of use plays an
important role in validation between quality in use requirements and quality influence. In runtime
layer, the quality requirements using external property are validated and verified based on quality
of external property, and vice versa. In implementation layer, the quality requirements using internal
property are verified based on quality of internal property, and vice versa.
NOTE QMs for quality in use indicate quality explained by the effect to stakeholders; QMs on external
property indicate quality explained by the behaviour of the target entity, during prototyping test, product
test, and when actually used; QMs on internal property indicate quality explained as a result of reviewing
specifications and/or source code.
6.3 Selecting QMs
QMs are specified to satisfy the needs of developers, acquirers, managers, direct and indirect users
and other stakeholders for information. Candidate QMs that potentially satisfy quality requirements
should be identified from standards providing specific QMs in Quality measurement division of SQuaRE
series, such as ISO/IEC 25022, ISO/IEC 25023 or ISO/IEC 25024. And then, candidate QMs can be further
refined as applicable. At a minimum, one of the criteria for selecting QMs, including their employing
combinations of measures, shall be stated that provides the reason why the selected QMs are chosen.
Many different combinations of base measures and derived measures may be selected to construct
additional QMs, that act as indicators or address specific quality requirements. The following factors
are to be considered when selecting among the alternatives: relevance to the prioritised information
needs; feasibility of collecting data in the organizational unit; availability of human resources to collect
and manage data; and ease of data collection. When QMs are used to define quality requirements, the
criticality of such quality requirements or risk of their insufficiency can be considered as one of the
criteria for selecting QMs. When QMs are used to evaluate quality, applicable rigorousness and timing
can be considered as criteria.
The measures selected will influence human behavior. In many cases the behavior may lead to
dysfunctional outcomes, from individuals trying to “game the system”. Users should anticipate such
challenges and take actions to mitigate risks, including training, mentoring and additional strategic
governance.
Criteria for selecting QMs to fulfil those information needs shall be documented.
Annex A provides examples of criteria for selecting QMs.
NOTE ISO/IEC 25030 and ISO/IEC 25040 provide guidance on quality requirements specifications and
product quality evaluations, respectively.
When using a modified or a new measure that is not identified in all specific quality measurement
standards, such as ISO/IEC 25022, ISO/IEC 25023 or ISO/IEC 25024, the user shall specify how the
measure relates to its corresponding quality model and how it is constructed from QMEs.
Annex C provides an example of how to document a QM.
NOTE ISO/IEC 25010 provides guidance on defining and using a system and software product quality model.
6.4 Constructing QMs
6.4.1 Identify QMs needed to be constructed
The quality of a system is the degree to which the system satisfies the stated and implied needs of
its various stakeholders and thus provides value. These stated and implied needs are represented
in the SQuaRE series of International Standards by quality models that categorize quality into
characteristics, which, in a few cases, are further subdivided into sub-characteristics. The full set of
quality characteristics across these models will not be relevant to every stakeholder. Nonetheless, each
category of stakeholder shall be represented in reviewing and considering the relevance of the quality
10 © ISO/IEC 2019 – All rights reserved

characteristics in each model before finalising the set of quality characteristics that will be used, for
example, to establish software product and system performance requirements or evaluation criteria.
Applicable QMs are not limited to those listed in ISO/IEC 25022, ISO/IEC 25023 and ISO/IEC 25024. If
needed, a new QM may be constructed and included in the QM set of a specific characteristic or sub-
characteristic to satisfy a user’s additional quality requirements. The new QM should be described
according to 6.4.2, and appropriate QMEs should be selected and combined using the measurement
function (See Annex D).
Definitions of any new QMs, including QMs in ISO/IEC 2502n that are modified, shall be documented.
The definition of the QM should contain information included in the example format provided in
Annex C.
NOTE 1 A suggested set of quality in use measures along with their definitions is given in ISO/IEC 25022.
NOTE 2 A suggested set of system and software product quality measures along with their definitions is given
in ISO/IEC 25023.
NOTE 3 A suggested set of data quality measures along with their definitions is given in ISO/IEC 25024.
6.4.2 Description of the QM
The following information is important to document the definition of each QM, when the user performs
the measurement of system, software product, data and IT service. The user should document additional
detailed information, when describing the QM for more operational. Such more detailed information of
the QM is provided in Annex C.
a) ID: Identification code of the QM. Each ID consists of the following three parts:
— abbreviated alphabetic representing quality characteristics and possibly sub-characteristics.
(for example, “PTb” denotes “Time behaviour” which measures for “Performance efficiency”,
“Acc” denotes measures for accuracy);
— serial number of sequential order within the quality sub-characteristic.
— usage tag:
— G: generally applicable, could be used in a wide range of situations;
— S: specialised for specific needs.
NOTE The ID can include additional parts (e.g. PTb-1-G-IT-1 identifies a modification of PTb-1-G).
b) Name: Name of the QM.
c) Description: The information including the information needed (purpose of the measure) and
quality characteristic/sub-characteristic provided by the QM and (when useful) the purpose of the
measure.
d) Measurement function: Formula showing how the QMEs are combined to produce the QM.
e) Measurement method: The type of method that can be used to obtain the measure.
6.4.3 Definitions of the QMEs
QMEs are used throughout the ICT system life cycle to construct QMs of system and software product
quality, quality in use, data quality and IT service quality by applying measurement methods to specified
attributes and, when necessary, the measures combining QMEs via a measurement function shall be
documented. The QMEs are used to measure the attributes of the system and software product itself,
© ISO/IEC 2019 – All rights reserved 11

effects of using the system and software product in a specific context and the resources consumed or
activities performed during system and software product development, testing and maintenance.
NOTE 1 The ICT system is a system that uses information and communication technologies.
NOTE 2 A suggested set of QMEs along with their definitions is given in ISO/IEC 25021.
6.5 Plan and perform measurement
The user of the ISO/IEC 2502n division of standards shall plan and perform measurement to determine
the values of QMEs and QMs following the reference model in Figure 2.
The quality measurement shall be scheduled considering resources such as personnel, measurement
automation, software and hardware environments. The measurement plan should contain no duplicated
tasks to take the same measures to address different information needs.
NOTE 1 Some of QMEs and QMs are often planned to be taken repeatedly, iteratively or periodically to monitor
trends or improvements of quality, during specific stage or along with the product life cycle.
The criteria for selecting QMs and QMEs should be considered in the measurement plan to decrease the
risk of errors and reduce the planned effort, considering at least the following:
a) measurement budget;
b) the priority and strictness of QMs and QMEs that reflect critical quality requirements;
c) schedule and resources involved;
d) application of measurement result;
e) the relevance and importance of QMs based on the quality requirements and context of use.
NOTE 2 The above concerns in an individual project are often resolved by coordinating and sharing with
an organizational measurement strategy providing trainings, tools, environments, personnel and so on for
measurement and analysis.
The main activities associated with performing measurement are as follows:
a) identify the quality model according to the different information needs relating to quality
characteristics of systems or software product, IT service, data or quality in use;
b) identify candidate and select QMs with QMEs to construct these QMs, for the identified quality model;
c) communicate measurer or data provider to take adequately QMs and QMEs, plan and collaborate
with relevant stakeholders to capture QMs and QMEs;
d) generate the values of QMEs by using the measurement method;
e) calculate the value of QMs by using the measurement function;
f) verify and store the values of QMs and their QMEs with context information of measurement;
NOTE 3 Verification of the values of QMs and QMEs can be done using various techniques, for example,
range and type of values, singular values, classification errors, or large fluctuation.
g) measure quality characteristics and/or sub-characteristics by using QMs;
h) record results and inform the users who need information relating to quality for decision makings
during the project or operation.
Users of the ISO/IEC 2502n division of standards for quality measurement are encouraged to comply
with the quality evaluation procedures contained in the ISO/IEC 2
...