Systems and software engineering -- Systems and software Quality Requirements and Evaluation (SQuaRE) -- Quality measurement framework

This document provides a framework for developing quality measurement. The contents of this document are as follows: — quality measurement reference model; — relationships among different types of quality measures; — guidelines for selecting quality measures; — guidelines for constructing quality measures; — guidelines for planning and performing measurements; — guidelines for the application of measurement results. It includes considerations for selecting quality measures and quality measure elements (Annex A), assessing the reliability of measurement and the validity of quality measures (Annex B), elements for documenting quality measures (Annex C), normalized measurement function for quality measures (Annex D) and the measurement information model in ISO/IEC/IEEE 15939 (Annex E). This document can be applied for designing, identifying, evaluating and executing the measurement model of system and software product quality, quality in use, data quality and IT service quality. This reference model can be used by developers, acquirers, quality assurance staff and independent evaluators—essentially by people responsible for specifying and evaluating the quality of information and communication technology (ICT) systems and services.

Ingénierie des systèmes et du logiciel -- Exigences de qualité du produit logiciel et évaluation (SQuaRE) -- Modèle de référence de mesure et guide

General Information

Status
Published
Publication Date
11-Jul-2019
Current Stage
6060 - International Standard published
Start Date
10-May-2019
Completion Date
12-Jul-2019
Ref Project

RELATIONS

Buy Standard

Standard
ISO/IEC 25020:2019 - Systems and software engineering -- Systems and software Quality Requirements and Evaluation (SQuaRE) -- Quality measurement framework
English language
27 pages
sale 15% off
Preview
sale 15% off
Preview

Standards Content (sample)

INTERNATIONAL ISO/IEC
STANDARD 25020
Second edition
2019-07
Systems and software engineering —
Systems and software Quality
Requirements and Evaluation
(SQuaRE) — Quality measurement
framework
Ingénierie des systèmes et du logiciel — Exigences de qualité du
produit logiciel et évaluation (SQuaRE) — Modèle de référence de
mesure et guide
Reference number
ISO/IEC 25020:2019(E)
ISO/IEC 2019
---------------------- Page: 1 ----------------------
ISO/IEC 25020:2019(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2019

All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting

on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address

below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2019 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC 25020:2019(E)
Contents Page

Foreword ........................................................................................................................................................................................................................................iv

Introduction ..................................................................................................................................................................................................................................v

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ...................................................................................................................................................................................... 1

3 Terms and definitions ..................................................................................................................................................................................... 1

4 Abbreviated terms .............................................................................................................................................................................................. 5

5 Conformance ............................................................................................................................................................................................................. 5

6 Quality measurement ...................................................................................................................................................................................... 5

6.1 Quality measurement reference model ............................................................................................................................. 5

6.2 Different QMs and their relationships ................................................................................................................................ 7

6.3 Selecting QMs ........................................................................................................................................................................................10

6.4 Constructing QMs ..............................................................................................................................................................................10

6.4.1 Identify QMs needed to be constructed ....................................................................................................10

6.4.2 Description of the QM ...............................................................................................................................................11

6.4.3 Definitions of the QMEs ..........................................................................................................................................11

6.5 Plan and perform measurement ...........................................................................................................................................12

6.6 Application of the measurement results .......................................................................................................................13

Annex A (informative) Considerations for selecting QMs and QMEs ...............................................................................14

Annex B (informative) Assessing the reliability of measurement and the validity of QMs ......................16

Annex C (informative) Elements for documenting QMs ................................................................................................................18

Annex D (informative) Normalized measurement function for QMs ...............................................................................21

Annex E (informative) Measurement information model in ISO/IEC/IEEE 15939 ...........................................24

Bibliography .............................................................................................................................................................................................................................27

© ISO/IEC 2019 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC 25020:2019(E)
Foreword

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical

Commission) form the specialized system for worldwide standardization. National bodies that are

members of ISO or IEC participate in the development of International Standards through technical

committees established by the respective organization to deal with particular fields of technical

activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international

organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the

work. In the field of information technology, ISO and IEC have established a joint technical committee,

ISO/IEC JTC 1.

The procedures used to develop this document and those intended for its further maintenance are

described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for

the different types of document should be noted. This document was drafted in accordance with the

editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/directives).

Attention is drawn to the possibility that some of the elements of this document may be the subject

of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent

rights. Details of any patent rights identified during the development of the document will be in the

Introduction and/or on the ISO list of patent declarations received (see www .iso .org/patents).

Any trade name used in this document is information given for the convenience of users and does not

constitute an endorsement.

For an explanation on the meaning of ISO specific terms and expressions related to conformity

assessment, as well as information about ISO's adherence to the World Trade Organization (WTO)

principles in the Technical Barriers to Trade (TBT) see www .iso .org/iso/foreword .html.

This document was prepared by Technical Committee ISO/IEC JTC 1, Information technology,

Subcommittee SC 7, Software and systems engineering.

This second edition cancels and replaces the first edition (ISO/IEC 25020:2007), which has been

technically revised.
The main changes compared to the previous edition are as follows:
— relationships among different types of quality measures have been added;

— application of measurement results and description of quality measure have been added;

— elements for documenting quality measures in Annex C have been supplemented and categorized;

— Annex D has been added showing a normalized measurement function for QMs;

— Annex E has been added showing the measurement information model in ISO/IEC/IEEE 15939;

— harmonized with ISO/IEC 25000:2014, ISO/IEC 25022:2016, ISO/IEC 25023:2016, ISO/IEC 25024:2015

and ISO/IEC/IEEE 15939:2017.

Any feedback or questions on this document should be directed to the user’s national standards body. A

complete listing of these bodies can be found at https: //www .iso .org/members .html.

iv © ISO/IEC 2019 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC 25020:2019(E)
Introduction
0.1 General

This document is a part of the SQuaRE series of International Standards. It provides a framework

for measuring the quality characteristics and sub-characteristics (defined in ISO/IEC 2501n). This

document serves as a guideline for developing and selecting quality measures for quality in use

(in conjunction with ISO/IEC 25022), system and software product quality (in conjunction with

ISO/IEC 25023), data quality (in conjunction with ISO/IEC 25024) and IT service quality (in conjunction

with ISO/IEC TS 25025 ).
0.2 Quality measurement division

This document is a part of ISO/IEC 2502n Quality Measurement Division of the SQuaRE series that

consists of the following International Standards:

— ISO/IEC 25020 — Quality measurement framework: provides a framework for developing quality

measurement;

— ISO/IEC 25021 — Quality measure elements: provides a format for specifying QMEs (Quality Measure

Elements) and a few examples of QMEs that can be used to construct software quality measures;

— ISO/IEC 25022 — Measurement of quality in use: provides measures, including associated

measurement functions for the quality characteristics in the quality in use model;

— ISO/IEC 25023 — Measurement of system and software product quality: provides measures,

including associated measurement functions and QMEs for the quality characteristics in the

product quality model;

— ISO/IEC 25024 — Measurement of data quality: provides measures, including associated measurement

functions and QMEs for the quality characteristics in the data quality model;

— ISO/IEC TS 25025 — Measurement of IT service quality: provides measures for the IT service

quality model.

Figure 1 shows the relationship between this document and other standards in the ISO/IEC 2502n

division.
Figure 1 — Structure of the Quality Measurement Division
1) To be developed.
© ISO/IEC 2019 – All rights reserved v
---------------------- Page: 5 ----------------------
ISO/IEC 25020:2019(E)
0.3 Outline and organization of the SQuaRE series

The SQuaRE series consists of five main divisions and one extension division. The outlines of each

division within the SQuaRE series are as follows.

— ISO/IEC 2500n - Quality Management Division. The standards comprising this division define

all common models, terms and definitions referred further by all other standards in the SQuaRE

series. The division also provides requirements and guidance for the planning and management of

a project.

— ISO/IEC 2501n - Quality Model Division. The standards comprising this division present quality

models for system/software products, quality in use and data. The IT service quality model is

published as a Technical Specification.

— ISO/IEC 2502n - Quality Measurement Division. The standards comprising this division include

a system/software product quality measurement reference model, definitions of quality measures,

and practical guidance for their application. This division presents QMs on internal and external

property of a system and software product, QMs for quality in use, QMs for data quality and QMs for

IT service. Quality measure elements forming the foundations of the quality measures are defined

and presented.

— ISO/IEC 2503n - Quality Requirements Division. The standards comprising this division help

specify quality requirements. These quality requirements can be used in the process of quality

requirements elicitation for a system/software product to be developed, designing a process for

achieving necessary quality, or as inputs for an evaluation process.

— ISO/IEC 2504n - Quality Evaluation Division. The standards comprising this division provide

requirements, recommendations and guidelines for system/software product evaluation, whether

performed by independent evaluators, acquirers or developers. The support for documenting a

quality measure as an Evaluation Module is presented as well.

— ISO/IEC 25050-25099 - SQuaRE Extension Division. These standards are reserved for SQuaRE

extension International Standards, which currently include ISO/IEC 25051 and ISO/IEC TR 25060

to ISO/IEC 25069.
vi © ISO/IEC 2019 – All rights reserved
---------------------- Page: 6 ----------------------
INTERNATIONAL STANDARD ISO/IEC 25020:2019(E)
Systems and software engineering — Systems and software
Quality Requirements and Evaluation (SQuaRE) — Quality
measurement framework
1 Scope
This document provides a framework for developing quality measurement.
The contents of this document are as follows:
— quality measurement reference model;
— relationships among different types of quality measures;
— guidelines for selecting quality measures;
— guidelines for constructing quality measures;
— guidelines for planning and performing measurements;
— guidelines for the application of measurement results.

It includes considerations for selecting quality measures and quality measure elements (Annex A),

assessing the reliability of measurement and the validity of quality measures (Annex B), elements for

documenting quality measures (Annex C), normalized measurement function for quality measures

(Annex D) and the measurement information model in ISO/IEC/IEEE 15939 (Annex E).

This document can be applied for designing, identifying, evaluating and executing the measurement

model of system and software product quality, quality in use, data quality and IT service quality.

This reference model can be used by developers, acquirers, quality assurance staff and independent

evaluators—essentially by people responsible for specifying and evaluating the quality of information

and communication technology (ICT) systems and services.
2 Normative references

The following documents are referred to in the text in such a way that some or all of their content

constitutes requirements of this document. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any amendments) applies.

ISO/IEC/IEEE 15939, Systems and software engineering — Measurement process

ISO/IEC 25000, Systems and software engineering — Systems and software Quality Requirements and

Evaluation (SQuaRE) — Guide to SQuaRE
3 Terms and definitions

For the purposes of this document, the terms and definitions given in ISO/IEC 25000 and

ISO/IEC/IEEE 15939 and the following apply.

ISO and IEC maintain terminological databases for use in standardisation at the following addresses:

— ISO Online browsing platform: available at https: //www .iso .org/obp
— IEC Electropedia: available at http: //www .electropedia .org/
© ISO/IEC 2019 – All rights reserved 1
---------------------- Page: 7 ----------------------
ISO/IEC 25020:2019(E)
3.1
attribute

inherent property or characteristic of an entity that can be distinguished quantitatively or qualitatively

by human or automated means

Note 1 to entry: ISO 9000 distinguishes two types of attributes: a permanent characteristic existing inherently in

something; and an assigned characteristic of a product, process or system (e.g. the price of a product, the owner of

a product). The assigned characteristic is not an inherent quality characteristic of that product, process or system.

[SOURCE: ISO/IEC 25000:2014, 4.1, modified — Note 1 to entry has been removed; Note 2 to entry has

become Note 1 to entry.]
3.2
base measure

measure (3.6) defined in terms of an attribute (3.1) and the method for quantifying it

Note 1 to entry: A base measure is functionally independent of other measures.

Note 2 to entry: Based on the definition of “base quantity” in the International Vocabulary of Metrology – Basic

and General Concepts and Associated Terms, 2012.
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.3]
3.3
derived measure
measure (3.6) defined as a function of two or more values of base measures (3.2)

Note 1 to entry: Adapted from the definition of “derived quantity” in the International Vocabulary of Metrology –

Basic and General Concepts and Associated Terms, 2012.
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.8]
3.4
indicator

measure (3.6) that provides an estimate or evaluation of specified attributes (3.1) derived from a model

with respect to defined information needs (3.5)
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.10]
3.5
information need
insight necessary to manage objectives, goals, risks and problems
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.12]
3.6
measure, noun
variable to which a value is assigned as the result of measurement (3.8)

Note 1 to entry: The plural form “measures” is used to refer collectively to base measures (3.2), derived measures

(3.3) and indicators (3.4).
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.15]
3.7
measure, verb
make a measurement (3.8)
[SOURCE: ISO/IEC 25000:2014, 4.19]
2 © ISO/IEC 2019 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/IEC 25020:2019(E)
3.8
measurement
set of operations having the objective of determining a value of a measure (3.6)

Note 1 to entry: Measurement can include assigning a qualitative category such as the language of a source

program (ADA, C, JAVA, etc.).
[SOURCE: ISO/IEC 25000:2014, 4.20]
3.9
measurement function
algorithm or calculation performed to combine two or more base measures (3.2)
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.20]
3.10
measurement method

logical sequence of operations, described generically, used in quantifying an attribute (3.1) with respect

to a specified scale

Note 1 to entry: The type of measurement method depends on the nature of the operations used to quantify an

attribute. Two types can be distinguished:
— subjective: quantification involving human judgment;
— objective: quantification based on numerical rules.

Note 2 to entry: Based on the definition of “method of measurement” in the International Vocabulary of Metrology

– Basic and General Concepts and Associated Terms, 2012.
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.21]
3.11
property to quantify

property of a target entity that is related to a quality measure element (3.14) and which can be quantified

by a measurement method (3.10)
Note 1 to entry: A software artifact is an example of a target entity.
[SOURCE: ISO/IEC 25023:2016, 4.7]
3.12
quality in use

degree to which a product or system can be used by specific users to meet their needs to achieve specific

goals with effectiveness, efficiency, freedom from risk and satisfaction in specific contexts of use

Note 1 to entry: Before the product is released, quality in use can be specified and measured (3.7) in a test

environment designed and used exclusively by the intended users for their goals and contexts of use, e.g. User

Acceptance Testing Environment.
[SOURCE: ISO/IEC 25000:2014, 4.24]
3.13
quality measure

derived measure (3.3) that is defined as a measurement function (3.9) of two or more values of quality

measure elements (3.14)

[SOURCE: ISO/IEC 25021:2012, 4.13, modified — The abbreviated term “QM” has been added.]

© ISO/IEC 2019 – All rights reserved 3
---------------------- Page: 9 ----------------------
ISO/IEC 25020:2019(E)
3.14
quality measure element
QME

measure (3.6) defined in terms of a property and the measurement method (3.10) for quantifying it,

including optionally the transformation by a mathematical function

Note 1 to entry: The system or software quality characteristic or subcharacteristic of the entity is derived

afterwards by calculating a software quality measure (3.13).

[SOURCE: ISO/IEC 25000:2014, 4.26, modified — The abbreviated term “QME” has been added.]

3.15
quality measure on external property
QM on external property

measure (3.6) of the degree to which a system or software product enables its behaviour to satisfy

stated and implied needs for the system including the software to be used under specified conditions

Note 1 to entry: Attributes (3.1) of the behaviour can be measured (3.7), verified and/or validated by executing

the system or software product during testing and operation.

EXAMPLE The failure density against test cases found during testing is a quality measure on external

property related to the number of faults present in the computer system. The two measures are not necessarily

identical since testing may not find all faults, and a fault may give rise to apparently different failures in different

circumstances.

[SOURCE: ISO/IEC 25000:2014, 4.11, modified — The term has been changed from “external measure of

system or software quality" to "quality measure on external property”; "QM on external property" has

been added as an alternative; in Note 1 to entry, the word "measured" has been added; in EXAMPLE,

"number of failures" has been changed to "failure density against test cases".]
3.16
quality measure on internal property
QM on internal property

measure (3.6) of the degree to which a set of static attributes (3.1) of a software product satisfies stated

and implied needs for the software product to be used under specified conditions

Note 1 to entry: Static attributes include those that relate to the software architecture, structure and its

components, data structure and its formats, structure and appearance of graphical display on screen and menus

for users or recipients of service.

Note 2 to entry: Static attributes can be verified by review, inspection, simulation and/or automated tools.

Note 3 to entry: Quality measures on internal property are typically associated with quality requirements on

static properties and attributes that can be specified in or derived from requirements.

EXAMPLE Complexity measures and the number, severity, and failure frequency of faults found in a walk

through are typical quality measures on internal property made on the product itself.

[SOURCE: ISO/IEC 25000:2014, 4.16, modified — The term has been changed from “internal measure of

software quality" to "quality measure on internal property"; "QM on internal property" has been added

as an alternative; in Note 1 to entry, more information on static attributes has been added; Note 3 to

entry has been added.]
3.17
system and software product quality
product quality

capability of a system and/or software to satisfy stated and implied needs when used under specified

conditions

Note 1 to entry: Product quality model refers to the system and software product quality model defined in

ISO/IEC 25010.
4 © ISO/IEC 2019 – All rights reserved
---------------------- Page: 10 ----------------------
ISO/IEC 25020:2019(E)
4 Abbreviated terms
ICT Information and Communication Technology
QM-RM Quality Measurement Reference Model
QM Quality Measure
QME Quality Measure Element
5 Conformance

Any measurement process for system and software product quality and quality in use, data quality and

IT service quality that conforms to this document shall fulfil the requirements of Clause 6.

6 Quality measurement
6.1 Quality measurement reference model

The Quality Measurement Reference Model (QM-RM) describes the relationship between a quality

model and the construction of QMs from QMEs, as shown in Figure 2. The relationship constitutes

the reference model for the measurement of system and software product quality, quality in use, data

quality and IT service quality. The measurement information model presented in Annex E describes the

relationship between attributes and measurement.
© ISO/IEC 2019 – All rights reserved 5
---------------------- Page: 11 ----------------------
ISO/IEC 25020:2019(E)
NOTE Target entity can be a system, a software product, data or IT service.
Figure 2 — Quality Measurement Reference Model (QM-RM)

The quality of a system, software product, data or IT service is the degree to which it satisfies the stated

and implied needs of various stakeholders, and thus provides value. User needs for quality include

requirements for system quality in specific contexts of use. These stated and implied needs are represented

in the SQuaRE series of standards by quality models that categorise quality into characteristics, which

are further subdivided into sub-characteristics. Quality properties are measured using a measurement

method. A measurement method is a logical sequence of operations used to quantify a property against a

specified scale. The result of applying a measurement method is called a QME.

QMs are constructed by applying a measurement function to a set of QMEs. A measurement function is

an algorithm used to combine QMEs. The result of applying a measurement function is called a QM. In

this way, QMs serve as quantifications of quality characteristics (and sub-characteristics). More than

one QM may be used for measuring a quality characteristic (and sub-characteristics).

In the special case where the QME serves as a QM as well, the measurement function applied would

be the identity function. QMEs may either be base or derived measures. Annex B provides assessment

information for the validation and verification of the measure. QMEs are constructed based on the

guidance provided in ISO/IEC/IEEE 15939. Refer to ISO/IEC 25030 for guidance on selecting quality

characteristics and sub-characteristics of interest in conjunction with the specification of quality

requirements and ISO/IEC 25040 for guidance on using software QMs for software product evaluation.

6 © ISO/IEC 2019 – All rights reserved
---------------------- Page: 12 ----------------------
ISO/IEC 25020:2019(E)
6.2 Different QMs and their relationships
There are four quality models in the SQuaRE series:

a) the quality in use model in ISO/IEC 25010 that can be applied to software products, systems and IT

services;

b) the product quality model in ISO/IEC 25010 that can be applied to systems and software products;

c) the data quality model in ISO/IEC 25012 that can be applied to data within a computer system and

used by humans and systems;

d) the IT service quality model defined in ISO/IEC/TS 25011 that can be applied to IT services that

support the needs of an individual user or a business.

These models provide a set of quality characteristics and sub-characteristics, as well as their definitions.

The relationships among various QMs for different quality models from the SQuaRE series are shown in

Figure 3.
Figure 3 — Relationships among different QMs
© ISO/IEC 2019 – All rights reserved 7
---------------------- Page: 13 ----------------------
ISO/IEC 25020:2019(E)

QMs for quality in use are defined or selected to specify stakeholder requirements in a specific context

of use, when quality requirements are derived from user needs by analysing the concept of operation.

QMs for quality in use are to measure the extent to which a product meets the needs of specific

users with respect to their specific personal or business goals by means of quantifying outcomes of

interaction between a user and a system or of effects to stakeholders, including indirect users as well as

direct users. These measures can only be prepared in a realistic and operational system environment.

QMs on external and internal property of product are for the user (including executing testing

engineer) and the developer, respectively. There is no distinction between the two, even at the level of

characteristics and sub-characteristics. However, when users apply the QMs depending on the purpose

and stage of the software product life cycle, the QME and QM selected should be related and suitable

to either the user or the developer. QMs on external property are used to measure the quality of the

system and software product based on the behaviour of the system. QMs on external property are used

in the testing and operational stages of the product life cycle. QMs on internal property allow users to

measure the quality of intermediate deliverables or work products. Additionally, these measures may

be used with an analysis model to predict the quality of the final system and software product. This

allows users to detect system and software product quality issues and take corrective and preventive

actions during the early stages of the development life cycle.

Data quality measures can be transformed from quality in use, system and software product quality

requirements and measures. Then, these measures representing the targeted data quality requirements

are used to evaluate the data quality of system and software product, to verify, validate and improve

data and product phase-by-phase during design, implementation, testing or in use. QMs for data quality

are to measure data in the system and software product from two viewpoints, namely, “Inherent” and

“System-dependent”, to detect potential quality problems related to the data and database. These QMs

can be applied during the development, testing and operation stages. Data quality has a big influence on

quality in use, in particular for effectiveness, usefulness and risks management.

QMs for IT service quality quantify the degree to which the properties of an IT service can satisfy the

stated and implied needs of the IT service when used under specified conditions. IT service has its own

provision system. QMs for IT service quality typically measure interactions between the s

...

Questions, Comments and Discussion

Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.