ISO/IEC 25021:2012

Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - Quality measure elements

ISO/IEC 25021:2012 provides guides to specify Quality Measure Elements (QME) and initial set of QME as examples. QME is a measure defined in terms of a property and the measurement method for quantifying it, including optionally the transformation by a mathematical function. ISO/IEC 25021:2012 is intended to be used throughout the system and software product life cycle used with other documents of the ISO/IEC 250nn SQuaRE series, especially ISO/IEC 25022, ISO/IEC 25023 and ISO/IEC 25024.

Ingénierie des systèmes et du logiciel — Exigences de qualité et évaluation des systèmes et du logiciel (SQuaRE) — Éléments de mesure de la qualité

General Information

• Status: Published
• Publication Date: 22-Oct-2012
• ICS: 35.080 - Software
• Technical Committee: ISO/IEC JTC 1/SC 7 - Software and systems engineering
• Drafting Committee: ISO/IEC JTC 1/SC 7/WG 6 - Software Product and System Quality
• Current Stage: 9093 - International Standard confirmed
• Start Date: 09-Aug-2024
• Completion Date: 30-Oct-2025

Relations

• Revises: ISO/IEC TR 25021:2007 - Software engineering - Software product Quality Requirements and Evaluation (SQuaRE) - Quality measure elements
• Effective Date: 07-Nov-2009

Overview

ISO/IEC 25021:2012 - part of the SQuaRE (Systems and software Quality Requirements and Evaluation) series - specifies the concept, format and examples of Quality Measure Elements (QME). A QME is a measure defined by a property of a target entity (artifact, component, behaviour) together with a measurement method to quantify that property, optionally including a mathematical transformation. ISO/IEC 25021:2012 provides guidance and an initial set of QMEs to be used throughout the system and software product life cycle for consistent quality measurement.

Key topics and requirements

• Definition of QME: A QME links a property-to-quantify with a measurement method and, optionally, a transformation function.
• Measurement model: QMEs are inputs for a measurement function that composes two or more QMEs into a derived quality measure (QM) to indicate product quality characteristics or subcharacteristics.
• Table format and conformance: When defining or modifying QMEs, each QME must be described using the information items and table format specified in the standard (see Clause 6.2). This ensures consistent specification and traceability.
• Guidance and examples: Informative annexes provide an initial set of QME examples, a guide for designing QMEs, additional examples and a discussion of measurement scale types.
• Linkage with measurement processes: The standard references measurement process and terminology standards (ISO/IEC 15939, ISO/IEC Guide 99) and the SQuaRE measurement reference model (ISO/IEC 25020).

Practical applications

• Defining quality requirements: Use QMEs to create measurable quality requirements for systems, software products and data.
• Designing metrics and measures: Metric designers and measurement engineers use QMEs as building blocks to derive internal, external and quality-in-use measures (via measurement functions).
• Product evaluation and certification: Acquirers, evaluators and QA teams adopt standardized QMEs to perform reproducible evaluations and to compare products consistently.
• Tooling and automation: QMEs guide the selection and specification of automated measurement procedures, data collection rules and reporting formats.

Who should use this standard

• Software and systems developers, quality managers, measurement engineers, acquirers, independent evaluators, and test teams responsible for defining, implementing or using quality measures across the product life cycle.

Related standards

• ISO/IEC 25000 (SQuaRE guide)
• ISO/IEC 25010 (quality models: characteristics and subcharacteristics)
• ISO/IEC 25020 (measurement reference model)
• ISO/IEC 25022, 25023, 25024 (quality measures for internal, external and quality-in-use)
• ISO/IEC 15939 and ISO/IEC Guide 99 (measurement process and metrology vocabulary)

Keywords: ISO/IEC 25021:2012, SQuaRE, Quality Measure Elements, QME, quality measures, measurement method, ISO/IEC 25022, ISO/IEC 25023, ISO/IEC 25024, software quality, systems engineering.

Standards Content (Sample)

INTERNATIONAL ISO/IEC
STANDARD 25021
First edition
2012-11-01
Systems and software engineering —
Systems and software Quality
Requirements and Evaluation
(SQuaRE) — Quality measure elements
Ingénierie des systèmes et du logiciel — Exigences de qualité et
évaluation des systèmes et du logiciel (SQuaRE) — Éléments de
mesure de la qualité
Reference number
©
ISO/IEC 2012
©  ISO/IEC 2012
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56  CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail [e-mail]
Web www.iso.org
Published in Switzerland
ii © ISO/IEC 2012 – All rights reserved

Contents Page
Foreword . iv
Introduction . v
1  Scope . 1
2  Conformance . 1
3  Normative references . 1
4  Terms and definitions . 2
5  Abbreviated terms . 4
6  Quality measure elements concept . 4
6.1  Presentation of the measurement method model . 4
6.2  Table format of QMEs . 7
Annex A (informative) Examples of QMEs . 12
Annex B (informative) Guide for Designing a Quality Measure Element (QME) . 27
Annex C (informative) Additional Examples of QME and proposed expansion . 30
Annex D (informative) Measurement scale type . 36
Bibliography . 37

© ISO/IEC 2012 – All rights reserved iii

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees
established by the respective organization to deal with particular fields of technical activity. ISO and IEC
technical committees collaborate in fields of mutual interest. Other international organizations, governmental
and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as
an International Standard requires approval by at least 75 % of the national bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC 25021 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 7, Software and System Engineering.
This first edition of International Standard ISO/IEC 25021 cancels and replaces the first edition of
Technical Report ISO/IEC TR 25021:2007.
The SQuaRE series of standards consists of the following divisions under the general title Systems and
Software Quality Requirements and Evaluation (SQuaRE):
 ISO/IEC 2500n, Quality Management Division,
 ISO/IEC 2501n, Quality Model Division,
 ISO/IEC 2502n, Quality Measurement Division,
 ISO/IEC 2503n, Quality Requirements Division, and
 ISO/IEC 2504n, Quality Evaluation Division.
iv © ISO/IEC 2012 – All rights reserved

Introduction
The purpose of this International Standard is to define and/or design an initial set of Quality Measure
Elements (QME) to be used throughout the product life cycle for the purpose of Systems and Software Quality
Requirements and Evaluation (SQuaRE). The document also gives a set of rules to design a QME or verify
the design of an existing QME. The content of this document constitutes the link between the ISO/IEC 9126
series of standards and the subsequent SQuaRE series of standards.
A number of QMEs for quality measures that quantify some of the characteristic and subcharacteristic
represent an initial list, which is to be used during the construction of the quality measures as referenced in
ISO/IEC TR 9126-2, ISO/IEC TR 9126-3 and ISO/IEC TR 9126-4. Quality measures presented in the
SQuaRE series (Figures 1, 2) were extracted from ISO/IEC TR 9126 series but it is not the only source. When
evaluating selected quality measures, the user should first understand the definition of each property related
to a QME used within the selected quality measures.
The main purposes of defining and using the Quality Measures Elements (QMEs) in this document are:
 To provide guidance for organisations developing and implementing their own QMEs;
 To promote the consistent use of specific QME for measuring and using the product properties that are
relevant to different product quality characteristics and subcharacteristics;
 To help identify a set of QMEs that are uniquely required to derive all the quality measures for a given set
of characteristics or a set of subcharacteristics of a product.
The QMEs are the common components of a number of quality measures. The intended usage of this
International standard is that users will be able to select and define relevant valid QMEs to define internal,
external, data or quality-in-use quality measures. Then, these can be used for quality requirements definition,
products evaluation and quality assessment but not necessary limited to those. It is therefore recommended to
use this document prior or together with the ISO/IEC 2502n series of standards.
Quality Model
Division
2501n
Quality Quality
Quality Management
Requirements Evaluation
Division
Division Division
2500n
2503n 2504n
Quality Measurement
Division
2502n
Extension Division 25050-25099

Figure 1 — Organisation of the SQuaRE series of international standards
© ISO/IEC 2012 – All rights reserved v

Figure 1 illustrates the organisation of the SQuaRE series representing families of standards, further called
Divisions.
Figure 2 — Structure of the Quality Measurement Division
ISO/IEC TR 9126-4
ISO/IEC 25021
Quality in use measures
Quality measure elements
ISO/IEC TR 9126-3
Internal measures
(QME)
ISO/IEC TR 9126-2
External measures
ISO/IEC 9126-1
Quality model
Quality characteristics
SQuaRE Quality measures
Quality subcharacteristics
Quality measure 1
Quality measure 2
Quality measure 3
Quality measures
----- - -- -
Quality measure n
ISO/IEC 2502n
Figure 3 — The relationship of ISO/IEC 25021 as a link between the 9126 series and the SQuaRE series
of standard
vi © ISO/IEC 2012 – All rights reserved

The ISO/IEC 9126 series is composed of four documents that list and describe the characteristics,
subcharacteristics and quality measures that are referred to as the quality model. The SQuaRE quality models
categorize product quality into characteristics which are further subdivided into subcharacteristics and quality
properties (ISO/IEC 25010). Each quality measure within ISO/IEC 9126 series is composed of at least two
QMEs. The properties (of a product) are linked to the QME (ISO/IEC 25020), using a measurement method.
The 2502n series designs and describes quality measures and associated QMEs for all the quality
(sub)characteristics in the quality model.
© ISO/IEC 2012 – All rights reserved vii

INTERNATIONAL STANDARD ISO/IEC 25021:2012(E)

Systems and software engineering — Systems and software
Quality Requirements and Evaluation (SQuaRE) — Quality
measure elements
1 Scope
This International Standard contains the following information:
 Requirements for defining QMEs as part of the specification of the product quality requirements with
examples (see 6.2 Tables 1 and 2);
NOTE Product quality includes system quality, software product quality, data quality and eventually system service
quality.
 An initial set of QMEs, as examples (see Annex A Table A.1);
 A guideline for defining and quantifying the property of the product (target entity) for QMEs (see Annex B)
This document is intended for, but not limited to, developers, acquirers and independent evaluators of
products, particularly those responsible for defining product quality requirements and for product evaluation.
This International Standard is applicable when defining the QMEs to be used to implement quality measures
such as those specified in ISO/IEC 25022, ISO/IEC 25023 and ISO/IEC 25024.
2 Conformance
When users define quality measures for a product, each of the referred QME shall be described according to
the information items of format specified in Table 1 (see 6.2). The same should be applied for modifying an
existing QME.
3 Normative references
The following documents, in whole or in part, are normatively referenced in this document and are
indispensable for its application. For dated references, only the edition cited applies. For undated references,
the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 25000:2005, Software engineering — Software product Quality Requirements and Evaluation
(SQuaRE) — Guide to SQuaRE
ISO/IEC 25010:2011, Systems and software engineering — Systems and software product Quality
Requirements and Evaluation (SQuaRE) — System and software quality models
ISO/IEC 25020:2007, Software engineering — Software product Quality Requirements and Evaluation
(SQuaRE) — Measurement reference model and guide
ISO/IEC 15939:2007, Systems and software engineering — Measurement process
ISO/IEC Guide 99:2007, International vocabulary of metrology — Basic and general concepts and associated
terms (VIM)
© ISO/IEC 2012 – All rights reserved 1

4 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC 25000, ISO/IEC 25010,
ISO/IEC 25020, ISO/IEC 15939, ISO/IEC Guide 99, and the following apply. The following definitions are
replicated here for the convenience of the user of this document.
4.1
data quality
degree to which the characteristics of data satisfy stated and implied needs when used under specified
conditions
[ISO/IEC 25012:2008]
4.2
external measure of software quality
measure of the degree to which a software product enables the behavior of a system to satisfy stated and
implied needs for the system including the software to be used under specified conditions
NOTE 1 The behavior can be verified and/or validated by executing the software product during testing and operation.
NOTE 2 Based on the ISO/IEC 25000:2005 definition of external software quality.
NOTE 3 This definition was adapted from ISO/IEC 25010:2011.
4.3
internal measure of software quality
measure of the degree to which a set of static properties of a software product satisfies stated and implied
needs for the software product to be used under specified conditions
NOTE 1 Static properties include those that relate to the software architecture, structure and its components.
NOTE 2 Static properties can be verified by review, inspection, simulation and/or automated tools.
NOTE 3 This definition was adapted from ISO/IEC 25010:2011.
EXAMPLE Depending of the context specifications faults, design faults and code faults could be used as internal
quality measures.
NOTE 4 Based on the ISO/IEC 25000:2005 definition of internal software quality.
4.4
measure (noun)
variable to which a value is assigned as the result of measurement
NOTE The term “measures” is used to refer collectively to base measures, measures, and indicators.
[ISO/IEC 15939:2007]
4.5
measure (verb)
make a measurement
[ISO/IEC 25000:2005]
4.6
measurement
set of operations having the object of determining a value of a measure
[ISO/IEC 15939:2007]
NOTE Measurement can be nominal, ordinal, interval and ratio scale type.
2 © ISO/IEC 2012 – All rights reserved

4.7
measurement function
algorithm or calculation performed to combine two or more quality measure elements
NOTE This definition is modified from ISO/IEC 15939:2007 definition of measurement method.
4.8
measurement method
logical organisation of operations, described generically, used in measurement
NOTE This definition is modified from ISO/IEC 15939:2007 definition of measurement method.
4.9
measurement procedure
logical organisation of operations, applied specifically, used in the performance of particular measurements
according to a given measurement method
NOTE 1 This definition is modified from ISO/IEC 15939:2007 definition of measurement procedure.
NOTE 2 A measurement procedure is usually recorded in a document that is sometimes itself called a "measurement
procedure" and is usually in sufficient detail to enable an operator to carry out a measurement without additional
information.
4.10
model
specification of the concepts, relationships and rules that are used to define a methodology
[ISO/IEC 24744:2007, Software Engineering — Model for Development Methodologies]
4.11
property to quantify
property of a target entity that is related to a quality measure element and which can be quantified by a
measurement method
NOTE 1 A software artifact is an example of a target entity.
NOTE 2 A sub-property is related to a property.
4.12
quality in use measure
measure of the degree to which a product or system can be used by specific users to meet their needs to
achieve specific goals with effectiveness, efficiency, freedom from risk, satisfaction and context coverage in
specific contexts of use
NOTE Based on the ISO/IEC 25010:2011 definition of quality in use.
4.13
quality measure
derived measure that is defined as a measurement function of two or more values of quality measure
elements
4.14
quality measure element (QME)
measure defined in terms of a property and the measurement method for quantifying it, including optionally
the transformation by a mathematical function
© ISO/IEC 2012 – All rights reserved 3

4.15
repeatability (of results of measurement)
closeness of the agreement between the results of successive measurements of the same measurand carried
out under the same conditions of measurement
[ISO/IEC TR 14143-3:2003]
4.16
reproducibility (of results of measurement)
closeness of the agreement between the results of measurements of the same measurand carried out under
changed conditions of measurement
[ISO/IEC TR 14143-3:2003]
NOTE Repeatability and reproducibility may be expressed quantitatively in terms of the dispersion characteristics of
the results.
4.17
target entity
fundamental thing of relevance to the user, about which information is kept, and need to be measured
4.18
unit (of measure)
a particular quantity defined and adopted by convention, with which other quantities of the same kind are
compared in order to express their magnitude relative to that quantity
NOTE 1 Only quantities expressed in the same units of measurement are directly comparable. Examples of units
include the number of faults and the number of failures. Hour and meter are also unit of measure.
NOTE 2 Units of measurement have conventionally assigned names and symbols.
NOTE 3 Based on the ISO/IEC 25000:2005 definition of unit of measurement.
5 Abbreviated terms
In this International Standard, the following abbreviations are used:
a) QME - Quality measure element;
b) QM - Quality measure;
6 Quality measure elements concept
6.1 Presentation of the measurement method model
To understand and indicate quality (sub)characteristics, QM is defined and then QMEs are defined.
A measurement function is applied to QME to generate QM. A measurement method shall be applied to a
property to define and identify a way to quantify a QME.
4 © ISO/IEC 2012 – All rights reserved

Figure 4 — Relationship between property to quantify, measurement method and QME
The user of the measurement method shall identify and collect data related to quantifying the property
(Figure 4). Depending on the context of usage and objective(s) of the QME, a number of properties and sub
properties can be identified. These are the input of the measurement method. Those properties are extracted
and defined from the artefacts, components, content or behaviour of the target entity (e.g. documentation,
code).
ISO/IEC 25022, 23
ISO/IEC 25010
and 24
(TR9126-2,3 and 4)
System and
QM (Quality Measure)
ISO/IEC 25012
Software Product
Data Quality
generates
Quality Indicate
Measurement Function
Composed of
are applied to
Quality
Characteristics
Quality Measure Elements
Composed of
QME
QME
Quality
QME
(Quality Measure Element)
Subcharacteristics
Measurement Method
ISO/IEC 25021
Property to Quantify
Target Entity
Figure 5 — Relationship among property to quantify, measurement method, QME and QM
© ISO/IEC 2012 – All rights reserved 5

Figure 5 shows that
a) product quality is expressed as a set of quality characteristics which in turn are composed of
subcharacteristics;
b) product quality measures are used to indicate the quality characteristics and subcharacteristics of interest;
c) the relationship between the property to quantify, the measurement method and QME.
NOTE This Figure 5 is based on the Systems and Software Product Quality Measurement Reference Model defined
in ISO/IEC 25020.
ISO/IEC 25010
QM =MTBF
System and
ISO/IEC 25012
(Mean Time between Failure)
Software Product
Data Quality
generates
Indicate
Quality
Number of Failures
Composed of
/ Duration period
are applied to
Reliability
Quality Measure Elements
Composed of
QME= Number of Failures Duration
Maturity
(Quality Measure Element)
period
Measurement Method
Measurement Method
Failures Duration
system/software product
executing tasks during testing or operation

Figure 6 — An Example of Relationship among property to quantify, measurement method, QME
and QM
Figure 6 shows an example that a QME is derived by applying a measurement method to a property to
quantify.
Table 1 shows the measurement information items for QME which shall be used to describe QME.
NOTE 1 A QME can be identified when quality characteristic or subcharacteristic are selected and/or a QM is defined
to indicate it. The same QME may be used by different QMs.
NOTE 2 Guide for designing a QME is provided in Annex B.
6 © ISO/IEC 2012 – All rights reserved

6.2 Table format of QMEs
The information items listed in table format of the QMEs (see Table 1) shall be used to define and/or design
a QME to provide necessary or helpful information.
Table 1 — Table format for QMEs
NOTE Items in the below table are organized into four groups which are a) for QME identification, b) – d) for “what
the QME is”, e) – k) for “how to measure the QME” and l) – n) for management of QME application.
a) QME Name A QME should have a unique name and should be identified with a serial number, if
necessary. Most of the time it begins with "number of… (ratio scale)”.
b) Target entity A QME shall have a target object that is to be characterized by measuring its property
Target entity should be a work product or behavior of a system, software, or
stakeholders such as users, operators, developers, testers, or maintainers.
c) Objectives Identification of the property to quantify is usually related to the name of the QME.
and property to Selected property to quantify should be the one which is most relevant to the
quantify measurement of information needed. A given property may be incorporated in multiple
measurement constructs. For example, “number of software faults” is the QME and
“fault” is the property of the software to quantify.
Objective of QME should be specified with the definition of the property to quantify to
describe such as the followings:
- What is intended to know by making definition of the property to quantify of this
QME?
- What is the information needed that is expected to be represented by this QME?
From the identification and definition of the property to quantify, what needs to be
measured is determined (e.g. line of code, defects, duration).
It is helpful to describe what kinds of component or event in the designated target
entity need to be identified, defined and quantified.
The followings are examples:
1) Lines, functions, paths or tokens having specified feature in program source code
may be identified, defined and quantified;
2) Events every when the software under testing fails to specified test cases may be
identified, defined and quantified;
3) Events every when the user of the system fails to user’s intended tasks may be
identified, defined and quantified.
d) Relevant Reference to specific quality measure(s) which use this QME shall be specified.
Quality
measure(s) Examples of quality measures can be found in the ISO/IEC 9126 series, 25000
SQuaRE series and other documents. An exhaustive list of quality measures is not
required.
For the definitions see c) (objectives and properties to quantify) and g) (definition of each sub-properties).
© ISO/IEC 2012 – All rights reserved 7

e) Measurement Measurement method explains how to collect data and how to transform it to a value
method quantifying the property through a numerical rule. The following information: context of
QME, Software Life Cycle process, measurement constraints and numerical rules are
parts of the measurement method.
The measurer can give optionally a name to the measurement method to facilitate the
distinction between QME name, property to quantify name and measurement method.
For example, the functional measurement methods could have names: IFPUG FPA,
COSMIC, Mark II, etc.
f) List of sub An identified property to quantify can be related to different sub properties, if
properties necessary. This relationship between properties should be expressed as a schema or
related to the a formula. This constitutes the measurement method model.
property to
quantify For example, within the COSMIC method, a functional process is one property that can
(optional)
be expressed in a model with some sub properties like entry, read, write and exit. This
can help to identify the property to quantify of “data movement” which is relevant to
functional size based measure.
g) Definition of If there is a list of sub properties, each sub property should be defined.
each sub
property
(optional)
h) Input for the The input shall be described in enough details to identify what quantitative information
QME is used to measure the QME. Any sources providing the input should also be identified
such as the documented work products, behavior of system and software, or human
behavior of users, operators, developers, testers, or maintainers.
Then, the input may be sub properties or quantitative information relating to them.
For example, the measurer could identify in a data model the information to retrace the
entity of a read type (data movement) in COSMIC function point.
i) Unit of The unit of measurement and, if appropriate, the formula used. Examples of units
measurement for include number of X, percentage and rank.
the QME
j) Numerical A numerical assignment rule shall be described from a practitioner view (generally a
rules text form) or from a theoretical point of view (generally a mathematical expression).
The internal consistency is often a problem when assigning a numerical rule.
It is important to have consistency between property and sub properties that need to
be measured. For this reason it is important to demonstrate that when adding two
entities they are related by a common property.
For example, measuring faults will give the number of faults. But if there is a distinction
between major and minor faults, a more precise measure will be obtained by adding
separately the major and minor faults. The interpretation consider the limit of the result
applied to each property and sub properties.
k) Scale type Scale type shall be identified. Scale type could be nominal, ordinal, interval or ratio
(see Annex D).
8 © ISO/IEC 2012 – All rights reserved

l) Context of This gives information about the intended use of the measurement results.
QME
It is helpful to understand possibility of using QME to express quality (sub)
characteristics by describing typical examples of quality characteristics, quality
subcharacteristics or quality measures (QM) which are mainly intended to use of the
QME measurement results.
NOTE The QME is able to be employed by a number of quality measure (QM) to measure
any of quality (sub)characteristics. Assumptions and prerequisite conditions of target entities, its
environments and circumstances to which the measurement method of QME is to be applied is
described here.
m) Software Life The typical appropriate life cycle process(es) that are suited for actual measurement of
Cycle this QME with respect to a target entity should be identified here (e.g. process(es) in
process(es) which a specific target entity is created or realized enough to enable the obtaining an
actual measured value of QME).

NOTE 1 In some cases, the estimation may be available in some of life cycle processes
based on historical data before actual measurement of QME. However, life cycle process(es)
listed here are those where we can obtain actual measured results of the QME. Related life
cycle process(es) after obtaining actual data, additional actual measurement or use of the
measured results are also specified here.
For example, the number of faults in code is actually measurable by applying code review, code
analyzing tools or unit testing during construction process (coding and unit testing). Also the
number of faults in code is also able to be additionally measured when code is corrected to
resolve failures in integration or qualification testing processes. Besides, the number of faults in
code may be estimated through estimated coding size from page volume of requirements
specifications based on historical data.
NOTE 2 The basic software life cycle process(es), such as stakeholder requirements
definition, software requirements analysis, software architectural design, software detailed
design, software construction, software integration, software qualification testing, software
installation, software acceptance support, software operation, software maintenance, software
disposal and so on, are Defined in ISO/IEC 12207:2008, Systems and Software Engineering -
Software Life Cycle Processes. The basic system process(es), such as stakeholder
requirements definition, requirements analysis, architectural design, implementation, integration,
verification, transition, validation, operation, maintenance, disposal and so on, are defined in
ISO/IEC 15288:2008, Systems and Software Engineering - System Life Cycle Processes.
NOTE 3 If the methodology in use does not include any life cycle process(es) described in
neither ISO/IEC 12207 nor ISO/IEC 15288 then the measurer may also mention the
methodology and the particular process(es) that will be used.
n) Measurement If necessary, any constraints related to the measurement method should be described,
Constraints if necessary.
(optional) The QME may have measurement constraints such as measurement errors or
fluctuations due to dependency of the followings: scope of investigation, way of
investigation, volatility of specification or tactics of test cases.
NOTE 1 For example, the number of faults in code may vary between newly developed code
and reused code in scope.
Each of the various ways of code investigation such as review, walk-through,
inspection, individual inspection by expertise, pair programming, code analysis tools,
unit testing, causal analysis of failure in integration testing etc gives different number of
faults in code.
NOTE 2 For example, in case of counting the number of specification defects, the
specification document should be available and not very volatile.
© ISO/IEC 2012 – All rights reserved 9

The following table is an example on how to use the format of Table 1.
Table 2 — EXAMPLE: application of Table 1 for fault (of code)
a) QME Name Number of faults (of code)
b) Target entity
Program source code
c) Objectives The objective is to measure the number of faults in code with reference to the design
and property to specifications and / or coding standards.
quantify
What needs to be measured is the number of erroneous lines of code.
Fault is the property to quantify.
Definition of fault:(1) a manifestation of an error in software (ISO/IEC 24765:2010
Systems and software engineering vocabulary) (2) an incorrect step, process, or data
definition in a computer program (ISO/IEC 24765:2010 Systems and software
engineering vocabulary).
NOTE A fault, if encountered, may cause a failure.
d) Relevant To measure the reliability of the software by utilizing fault density
Quality
 Finding fault detection rate during coding phase
measure(s)
 Finding fault removal rate with corrected faults in code phase

Maturity (subcharacteristic) and Reliability (characteristic) level of the software
e) Measurement Software Fault Measurement Method in the code
method
Review or analyze differences of the revised program source code and identify
corrected lines of code which consist of modified lines, added lines and deleted lines of
code.
NOTE Program source code is usually revised as a result of verification and validation
activities such as code review, unit testing, causal analysis to resolve failures in integration
testing.
f) List of sub Related sub properties: executable statements, erroneous lines of code, corrected
properties lines of code.
related to the
property to
quantify
(optional)
10 © ISO/IEC 2012 – All rights reserved

g) Definition of Executable Statements: The statements which can be categorized as labeled
each sub statements, expressions, selection statements, iteration statements and jump
property statements.
(optional)
Non-Executable Statements: The statements that can be categorized as declarations
and declaration specifiers.
Erroneous Lines of Code: Lines of code which contains fault(s). Specifications should
tell if the source code is erroneous or not.
Correct Lines of Code: Lines of code with no fault.
NOTE It is also possible that the actual lines of code are correct and the specifications
should be changed. This should not be counted as erroneous lines of code.
h) Input for the Source code, design specifications and coding standards
QME
i) Unit of Lines of code
measurement
for the QME
j) Numerical Adding total erroneous lines of code
rules
A numerical assignment rule from a practitioner view using the following measurement
actions:
a) Review or analyze differences of the revised program source code and identify
corrected lines of code which consist of modified lines, added lines and deleted lines of
code
K) Scale type Ratio
l) Context of This QME is mainly chosen to measure the Maturity (subcharacteristic) and Reliability
QME (characteristic) level of the software
m) Software Life Software Construction (coding and unit testing), Implementation process.
Cycle
process(es)
n) Measurement Source codes shall be available to be able to compare the actual lines of code with the
Constraints design specifications.
(optional)
Stable design specifications shall be available to be able to compare verify the actual
lines of code with the design specifications and identify faults.
A tool or checklist shall be available to check for coding standards.
© ISO/IEC 2012 – All rights reserved 11

Annex A
(informative)
Examples of QMEs
Various QMEs can be used together and combined to define QMs. Some QMEs are from the ISO/IEC 9126s
and the others are from industry market needs and existing standard such as functional size measurement.
QMEs listed in this sample set are related to quality (sub)characteristics of product quality model defined in
ISO/IEC 25010. This sample set of QMEs is suggested for the user of this document to consider its
applicability, when one is preparing QMs for evaluation of product quality.
Table A.1 — List of Initial set of QMEs
QME
NO Description
a) QME name Number of accessible functions
b) Target entity User callable functions
c) Objectives and property to To know how many user callable functions are accessible by disabled users.
quantify Definition of callable function: function provided to users by a system for users to
access, call and use to perform specified tasks.
d) Relevant quality QMs such as number of (not) accessible functions under specific context in use, or by
measure(s) specific type of user during testing or operation for quantifying context coverage in
quality in use and accessibility in usability
e) Measurement method Review or test specified cases that disabled users callable and operable functions of
system/software and count the number of functions that they could not be able to
successfully use.
f) List of sub properties -to be completed, if necessary
related to the property to
quantify (optional)
g) Definition of each sub -to be completed, if necessary
property (optional)
h) Input for the QME Review or test results on specified cases of operation by disabled users, User
manuals
i) Unit of measurement for Number of user callable functions
the QME
j) Numerical rules x-y,
x: The number of user callable functions which are reviewed or tested on specified
cases of operation by disabled users
y: The number of user callable functions that disabled user could not be able to
successfully use.
k) Scale type Ratio
l) Context of QME This QME is usable to context coverage in quality in use and accessibility in usability
m) Software life cycle Implementation and Operation
process(es):
n) Measurement constraints -to be completed, if necessary
(optional)
12 © ISO/IEC 2012 – All rights reserved

QME
NO Description
a) QME name Number of user problems
b) Target entity User problems during operation
c) Objectives and property to To know how many occurrences of problems system/software users recognize during
quantify operation. User problem during system/software operation is the property to quantify
Definition of user problem: Each time a user complains about a product, it is
registered by the organisation (normally at the helpdesk level). Knowing the
complaint could help to measure the degree of the user satisfaction over time period.
For example, technical problems or functional problems are extracted from user
complaints and sorted out by help desk.
This QME can help users to determine problem during operation of the software but
not necessarily limited to that properties.
d) Relevant quality QMs such as number of (sever) user problems reported per week or distribution of
measure(s) spent days until resolution of user problems for quantifying usability, reliability and
satisfaction
e) Measurement method List up problems from user claim reports, categorize them by severity and count
them.
f) List of sub properties -to be completed, if necessary
related to the property to
quantify (optional)
g) Definition of each sub -to be completed, if necessary
property (optional)
h) Input for the QME User claim report sent to help desk
i) Unit of measurement for Number of problems
the QME
j) Numerical rules Count the number of problems for individual severity level
k) Scale type Ratio
l) Context of QME This QME is usable to QMs applied during operation for usability, reliability and
satisfaction.
m) Software life cycle Operation
process(es):
n) Measurement constraints -to be completed, if necessary
(optional)
a) QME name Number of records
b) Target entity Data items treated as unit or records
c) Objectives and property to It is used to quantify the complexity of a database.
quantify Definition of record: a set of related data items treated as a unit.
(ISO/IEC 24765:2010 Systems and software engineering vocabulary)
d) Relevant quality A large number of records can affect the maintainability.
measure(s)
e) Measurement method List up records and count up them
f) List of sub properties -to be completed, if necessary
related to the property to
quantify (optional)
g) Definition of each sub -to be completed, if necessary
property (optional)
h) Input for the QME Data items
i) Unit of measurement for Each record
the QME
j) Numerical rules Adding each record
k) Scale type Ratio
l) Context of QME This QME is usable to QMs for data quality characteristics.
m) Software life cycle Maintenance
process(es):
© ISO/IEC 2012 – All rights reserved 13

QME
NO Description
n) Measurement constraints Need to obtain data items
(optional)
a) QME name Duration
b) Target entity A time period
c) Objectives and property to Definition of duration: total number of work periods (not including holidays or other
quantify nonworking periods) required to complete a schedule activity or work breakdown
structure component. Usually expressed as days weeks or months. Sometimes
incorrectly equated with elapsed time.(A Guide to the Project Management Body of
Knowledge, (PMBOK® Guide) — Fourth Edition)
d) Relevant quality
This QME is useful to QMs for performance efficiency characteristic. This is also
measure(s)
useful to QMs such as time to complete user’s intended task, throughput, mean time
between failures or mean time to repair which is using time period spent by operator,
user, maintainer or system.
e) Measurement method Duration is based on the definition of the base quantity of time and related to the
International System of Quantities (VIM)
f) List of sub properties -to be completed, if necessary
related to the property to
quantify (optional)
g) Definition of each sub -to be completed, if necessary
property (optional)
h) Input for the QME From time sheet in the organisation
i) Unit of measurement for Days, weeks or months
the QME
j) Numerical rules Accumulating the work periods
k) Scale type Ratio
l) Context of QME
This QME is usable for all QMs relating duration time, such as MTBF and average
throughput per unit time for reliability and performance efficiency. Besides,
combination with effort provides productivity measure.
m) Software life cycle All
process(es):
n) Measurement constraints
-to be completed, if necessary
(optional)
a) QME name Effort (in unit of time)
b) Target entity Effort in hours or days
c) Objectives and property to For productivity measures in hours.
quantify
Definition of effort: The number of labor units required to complete a scheduled
activity or work breakdown structure component. Usually expressed as hours, days,
or weeks. (‘A Guide to the Project Management Body of Knowledge’ (PMBOK®
Guide) — Fourth Edition)
d) Relevant quality
This QME is useful to QMs for performance efficiency characteristic. This is also
measure(s)
useful to QMs such as effort to complete user’s intended task, effort for system
recovery, or effort for maintenance which is using people effort spent by operator,
user, developer, tester or maintainer.
e) Measurement method Effort is based on the definition of the base quantity of time and related to the
International System of Quantities (VIM)
14 © ISO/IEC 2012 – All rights reserved

QME
NO Description
f) List of sub properties -to be completed, if necessary
related to the property to
quantify (optional)
g) Definition of each sub -to be completed, if necessary
property (optional)
h) Input for the QME Time sheet from the organisation
i) Unit of measurement for Generally in hours and days
the QME
j) Numerical rules Adding
k) Scale type Ratio
l) Context of QME
This QME for all QMs relating effort for especially efficiency, effectiveness, reliability,
performance efficiency, usability and maintainability.
Besides, combination with effort provides productivity measure or estimation.
m) Software life cycle All
process(es):
n) Measurement constraints
-to be completed, if necessary
(optional)
a) QME name Number of system failures
b) Target entity System failure
c) Objectives and property to ‘System Failure Count’ is intended to be used in the quality (derived) measures such
quantify as reliability, efficiency and quality of the software and applicable to system
engineering and software engineering and management disciplines. Property if
failure.
Definition of system failure: A complete system includes all of the
associated equipment, facilities, material; computer programs, firmware, technical
documentation, services, and personnel required for operations and support to the
degree necessary for self-sufficient use in its intended environment previously
specified limits.
Software failure: Termination of the ability of a product to perform a required
function or its inability to perform within previously specified limits.
d) Relevant quality QMs such as system failure frequency or MTBF during testing or operation for
measure(s) reliability and efficiency of quality in use.
e) Measurement method Adding the number of System failures
f) List of sub properties to be completed, if necessary
related to the property to
quantify (optional)
g) Definition of each sub Structural System Failures: Subsy
...