ISO/IEC TR 25021:2007

Software engineering - Software product Quality Requirements and Evaluation (SQuaRE) - Quality measure elements

ISO/IEC TR 25021:2007 defines the set of quality measure elements to be used throughout the software product life cycle for the purpose of Software Product Quality Requirement and Evaluation (SQuaRE). While some quality measure elements can be used as standalone quality measures, their main purpose is to be the building blocks for other SQuaRE measurements as described in ISO/IEC TR 9126-2, ISO/IEC TR 9126-3 and ISO/IEC TR 9126-4. ISO/IEC TR 25021:2007 constitutes the link between ISO/IEC 9126 and the subsequent SQuaRE series of standards. ISO/IEC TR 25021:2007 contains informative annexes documenting the cross-reference relationship between the quality measure elements and quality measures, characteristics and subcharacteristics defined in ISO/IEC 9126. The Quality Measurement Division, of which ISO/IEC TR 25021:2007 is a member, also offers examples of quality measures that can be used across the product development life-cycle. These measures are defined in the other documents in the division and correspond to the quality characteristics in a software product quality model such as that described in the future International Standard ISO/IEC 25010 (replacing ISO/IEC 9126-1). ISO/IEC TR 25021:2007 is designed to be used, in particular, with other standards in the SQuaRE series that address quality requirements (ISO/IEC 25030) and product quality evaluation (the future International Standard ISO/IEC 25040).

Ingénierie du logiciel — Exigences de qualité et évaluation du produit logiciel (SQuaRE) — Éléments de mesure de la qualité

General Information

• Status: Withdrawn
• Publication Date: 14-Oct-2007
• Withdrawal Date: 14-Oct-2007
• ICS: 35.080 - Software
• Technical Committee: ISO/IEC JTC 1/SC 7 - Software and systems engineering
• Drafting Committee: ISO/IEC JTC 1/SC 7/WG 6 - Software Product and System Quality
• Current Stage: 9599 - Withdrawal of International Standard
• Start Date: 23-Oct-2012
• Completion Date: 30-Oct-2025

Relations

• Revised: ISO/IEC 25021:2012 - Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - Quality measure elements
• Effective Date: 07-Nov-2009

Standards Content (Sample)

TECHNICAL ISO/IEC
REPORT TR
First edition
2007-10-15
Software engineering — Software product
Quality Requirements and Evaluation
(SQuaRE) — Quality measure elements
Ingénierie du logiciel — Exigences de qualité et évaluation du produit
logiciel (SQuaRE) — Éléments de mesure de la qualité

Reference number
©
ISO/IEC 2007
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but
shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In
the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.

©  ISO/IEC 2007
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail [e-mail]
Web www.iso.org
Published in Switzerland
ii © ISO/IEC 2007 – All rights reserved

Contents Page
Foreword. iv
Introduction . v
1 Scope .1
2 Conformance.1
3 Normative references .1
4 Terms and definitions .2
5 Symbols and abbreviated terms .4
6 Quality measure elements concept .4
6.1 Quality measure elements in the Software Product Quality Measurement Reference Model
(SPQM-RM) .4
6.2 Quality measure elements and their categories.6
6.3 Aspects of quality measure elements .6
6.4 Table format of quality measure elements.8
7 Quality measure element categories .9
8 Set of quality measure elements.11
8.1 Data Size .11
8.2 Number of Data Items.13
8.3 Number of Failures .17
8.4 Number of Faults .19
8.5 Number of Function.21
8.6 Number of I/O Events .29
8.7 Number of Requirements.31
8.8 Number of Restarts.32
8.9 Number of Tasks.33
8.10 Number of Test Cases.35
8.11 Number of Trials .37
8.12 Product Size .38
8.13 Time Duration.39
Annex A (informative) Selected quality measures.42
Annex B (informative) Cross reference table between quality measures and quality measure
element categories .45
Annex C (informative) Cross reference table between set of quality measure elements and quality
measures .50
Bibliography .59

© ISO/IEC 2007 – All rights reserved iii

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees
established by the respective organization to deal with particular fields of technical activity. ISO and IEC
technical committees collaborate in fields of mutual interest. Other international organizations, governmental
and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as
an International Standard requires approval by at least 75 % of the national bodies casting a vote.
In exceptional circumstances, the joint technical committee may propose the publication of a Technical Report
of one of the following types:
— type 1, when the required support cannot be obtained for the publication of an International Standard,
despite repeated efforts;
— type 2, when the subject is still under technical development or where for any other reason there is the
future but not immediate possibility of an agreement on an International Standard;
— type 3, when the joint technical committee has collected data of a different kind from that which is
normally published as an International Standard (“state of the art”, for example).
Technical Reports of types 1 and 2 are subject to review within three years of publication, to decide whether
they can be transformed into International Standards. Technical Reports of type 3 do not necessarily have to
be reviewed until the data they provide are considered to be no longer valid or useful.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC TR 25021, which is a Technical Report of type 2, was prepared by Joint Technical Committee
ISO/IEC JTC 1, Information technology, Subcommittee SC 7, Software and systems engineering.
The SQuaRE series of standards consists of the following divisions under the general title Software product
Quality Requirements and Evaluation:
⎯ Quality Management Division,
⎯ Quality Model Division,
⎯ Quality Measurement Division,
⎯ Quality Requirements Division, and
⎯ Quality Evaluation Division.

iv © ISO/IEC 2007 – All rights reserved

Introduction
The purpose of this Technical Report is to define an initial set of quality measure elements to be used
throughout the software product life cycle for the purpose of Software Product Quality Requirement and
Evaluation (SQuaRE). While the quality measure elements can be used for standalone measurement, their
main purpose is to be used as the building blocks for other SQuaRE measures as described in
ISO/IEC TR 9126-2, ISO/IEC TR 9126-3 and ISO/IEC TR 9126-4. The content of this Technical Report
constitutes the link between ISO/IEC 9126 and the subsequent SQuaRE series of standards (Figure 1).
Quality Model Division
2501n
Quality
Quality
Quality
Evaluation
Requirements
Management Division
Division
Division
2500n
2504n
2503n
Quality
Measurement Division
2502n
Figure 1 — Organization of SQuaRE series of standards
During the period of transition from a Technical Report to an International Standard, the intention of this
Technical Report is to be only a source of support for the collection of quality measure elements. During this
trial period, the applicability of this Technical Report and its content will be evaluated in the user environment.
The support provided by this Technical Report for the measurement of software product quality is twofold: (a)
assist in the selection of the required quality measure elements for a given quality measure, and (b) provide
guidance for collecting the selected measurements.
The set of quality measures listed in this Technical Report arise from surveys involving several large
commercial and academic institutions conducted during the following research projects:
⎯ Quality Measure Validation Survey (Prague University of Economics research project GACR 201/06/0175
and Czech University of Life Sciences in Prague research projects MSM6046070904 and 2C06004);
⎯ Conformity of Industrial Software to International Standards ES (Excellent Software) Mark (Korean
Agency for Technology and Standards, Korea);
⎯ Project for Development fund of MII (China Ministry of Information Industry): software engineering
standardization.
© ISO/IEC 2007 – All rights reserved v

NOTE  This set does not contain quality measure elements for all ISO/IEC 9126-1 Quality Model subcharacteristics.
Some subcharacteristics were omitted because survey results did not find evidence of their use. It does not imply the
ISO/IEC 9126-1 Quality Model should be changed, but rather new quality measures may need to be defined in the future.
Subsequently the quality measure elements necessary for measurement of the above quality measures set
have been identified and documented. These quality measure elements represent an initial set of measures
which can be used during the construction of quality measures referenced in ISO/IEC TR 9126-2,
ISO/IEC TR 9126-3 and ISO/IEC TR 9126-4, as well as other measures for other purposes. Quality measures
described in the SQuaRE series (Figure 2) are derived from one or more quality measure elements described
in this Technical Report. When evaluating a selected quality measure, the user should first review and
evaluate the relevant quality measure element(s) listed in this Technical Report.
Some important benefits from using the measures in this Technical Report are the following.
⎯ To improve measurement productivity and consistency which will minimize measurement effort: When
using quality measures such as internal, external and quality in use, there is a possibility of duplicating
the attribute measurement tasks, because they are usually performed as separate activities. However by
identifying the set of quality measure elements that are uniquely required to derive all the quality
measures for a given product, these measurements need to be performed only once at the attribute level.
This approach will improve the productivity of the measurement processes.
⎯ Usability (guidance, cross reference): This Technical Report allows users to identify the possible
indicators of quality (quality measures) that can be derived by measuring one or more measures from a
selected set of quality measure elements, and thereby maximizing the benefits from the measurement
process.
The quality measure elements are the common components of quality measures. The intention here is that
users of this Technical Report will select measures from quality measure elements for the purpose of defining
internal, external or quality-in-use measures. These are then used for the definition of quality requirements
(ISO/IEC 25030), software product evaluation (ISO/IEC 2504n), quality assessment and other purposes. It is
therefore strongly recommended to use this Technical Report and the other documents in the Quality
Measurement Division together with the other relevant documents in the ISO/IEC 25000 series.

Figure 2 — Structure of the Quality Measurement Division
vi © ISO/IEC 2007 – All rights reserved

Quality measure elements, together with the other support documentation offered in this Technical Report,
provide added value to the ISO/IEC 9126 Technical Reports and International Standards by making them
more understandable to users by clearly defining the relevant quality measure elements (Figure 3). In this
sense, this Technical Report acts effectively as the common link between ISO/IEC 9126 and its follower, the
SQuaRE series.
9126-4 Quality in Use 25021 Quality measure
elements
9126-3 Internal Metrics
Quality measure element
categories
Quality measure elements
9126-2 External Metrics
9126-1 Quality Model
Quality characteristics
SQuaRE
Quality measures
Measure 1
Measure 2
Quality subcharacteristics
Measure 3
…
2502n
Figure 3 — The relationship of ISO/IEC TR 25021 as a link between the ISO/IEC 9126 and the SQuaRE
series of standards
© ISO/IEC 2007 – All rights reserved vii

TECHNICAL REPORT ISO/IEC TR 25021:2007(E)

Software engineering — Software product Quality
Requirements and Evaluation (SQuaRE) — Quality measure
elements
1 Scope
This Technical Report specifies an initial set of quality measure elements in order to assist users of
ISO/IEC TR 9126-2, ISO/IEC TR 9126-3 and ISO/IEC TR 9126-4 and users of the SQuaRE series of quality
measurement standards ISO/IEC 2502n in the selection and use of the quality measures for software product
quality evaluation and in the selection of the entities to be measured in the software product lifecycle.
This Technical Report contains
a) a description of the concept of quality measure elements,
b) considerations for using quality measure elements,
c) a set of quality measure elements.
This Technical Report is intended for, but not limited to, developers, acquirers and independent evaluators of
software product, particularly those responsible for defining software product quality requirements and for
software product evaluation.
2 Conformance
There are no conformance requirements in this Technical Report.
3 Normative references
The following referenced documents are indispensable for the application of this document. For dated
references, only the cited edition applies. For undated references, the latest edition of the referenced
document (including any amendments) applies.
ISO/IEC 25000, Software Engineering – Software product Quality Requirements and Evaluation (SQuaRE) –
Guide to SQuaRE
ISO/IEC 25020, Software engineering – Software product Quality Requirements and Evaluation (SQuaRE) –
Measurement reference model and guide
ISO/IEC 15939, Systems and software engineering – Measurement process
ISO/IEC TR 9126-2, Software engineering — Product quality — Part 2: External metrics
ISO/IEC TR 9126-3, Software engineering — Product quality — Part 3: Internal metrics
ISO/IEC TR 9126-4, Software engineering — Product quality — Part 4: Quality in use metrics
© ISO/IEC 2007 – All rights reserved 1

4 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC 25000, ISO/IEC 25020 and
ISO/IEC 15939 apply.
NOTE The following definitions are replicated here for the convenience of the user.
4.1
attribute
inherent property or characteristic of an entity that can be distinguished quantitatively or qualitatively by
human or automated means
NOTE 1 Based on ISO/IEC 15939:2007.
NOTE 2 ISO 9000 distinguishes two types of attributes: a permanent characteristic existing inherently in something;
and an assigned characteristic of a product, process or system (e.g. the price of a product, the owner of a product). The
assigned characteristic is not an inherent quality characteristic of that product, process or system.
4.2
base measure
measure defined in terms of an attribute and the method for quantifying it
NOTE  A base measure is functionally independent of other measures.
[ISO/IEC 15939:2007, based on the definition in the International Vocabulary of Basic and General Terms in
Metrology, 1993]
4.3
derived measure
measure that is defined as a function of two or more values of base measures
[ISO/IEC 15939:2007, based on the definition in the International Vocabulary of Basic and General Terms in
Metrology, 1993]
NOTE A transformation of a base measure using a mathematical function can also be considered as a derived
measure.
4.4
external software quality
capability of a software product to enable the behaviour of a system to satisfy stated and implied needs when
the system is used under specified conditions
NOTE Attributes of the behaviour can be verified and/or validated by executing the software product during testing
and operation.
EXAMPLE The number of failures found during testing is an external software quality measure related to the number
of faults present in the program. The two measures are not necessarily identical since testing may not find all faults, and a
fault may give rise to apparently different failures in different circumstances.
4.5
indicator
measure that provides an estimate or evaluation of specified attributes derived from a model with respect to
defined information needs
[ISO/IEC 15939:2007]
NOTE In ISO/IEC 14598, this definition of “indicator” was: “a measure that can be used to estimate or predict another
measure”.
2 © ISO/IEC 2007 – All rights reserved

4.6
information need
insight necessary to manage objectives, goals, risks and problems
[ISO/IEC 15939:2007]
4.7
internal software quality
capability of a set of static attributes of a software product to satisfy stated and implied needs when the
software product is used under specified conditions
NOTE 1 Static attributes include those that relate to the software architecture, structure and its components.
NOTE 2 Static attributes can be verified by review, inspection and/or automated tools.
EXAMPLE The number of lines of code, complexity measures and the number of faults found in a walk through are
all internal software quality measures made on the product itself.
4.8
measure, noun
variable to which a value is assigned as the result of measurement
NOTE The term “measures” is used to refer collectively to base measures, derived measures, and indicators.
[ISO/IEC 15939:2007]
4.9
measure, verb
make a measurement
[ISO/IEC 14598-1:1999]
4.10
measurement
set of operations having the object of determining a value of a measure
[ISO/IEC 15939:2007, based on the definition in the International Vocabulary of Basic and General Terms in
Metrology, 1993]
NOTE Measurement can include assigning a qualitative category such as the language of a source program (ADA, C,
COBOL, etc.).
4.11
measurement function
algorithm or calculation performed to combine two or more base measures
[ISO/IEC 15939:2007]
4.12
measurement method
logical sequence of operations, described generically, used in quantifying an attribute with respect to a
specified scale
[ISO/IEC 15939:2007, based on the definition in the International Vocabulary of Basic and General Terms in
Metrology, 1993]
4.13
quality in use (measure)
extent to which a product used by specific users meets their needs to achieve specific goals with effectiveness,
productivity, safety and satisfaction in specific contexts of use
© ISO/IEC 2007 – All rights reserved 3

4.14
quality measure element
base measure or derived measure that is used for constructing software quality measures
NOTE The software quality characteristics or subcharacteristics of the entity are derived afterwards by calculating a
software quality measure.
5 Symbols and abbreviated terms
For the purposes of this Technical Report, the symbols and abbreviations given in ISO/IEC 25000, ISO/IEC
25020 and the following apply.
QME Quality Measure Element
SPQM-RM Software Product Quality Measurement Reference Model
6 Quality measure elements concept
Quality measure elements are used throughout the software product lifecycle as an input for the internal,
external and quality in use measures listed in ISO/IEC TR 9126-2, ISO/IEC TR 9126-3 and ISO/IEC TR
9126-4. They measure:
⎯ Attributes of resources consumed, or activities performed during the software product development,
testing, and maintenance that relate to software product quality,
⎯ Attributes of the software product itself,
⎯ Attributes of the specific context of use,
⎯ Attributes of the software product when used in a specific context of use, and the effects connected with
such use (effects on the user and the result of their task).
6.1 Quality measure elements in the Software Product Quality Measurement Reference
Model (SPQM-RM)
International Standard ISO/IEC 25020 defines the Software Product Quality Measurement Reference Model
(SPQM-RM) to be used for the software product quality requirements specification process (top-down
approach) and for the software product quality evaluation process (bottom-up approach).
SPQM-RM (Figure 4) defines the position of quality measure elements in the software product quality
measures definition process. Any single quality measure element by itself generally will not indicate the quality
of the measured entity. The quality measure is derived afterwards by computing the results of quality measure
element values as stated in the formula of the quality measure that is being measured.
4 © ISO/IEC 2007 – All rights reserved

SoftSoftwwaarree Pr Proodductuct
QuQualitalityy
comcomposed ofposed of
indicateindicate
QQuualaliittyy M Meeasurasureess
QuQualitalityy
genergeneraatetess
CChharacaractteerriisstticsics
MMeeasasuurremeemenntt
comcomposed ofposed of
FuFuncncttiioonn
indicateindicate
arare ape applied plied ttoo
QuQualitalityy
QuQualitalityy M Meeasurasuree
SSub-ub-ChaCharracactteeririststiiccss
ElElemenementsts
Figure 4 — Quality measure elements concept in SPQM-RM
A quality measure is derived from one or more quality measure elements. A quality measure element can be
an input to more than one quality measure. This relationship can be represented in the form of a two-
dimensional cross-reference table, which allows a user of this Technical Report to select the proper quality
measure elements that are necessary for measuring an individual set of quality measures. It also allows the
user of this Technical Report to identify quality measures, which could be derived from a given set of quality
measure elements.
The layout of the cross-reference table is shown in Table 1. In this table the quality measures and quality
measure elements can be any of those stated in this document. This table works in both ways: by row, a cross
associates a quality measure element to all the quality measures where its usage applies; by column, all the
crosses indicate the quality measure elements that are required for the quality measure in that column.
Table 1 — Layout of the cross-reference table representing the relationship between QMEs and
quality measures
Quality Quality Quality
measure 1 measure 2 measure 3
Quality measure
X X
element 1
Quality measure
X X
element 2
Quality measure
X
element 3
When evaluating the quality measure elements, a user should be familiar with the information given in the
quality measure element table and consider all the limitations and rules given for a listed quality measure
element, prior to applying the formula to obtain the value of the quality measure.
© ISO/IEC 2007 – All rights reserved 5

6.2 Quality measure elements and their categories
Quality measures in ISO/IEC TR 9126-2, ISO/IEC TR 9126-3 and ISO/IEC TR 9126-4 are described in
different level of detail. Some of them are specified precisely, such that the QME and their measurement
methods could be easily derived from them; others are described in a way that permits different QME and
measurement methods to be used. The concept of the QME Category is used to indicate this difference.
6.3 Aspects of quality measure elements
Users of this Technical Report should consider each of the aspects listed in this section (while evaluating the
quality measure elements) in order to avoid accidental misuse of the measurement process, and possible
omission of essential measurement steps.
The purpose of quality measure elements is to measure different attributes of a software product or process at
any stage during the software product lifecycle. This requires consideration of different aspects of the quality
measure elements construction and application of specific practices, such as data selection, measurement
process, documentation, arithmetic operations, etc. that are described in each clause.
For the purpose of this Technical Report these aspects are:
⎯ Measurement scale type – the scale type used for measurement,
⎯ Measurement focus in the software product quality lifecycle – the scope and objective of the
measurement (e.g. the software product itself, software product in a system, software product in a system
used by a specified user in a specified scenario),
⎯ Measurement method type – the measurement method type relating to the quality measure element used
for measurement.
These aspects are followed with an essential, but not exclusive set of considerations. While measuring any
quality measure element a user of this Technical Report should examine relevant considerations from all the
above listed aspects.
NOTE 1 The following aspects and considerations relate to common issues that are relevant to the collection or
measurement of (non-exclusive) sets of QMEs.
NOTE 2 Some of the described considerations (e.g. nominal scale etc.) are not relevant for any of the QME listed in
this Technical Report. They have bee included to bring the complete description of each aspect and allow measurement of
QMEs not listed in this Technical Report.
6.3.1 Measurement scale type
The type of scale depends on the nature of the relationship between values on the scale. Five types of scales
are commonly used and connected considerations are:
NOTE Scale is defined in ISO/IEC 25000. The following are examples of types of scale.
Nominal scale - The purpose of nominal scale type measures is to classify measured attributes. No ordering
is implied even if numbers are used. The numbers assigned to measures in nominal scale type measurement
identify only the category (type) of measured attribute. Therefore the order, minimum, maximum, median,
arithmetic mean, percentage, etc. derived from these numbers do not have any empirical meaning.
EXAMPLE Software product fault types or categories. Type of function based on the software product quality model
(e.g., type of quality characteristics or subcharacteristics)
Ordinal scale - The purpose of ordinal scale type measures is to assign an order to measured attributes. The
ordinal scale is often useful to augment the nominal scale with information about an ordering of the classes or
categories. The minimum, maximum and median derived from the measures in the ordinal scale type
6 © ISO/IEC 2007 – All rights reserved

measurement have empirical meaning. The arithmetic mean, percentage etc. do not have an empirical
meaning.
EXAMPLE Software product failure by severity (e.g. negligible, marginal, critical, catastrophic)
Interval scale - The purpose of interval scale type measures is to measure a difference between measures. If
a ratio is calculated from the quality measure elements of the same type, it does not have an empirical
meaning.
EXAMPLE Cyclomatic complexity has a minimum value of one, but each increment represents an additional path.
The value of zero is not possible.
Ratio scale - The ratio scale is similar to the interval scale but includes the value zero, representing total lack
of an attribute. Ratio scale quality measure elements include ordered rating scales, where the difference
between two measures, and the ratio of two measures, has the same empirical meaning. Ratio and average
are giving meaning to the values.
EXAMPLE Effort (time) spent to change, Buffer size, Number of detected faults
Absolute scale - The purpose of absolute scale type measures is to measure a proportion between
measures with the same units. Absolute scale quality measure elements include measures resulting from
dividing one ratio scale measure to another ratio scale measure where the measurement unit is the same in
both cases.
EXAMPLE Number of comment lines divided by total lines of code
6.3.2 Measurement focus in software product quality lifecycle
NOTE ISO/IEC 25020 defines the relationship between the software product quality lifecycle and the quality
measures. A short summary of these relationships is also described in clause 6.1 Quality measure elements in the
Software Product Quality Measurement Reference Model (SPQM-RM).
Internal software quality measure - The purpose of internal software quality measures is to measure
internal attributes of a software product. Internal attributes are those that can be measured in terms of the
software product itself, i.e., separate from its behaviour. Internal software quality measures provide the users
with the ability to measure the internal quality characteristics of the intermediate deliverables and thereby
manage the development of the product to achieve the specified quality requirements. The quality measure
element can be measured only if the user has access to the software product structure (source code, design
specifications etc.).
NOTE Internal software quality is defined in ISO/IEC 25000
EXAMPLE The number of lines of code, complexity measures and the number of faults found in a walk through are
all internal software quality measures made on the product itself.
External software quality measure - The purpose of external software quality measures is to measure
external attributes of a software product. External attributes are those that can be measured only with regard
to how the product relates to its environment, i.e., by observing its behaviour in the system environment in
which it is intended to operate, or in a similar environment. External software quality measures provide users,
evaluators, testers, and developers with the insight into the quality of the software product by collecting
measures during the execution of the software product. The value of a quality measure element of that type is
influenced either by the attributes of the product itself or by the attributes of the process of using it.
NOTE External software quality is defined in ISO/IEC 25000
EXAMPLE The number of failures found during testing – it could be influenced by quality of the software product but
also by the testing process.
Quality in use measure - The purpose of quality in use measures is to measure quality in use attributes.
Quality in use attributes are those that can be measured when executing the software product in a specified
© ISO/IEC 2007 – All rights reserved 7

context of use. Quality in use measures provides users with the benefit that they are able to evaluate the
results of using the software product rather than the properties of the software product itself. The value of the
quality measure element is influenced by the context of use, which includes skills of the user who is using the
software product. The context of use has to be stated while presenting the measurement results.
NOTE Quality in use (measure) is defined in ISO/IEC 25000
EXAMPLE Number of times that specific software product functions/applications/systems are used to fulfil a specific
user task, expert user’s task efficiency
6.3.3 Measurement method type
Considerations related to the measurement method type with regard to the quality measure element used for
measurement are:
Subjective measurement method - Subjective measurements are those where quantification is influenced
by human judgment. Subjective measures are used when no formal objective procedures of measurement
can be applied. The value of the quality measure element is influenced by human judgment as an evaluator.
Therefore it is necessary to interpret the results with respect to the number of evaluators and statistical
methods used for the measurement result calculation. Both should be stated while presenting the
measurement results.
EXAMPLE Number of times the user pauses for a long period, expert user’s task efficiency
Objective measurement method - Objective measurements provide quantification that is based on
numerical rules for selection, such as counting. The application of these rules may be implemented via human
or automated means. When using this type of measures, no additional considerations are necessary.
EXAMPLE Number of lines of code, total number of tasks tested
6.4 Table format of quality measure elements
The following information is given in this Technical Report for each quality measure element in the form of a
table:
a) QME category name: Name of quality measure element category, measured by this QME,
b) QME name: Name of quality measure element,
c) QME ID: Unique Identification of quality measure element,
d) Detail: Detail specification of measurement method,
e) Input: Source of data which could be used to measure the quality measure element,
f) Documentation: Documentation which should be prepared in order to measure the quality measure
element. This is also used while interpreting the measurement for proper justification of measurement
results (e.g. for justification of type of functions counted),
g) Considerations connected with the quality measure element use:
⎯ Measurement concept,
⎯ Measurement scale,
⎯ Measurement focus in Software Product Quality Lifecycle,
⎯ Measurement method type.
NOTE A more detailed explanation is given in clause 6.2 Quality measure elements and their categories
8 © ISO/IEC 2007 – All rights reserved

h) Used for: Specifies the relevant internal, external, or quality in use characteristics, subcharacteristics and
measure where a given QME is used.
NOTE Mapping of QME or QME Category to specific quality characteristics or subcharacteristics is not provided in
this table; they are in special annexes of this TR (Error! Reference source not found. and 0).
7 Quality measure element categories
Quality measure element categories are listed in the following table:
Table 2 — Quality measure element categories
Quality measure element Description
category name
Data size The number of records of the same structure, class or format that
satisfy the conditions given in the relevant QME definitions. This
number of records can be expressed as:
⎯ Count of records,
⎯ Size in bytes.
Number of Data Items The count of different types of structures, classes, or formats of
data that satisfy the condition given in the relevant QME
definitions.
Number of Failures The count of all failures which occur in a given time span and
which also satisfy the condition given in the relevant QME
definitions.
EXAMPLE # of expected failures, # of detected failures, # of
resolved failures, # of failures of a given severity level.
Number of Faults The count of software product faults detected (or estimated) in a
given software product component and satisfy the condition given
in the relevant QME definitions.
NOTE # fault of a given category, # faults of a given severity, # faults
successfully corrected, etc.
Number of Functions The count of all the functions that satisfy the condition given in the
relevant QME definitions.
NOTE Functions can be for example, required, implemented, tested,
essential, optional, or any combination of these and more.
Number of I/O Events
The count of the number of I/O events that satisfy the condition
given in the relevant QME definitions.
NOTE We can distinguish between messages from the system to the
observer
- Interaction between observer and the system e.g. a dialogue,
- Transaction: the sequence of interactions between the observer and
system which must be executed atomically to accomplish an operation,
e.g. a wizard (with options).
© ISO/IEC 2007 – All rights reserved 9

Quality measure element Description
category name
Number of Requirements The count of requirement clauses that satisfy the condition given
in the relevant QME definitions.
NOTE Requirements can be i.e. essential, optional, validated, or any
combination of these and more.
Number of Restarts The count of the number of attempts for a system to resume
computation after a critical failure, and satisfies the condition given
in the relevant QME definitions. We distinguish between System
restart Recover
Number of System Operations The count of complete operations performed by the system that
satisfy the condition given in the relevant QME definitions.
NOTE We count the number of complete operations, not the
individual steps required in each operation.
Number of Tasks A Task is the set or sequence of activities required to achieve a
given goal. The number of tasks is the count of tasks that satisfy
the condition given in the relevant QME definitions. We
distinguish between
⎯ User tasks: activities performed by the user (using software
product) towards a specified goal,
⎯ System tasks: activities performed by the system to support
user tasks.
Number of Test Cases Refers to the count of different test input data and scenarios that
satisfy the condition given in the relevant QME definitions,
e.g.  test cases designed, required, executed (successfully or
failed), etc.
Number of Trials
The count of attempts to perform the same operation to satisfy the
condition given in the relevant QME definitions. These attempts
can be
⎯ Evaluation: iterations with same input and same scenario (e.g.
stress testing),
⎯ Cases: iterations with different input and/or different
scenarios.
Number of User Operations The count of number of operations performed by the user that
satisfy the condition given in the relevant QME definitions, where
an operation is a sequence of steps required to perform a task.
10 © ISO/IEC 2007 – All rights reserved

Quality measure element Description
category name
Product Size The count of software product components according to a desired
criterion. This can be lines of code (LOC), function points (see
note), modules, classes, or visual structures such as diagrams or
their parts.
NOTE 1 Software product components can be counted only if some
additional properties are satisfied e.g. only executable lines of code, lines
of code which also contain commenting, only comment lines, declaration
or type casting, bracket/braces only, etc.
NOTE 2 There are four ISO/IEC standards defining the function point
product size.
Time Duration Refers to the interval between a starting time and an end time of
any process described in the relevant QME definitions (Time
duration = end time – start time). We distinguish between
⎯ Execution Time: Refers to time measured internally by the
computer clock, e.g. CPU time, I/O time, etc., or any time
measured via inserted code or software tools (e.g. test
suites),
⎯ Observation time: Refers to time measured externally by the
observer, using an external clock, e.g. time to finish a
transaction or user task,
⎯ Set time: A fixed time process or observation, but meaningful
for a measure evaluation that is independent of action, e.g.
required response time.
8 Set of quality measure elements
8.1 Data Size
8.1.1 Number of change log data actually recorded
QME Category Data Size
QME Name Number of change log data actually recorded
QME ID QME0101
Detail Count the number of change log data that have been actually recorded (as internal
files, as database records) during the software product change.
Input Maintenance report
Documentation
Measurement scale Ratio
Measurement focus External
© ISO/IEC 2007 – All rights reserved 11

Measurement method Objective
Used for Software change control capability

8.1.2 Number of change log data planned to be recorded as sufficient to trace software changes
QME Category Data Size
QME Name Number of change log data planned to be recorded as sufficient to trace software
changes
QME ID QME0102
Detail Derive from specification of required change log data items to be recorded.
Input REQ Spec.
Documentation
Measurement scale Ratio
Measurement focus External
Measurement method Objective
Used for Software change control capability

8.1.3 Number of data actually recorded during operation
QME Category Data Size
QME Name Number of data actually recorded during operation
QME ID QME0103
Detail Document the data size that has been recorded during operation ite
...