ISO/IEC 25022:2016
(Main)Systems and software engineering — Systems and software quality requirements and evaluation (SQuaRE) — Measurement of quality in use
Systems and software engineering — Systems and software quality requirements and evaluation (SQuaRE) — Measurement of quality in use
ISO/IEC 25022:2016 defines quality in use measures for the characteristics defined in ISO/IEC 25010, and is intended to be used together with ISO/IEC 25010. It can be used in conjunction with the ISO/IEC 2503n and the ISO/IEC 2504n standards or to more generally meet user needs with regard to product or system quality. ISO/IEC 25022:2016 contains the following: - a basic set of measures for each quality in use characteristic; - an explanation of how quality in use is measured. It provides a suggested set of quality in use measures to be used with the quality in use model in ISO/IEC 25010. They are not intended to be an exhaustive set. It includes as informative annexes examples of how to measure context coverage (Annex A), options for normalising quality in use measures (Annex B), use of ISO/IEC 25022 for measuring usability in ISO 9241‑11 (Annex C), a quality in use evaluation process (Annex D), the relationship between different quality models (Annex E), and quality measurement concepts (Annex F). The measures are applicable to the use of any human-computer system, including both computer systems in use and software products that form part of the system. It does not assign ranges of values of the measures to rated levels or to grades of compliance because these values are defined for each system or product depending, on the context of use and users' needs. Some attributes could have a desirable range of values, which does not depend on specific user needs but depends on generic factors, for example, human cognitive factors. The proposed quality in use measures are primarily intended to be used for quality assurance and management of systems and software products based on their effects when actually used. The main users of the measurement results are people managing development, acquisition, evaluation, or maintenance of software and systems. The main users of ISO/IEC 25022:2016 are people carrying out specification and evaluation activities as part of the following: - development: including requirements analysis, design, and testing through acceptance during the life cycle process; - quality management: systematic examination of the product or computer system, for example, when evaluating quality in use as part of quality assurance and quality control; - supply: a contract with the acquirer for the supply of a system, software product, or software service under the terms of a contract, for example, when validating quality at qualification test; - acquisition: including product selection and acceptance testing, when acquiring or procuring a system, software product, or software service from a supplier; - maintenance: improvement of the product based on quality in use measures.
Ingénierie des systèmes et du logiciel — Exigences de qualité et évaluation des systèmes et du logiciel (SQuaRE) — Mesurage de la qualité lors de l'utilisation
General Information
Relations
Buy Standard
Standards Content (Sample)
INTERNATIONAL ISO/IEC
STANDARD 25022
First edition
2016-06-15
Systems and software engineering —
Systems and software quality
requirements and evaluation (SQuaRE)
— Measurement of quality in use
Ingénierie des systèmes et du logiciel — Exigences de qualité et
évaluation des systèmes et du logiciel (SQuaRE) — Mesurage de la
qualité lors de l’utilisation
Reference number
ISO/IEC 25022:2016(E)
©
ISO/IEC 2016
---------------------- Page: 1 ----------------------
ISO/IEC 25022:2016(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2016, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. +41 22 749 01 11
Fax +41 22 749 09 47
copyright@iso.org
www.iso.org
ii © ISO/IEC 2016 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC 25022:2016(E)
Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Conformance . 2
3 Normative references . 2
4 Terms and definitions . 2
5 Abbreviated terms . 6
6 Use of quality in use measures. 6
6.1 Applications of quality in use . 6
6.2 Measurement of quality in use . 7
6.3 Interpretation of quality in use measures . 8
6.4 Selecting appropriate quality in use measures . 9
6.5 Aspects of quality in use described in other International Standards . 9
7 Format used for documenting the quality measures . 9
8 Quality in use measures .10
8.1 General .10
8.2 Effectiveness measures .10
8.3 Efficiency measures .12
8.4 Satisfaction measures .13
8.4.1 General.13
8.4.2 Usefulness measures .14
8.4.3 Trust measures .15
8.4.4 (User experience) pleasure measures .15
8.4.5 (Ergonomic) comfort measures .16
8.5 Freedom from risk measures .16
8.5.1 General.16
8.5.2 Economic risk mitigation measures .18
8.5.3 Health and safety risk mitigation measures .19
8.5.4 Environmental risk mitigation measures .20
8.6 Context coverage measures .21
8.6.1 General.21
8.6.2 Context completeness measures .21
8.6.3 Flexibility measures .21
Annex A (informative) Examples of how to measure context coverage.23
Annex B (informative) Normalization of quality in use measures .25
Annex C (informative) Use of ISO/IEC 25022 for measuring usability in ISO 9241‑11 .30
Annex D (informative) Quality in use evaluation process .31
Annex E (informative) Relationship between different quality models .37
Annex F (informative) Quality measurement concepts .38
Annex G (informative) QMEs used to define quality measures .39
Bibliography .40
© ISO/IEC 2016 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC 25022:2016(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the meaning of ISO specific terms and expressions related to conformity
assessment, as well as information about ISO’s adherence to the WTO principles in the Technical
Barriers to Trade (TBT) see the following URL: Foreword - Supplementary information
The committee responsible for this document is ISO/IEC JTC 1, Information technology, SC 7, Software
and systems engineering.
This first edition of ISO/IEC 25022, which is a part of the SQuaRE series of standards, cancels and
replaces ISO/IEC 9126-4:2004, with the following changes:
— measures are given for the revised quality model for quality in use in ISO/IEC 25010;
— measures are categorized as generally applicable, could be used in a wide range of situations, or
specialized for specific needs;
— annexes that were common to ISO/IEC 9126-2, ISO/IEC 9126-3, and ISO/IEC 9126-4 have been
removed (and might be included in a future revision of ISO/IEC 25020).
The SQuaRE series of standards consists of the following divisions under the general title Systems and
software quality requirements and evaluation:
— ISO/IEC 2500n — Quality Management Division
— ISO/IEC 2501n — Quality Model Division
— ISO/IEC 2502n — Quality Measurement Division
— ISO/IEC 2503n — Quality Requirements Division
— ISO/IEC 2504n — Quality Evaluation Division
— ISO/IEC 25050 — 25099 SQuaRE Extension Division
Annexes A, B, C, D, E, F and G are for information only.
iv © ISO/IEC 2016 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC 25022:2016(E)
Introduction
This International Standard is a part of the SQuaRE series of International Standards. It provides a
set of measures for the characteristics of quality in use (defined in ISO/IEC 25010) that can be used
for specifying quality in use requirements (in conjunction with ISO/IEC 25030) and measuring and
evaluating quality in use (in conjunction with ISO/IEC 25040 and ISO/IEC 25041).
The quality measures included in this International Standard were selected based on their practical
value. They are based on established practice (including, for example, Reference [17]). They are not
intended to be exhaustive, and users of this International Standard are encouraged to refine them, if
necessary.
Quality Measurement Division
This International Standard is a part of ISO/IEC 2502n Quality Measurement Division of SQuaRE series
that currently consists of the following International Standards:
— ISO/IEC 25020 — Measurement reference model and guide: provides a reference model and guide
for measuring the quality characteristics defined in ISO/IEC 2501n Quality Model Division;
— ISO/IEC 25021 — Quality measure elements: provides a format for specifying Quality Measure
Elements and some examples of QMEs that can be used to construct software quality measures;
— ISO/IEC 25022 — Measurement of quality in use: provides measures, including associated
measurement functions for the quality characteristics in the quality in use model;
— ISO/IEC 25023 — Measurement of system and software product quality: provides measures,
including associated measurement functions and QMEs for the quality characteristics in the product
quality model;
— ISO/IEC 25024 — Measurement of data quality: provides measures, including associated
measurement functions and QMEs for the quality characteristics in the data quality model.
Figure 1 depicts the relationship between this International Standard and the other standards in the
ISO/IEC 2502n division.
© ISO/IEC 2016 – All rights reserved v
---------------------- Page: 5 ----------------------
ISO/IEC 25022:2016(E)
ISO/IEC 25020:
Measurement reference
model and guide
25020 gives guide for developing and
specifying Quality Measures
ISO/IEC 25022: ISO/IEC 25023:
Measurement of system
Measurement of ISO/IEC 25024:
ISO/IEC 25025:
and software product quality
quality in use
Measurement of Measurement of
-Revision
-Revision of 9126-4
data quality IS Service quality
of ISO/IEC 9126-2 & 9126-3
ISO/IEC 9126-4
ISO/IEC 25021 is used for developing and
specifying
ISO/IEC 25021:
Quality measure
Revision of
Figure 1 — Structure of the Quality Measurement Division
Outline and Organization of SQuaRE Series
The SQuaRE series consists of five main divisions and extension division. The outline of each divisions
within the SQuaRE series are as follows.
— ISO/IEC 2500n — Quality Management Division. The standards that form this division define
all common models, terms, and definitions referred further by all other standards from SQuaRE
series. The division also provides requirements and guidance for the planning and management of
a project.
— ISO/IEC 2501n — Quality Model Division. The standards that form this division provide quality
models for system/software products, quality in use, and data. An IT service quality model is under
development. Practical guidance on the use of the quality model is also provided.
— ISO/IEC 2502n — Quality Measurement Division. The standards that form this division include a
system/software product quality measurement reference model, definitions of quality measures,
and practical guidance for their application. This division presents internal measures of software
quality, external measures of software quality, and quality in use measures. Quality measure
elements forming foundations for the quality measures are defined and presented.
— ISO/IEC 2503n — Quality Requirements Division. The standard that forms this division helps
specifying quality requirements. These quality requirements can be used in the process of quality
requirements elicitation for a system/software product to be developed, designing a process for
achieving necessary quality, or as inputs for an evaluation process.
— ISO/IEC 2504n — Quality Evaluation Division. The standards that form this division provide
requirements, recommendations, and guidelines for system/software product evaluation, whether
performed by independent evaluators, acquirers, or developers. The support for documenting a
measure as an Evaluation Module is also presented.
ISO/IEC 25050 to ISO/IEC 25099 are reserved for SQuaRE extension International Standards, which
currently include ISO/IEC 25051 and the ISO/IEC 25060 to ISO/IEC 25069.
vi © ISO/IEC 2016 – All rights reserved
---------------------- Page: 6 ----------------------
INTERNATIONAL STANDARD ISO/IEC 25022:2016(E)
Systems and software engineering — Systems and
software quality requirements and evaluation (SQuaRE) —
Measurement of quality in use
1 Scope
This International Standard defines quality in use measures for the characteristics defined in
ISO/IEC 25010, and is intended to be used together with ISO/IEC 25010. It can be used in conjunction
with the ISO/IEC 2503n and the ISO/IEC 2504n standards or to more generally meet user needs with
regard to product or system quality.
This International Standard contains the following:
— a basic set of measures for each quality in use characteristic;
— an explanation of how quality in use is measured.
This International Standard provides a suggested set of quality in use measures to be used with the
quality in use model in ISO/IEC 25010. They are not intended to be an exhaustive set.
It includes as informative annexes examples of how to measure context coverage (Annex A), options
for normalising quality in use measures (Annex B), use of ISO/IEC 25022 for measuring usability in
ISO 9241-11 (Annex C), a quality in use evaluation process (Annex D), the relationship between different
quality models (Annex E), and quality measurement concepts (Annex F).
The measures are applicable to the use of any human-computer system, including both computer
systems in use and software products that form part of the system.
This International Standard does not assign ranges of values of the measures to rated levels or to grades
of compliance because these values are defined for each system or product depending, on the context of
use and users’ needs.
Some attributes could have a desirable range of values, which does not depend on specific user needs
but depends on generic factors, for example, human cognitive factors.
The proposed quality in use measures are primarily intended to be used for quality assurance and
management of systems and software products based on their effects when actually used. The main
users of the measurement results are people managing development, acquisition, evaluation, or
maintenance of software and systems.
The main users of this International Standard are people carrying out specification and evaluation
activities as part of the following:
— development: including requirements analysis, design, and testing through acceptance during the
life cycle process;
— quality management: systematic examination of the product or computer system, for example, when
evaluating quality in use as part of quality assurance and quality control;
— supply: a contract with the acquirer for the supply of a system, software product, or software service
under the terms of a contract, for example, when validating quality at qualification test;
— acquisition: including product selection and acceptance testing, when acquiring or procuring a
system, software product, or software service from a supplier;
— maintenance: improvement of the product based on quality in use measures.
© ISO/IEC 2016 – All rights reserved 1
---------------------- Page: 7 ----------------------
ISO/IEC 25022:2016(E)
2 Conformance
Any quality requirement specification or quality evaluation that conforms to this International
Standard shall:
a) select the quality in use characteristics and/or subcharacteristics to be specified or evaluated as
defined in ISO/IEC 25010;
b) for each selected characteristic or subcharacteristic, all the General (G) quality measures defined
in Clause 8 should be used. If any are excluded, then provide a rationale;
c) select any Special (S) quality measures that are relevant;
d) if any quality measure is modified, provide the rationale for any changes;
e) define any additional quality measures to be used that are not included in this International
Standard;
f) define precisely how each quality measure is operationalized (for example details of the
measurement method or questionnaire used).
NOTE It is important to use the same measurement method when making comparisons.
3 Normative references
The following documents, in whole or in part, are normatively referenced in this document and are
indispensable for its application. For dated references, only the edition cited applies. For undated
references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 25010, Systems and software engineering — Systems and software Quality Requirements and
Evaluation (SQuaRE) — System and software quality models
4 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC 25000 and ISO/IEC 25010
and the following apply.
NOTE The essential definitions from ISO/IEC 25000 SQuaRE series and the other ISO standards are
reproduced here.
4.1
context completeness
degree to which a product or system can be used with the required levels of effectiveness, efficiency,
satisfaction, and freedom from risk in each of the specified contexts of use
Note 1 to entry: Context completeness is a subcharacteristic of context coverage.
[SOURCE: ISO/IEC 25010:2011, modified — Added “the required levels of” and changed “all” to “each of”
for clarification.]
4.2
context coverage
degree to which a product or system can be used with effectiveness, efficiency, satisfaction, and freedom
from risk in both specified contexts of use and in contexts beyond those initially explicitly identified
Note 1 to entry: Context of use is relevant to both quality in use and some product quality (sub)characteristics
(where it is referred to as “specified conditions”).
[SOURCE: ISO/IEC 25010:2011, 4.1.5]
2 © ISO/IEC 2016 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/IEC 25022:2016(E)
4.3
customer
relationship with the supplier of an organization or person that receives or uses a product or service
Note 1 to entry: The relationship can include a warranty or agreeing to the terms and conditions of a service.
4.4
effectiveness
accuracy and completeness with which users achieve specified goals
[SOURCE: ISO 9241-11:1998]
4.5
efficiency
resources expended in relation to the accuracy and completeness with which users achieve goals
Note 1 to entry: Relevant resources can include time to complete the task (human resources), materials, or the
financial cost of usage.
[SOURCE: ISO 9241-11:1998, modified — note 1 to entry added]
4.6
context of use
users, tasks, equipment (hardware, software and materials), and the physical and social environments
in which a system, product or service is used
[SOURCE: ISO 9241-11:1998, 3.5, modified — With “product” replaced by “system, product or service”.]
4.7
flexibility
degree to which a product or system can be used with acceptable levels of effectiveness, efficiency,
freedom from risk, and satisfaction in contexts beyond those initially specified in the requirements
Note 1 to entry: Flexibility is a subcharacteristic of context coverage.
[SOURCE: ISO/IEC 25010:2011, modified — Added “acceptable levels of” for clarification.]
4.8
formative evaluation
evaluation designed and used to improve the object of evaluation, especially when it is still being
developed
[SOURCE: ISO/TS 18152:2010, 4.6]
4.9
freedom from risk
degree to which the quality of a product or system mitigates or avoids potential risks to economic
status, human life, health, or the environment
Note 1 to entry: Risk is a function of the probability of occurrence of a given threat and the potential adverse
consequences of that threat’s occurrence.
Note 2 to entry: The risks considered by the SQuaRE series are those arising from insufficient product quality.
Note 3 to entry: Freedom from risk includes reduction of potential risks to the user, organisation or project.
[SOURCE: ISO/IEC 25010:2011, modified — Added “quality of” and “or avoids” for clarification.]
4.10
goal
intended outcome
[SOURCE: ISO 9241-11:1998]
© ISO/IEC 2016 – All rights reserved 3
---------------------- Page: 9 ----------------------
ISO/IEC 25022:2016(E)
4.11
measure (noun)
variable to which a value is assigned as the result of measurement
Note 1 to entry: The term “measures” is used to refer collectively to base measures, derived measures, and
indicators.
Note 2 to entry: In this International Standard, when the word “measure” is used qualified by a characteristic or
subcharacteristic, it refers to a quality measure.
[SOURCE: ISO/IEC 15939:2007, modified — note 2 to entry added]
4.12
measurement
set of operations having the object of determining a value of a measure
Note 1 to entry: Measurement can include assigning a qualitative category such as the language of a source
program (ADA, C, COBOL, etc.).
[SOURCE: ISO/IEC 15939:2007, modified — note 1 to entry modified from original]
4.13
measurement function
algorithm or calculation performed to combine two or more quality measurement elements
[SOURCE: ISO/IEC 25021:2012]
4.14
psychometrics
field of study concerned with the theory and technique for developing valid and reliable psychological
measures
4.15
quality in use
degree to which a product or system can be used by specific users to meet their needs to achieve specific
goals with effectiveness, efficiency, satisfaction, and freedom from risk in specific contexts of use
Note 1 to entry: The quality in use of a software product or system can be measured and evaluated by the effect
of the target system or software products when used by users of the implemented system or during field testing
or prototype testing.
Note 2 to entry: When quality in use is specified, it relates to specified users meeting their needs to achieve
specified goals with effectiveness, efficiency, satisfaction, and freedom from risk in specified contexts of use.
[SOURCE: ISO/IEC 25010:2011, modified — notes 1 and 2 to entry added]
4.16
quality measure
measure that is defined as a measurement function of two or more values of quality measure elements
[SOURCE: ISO/IEC 25021:2012]
4.17
quality measure element
measure defined in terms of a property and the measurement method for quantifying it, including
optionally the transformation by a mathematical function
[SOURCE: ISO/IEC 25021:2012]
4 © ISO/IEC 2016 – All rights reserved
---------------------- Page: 10 ----------------------
ISO/IEC 25022:2016(E)
4.18
quality model
defined set of characteristics, and of relationships between them, which provides a framework for
specifying quality requirements and evaluating quality
[SOURCE: ISO/IEC 25000:2014]
4.19
satisfaction
degree to which user needs are satisfied when a product or system is used in a specified context of use
Note 1 to entry: For a user who does not directly interact with the product or system, only purpose accomplishment
and trust are relevant.
Note 2 to entry: Satisfaction is the user’s response to interaction with the product or system, and includes
attitudes towards use of the product.
Note 3 to entry: Users include: primary users who interact with the system to achieve the primary goals,
secondary users who provide support, and indirect users who receive output, but do not interact with the system.
Note 4 to entry: In this International Standard, user’s needs include their desires and expectations associated
with use of a product, system, or service. Exceeding desires and expectations is a means of significantly
increasing satisfaction and improving the user experience.
[SOURCE: ISO/IEC 25010:2011, modified — notes 3 and 4 to entry added]
4.20
stakeholder satisfaction
degree to which stakeholder needs are satisfied when a product or system is used in a specified
context of use
Note 1 to entry: Users of a product or system are one type of stakeholder, so user satisfaction is one type of
stakeholder satisfaction.
[SOURCE: ISO/IEC 25010:2011, modified — Definition for the term “satisfaction” modified to refer to
stakeholders.]
4.21
summative evaluation
evaluation designed to present conclusions about the merit or worth of the object of evaluation
Note 1 to entry: The results can be used to produce recommendations about whether it should be retained,
altered, or eliminated.
Note 2 to entry: It is possible to design a method to provide a combined formative and summative evaluation.
Note 3 to entry: A summative test method is used to perform a summative evaluation.
[SOURCE: ISO/TS 20282-2:2013, 4.17]
4.22
system
combination of interacting elements organised to achieve one or more stated purposes
Note 1 to entry: A system may be considered as a product or as the services it provides.
Note 2 to entry: In practice, the interpretation of its meaning is frequently clarified by the use of an associative
noun, e.g. aircraft system. Alternatively, the word system may be substituted simply by a context dependent
synonym, e.g. aircraft, though this may then obscure a system principles perspective.
[SOURCE: ISO/IEC 15288:2015, modified — note 3 to entry deleted.]
© ISO/IEC 2016 – All rights rese
...
DRAFT INTERNATIONAL STANDARD
ISO/IEC DIS 25022.2
ISO/IEC JTC 1/SC 7 Secretariat: SCC
Voting begins on: Voting terminates on:
2015-08-03 2015-10-03
Systems and software engineering — Systems and
software quality requirements and evaluation (SQuaRE) —
Measurement of quality in use
Titre manque
ICS: 35.080
THIS DOCUMENT IS A DRAFT CIRCULATED
FOR COMMENT AND APPROVAL. IT IS
THEREFORE SUBJECT TO CHANGE AND MAY
NOT BE REFERRED TO AS AN INTERNATIONAL
STANDARD UNTIL PUBLISHED AS SUCH.
IN ADDITION TO THEIR EVALUATION AS
BEING ACCEPTABLE FOR INDUSTRIAL,
TECHNOLOGICAL, COMMERCIAL AND
USER PURPOSES, DRAFT INTERNATIONAL
STANDARDS MAY ON OCCASION HAVE TO
BE CONSIDERED IN THE LIGHT OF THEIR
POTENTIAL TO BECOME STANDARDS TO
WHICH REFERENCE MAY BE MADE IN
Reference number
NATIONAL REGULATIONS.
ISO/IEC DIS 25022.2:2015(E)
RECIPIENTS OF THIS DRAFT ARE INVITED
TO SUBMIT, WITH THEIR COMMENTS,
NOTIFICATION OF ANY RELEVANT PATENT
RIGHTS OF WHICH THEY ARE AWARE AND TO
©
PROVIDE SUPPORTING DOCUMENTATION. ISO/IEC 2015
---------------------- Page: 1 ----------------------
ISO/IEC DIS 25022.2:2015(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2015, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. +41 22 749 01 11
Fax +41 22 749 09 47
copyright@iso.org
www.iso.org
ii © ISO/IEC 2015 – All rights reserved
---------------------- Page: 2 ----------------------
Contents
1 Scope . 8
2 Conformance . 9
3 Normative references . 9
4 Terms and definitions . 9
5 Abbreviated terms .13
6 Use of quality in use measures .13
6.1 Applications of quality in use .13
6.2 Measurement of quality in use .13
6.3 Normalising measures .14
6.4 Selecting appropriate quality in use measures .14
6.5 Aspects of quality in use described in other International Standards .15
7 Format used for documenting the quality measures.15
8 Quality in use measures .16
8.1 General .16
8.2 Effectiveness measures.16
8.3 Efficiency measures .17
8.4 Satisfaction measures .18
8.4.1 General .18
8.4.2 Usefulness measures .19
8.4.3 Trust measures .20
8.4.4 Pleasure measures .20
8.4.5 Comfort measures .20
8.5 Freedom from risk measures .20
8.5.1 General .20
8.5.2 Economic risk mitigation measures .21
8.5.3 Health and safety risk mitigation measures .22
8.5.4 Environmental risk mitigation measures .23
8.6 Context coverage measures .23
8.6.1 General .23
8.6.2 Context completeness measures .23
8.6.3 Flexibility measures .24
Annex A (Informative) Examples of how to measure context coverage .25
Annex B (Informative) Normalisation of quality in use measures .27
Annex C (Informative) Use of ISO/IEC 25022 for measuring usability in ISO 9241-11 .31
Annex D (Informative) Quality in use evaluation process .32
D.1 Establish evaluation requirements .32
D.2 Specify the evaluation .32
D.3 Design the evaluation .35
D.4 Execute the evaluation .35
Annex E (Informative) Relationship between different quality models .36
Annex F (Informative) Quality measurement concepts .37
Annex G (Informative) QMEs used to define quality measures .38
Bibliography .39
© ISO/IEC 2015 – All rights reserved
4
---------------------- Page: 3 ----------------------
ISO/IEC DIS 25022.2(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical Commission)
form the specialized system for worldwide standardization. National bodies that are members of ISO or IEC
participate in the development of International Standards through technical committees established by the
respective organization to deal with particular fields of technical activity. ISO and IEC technical committees
collaborate in fields of mutual interest. Other international organizations, governmental and non-governmental, in
liaison with ISO and IEC, also take part in the work. In the field of information technology, ISO and IEC have
established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International Standards
adopted by the joint technical committee are circulated to national bodies for voting. Publication as an International
Standard requires approval by at least 75 % of the national bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights.
ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC 25022, which replaces ISO/IEC 9126-4, is a part of the SQuaRE series of standards and was prepared by
Joint Technical Committee ISO/IEC JTC 1, information technology, Subcommittee SC 7, Software and Systems
Engineering
The SQuaRE series of standards consists of the following divisions under the general title Systems and software
Quality Requirements and Evaluation:
ISO/IEC 2500n - Quality Management Division,
ISO/IEC 2501n - Quality Model Division,
ISO/IEC 2502n - Quality Measurement Division,
ISO/IEC 2503n - Quality Requirements Division,
ISO/IEC 2504n - Quality Evaluation Division,
ISO/IEC 25050 - 25099 SQuaRE Extension Division.
Annexes A, B, C, D, E, F and G are for information only.
5
© ISO/IEC 2015 – All rights reserved 5
---------------------- Page: 4 ----------------------
Introduction
This International Standard is a part of the SQuaRE series of international standards. It provides a set of measures
for the characteristics of quality in use (defined in ISO/IEC 25010) that can be used for specifying quality in use
requirements (in conjunction with ISO/IEC 25030) and measuring and evaluating quality in use (in conjunction with
ISO/IEC 25040 and ISO/IEC 25041).
The quality measures included in this International Standard were selected based on their practical value. They are
based on established practice (including for example [4] in the Bibliography), They are not intended to be
exhaustive, and users of this International Standard are encouraged to refine them if necessary.
This International Standard replaces ISO/IEC 9126-4, and has the following changes:
Measures are given for the revised quality model for quality in use in ISO/IEC 25010.
Measures are categorised as generally applicable, could be used in a wide range of situations or
specialised for specific needs.
Annexes that were common to ISO/IEC 9126-2, -3 and -4 have been removed (and might be included in a
future revision of ISO/IEC 25020).
Quality Measurement Division
This International Standard is a part of ISO/IEC 2502n Quality Measurement Division of SQuaRE series that
currently consists of the following International Standards:
ISO/IEC 25020 – Measurement reference model and guide: provides a reference model and guide for
measuring the quality characteristics defined in ISO/IEC 2501n Quality Model Division.
ISO/IEC 25021 – Quality measure elements: provides a format for specifying Quality Measure Elements
and some examples of QMEs that can be used to construct software quality measures.
ISO/IEC 25022 – Measurement of quality in use: provides measures, including associated measurement
functions for the quality characteristics in the quality in use model.
ISO/IEC 25023 – Measurement of system and software product quality: provides measures, including
associated measurement functions and QMEs for the quality characteristics in the product quality model.
ISO/IEC 25024 – Measurement of data quality: provides measures, including associated measurement
functions and QMEs for the quality characteristics in the data quality model.
Figure 1 depicts the relationship between this International Standard and the other standards in ISO/IEC 2502n
division.
© ISO/IEC 2015 – All rights reserved
6
---------------------- Page: 5 ----------------------
ISO/IEC DIS 25022.2(E)
Figure 1 - Structure of the Quality Measurement division
Outline and Organization of SQuaRE Series
The SQuaRE series consists of five main divisions and extension division. Outline of each divisions within SQuaRE
series are as follows.
ISO/IEC 2500n - Quality Management Division. The standards that form this division define all common
models, terms and definitions referred further by all other standards from SQuaRE series. The division also
provides requirements and guidance for the planning and management of a project.
ISO/IEC 2501n - Quality Model Division. The standards that forms this division provides quality models for
system/software products, quality in use and data. A service quality model is under development. Practical
guidance on the use of the quality model is also provided.
ISO/IEC 2502n - Quality Measurement Division. The standards that form this division include a
system/software product quality measurement reference model, definitions of quality measures, and
practical guidance for their application. This division presents internal measures of software quality,
external measures of software quality and quality in use measures. Quality measure elements forming
foundations for the quality measures are defined and presented.
ISO/IEC 2503n - Quality Requirements Division. The standard that forms this division helps specifying
quality requirements. These quality requirements can be used in the process of quality requirements
elicitation for a system/software product to be developed, designing a process for achieving necessary
quality, or as inputs for an evaluation process.
ISO/IEC 2504n - Quality Evaluation Division. The standards that form this division provide requirements,
recommendations and guidelines for system/software product evaluation, whether performed by
independent evaluators, acquirers or developers. The support for documenting a measure as an Evaluation
Module is also presented.
ISO/IEC 25050 to ISO/IEC 25099 are reserved for SQuaRE extension International Standards, which currently
include 25051 Requirements for quality of Ready to Use Software Products (RUSP) and instructions for testing,
and the ISO/IEC 25060 – 25069 Common industry format for usability series of standards.
7
© ISO/IEC 2015 – All rights reserved 7
---------------------- Page: 6 ----------------------
Systems and software engineering – Systems and software Quality
Requirements and Evaluation (SQuaRE) – Measurement of quality in use
1 Scope
This International Standard defines quality in use measures for the characteristics defined in ISO/IEC 25010, and is
intended to be used together with ISO/IEC 25010. It can be used in conjunction with the ISO/IEC 2503n – Quality
requirements division and the ISO/IEC 2504n – Quality evaluation division standards or to more generally meet
user needs with regard to product or system quality.
This International Standard contains:
a basic set of measures for each quality in use characteristic;
an explanation of how quality in use is measured.
This International Standard provides a suggested set of quality in use measures to be used with the quality in use
model in ISO/IEC 25010. They are not intended to be an exhaustive set.
It includes as informative annexes examples of how to measure context coverage (Annex A), options for
normalising quality in use measures (Annex B), Use of ISO/IEC 25022 for measuring usability in ISO 9241-11,
(Annex C), a quality in use evaluation process (Annex D), the relationship between different quality models (Annex
E) and quality measurement concepts (Annex F).
The measures are applicable to the use of any human-computer system, including both computer systems in use
and software products that form part of the system.
This International Standard does not assign ranges of values of the measures to rated levels or to grades of
compliance, because these values are defined for each system, product depending on the context of use and
users' needs.
Some attributes could have a desirable range of values, which does not depend on specific user needs but
depends on generic factors; for example, human cognitive factors.
The proposed quality in use measures are primarily intended to be used for quality assurance and management of
systems and software products based on their effects when actually used. The main users of the measurement
results are people managing development, acquisition, evaluation or maintenance of software and systems.
The main users of this International Standard are people carrying out specification and evaluation activities as part
of:
Development: including requirements analysis, design, and testing through acceptance during the life cycle
process.
Quality management: systematic examination of the product or computer system, for example when
evaluating quality in use as part of quality assurance and quality control.
Supply: a contract with the acquirer for the supply of a system, software product or software service under
the terms of a contract, for example when validating quality at qualification test.
Acquisition: including product selection and acceptance testing, when acquiring or procuring a system,
software product or software service from a supplier.
Maintenance: improvement of the product based on quality in use measures.
© ISO/IEC 2015 – All rights reserved
8
---------------------- Page: 7 ----------------------
ISO/IEC DIS 25022.2(E)
2 Conformance
Any quality requirement specification or quality evaluation that conforms to this International Standard shall:
a) Select the quality in use characteristics and/or subcharacteristics to be specified or evaluated as defined in
ISO/IEC 25010.
b) For each selected characteristic or subcharacteristic, select all the General (G) quality measures defined in
clause 8 or provide a rationale for any that are excluded.
c) Optionally select any other quality measures that are relevant.
d) If any quality measure is modified, provide the rationale for any changes.
e) Define precisely how each quality measure is operationalised (for example details of the measurement
method or questionnaire used).
f) Define any additional quality measures used that are not included in this International Standard.
3 Normative references
The following documents, in whole or in part, are normatively referenced in this document and are indispensable
for its application. For dated references, only the edition cited applies. For undated references, the latest edition of
the referenced document (including any amendments) applies.
ISO/IEC 25010:2011, Systems and software engineering - Systems and software Quality Requirements and
Evaluation (SQuaRE) - System and software quality models
4 Terms and definitions
For the purposes of this International Standard, the following terms and definitions and those given in ISO/IEC
25000 and in ISO/IEC 25010 apply.
NOTE: The essential definitions from ISO/IEC 25000 SQuaRE series and the other ISO standards are reproduced here.
4.1
context completeness
degree to which a product or system can be used with the required levels of effectiveness, efficiency, satisfaction
and freedom from risk in each of the specified contexts of use
[SOURCE: ISO/IEC 25010:2011 modified – added "the required levels of" and changed "all" to "each of" for
clarification]
Note 1 to entry: Context completeness is a subcharacteristic of context coverage.
4.2
context coverage
degree to which a product or system can be used with effectiveness, efficiency, satisfaction and freedom from risk
in both specified contexts of use and in contexts beyond those initially explicitly identified
Note 1 to entry: Context of use is relevant to both quality in use and some product quality (sub)characteristics (where it is
referred to as “specified conditions”).
[SOURCE: ISO/IEC 25010:2011 4.1.5]
4.3
effectiveness
accuracy and completeness with which users achieve specified goals
[SOURCE: ISO 9241-11:1998]
4.4
efficiency
resources expended in relation to the accuracy and completeness with which users achieve goals
[SOURCE: ISO 9241-11:1998]
9
© ISO/IEC 2015 – All rights reserved 9
---------------------- Page: 8 ----------------------
Note 1 to entry: Relevant resources can include time to complete the task (human resources), materials, or the financial cost of
usage.
4.5
context of use
users, tasks, equipment (hardware, software and materials), and the physical and social environments in which a
system, product or service is used
[SOURCE: ISO 9241-11:1998, 3.5, with "product" replaced by "system, product or service".]
4.6
flexibility
degree to which a product or system can be used with acceptable levels of effectiveness, efficiency, freedom from
risk in contexts beyond those initially specified in the requirements satisfaction and
[SOURCE: ISO/IEC 25010:2011, modified – added "acceptable levels of" for clarification.]
Note 1 to entry: Flexibility is a subcharacteristic of context coverage.
4.7
formative evaluation
evaluation designed and used to improve the object of evaluation, especially when it is still being developed
[SOURCE: ISO/TS 18152, 2010, 4.6]
4.8
freedom from risk
degree to which the quality of a product or system mitigates or avoids potential risks to economic status, human life,
health, or the environment
Note 1 to entry: Risk is a function of the probability of occurrence of a given threat and the potential adverse consequences of
that threat's occurrence.
[SOURCE: ISO/IEC 25010:2011, modified – added "quality of" and "or avoids" for clarification]
Note 2 to entry: The risks considered by the SQuaRE series are those arising from insufficient product quality.
Note 3 to entry: Freedom from risk includes reduction of potential risks to the user, organisation or project.
4.9
goal
intended outcome
[SOURCE: ISO 9241-11:1998]
4.10
measure (noun)
variable to which a value is assigned as the result of measurement
Note 1 to entry: The term “measures” is used to refer collectively to base measures, derived measures, and indicators.
[SOURCE: ISO/IEC 15939:2007]
Note 2 to entry: In this document when the word "measure" is used qualified by a characteristic or subcharacteristic it refers to
a quality measure.
4.11
measurement
set of operations having the object of determining a value of a measure
[SOURCE: ISO/IEC 15939:2007]
Note 1 to entry: Measurement can include assigning a qualitative category such as the language of a source program (ADA, C,
COBOL, etc.).
© ISO/IEC 2015 – All rights reserved
10
---------------------- Page: 9 ----------------------
ISO/IEC DIS 25022.2(E)
4.12
measurement function
algorithm or calculation performed to combine two or more quality measurement elements
[SOURCE: ISO/IEC 25021:2012]
4.13
psychometrics
field of study concerned with the theory and technique for developing valid and reliable psychological measures
4.14
quality in use
degree to which a product or system can be used by specific users to meet their needs to achieve specific goals
with effectiveness, efficiency, satisfaction and freedom from risk in specific contexts of use
[SOURCE: ISO/IEC 25010: 2011]
Note 1 to entry: The quality in use of a software product or system can be measured and evaluated by the effect of the target
system or software products when used by users of the implemented system, or during field testing or prototype testing.
Note 2 to entry: When quality in use is specified it relates to specified users meeting their needs to achieve specified goals with
effectiveness, efficiency, satisfaction and freedom from risk in specified contexts of use.
4.15
quality measure
measure that is defined as a measurement function of two or more values of quality measure elements
[SOURCE: ISO/IEC 25021:2012]
4.16
quality measure element
measure defined in terms of a property and the measurement method for quantifying it, including optionally the
transformation by a mathematical function
[SOURCE: ISO/IEC 25021:2012]
4.17
quality model
defined set of characteristics, and of relationships between them, which provides a framework for specifying quality
requirements and evaluating quality
[SOURCE: ISO/IEC 25000:2014]
4.18
satisfaction
degree to which user needs are satisfied when a product or system is used in a specified context of use
Note 1 to entry: For a user who does not directly interact with the product or system, only purpose accomplishment and trust
are relevant.
Note 2 to entry: Satisfaction is the user’s response to interaction with the product or system, and includes attitudes towards use
of the product.
[SOURCE: ISO/IEC 25010:2011]
Note 3 to entry: Users include: primary users who interacts with the system to achieve the primary goals, secondary users who
provide support and indirect users who receives output, but does not interact with the system.
Note 4 to entry: In this International Standard, user's needs include their desires and expectations associated with use of a
product, system or service. Exceeding desires and expectations is a means of significantly increasing satisfaction and improving
the user experience.
11
© ISO/IEC 2015 – All rights reserved 11
---------------------- Page: 10 ----------------------
4.19
stakeholder satisfaction
degree to which stakeholder needs are satisfied when a product or system is used in a specified context of use
[SOURCE: ISO/IEC 25010:2011 definition for the term "satisfaction" modified to refer to stakeholders]
Note 1 to entry: Users of a product or system are one type of stakeholder, so user satisfaction is one type of stakeholder
satisfaction.
4.20
summative evaluation
evaluation designed to present conclusions about the merit or worth of the object of evaluation
Note 1 to entry: The results can be used to produce recommendations about whether it should be retained, altered or
eliminated.
Note 2 to entry: It is possible to design a method to provide a combined formative and summative evaluation.
Note 3 to entry: A summative test method is used to perform a summative evaluation.
[SOURCE: ISO TS 20282-2:2012, 4.16]
4.21
system
a combination of interacting elements organised to achieve one or more stated purposes
Note 1 to entry: A system may be considered as a product or as the services it provides.
Note 2 to entry: In practice, the interpretation of its meaning is frequently clarified by the use of an associative noun, e.g.
aircraft system. Alternatively the word system may be substituted simply by a context dependent synonym, e.g. aircraft, though
this may then obscure a system principles perspective.
[SOURCE: ISO/IEC 15288:2008]
4.22
task
activities required to achieve a goal
Note 1 to entry: These activities can be physical or cognitive.
Note 2 to entry: Job responsibilities can determine goals and tasks.
[SOURCE: ISO 9241-11:1998]
4.23
usability
degree to which a product or system can be used by specified users to achieve specified goals with effectiveness,
efficiency and satisfaction in a specified context of use
Note 1 to entry: Based on ISO 9241-210.
Note 2 to entry: Usability can either be specified or measured as a product quality characteristic in terms of its
subcharacteristics, or specified or measured directly by measures that are a subset of quality in use.
[SOURCE: ISO/IEC 25010:2011]
4.24
use error
act or omission of an act that results in a different system response than intended by the manufacturer or expected
by the user
[SOURCE: IEC 62366:2007 with "medical device" replaced by "system"]
© ISO/IEC 2015 – All rights reserved
12
---------------------- Page: 11 ----------------------
ISO/IEC DIS 25022.2(E)
4.25
user
individual or group that benefits from a system during its utilization
[SOURCE: ISO/IEC 15939:2007]
5 Abbreviated terms
The following abbreviation is used in this International Standard.
QME – Quality Measure Element
6 Use of quality in use measures
6.1 Applications of quality in use
Quality in use is the degree to which a product or system can be used by specific users to meet their needs to
achieve specific goals with effectiveness, efficiency and satisfaction and reduced risk in specific contexts of use.
NOTE 1 Quality in use is thus a combination of usability (when specified or measured directly by measures of effectiveness,
efficiency and satisfaction) and freedom from risk.
Quality in use measures the outcomes of interaction between a user and a system. The final quality in use can be
measured when an implemented system is used in the user’s environment for its intended purpose. Quality in use
measures can also be used at conceptual design and development stages.
Quality in use requirements can be specified early in the design and development process, to give a high
level goal for the intended quality to be experienced by users and other stakeholders.
Specific problems with quality in use can be identified by testing prototypes with small numbers of
representative users during development using unnormalised measures in order to improve the product
(formative evaluation).
Estimates of the final quality in use of different design solutions can be obtained by testing prototypes with
larger numbers of representative users during development (summative evaluation using normalised
measures), and estimating how the resulting
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.