ISO/IEC TS 25052-2:2024

Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE): cloud services - Part 2: Quality measurement

This document defines quality measures for quantitatively evaluating cloud services quality in terms of characteristics and sub-characteristics defined in ISO/IEC TS 25052-1 and is intended to be used together with ISO/IEC TS 25052-1. This document contains the following: - a basic set of quality measures for each characteristic and sub-characteristics; - an explanation of how to apply quality measures to cloud services. Since the quality model defined in ISO/IEC TS 25052-1 is the extension to the existing quality models defined in ISO/IEC 25010 to ISO/IEC 25019, it can be used with the product quality model, IT service quality model, data quality model, and quality-in-use model according to evaluation purposes. For the same reason, the quality measures defined in this document can also be used with the quality measures for software ICT products, IT services, data, and quality-in-use. As there are several cloud service categories, this document focuses on the quality model of SaaS (software as a service). This document does not address PaaS (platform as a service) and IaaS (infrastructure as a service).

Ingénierie des systèmes et du logiciel — Exigences de qualité et évaluation des systèmes et du logiciel (SQuaRE): services en nuage — Partie 2: Mesure de la qualité

General Information

• Status: Published
• Publication Date: 15-Dec-2024
• ICS: 35.080 - Software
• Technical Committee: ISO/IEC JTC 1/SC 7 - Software and systems engineering
• Drafting Committee: ISO/IEC JTC 1/SC 7 - Software and systems engineering
• Current Stage: 6060 - International Standard published
• Start Date: 16-Dec-2024
• Due Date: 27-Sep-2026
• Completion Date: 16-Dec-2024

Overview

ISO/IEC TS 25052-2:2024 is a Technical Specification in the SQuaRE family that defines quality measures for quantitative evaluation of cloud services, intended to be used together with ISO/IEC TS 25052-1 (the cloud services quality model). Focused on SaaS (Software as a Service), this document provides a basic set of measurable metrics for each quality characteristic and guidance on how to apply those measures for objective assessment, benchmarking, and contractual use. It complements the ISO/IEC 25010–25019 quality-model series and adapts SQuaRE measurement concepts to cloud-specific properties such as multi-tenancy, elasticity, and measured service.

Key topics and technical scope

The standard organizes quality measurement across cloud-relevant characteristics and sub-characteristics. Major technical topics include:

• Service performance efficiency
  • Time behaviour (response and latency)
  • Aggregated resource utilization
  • Capacity, scalability and elasticity metrics
• Service compatibility
  • Cloud interoperability measures
• Service usability
  • Accessibility and user-oriented measures
• Service reliability
  • Availability, resilience and recoverability measures
• Service security
  • Confidentiality, accountability, isolation
  • PII protection conformance, security responsibility, asset protection
• Maintainability
  • Compliance and versioning, reversibility, monitorability
• Portability
  • Data portability and application portability metrics
• Service provisionability
  • Measurability, auditability, governability, self-provisioning
  • SLA information completeness and SLA satisfaction measures
• Service responsiveness
  • Supportiveness, reactiveness, continuity

The specification defines a consistent format for documenting quality measures and explains practical application methods to ensure measurable, repeatable cloud-service evaluations.

Applications and who uses it

ISO/IEC TS 25052-2:2024 is practical for:

• Cloud service consumers and procurement teams establishing SLA metrics and selection criteria
• Cloud service providers (CSPs) defining service KPIs, monitoring and reporting practices
• Cloud architects, DevOps/SRE teams and QA engineers implementing performance, reliability and security testing
• Auditors and compliance officers conducting objective audits and conformity assessments
• Vendors and integrators needing interoperability and portability evidence for migrations or multi-cloud strategies

Typical uses: requirements specification, acceptance testing, SLA negotiation, benchmarking, risk assessment, and continuous quality monitoring.

Related standards

• ISO/IEC TS 25052-1 - cloud services quality model (companion document)
• ISO/IEC 25010 – 25019 - core SQuaRE quality-model series for products, services and data
• ISO/IEC 22123-2 - referenced for cloud computing characteristics (e.g., elasticity, multi-tenancy)

Keywords: ISO/IEC TS 25052-2:2024, cloud services quality measurement, SQuaRE, SaaS quality metrics, SLA metrics, cloud interoperability, cloud performance, security measures.

Standards Content (Sample)

Technical
Specification
ISO/IEC TS 25052-2
First edition
Systems and software
2024-12
engineering — Systems and
software Quality Requirements
and Evaluation (SQuaRE): cloud
services —
Part 2:
Quality measurement
Ingénierie des systèmes et du logiciel — Exigences de qualité
et évaluation des systèmes et du logiciel (SQuaRE): services
en nuage —
Partie 2: Mesure de la qualité
Reference number
© ISO/IEC 2024
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on
the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below
or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: [phone]
Email: [e-mail]
Website: www.iso.org
Published in Switzerland
© ISO/IEC 2024 – All rights reserved
ii
Contents Page
Foreword .v
Introduction .vi
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Format used for documenting the quality measures . 2
5 Cloud service quality measures . 2
5.1 Overview .2
5.2 Service performance efficiency measures .3
5.2.1 General .3
5.2.2 Time behaviour measures .3
5.2.3 Aggregated resource utilization measures .3
5.2.4 Capacity measures .3
5.2.5 Scalability measures .4
5.2.6 Elasticity measures .4
5.3 Service compatibility measures .5
5.3.1 General .5
5.3.2 Cloud interoperability measures .5
5.4 Service usability measures . .5
5.4.1 General .5
5.4.2 Accessibility measures . . .6
5.5 Service reliability measures .6
5.5.1 General .6
5.5.2 Availability measures .6
5.5.3 Resilience measures .6
5.5.4 Recoverability measures .7
5.6 Service security measures .7
5.6.1 General .7
5.6.2 Confidentiality measures .7
5.6.3 Accountability measures .8
5.6.4 Isolation measures.8
5.6.5 PII protection conformance measures .9
5.6.6 Security responsibility measures .9
5.6.7 Asset protection measures.9
5.7 Service maintainability measures .10
5.7.1 General .10
5.7.2 Maintenance compliance and versioning measures .10
5.7.3 Reversibility measures .10
5.7.4 Monitorability measures .11
5.8 Portability measures .11
5.8.1 General .11
5.8.2 Cloud data portability measures .11
5.8.3 Cloud application portability measures . 12
5.9 Service provisionability measures . 13
5.9.1 General . 13
5.9.2 Service measurability measures . 13
5.9.3 Auditability measures .14
5.9.4 Governability measures .14
5.9.5 Self-provisioning measures . 15
5.9.6 SLA (service level agreement) information completeness measures . 15
5.9.7 SLA (service level agreement) satisfaction measures. 15
5.10 Service responsiveness measures . .16
5.10.1 General .16

© ISO/IEC 2024 – All rights reserved
iii
5.10.2 Service supportiveness measures .16
5.10.3 Reactiveness measures .16
5.10.4 Continuity measures .16
Bibliography .18

© ISO/IEC 2024 – All rights reserved
iv
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical activity.
ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations,
governmental and non-governmental, in liaison with ISO and IEC, also take part in the work.
The procedures used to develop this document and those intended for its further maintenance are described
in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the different types
of document should be noted. This document was drafted in accordance with the editorial rules of the ISO/
IEC Directives, Part 2 (see www.iso.org/directives or www.iec.ch/members_experts/refdocs).
ISO and IEC draw attention to the possibility that the implementation of this document may involve the
use of (a) patent(s). ISO and IEC take no position concerning the evidence, validity or applicability of any
claimed patent rights in respect thereof. As of the date of publication of this document, ISO and IEC had not
received notice of (a) patent(s) which may be required to implement this document. However, implementers
are cautioned that this may not represent the latest information, which may be obtained from the patent
database available at www.iso.org/patents and https://patents.iec.ch. ISO and IEC shall not be held
responsible for identifying any or all such patent rights.
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and expressions
related to conformity assessment, as well as information about ISO's adherence to the World Trade
Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www.iso.org/iso/foreword.html.
In the IEC, see www.iec.ch/understanding-standards.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 7, Software and systems engineering.
A list of all parts in the ISO/IEC 25052 series can be found on the ISO and IEC websites.
Any feedback or questions on this document should be directed to the user’s national standards
body. A complete listing of these bodies can be found at www.iso.org/members.html and
www.iec.ch/national-committees.

© ISO/IEC 2024 – All rights reserved
v
Introduction
In the standards on SQuaRE developed by ISO/IEC JTC 1/SC 7, there are well-defined quality models for
measuring and evaluating system and software products, IT services, data, etc. Although the standards on
SQuaRE developed by ISO/IEC JTC 1/SC 7 provide practical quality models, they do not fit new technologies
well. To support the evaluation of new technologies, ISO/IEC TS 25052-1 defines the quality model of cloud
services, which is an extension of the quality models defined in ISO/IEC 25010 to ISO/IEC 25019. To support
the practical measurement of cloud services, this document provides quality measures based on the quality
model defined in ISO/IEC TS 25052-1.
Compared to information and communication technology (ICT) systems, cloud computing has different
characteristics. This document reflects the characteristics of cloud computing. The following are the key
characteristics of cloud computing described in ISO/IEC 22123-2.
— Broad network access: physical and virtual resources are available over a network and accessed through
standard mechanisms that promote the use of cloud service customers (CSC).
— Measured service: characteristic in which the metered delivery of cloud services is such that usage can
be monitored, controlled, reported, and billed.
— Multi-tenancy: characteristic in which physical or virtual resources are allocated in such a way that
multiple tenants and their computations and data are isolated from and inaccessible to one another.
— On-demand self-service: characteristic in which a CSC can provision cloud services, as needed,
automatically or with minimal interaction with cloud service providers(CSP).
— Rapid elasticity and scalability: resources can be rapidly and elastically adjusted, in some cases
automatically, to quickly increase or decrease capacity.
— Resource pooling: characteristic in which a CSP’s physical or virtual resources can be aggregated to
serve one or more CSCs.
The quality model in this document is to support the non-functional specification and evaluation of cloud
services from different perspectives by those associated with cloud service selection, requirements analysis,
development, use, evaluation, support, maintenance, quality assurance and control, and audit.
For example, activities during cloud service selection that can benefit from the use of the quality model
include:
— identifying cloud services requirements;
— establishing cloud service selection criteria;
— defining service coverage and service objectives;
— establishing service level agreements;
— establishing measures of quality characteristics in support of these activities.
Activities during cloud service development that can benefit from the use of the quality model include:
— identifying cloud service requirements;
— validating comprehensiveness of requirement definitions;
— identifying cloud service design objectives;
— identifying cloud service testing objectives;
— identifying quality control criteria as part of quality assurance;
— identifying acceptance criteria for a cloud service;

© ISO/IEC 2024 – All rights reserved
vi
— establishing measures of quality characteristics in support of these activities.
Figure 1 illustrates the organization of the standards on SQuaRE developed by ISO/IEC JTC 1/SC 7. Similar
standards are grouped into divisions. Each division provides guidance and resources for performing a
different function in ensuring system and software product quality. This document belongs to extension
division 25050 to 25099.
Figure 1 — Organization of the standards on SQuaRE developed by ISO/IEC JTC 1/SC 7
The divisions within standards on SQuaRE are:
— ISO/IEC 25000 to ISO/IEC 25009 - Quality management division. The International Standards that form
this division define all common models, terms, and definitions referred to by all other standards on
SQuaRE developed by ISO/IEC JTC 1/SC 7. This division also provides requirements and guidance for
a supporting function that is responsible for the management of the requirements, specification, and
evaluation of software product quality. Practical guidance on the use of the quality models is also
provided.
— ISO/IEC 25010 to ISO/IEC 25019 - Quality model division. The International Standards that form this
division present detailed quality models for computer systems and software products, data, IT services,
and quality-in-use.
— ISO/IEC 25020 to ISO/IEC 25029 - Quality measurement division. The International Standards that form
this division include a quality measurement framework, mathematical definitions of quality measures,
and practical guidance for their application. Examples are given of quality measures for the internal
and external properties of products, data, IT services, and quality-in-use. Quality measure elements
(QME) forming foundations for quality measures for the internal and external properties of products
are defined and presented.
— ISO/IEC 25030 to ISO/IEC 25039- Quality requirements division. The International Standards that
form this division help specify quality requirements based on quality models and quality measures.
These quality requirements can be used in the process of eliciting quality requirements for information
systems and IT services to be developed or as input for an evaluation process.
— ISO/IEC 25040 to ISO/IEC 25049 - Quality evaluation division. The International Standards that form
this division provide requirements, recommendations, and guidelines for software product evaluation,

© ISO/IEC 2024 – All rights reserved
vii
whether performed by evaluators, acquirers, or developers. The guideline for documenting a measure as
an evaluation module is also provided.
— ISO/IEC 25050 to ISO/IEC 25099 - SQuaRE extension division. These International Standards currently
include requirements for quality of ready-to-use software product (RUSP), Common Industry Formats
for usability reports, and quality models and measures for new technologies such as cloud services and
artificial intelligence.
© ISO/IEC 2024 – All rights reserved
viii
Technical Specification ISO/IEC TS 25052-2:2024(en)
Systems and software engineering — Systems and software
Quality Requirements and Evaluation (SQuaRE): cloud
services —
Part 2:
Quality measurement
1 Scope
This document defines quality measures for quantitatively evaluating cloud services quality in terms of
characteristics and sub-characteristics defined in ISO/IEC TS 25052-1 and is intended to be used together
with ISO/IEC TS 25052-1.
This document contains the following:
— a basic set of quality measures for each characteristic and sub-characteristics;
— an explanation of how to apply quality measures to cloud services.
Since the quality model defined in ISO/IEC TS 25052-1 is the extension to the existing quality models defined
in ISO/IEC 25010 to ISO/IEC 25019, it can be used with the product quality model, IT service quality model,
data quality model, and quality-in-use model according to evaluation purposes. For the same reason, the
quality measures defined in this document can also be used with the quality measures for software ICT
products, IT services, data, and quality-in-use.
As there are several cloud service categories, this document focuses on the quality model of SaaS (software
as a service). This document does not address PaaS (platform as a service) and IaaS (infrastructure as a
service).
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content constitutes
requirements of this document. For dated references, only the edition cited applies. For undated references,
the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 25000, Systems and software engineering — Systems and software Quality Requirements and
Evaluation (SQuaRE) — Guide to SQuaRE
ISO/IEC 22123-1, Information technology — Cloud computing — Part 1: Vocabulary
3 Terms and definitions
For this document, the terms and definitions given in ISO/IEC 25000, ISO/IEC 22123-1 and the following apply.
ISO and IEC maintain terminology databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at https:// www .electropedia .org/

© ISO/IEC 2024 – All rights reserved
3.1
measure
variable to which a value is assigned as the result of measurement (3.2)
Note 1 to entry: The plural form “measures” is used to refer collectively to base measures, derived measures and
indicators.
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.15]
3.2
measurement
set of operations having the object of determining a value of a measure (3.1)
Note 1 to entry: Adapted from the International Vocabulary of Metrology – Basic and General Concepts and Associated
Terms, 2012.
[SOURCE: ISO/IEC/IEEE 15939:2017, 3.17]
3.3
measurement function
algorithm or calculation performed to combine two or more quality measure (3.4) elements
[SOURCE: ISO/IEC 25021:2012, 4.7]
3.4
quality measure
derived measure that is defined as a measurement function (3.3) of two or more values of quality measure
elements
[SOURCE: ISO/IEC 25021:2012, 4.13]
4 Format used for documenting the quality measures
The following information is given for each quality measure from Table 1 to 30:
a) ID: identification code of quality measure; each ID consists of the following three parts:
— abbreviated alphabetic code representing the quality characteristics as capital X and sub-characteristics
as one capital X followed by lowercase x (for example, “PTb” denotes “time behaviour” measures for
“performance efficiency”);
— serial number of sequential orders within quality sub-characteristic;
— G (generic) or S (specific) expressing potential categories of quality measure; where generic measures
can be used whenever appropriate and specific measures can be used when relevant in a particular
situation;
b) name: quality measure name;
c) description: the information provided by the quality measure;
d) measurement function: algorithm or calculation performed to combine two or more quality measure
elements.
5 Cloud service quality measures
5.1 Overview
The quality measures in this clause are listed by quality characteristics and sub-characteristics, in the order
used in ISO/IEC TS 25052-1; and the word “measures” in this clause means quality measures.

© ISO/IEC 2024 – All rights reserved
5.2 Service performance efficiency measures
5.2.1 General
Service performance efficiency measures are used to assess the degree to which a cloud service meets
performance requirements under stated conditions.
5.2.2 Time behaviour measures
Time behaviour measures are used to assess the degree to which a cloud service meets the requirements
of the response times and throughput rates of a cloud service when concurrent users take advantage of the
cloud service. Time behaviour measures are measured in Table 1.
Table 1 — Time behaviour measures
ID Name Description Measurement function
PTb-1-G Mean response How long does it take to receive a n
time response for user requests to cloud
XA= ()/n
∑ i
services?
i=1
A = Time taken by the cloud service to
i
respond to a user request
n = Number of responses measured
PTb-2-G Response time How well does a cloud service meet X = 1 – A/B
adequacy response time requirements speci-
A = Number of responses that do not meet
fied in SLA?
the response time requirements
B = Number of responses
NOTE If X is less than 1, it means the system fails to meet response time requirements.
5.2.3 Aggregated resource utilization measures
Aggregated resource utilization measures are used to the degree to which a cloud service utilizes efficiently
aggregated resources from resource pooling to support multi-tenancy. Aggregated resource utilization
measures are measured in Table 2.
Table 2 — Aggregated resource utilization measures
ID Name Description Measurement function
PRu-1-G Aggregated re- How efficiently are virtual resourc- X = A/B
source utilization es, assigned from resource pools,
A = Amount of the resource used and re-
utilized to provide a cloud service
turned after they are no longer in use
in a multi-tenant environment?
B = Amount of the resource required to be
used to provide a cloud service
NOTE 1 The efficient utilization of resources means that available virtual resources are used by users, and once they are no
longer in use, the resources are returned.
NOTE2 Resource utilization can be monitored using the special monitoring services provided by IaaS. The resource allocation
and release can be indirectly monitored by observing the usage of resources such as CPU and memory.
NOTE 3 Due to the characteristics of resource abstraction, it can be challenging to monitor resource utilization, which can make
this quality measure inapplicable in some cases.
5.2.4 Capacity measures
Capacity measures are used to assess the degree to which the maximum limits of a cloud service’s parameters
meet requirements in SLA (service level agreement). Capacity measures are measured in Table 3.
NOTE Parameters can include the limit of simultaneous cloud service connections, the limit of available cloud
service resources, cloud service throughput, and cloud service bandwidth.

© ISO/IEC 2024 – All rights reserved
Table 3 — Capacity measures
ID Name Description Measurement function
PCa-1-G Cloud service con- How many simultaneous connec- X = A/B
nection capacity tions can a cloud service accommo-
A = Maximum amount of simultaneous con-
date over a given period with the
nections that a cloud service can accommo-
available resources?
date
B = Maximum amount of simultaneous con-
nections according to capacity requirements
of SLA
PCa-2-G Cloud service re- How much resource can be used for X = A/B
source capacity the cloud service compared to the
A = Maximum amount of the resource used
resource allocated?
B = Amount of the resource allocated accord-
ing to capacity requirements specified in
SLA
PCa-3-G Cloud service How many transactions can a cloud X = A/B
throughput ca- service process within a given
A = Maximum amount of transactions pro-
pacity period?
cessed by a cloud service
B = Capacity requirements specified in SLA
PCa-4-G Cloud service How much data can be transferred X = A/B
bandwidth capac- at a period?
A = Maximum amount of data transferred
ity
B = Observation time
5.2.5 Scalability measures
Scalability measures are used to assess the degree to which physical and virtual resources are available
automatically and immediately when they are needed, subject to constraints of service agreements.
Scalability measures are measured in Table 4.
Table 4 — Scalability measures
ID Name Description Measurement function
PSa-1-G Scale-out To what extent do a cloud service X = A/B
support scale-out compared to
(horizontal scal- A = Number of the resources that are availa-
configuration settings?
ing) ble automatically and immediately
B = Number of the resources according to
configuration settings
PSa-2-G Scale-up To what extent does a cloud service X = A/B
support scale-up compared to con-
(vertical scaling) A = Amount of the resource that is available
figuration settings?
automatically and immediately
B = Amount of the resource according to
configuration settings
A cloud service should be designed to handle larger workloads and is capable of accommodating additional resources to support
scalability.
NOTE “Scale out" involves adding multiple identical servers or nodes to handle larger workloads or traffic, with each additional
server or node performing the same tasks, and new servers being added as the load increases. In contrast, "scale up" is a method
of improving the hardware performance of a single server or node by upgrading components such as CPU, memory, storage, and
so on to enhance the performance of that single server.
5.2.6 Elasticity measures
Elasticity measures are used to assess the degree to which a cloud service adjusts rapidly and elastically
the amount of resource that are allocated to an instance of the service. Elasticity measures are measured in
Table 5.
© ISO/IEC 2024 – All rights reserved
Table 5 — Elasticity measures
ID Name Description Measurement function
PEl-1-G Manual elasticity How fast a cloud service can react X = A
Speed manually to resource requests?
A = Time duration to allocate additional
resources manually when the allocation is
requested
PEl-2-G Automatic elastic- How fast a cloud service can react X = A
ity speed automatically to resource requests
A = Time duration to allocate additional
when the workload is increased or
resources automatically when workload is
decreased?
increased or decreased
PEl-3-G Elasticity preci- How precisely does the manual or X = A/B
sion automatic resource allocation meet
A = Amount of the actually allocated
elasticity precision requirements?
resource that meets elasticity precision
requirements
B = Amount of the resource required to be
allocated according to elasticity precision
requirements
5.3 Service compatibility measures
5.3.1 General
Service compatibility measures are used to assess the degree to which a cloud service can exchange
information with CSC’s systems or other cloud services and/or perform its required functions.
5.3.2 Cloud interoperability measures
Cloud interoperability measures are used to assess the degree to which a cloud service
...