ISO/IEC TR 23951:2020
(Main)Information technology — Cloud computing — Guidance for using the cloud SLA metric model
Information technology — Cloud computing — Guidance for using the cloud SLA metric model
The scope of this document is to describe guidance for using the ISO/IEC 19086-2 metric model, illustrated with examples.
Technologies de l'information — Informatique en nuage — Recommandations pour l'utilisation du modèle métrique d'accord de niveau de service (SLA) dans le Cloud
General Information
Buy Standard
Standards Content (Sample)
TECHNICAL ISO/IEC TR
REPORT 23951
First edition
2020-06
Information technology — Cloud
computing — Guidance for using the
cloud SLA metric model
Technologies de l'information — Informatique en nuage —
Recommandations pour l'utilisation du modèle métrique d'accord de
niveau de service (SLA) dans le Cloud
Reference number
ISO/IEC TR 23951:2020(E)
©
ISO/IEC 2020
---------------------- Page: 1 ----------------------
ISO/IEC TR 23951:2020(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2020
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2020 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC TR 23951:2020(E)
Contents Page
Foreword .v
Introduction .vi
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Symbols and abbreviated terms . 1
5 Structure of this document . 2
6 Motivation. 2
6.1 Preamble . 2
6.2 Audience and some user categories . 2
6.2.1 General. 2
6.2.2 Cloud service customer (CSC) . 3
6.2.3 Cloud service provider (CSP) . 3
6.2.4 Cloud service partner (CSN) . 3
6.2.5 Regulators and policy makers . 4
6.3 Usage patterns . 4
6.3.1 General. 4
6.3.2 Extract and clarify an existing metric description from an SLA . 4
6.3.3 Create and share a metric description . 4
6.3.4 Compare metric descriptions . 5
6.3.5 Share a common foundation for a set of metrics . 5
6.3.6 Build a metrics catalogue . 5
6.4 Examples of scenarios and roles involved in sharing metric definitions . 5
7 The metric model in practice: templates . 6
7.1 A brief reminder of the metric model . 6
7.2 A tabular representation . 7
7.2.1 General. 7
7.2.2 The tabular representation for the Metric element . 8
7.2.3 The tabular representation for the Expression elements . 9
7.2.4 The tabular representation for the Rule elements .10
7.2.5 The tabular representation for the Parameter elements .11
8 An example of metric definition: the cloud service mean response time metric .11
8.1 The cloud service mean response time metric: informal variant .11
8.1.1 Extracting metric elements from an SLA narrative .11
8.1.2 Using the tabular representation .12
8.1.3 Overall structure of the metric .14
8.2 The cloud service mean response time metric: more formal variant .14
8.2.1 A more formal variant of the metric .14
8.2.2 Adding a parameter .15
8.2.3 The metric rules .15
8.2.4 The metric expressions .15
8.2.5 Overall structure of the metric .17
8.2.6 Using constants .17
9 Guidelines for using the metric model with the tabular representation .19
9.1 General .19
9.2 Guideline 1 about defining expression and rule languages.20
9.3 Guideline 2 about associating rules with expressions .20
9.4 Guideline 3 about relating expressions to each other .20
9.5 Guideline 4 about the identifiers of metric elements .21
9.6 Guideline 5 about rules specifically designed to support an expression .21
9.7 Guideline 6 about the role of parameters .21
© ISO/IEC 2020 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC TR 23951:2020(E)
9.8 Guideline 7 about representing constants .22
10 The simple cloud service availability metric .22
10.1 Measuring cloud service availability .22
10.1.1 General.22
10.1.2 Overall design approach .23
10.1.3 SLA rules and metric rules .23
10.2 The simple cloud service availability metric variant Simple_SAM_1 .24
10.2.1 The Metric element .24
10.2.2 The metric rules .24
10.2.3 The metric expressions .25
10.2.4 The metric parameters .27
10.2.5 Overall structure of the metric .27
10.3 The simple cloud service availability metric variant Simple_SAM_2 .28
10.3.1 Differences in metric design and assumptions .28
10.3.2 The Metric element .29
10.3.3 The metric rules .29
10.3.4 The metric parameters .30
10.3.5 The metric expressions .31
10.3.6 Overall structure of the metric .32
10.3.7 An alternative metric design using the Configuration element option .32
Bibliography .34
iv © ISO/IEC 2020 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC TR 23951:2020(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that
are members of ISO or IEC participate in the development of International Standards through
technical committees established by the respective organization to deal with particular fields of
technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other
international organizations, governmental and non-governmental, in liaison with ISO and IEC, also
take part in the work.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC
list of patent declarations received (see http:// patents .iec .ch).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www .iso .org/
iso/ foreword .html.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 38, Cloud Computing and Distributed Platforms.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/ members .html.
© ISO/IEC 2020 – All rights reserved v
---------------------- Page: 5 ----------------------
ISO/IEC TR 23951:2020(E)
Introduction
In most cases, cloud service providers (CSPs) and cloud service customers (CSCs) negotiate service
level agreements (SLAs) which include service level objectives (SLOs) and service qualitative objectives
(SQOs) for which CSPs make commitments. The commitments described in SLAs are expected to be
measured against actual performance of the service to ensure compliance with the SLA. How actual
performance compares against commitments in SLAs is explained in ISO/IEC 19086-2. Cloud SLAs are
covered in ISO/IEC 19086-1 and in ISO/IEC 19086-4.
The metric model in ISO/IEC 19086-2 establishes common terminology, defines a model for specifying
metrics for cloud SLAs, and includes applications of the model with examples. This document provides
guidance and examples on using the metric model to compose the calculation of a cloud service
performance measure in order to compare against an SLA commitment. A few examples from the
SLOs listed in ISO/IEC 19086-1:2016, Clause 10 are given in the document, such as Cloud Service Mean
Response Time and Simple Cloud Service Availability. As specific, measurable characteristics of a cloud
service, SLOs are the basis for defining the metrics used to evaluate and compare agreements between
parties.
In Clauses 8, 9 and 10 of this document, a basic explanation of these examples is provided using a
practical method based on a tabular format that is a refinement of the informative tables provided
in ISO/IEC 19086-2:2018, Annex B. The tabular representation described in this document serves as
templates for designing metrics. Guidance in using the metric model with these templates is provided
while developing metric examples.
vi © ISO/IEC 2020 – All rights reserved
---------------------- Page: 6 ----------------------
TECHNICAL REPORT ISO/IEC TR 23951:2020(E)
Information technology — Cloud computing — Guidance
for using the cloud SLA metric model
1 Scope
The scope of this document is to describe guidance for using the ISO/IEC 19086-2 metric model,
illustrated with examples.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 17788, Information technology — Cloud computing — Overview and vocabulary
ISO/IEC 17789, Information technology — Cloud computing — Reference architecture
ISO/IEC 19086-1, Information technology — Cloud computing — Service level agreement (SLA)
framework — Part 1: Overview and concepts
ISO/IEC 19086-2, Cloud computing — Service level agreement (SLA) framework — Part 2: Metric model
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC 17788, ISO/IEC 17789,
ISO/IEC 19086-1 and ISO/IEC 19086-2 apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
4 Symbols and abbreviated terms
CCRA cloud computing reference architecture
CSC cloud service customer
CSN cloud service partner
CSP cloud service provider
SLA service level agreement
SLO service level objective
SQO service quality objective
© ISO/IEC 2020 – All rights reserved 1
---------------------- Page: 7 ----------------------
ISO/IEC TR 23951:2020(E)
5 Structure of this document
In supporting the scope presented in Clause 1, this document develops the rationale for a practical
metric representation to complement the metric model in ISO/IEC 19086-2 in the following clauses:
— Clause 6 states the rationale for complementing the metric model as defined in ISO/IEC 19086-2
with a practical representation and for providing related usage guidance as introduced by this
document. It identifies some usage patterns and highlights some usage scenarios where metric
definitions are shared across various parties. The users who benefit from this document include
parties with roles defined in ISO/IEC 17789 (Cloud computing — Reference architecture).
— Clause 7 introduces the tabular metric representation supportive of the metric model and derived
from the informative tables listed in ISO/IEC 19086-2:2018, Annex B. This representation is based on
tables intended to serve as templates for metric definitions. This clause represents initial guidance in
using the metric model, which is then illustrated and discussed throughout the examples developed
later in the document.
— Clause 8 introduces a simple case of metric definition that illustrates the use of the table templates
introduced in Clause 7. This example starts with the description of a metric as it would appear in
the narrative of an existing SLA and illustrates the extraction of this description toward a more
structured and distinct representation using the proposed tabular representation. The example
shows practical aspects when designing and developing metrics, such as how metric rules relate to
expressions, and how to parameterize rules and expressions.
— Clause 9 is a set of guidelines on how to use the metric model with the tabular templates (Clause 7).
This guidance is motivated and illustrated by the examples throughout the document. These
guidelines are best understood after developing a preliminary example (Clause 8). They explain how
to use the metric model with the tabular templates for metric use cases posing similar challenges or
using similar features.
— Clause 10 develops a more elaborate metric example for cloud service availability. It describes
two variants of the same metric that illustrate two different approaches in using the metric
model elements. Since it comes after the guideline items listed in Clause 9, it is easier to relate the
development of this second example to these guidelines.
6 Motivation
6.1 Preamble
This clause first identifies the audience of this document and for the tabular metric representation
described in this document. This clause then describes some metric usage patterns and then identifies
scenarios and roles for these metric usage patterns. Sharing common guidelines and conventions in
using the metric model improves the ability to reuse and compare metrics. These common guidelines
extend to the aspects of a metric that are part of the metric model but the details of which are out of
scope of the metric model in ISO/IEC 19086-2, such as the use of rule and expression languages and
how these constructs relate to each other. Supportive of the goal of harmonizing the usage of the metric
model across users, this document proposes a tabular representation for metric definitions that is
derived from and augments the tables provided in Annex B of the metric model in ISO/IEC 19086-2:2018,
as explained in 7.2.2.
6.2 Audience and some user categories
6.2.1 General
The audience for this document is expected to be diverse, as the metric representation proposed in
this document is intended for different parties involved in providing or using cloud services. However
not every clause is of interest to all. Those who read, negotiate or create SLA content, such as business
users and administrators, are expected to be interested in Clauses 1 to 7 and in the initial approach
2 © ISO/IEC 2020 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/IEC TR 23951:2020(E)
to the first metric example (see 8.1). In addition to these clauses, metric designers and developers
are expected to be interested in the remaining clauses including more elaborate examples of metrics
(starting from 8.2 and beyond).
The parties interested in this document include representatives of the following roles defined in
ISO/IEC 17788.
6.2.2 Cloud service customer (CSC)
This document helps the CSC to understand the metrics used for service quality and other assurances
described in SLAs. When blended into the narrative of the SLA, metrics are often ambiguous or
incomplete. A structured definition as described in the metric model and made practical with a tabular
representation helps to avoid or at least detect such issues.
Specific types of customers are interested in understanding how a service is measured without having
to read the entire SLA or prior to establishing an SLA. These customers are defined in the CCRA as
a cloud service users (who uses a cloud service to fulfil her/his role), a service administrator (who
oversees all the operational processes relating to the use of cloud services, serving as intermediary
between the user and the provider) and a business manager (who has overall responsibility for the
business aspects of using cloud services, including the purchase of the service under appropriate terms
and possibly the request of audit reports).
The tabular representation in this document is also an analysis tool for the CSC to identify and extract
the metric material found in an SLA in order to get a clearer understanding of how the service is
measured, as illustrated in 8.1.
6.2.3 Cloud service provider (CSP)
This document helps the CSP to describe the service metrics that support his or her SLAs, potentially
avoiding contentious claims afterward that result from CSCs misunderstanding the terms and conditions
of these SLAs. It also helps providers to harmonize their metrics across data-centre operators or world
regions. Among activities expected from CSPs as defined in ISO/IEC 17789, the following are facilitated
by metric definitions and evaluations: monitoring service, administering service security, providing
audit data on request, defining and gathering metrics, managing security and risks, and, finally,
handling support requests, reports and incidents from cloud service customers. For these activities,
this document helps to establish a common and unambiguous representation of metrics used between
parties involved in these activities and distinct from other SLA material.
6.2.4 Cloud service partner (CSN)
The following CSN sub-roles are expected to find value in a metric definition template and guidelines:
— Cloud service developer: this user is responsible for designing, developing, integrating, testing,
and maintaining cloud services. Developers need to understand the measurements used to evaluate
a cloud service. This role includes composing a new cloud service from existing separate cloud
services. By having access to precise metric definitions and their rules, such as those illustrated in
Clauses 8 and 10, developers understand what features are to be monitored, what is the expected
quality of the developed cloud service and its priorities, as well as how to evaluate the quality and
risks when integrating third party cloud services.
— Cloud auditor: the auditor has the responsibility of conducting an audit of the provision and the use
of cloud services. A cloud audit typically covers operations, performance and security and examines
whether a specified set of audit criteria are met. By using metrics, the auditor understands or
communicates clearly the details of the measurements to perform. Such precision and clarity
are provided by a distinct and detailed metric representation, as illustrated in the two examples
developed in Clauses 8 and 10.
© ISO/IEC 2020 – All rights reserved 3
---------------------- Page: 9 ----------------------
ISO/IEC TR 23951:2020(E)
6.2.5 Regulators and policy makers
Several aspects of policy definition and enforcement concern measurable properties both about
the cloud service usage (including cloud service usage duration and times, volume and type of data
involved), and the cloud service performance (such as cloud service quality, elasticity and scalability,
availability and reliability). Other policies (such as those about trust and transparency, security
procedures, privacy) concern the relationship, governance and risk management between parties,
especially CSCs and CSPs. Whether these policies involve automated monitoring or some human
assessment instead, they rely on some form of measurement for tracking their implementation. See
ISO/IEC TR 22678 for more information regarding the development of policies that govern or regulate
cloud service providers (CSPs) and cloud services, and those policies and practices that govern the use
of cloud services in organizations.
The expression of policies and rules sometimes translates into predefined metric elements that are
expected to be used even when defining a customized metric. An example is of a policy that determines
the formula (metric expression) to be used when assessing cloud service uptime percentage, while
leaving other details unconstrained. As another example, if there is agreement for sharing across CSPs
the common definitions of “natural disaster” or “service misuse”, the reuse of such definitions helps to
establish a common understanding of what a valid cloud service downtime means. Creating and sharing
predefined metric material is a usage pattern described in 6.3.5 as sharing a metric foundation.
6.3 Usage patterns
6.3.1 General
A summary of various usage patterns for the tabular metric representation given in 7.2 and a rationale
for doing so are provided in the next subclauses. Some of these usage patterns match usage categories
identified in ISO/IEC 19086-2:2018, 6.4.2.
6.3.2 Extract and clarify an existing metric description from an SLA
Often, the metric(s) information in a cloud service SLA is scattered over the SLA narrative. Parts of
metric material (such as measurement rules, exceptions, underlying quantities and metrics) is mixed
with related information that is not part of the metric definition (such as performance objectives,
remediation measures and penalties).
Distinguishing a metric definition apart from its context
...
TECHNICAL ISO/IEC TR
REPORT 23951
First edition
Information technology — Cloud
computing — Guidance for using the
cloud SLA metric model
PROOF/ÉPREUVE
Reference number
ISO/IEC TR 23951:2020(E)
©
ISO/IEC 2020
---------------------- Page: 1 ----------------------
ISO/IEC TR 23951:2020(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2020
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii PROOF/ÉPREUVE © ISO/IEC 2020 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC TR 23951:2020(E)
Contents Page
Foreword .v
Introduction .vi
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Symbols and abbreviated terms . 1
5 Structure of this document . 2
6 Motivation. 2
6.1 Preamble . 2
6.2 Audience and some user categories . 2
6.2.1 General. 2
6.2.2 Cloud service customer (CSC) . 3
6.2.3 Cloud service provider (CSP) . 3
6.2.4 Cloud service partner (CSN) . 3
6.2.5 Regulators and policy makers . 4
6.3 Usage patterns . 4
6.3.1 General. 4
6.3.2 Extract and clarify an existing metric description from an SLA . 4
6.3.3 Create and share a metric description . 4
6.3.4 Compare metric descriptions . 5
6.3.5 Share a common foundation for a set of metrics . 5
6.3.6 Build a metrics catalogue . 5
6.4 Examples of scenarios and roles involved in sharing metric definitions . 5
7 The metric model in practice: templates . 6
7.1 A brief reminder of the metric model . 6
7.2 A tabular representation . 7
7.2.1 General. 7
7.2.2 The tabular representation for the Metric element . 8
7.2.3 The tabular representation for the Expression elements . 9
7.2.4 The tabular representation for the Rule elements .10
7.2.5 The tabular representation for the Parameter elements .11
8 An example of metric definition: the cloud service mean response time metric .11
8.1 The cloud service mean response time metric: informal variant .11
8.1.1 Extracting metric elements from an SLA narrative .11
8.1.2 Using the tabular representation .12
8.1.3 Overall structure of the metric .14
8.2 The cloud service mean response time metric: more formal variant .14
8.2.1 A more formal variant of the metric .14
8.2.2 Adding a parameter .15
8.2.3 The metric rules .15
8.2.4 The metric expressions .15
8.2.5 Overall structure of the metric .17
8.2.6 Using constants .17
9 Guidelines for using the metric model with the tabular representation .19
9.1 General .19
9.2 Guideline 1 about defining expression and rule languages.20
9.3 Guideline 2 about associating rules with expressions .20
9.4 Guideline 3 about relating expressions to each other .20
9.5 Guideline 4 about the identifiers of metric elements .21
9.6 Guideline 5 about rules specifically designed to support an expression .21
9.7 Guideline 6 about the role of parameters .21
© ISO/IEC 2020 – All rights reserved PROOF/ÉPREUVE iii
---------------------- Page: 3 ----------------------
ISO/IEC TR 23951:2020(E)
9.8 Guideline 7 about representing constants .22
10 The simple cloud service availability metric .22
10.1 Measuring cloud service availability .22
10.1.1 General.22
10.1.2 Overall design approach .23
10.1.3 SLA rules and metric rules .23
10.2 The simple cloud service availability metric variant Simple_SAM_1 .24
10.2.1 The Metric element .24
10.2.2 The metric rules .24
10.2.3 The metric expressions .25
10.2.4 The metric parameters .27
10.2.5 Overall structure of the metric .27
10.3 The simple cloud service availability metric variant Simple_SAM_2 .28
10.3.1 Differences in metric design and assumptions .28
10.3.2 The Metric element .29
10.3.3 The metric rules .29
10.3.4 The metric parameters .30
10.3.5 The metric expressions .31
10.3.6 Overall structure of the metric .32
10.3.7 An alternative metric design using the Configuration element option .32
Bibliography .34
iv PROOF/ÉPREUVE © ISO/IEC 2020 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC TR 23951:2020(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that
are members of ISO or IEC participate in the development of International Standards through
technical committees established by the respective organization to deal with particular fields of
technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other
international organizations, governmental and non-governmental, in liaison with ISO and IEC, also
take part in the work.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC
list of patent declarations received (see http:// patents .iec .ch).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www .iso .org/
iso/ foreword .html.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 38, Cloud Computing and Distributed Platforms.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/ members .html.
© ISO/IEC 2020 – All rights reserved PROOF/ÉPREUVE v
---------------------- Page: 5 ----------------------
ISO/IEC TR 23951:2020(E)
Introduction
In most cases, cloud service providers (CSPs) and cloud service customers (CSCs) negotiate service
level agreements (SLAs) which include service level objectives (SLOs) and service qualitative objectives
(SQOs) for which CSPs make commitments. The commitments described in SLAs are expected to be
measured against actual performance of the service to ensure compliance with the SLA. How actual
performance compares against commitments in SLAs is explained in ISO/IEC 19086-2. Cloud SLAs are
covered in ISO/IEC 19086-1 and in ISO/IEC 19086-4.
The metric model in ISO/IEC 19086-2 establishes common terminology, defines a model for specifying
metrics for cloud SLAs, and includes applications of the model with examples. This document provides
guidance and examples on using the metric model to compose the calculation of a cloud service
performance measure in order to compare against an SLA commitment. A few examples from the SLOs
listed in Clause 10 of ISO/IEC 19086-1 are given in the document, such as Cloud Service Mean Response
Time and Simple Cloud Service Availability. As specific, measurable characteristics of a cloud service,
SLOs are the basis for defining the metrics used to evaluate and compare agreements between parties.
In Clauses 8, 9 and 10 of this document, a basic explanation of these examples is provided using a
practical method based on a tabular format that is a refinement of the informative tables provided
in ISO/IEC 19086-2:2018, Annex B. The tabular representation described in this document serves as
templates for designing metrics. Guidance in using the metric model with these templates is provided
while developing metric examples.
vi PROOF/ÉPREUVE © ISO/IEC 2020 – All rights reserved
---------------------- Page: 6 ----------------------
TECHNICAL REPORT ISO/IEC TR 23951:2020(E)
Information technology — Cloud computing — Guidance
for using the cloud SLA metric model
1 Scope
The scope of this technical report is to describe guidance for using the ISO/IEC 19086-2 metric model,
illustrated with examples.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 17788, Information technology — Cloud computing — Overview and vocabulary
ISO/IEC 17789, Information technology — Cloud computing — Reference architecture
ISO/IEC 19086-1, Information technology — Cloud computing — Service level agreement (SLA)
framework — Part 1: Overview and concepts
ISO/IEC 19086-2, Cloud computing — Service level agreement (SLA) framework — Part 2: Metric model
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC 17788, ISO/IEC 17789,
ISO/IEC 19086-1 and ISO/IEC 19086-2 apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
4 Symbols and abbreviated terms
CCRA cloud computing reference architecture
CSC cloud service customer
CSN cloud service partner
CSP cloud service provider
SLA service level agreement
SLO service level objective
SQO service quality objective
© ISO/IEC 2020 – All rights reserved PROOF/ÉPREUVE 1
---------------------- Page: 7 ----------------------
ISO/IEC TR 23951:2020(E)
5 Structure of this document
In supporting the scope presented in Clause 1, this document develops the rationale for a practical
metric representation to complement the metric model in ISO/IEC 19086-2 in the following clauses:
— Clause 6 states the rationale for complementing the metric model as defined in ISO/IEC 19086-2
with a practical representation and for providing related usage guidance as introduced by this
document. It identifies some usage patterns and highlights some usage scenarios where metric
definitions are shared across various parties. The users who benefit from this document include
parties with roles defined in ISO/IEC 17789 (Cloud computing — Reference architecture).
— Clause 7 introduces the tabular metric representation supportive of the metric model and derived
from the informative tables listed in ISO/IEC 19086-2:2018, Annex B. This representation is based on
tables intended to serve as templates for metric definitions. This clause represents initial guidance in
using the metric model, which is then illustrated and discussed throughout the examples developed
later in the document.
— Clause 8 introduces a simple case of metric definition that illustrates the use of the table templates
introduced in Clause 7. This example starts with the description of a metric as it would appear in
the narrative of an existing SLA and illustrates the extraction of this description toward a more
structured and distinct representation using the proposed tabular representation. The example
shows practical aspects when designing and developing metrics, such as how metric rules relate to
expressions, and how to parameterize rules and expressions.
— Clause 9 is a set of guidelines on how to use the metric model with the tabular templates (Clause 7).
This guidance is motivated and illustrated by the examples throughout the document. These
guidelines are best understood after developing a preliminary example (Clause 8). They explain how
to use the metric model with the tabular templates for metric use cases posing similar challenges or
using similar features.
— Clause 10 develops a more elaborate metric example for cloud service availability. It describes
two variants of the same metric that illustrate two different approaches in using the metric
model elements. Since it comes after the guideline items listed in Clause 9, it is easier to relate the
development of this second example to these guidelines.
6 Motivation
6.1 Preamble
This clause first identifies the audience of this document and for the tabular metric representation
described in this document. This clause then describes some metric usage patterns and then identifies
scenarios and roles for these metric usage patterns. Sharing common guidelines and conventions in
using the metric model improves the ability to reuse and compare metrics. These common guidelines
extend to the aspects of a metric that are part of the metric model but the details of which are out of
scope of the metric model in ISO/IEC 19086-2, such as the use of rule and expression languages and
how these constructs relate to each other. Supportive of the goal of harmonizing the usage of the metric
model across users, this document proposes a tabular representation for metric definitions that is
derived from and augments the tables provided in Annex B of the metric model in ISO/IEC 19086-2:2018,
as explained in 7.2.2.
6.2 Audience and some user categories
6.2.1 General
The audience for this document is expected to be diverse, as the metric representation proposed in
this document is intended for different parties involved in providing or using cloud services. However
not every clause is of interest to all. Those who read, negotiate or create SLA content, such as business
users and administrators, are expected to be interested in Clauses 1 to 7 and in the initial approach
2 PROOF/ÉPREUVE © ISO/IEC 2020 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/IEC TR 23951:2020(E)
to the first metric example (see 8.1). In addition to these clauses, metric designers and developers
are expected to be interested in the remaining clauses including more elaborate examples of metrics
(starting from 8.2 and beyond).
The parties interested in this document include representatives of the following roles defined in
ISO/IEC 17788.
6.2.2 Cloud service customer (CSC)
This document helps the CSC to understand the metrics used for service quality and other assurances
described in SLAs. When blended into the narrative of the SLA, metrics are often ambiguous or
incomplete. A structured definition as described in the metric model and made practical with a tabular
representation helps to avoid or at least detect such issues.
Specific types of customers are interested in understanding how a service is measured without having
to read the entire SLA or prior to establishing an SLA. These customers are defined in the CCRA as
a cloud service users (who uses a cloud service to fulfil her/his role), a service administrator (who
oversees all the operational processes relating to the use of cloud services, serving as intermediary
between the user and the provider) and a business manager (who has overall responsibility for the
business aspects of using cloud services, including the purchase of the service under appropriate terms
and possibly the request of audit reports).
The tabular representation in this document is also an analysis tool for the CSC to identify and extract
the metric material found in an SLA in order to get a clearer understanding of how the service is
measured, as illustrated in 8.1.
6.2.3 Cloud service provider (CSP)
This document helps the CSP to describe the service metrics that support his or her SLAs, potentially
avoiding contentious claims afterward that result from CSCs misunderstanding the terms and conditions
of these SLAs. It also helps providers to harmonize their metrics across data-centre operators or world
regions. Among activities expected from CSPs as defined in ISO/IEC 17789, the following are facilitated
by metric definitions and evaluations: monitoring service, administering service security, providing
audit data on request, defining and gathering metrics, managing security and risks, and, finally,
handling support requests, reports and incidents from cloud service customers. For these activities,
this document helps to establish a common and unambiguous representation of metrics used between
parties involved in these activities and distinct from other SLA material.
6.2.4 Cloud service partner (CSN)
The following CSN sub-roles are expected to find value in a metric definition template and guidelines:
— Cloud service developer: this user is responsible for designing, developing, integrating, testing,
and maintaining cloud services. Developers need to understand the measurements used to evaluate
a cloud service. This role includes composing a new cloud service from existing separate cloud
services. By having access to precise metric definitions and their rules, such as those illustrated in
Clauses 8 and 10, developers understand what features are to be monitored, what is the expected
quality of the developed cloud service and its priorities, as well as how to evaluate the quality and
risks when integrating third party cloud services.
— Cloud auditor: the auditor has the responsibility of conducting an audit of the provision and the use
of cloud services. A cloud audit typically covers operations, performance and security and examines
whether a specified set of audit criteria are met. By using metrics, the auditor understands or
communicates clearly the details of the measurements to perform. Such precision and clarity
are provided by a distinct and detailed metric representation, as illustrated in the two examples
developed in Clauses 8 and 10.
© ISO/IEC 2020 – All rights reserved PROOF/ÉPREUVE 3
---------------------- Page: 9 ----------------------
ISO/IEC TR 23951:2020(E)
6.2.5 Regulators and policy makers
Several aspects of policy definition and enforcement concern measurable properties both about
the cloud service usage (including cloud service usage duration and times, volume and type of data
involved), and the cloud service performance (such as cloud service quality, elasticity and scalability,
availability and reliability). Other policies (such as those about trust and transparency, security
procedures, privacy) concern the relationship, governance and risk management between parties,
especially CSCs and CSPs. Whether these policies involve automated monitoring or some human
assessment instead, they rely on some form of measurement for tracking their implementation. See
ISO/IEC TR 22678 for more information regarding the development of policies that govern or regulate
cloud service providers (CSPs) and cloud services, and those policies and practices that govern the use
of cloud services in organizations.
The expression of policies and rules sometimes translates into predefined metric elements that are
expected to be used even when defining a customized metric. An example is of a policy that determines
the formula (metric expression) to be used when assessing cloud service uptime percentage, while
leaving other details unconstrained. As another example, if there is agreement for sharing across CSPs
the common definitions of “natural disaster” or “service misuse”, the reuse of such definitions helps to
establish a common understanding of what a valid cloud service downtime means. Creating and sharing
predefined metric material is a usage pattern described in 6.3.5 as sharing a metric foundation.
6.3 Usage patterns
6.3.1 General
A summary of various usage patterns for the tabular metric representation given in 7.2 and a rationale
for doing so are provided in the next subclauses. Some of these usage patterns match usage categories
identified in ISO/IEC 19086-2:2018, 6.4.2.
6.3.2 Extract and clarify an existing metric description from an SLA
Often, the metric(s) information in a cloud service SLA is scattered over the SLA narrative. Parts of
metric material (such as measurement rules, exceptions, underlying quantities and metrics) is mixed
with related information that is not part of the metric definition (such as performance objectives,
remediation measures and penalties).
Distinguishing a metric definition apart from its context of usage in an SLA and us
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.