Information technology — Cloud computing — Guidance for using the cloud SLA metric model

The scope of this document is to describe guidance for using the ISO/IEC 19086-2 metric model, illustrated with examples.

Technologies de l'information — Informatique en nuage — Recommandations pour l'utilisation du modèle métrique d'accord de niveau de service (SLA) dans le Cloud

General Information

Status
Published
Publication Date
25-Jun-2020
Current Stage
6060 - International Standard published
Start Date
26-Jun-2020
Completion Date
26-Jun-2020
Ref Project

Buy Standard

Technical report
ISO/IEC TR 23951:2020 - Information technology -- Cloud computing -- Guidance for using the cloud SLA metric model
English language
34 pages
sale 15% off
Preview
sale 15% off
Preview
Draft
ISO/IEC PRF TR 23951 - Information technology -- Cloud computing -- Guidance for using the cloud SLA metric model
English language
34 pages
sale 15% off
Preview
sale 15% off
Preview

Standards Content (sample)

TECHNICAL ISO/IEC TR
REPORT 23951
First edition
2020-06
Information technology — Cloud
computing — Guidance for using the
cloud SLA metric model
Technologies de l'information — Informatique en nuage —
Recommandations pour l'utilisation du modèle métrique d'accord de
niveau de service (SLA) dans le Cloud
Reference number
ISO/IEC TR 23951:2020(E)
ISO/IEC 2020
---------------------- Page: 1 ----------------------
ISO/IEC TR 23951:2020(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2020

All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting

on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address

below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2020 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC TR 23951:2020(E)
Contents Page

Foreword ..........................................................................................................................................................................................................................................v

Introduction ................................................................................................................................................................................................................................vi

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ...................................................................................................................................................................................... 1

3 Terms and definitions ..................................................................................................................................................................................... 1

4 Symbols and abbreviated terms ........................................................................................................................................................... 1

5 Structure of this document ........................................................................................................................................................................ 2

6 Motivation.................................................................................................................................................................................................................... 2

6.1 Preamble ...................................................................................................................................................................................................... 2

6.2 Audience and some user categories ..................................................................................................................................... 2

6.2.1 General...................................................................................................................................................................................... 2

6.2.2 Cloud service customer (CSC) .............................................................................................................................. 3

6.2.3 Cloud service provider (CSP) ................................................................................................................................ 3

6.2.4 Cloud service partner (CSN) .................................................................................................................................. 3

6.2.5 Regulators and policy makers .............................................................................................................................. 4

6.3 Usage patterns ......................................................................................................................................................................................... 4

6.3.1 General...................................................................................................................................................................................... 4

6.3.2 Extract and clarify an existing metric description from an SLA ............................................. 4

6.3.3 Create and share a metric description .......................................................................................................... 4

6.3.4 Compare metric descriptions ............................................................................................................................... 5

6.3.5 Share a common foundation for a set of metrics ................................................................................. 5

6.3.6 Build a metrics catalogue ......................................................................................................................................... 5

6.4 Examples of scenarios and roles involved in sharing metric definitions ............................................. 5

7 The metric model in practice: templates .................................................................................................................................... 6

7.1 A brief reminder of the metric model ................................................................................................................................. 6

7.2 A tabular representation ................................................................................................................................................................ 7

7.2.1 General...................................................................................................................................................................................... 7

7.2.2 The tabular representation for the Metric element ........................................................................... 8

7.2.3 The tabular representation for the Expression elements ............................................................ 9

7.2.4 The tabular representation for the Rule elements ..........................................................................10

7.2.5 The tabular representation for the Parameter elements ...........................................................11

8 An example of metric definition: the cloud service mean response time metric ..........................11

8.1 The cloud service mean response time metric: informal variant ............................................................11

8.1.1 Extracting metric elements from an SLA narrative .........................................................................11

8.1.2 Using the tabular representation ...................................................................................................................12

8.1.3 Overall structure of the metric .........................................................................................................................14

8.2 The cloud service mean response time metric: more formal variant ...................................................14

8.2.1 A more formal variant of the metric ............................................................................................................14

8.2.2 Adding a parameter ....................................................................................................................................................15

8.2.3 The metric rules ............................................................................................................................................................15

8.2.4 The metric expressions ...........................................................................................................................................15

8.2.5 Overall structure of the metric .........................................................................................................................17

8.2.6 Using constants ..............................................................................................................................................................17

9 Guidelines for using the metric model with the tabular representation ...............................................19

9.1 General ........................................................................................................................................................................................................19

9.2 Guideline 1 about defining expression and rule languages...........................................................................20

9.3 Guideline 2 about associating rules with expressions .......................................................................................20

9.4 Guideline 3 about relating expressions to each other ........................................................................................20

9.5 Guideline 4 about the identifiers of metric elements .........................................................................................21

9.6 Guideline 5 about rules specifically designed to support an expression ...........................................21

9.7 Guideline 6 about the role of parameters .....................................................................................................................21

© ISO/IEC 2020 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC TR 23951:2020(E)

9.8 Guideline 7 about representing constants ...................................................................................................................22

10 The simple cloud service availability metric .......................................................................................................................22

10.1 Measuring cloud service availability .................................................................................................................................22

10.1.1 General...................................................................................................................................................................................22

10.1.2 Overall design approach .........................................................................................................................................23

10.1.3 SLA rules and metric rules ...................................................................................................................................23

10.2 The simple cloud service availability metric variant Simple_SAM_1 ....................................................24

10.2.1 The Metric element .....................................................................................................................................................24

10.2.2 The metric rules ............................................................................................................................................................24

10.2.3 The metric expressions ...........................................................................................................................................25

10.2.4 The metric parameters ............................................................................................................................................27

10.2.5 Overall structure of the metric .........................................................................................................................27

10.3 The simple cloud service availability metric variant Simple_SAM_2 ....................................................28

10.3.1 Differences in metric design and assumptions ...................................................................................28

10.3.2 The Metric element .....................................................................................................................................................29

10.3.3 The metric rules ............................................................................................................................................................29

10.3.4 The metric parameters ............................................................................................................................................30

10.3.5 The metric expressions ...........................................................................................................................................31

10.3.6 Overall structure of the metric .........................................................................................................................32

10.3.7 An alternative metric design using the Configuration element option ..........................32

Bibliography .............................................................................................................................................................................................................................34

iv © ISO/IEC 2020 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC TR 23951:2020(E)
Foreword

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical

Commission) form the specialized system for worldwide standardization. National bodies that

are members of ISO or IEC participate in the development of International Standards through

technical committees established by the respective organization to deal with particular fields of

technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other

international organizations, governmental and non-governmental, in liaison with ISO and IEC, also

take part in the work.

The procedures used to develop this document and those intended for its further maintenance are

described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for

the different types of document should be noted. This document was drafted in accordance with the

editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).

Attention is drawn to the possibility that some of the elements of this document may be the subject

of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent

rights. Details of any patent rights identified during the development of the document will be in the

Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC

list of patent declarations received (see http:// patents .iec .ch).

Any trade name used in this document is information given for the convenience of users and does not

constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and

expressions related to conformity assessment, as well as information about ISO's adherence to the

World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www .iso .org/

iso/ foreword .html.

This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,

Subcommittee SC 38, Cloud Computing and Distributed Platforms.

Any feedback or questions on this document should be directed to the user’s national standards body. A

complete listing of these bodies can be found at www .iso .org/ members .html.
© ISO/IEC 2020 – All rights reserved v
---------------------- Page: 5 ----------------------
ISO/IEC TR 23951:2020(E)
Introduction

In most cases, cloud service providers (CSPs) and cloud service customers (CSCs) negotiate service

level agreements (SLAs) which include service level objectives (SLOs) and service qualitative objectives

(SQOs) for which CSPs make commitments. The commitments described in SLAs are expected to be

measured against actual performance of the service to ensure compliance with the SLA. How actual

performance compares against commitments in SLAs is explained in ISO/IEC 19086-2. Cloud SLAs are

covered in ISO/IEC 19086-1 and in ISO/IEC 19086-4.

The metric model in ISO/IEC 19086-2 establishes common terminology, defines a model for specifying

metrics for cloud SLAs, and includes applications of the model with examples. This document provides

guidance and examples on using the metric model to compose the calculation of a cloud service

performance measure in order to compare against an SLA commitment. A few examples from the

SLOs listed in ISO/IEC 19086-1:2016, Clause 10 are given in the document, such as Cloud Service Mean

Response Time and Simple Cloud Service Availability. As specific, measurable characteristics of a cloud

service, SLOs are the basis for defining the metrics used to evaluate and compare agreements between

parties.

In Clauses 8, 9 and 10 of this document, a basic explanation of these examples is provided using a

practical method based on a tabular format that is a refinement of the informative tables provided

in ISO/IEC 19086-2:2018, Annex B. The tabular representation described in this document serves as

templates for designing metrics. Guidance in using the metric model with these templates is provided

while developing metric examples.
vi © ISO/IEC 2020 – All rights reserved
---------------------- Page: 6 ----------------------
TECHNICAL REPORT ISO/IEC TR 23951:2020(E)
Information technology — Cloud computing — Guidance
for using the cloud SLA metric model
1 Scope

The scope of this document is to describe guidance for using the ISO/IEC 19086-2 metric model,

illustrated with examples.
2 Normative references

The following documents are referred to in the text in such a way that some or all of their content

constitutes requirements of this document. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any amendments) applies.

ISO/IEC 17788, Information technology — Cloud computing — Overview and vocabulary

ISO/IEC 17789, Information technology — Cloud computing — Reference architecture

ISO/IEC 19086-1, Information technology — Cloud computing — Service level agreement (SLA)

framework — Part 1: Overview and concepts

ISO/IEC 19086-2, Cloud computing — Service level agreement (SLA) framework — Part 2: Metric model

3 Terms and definitions

For the purposes of this document, the terms and definitions given in ISO/IEC 17788, ISO/IEC 17789,

ISO/IEC 19086-1 and ISO/IEC 19086-2 apply.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
4 Symbols and abbreviated terms
CCRA cloud computing reference architecture
CSC cloud service customer
CSN cloud service partner
CSP cloud service provider
SLA service level agreement
SLO service level objective
SQO service quality objective
© ISO/IEC 2020 – All rights reserved 1
---------------------- Page: 7 ----------------------
ISO/IEC TR 23951:2020(E)
5 Structure of this document

In supporting the scope presented in Clause 1, this document develops the rationale for a practical

metric representation to complement the metric model in ISO/IEC 19086-2 in the following clauses:

— Clause 6 states the rationale for complementing the metric model as defined in ISO/IEC 19086-2

with a practical representation and for providing related usage guidance as introduced by this

document. It identifies some usage patterns and highlights some usage scenarios where metric

definitions are shared across various parties. The users who benefit from this document include

parties with roles defined in ISO/IEC 17789 (Cloud computing — Reference architecture).

— Clause 7 introduces the tabular metric representation supportive of the metric model and derived

from the informative tables listed in ISO/IEC 19086-2:2018, Annex B. This representation is based on

tables intended to serve as templates for metric definitions. This clause represents initial guidance in

using the metric model, which is then illustrated and discussed throughout the examples developed

later in the document.

— Clause 8 introduces a simple case of metric definition that illustrates the use of the table templates

introduced in Clause 7. This example starts with the description of a metric as it would appear in

the narrative of an existing SLA and illustrates the extraction of this description toward a more

structured and distinct representation using the proposed tabular representation. The example

shows practical aspects when designing and developing metrics, such as how metric rules relate to

expressions, and how to parameterize rules and expressions.

— Clause 9 is a set of guidelines on how to use the metric model with the tabular templates (Clause 7).

This guidance is motivated and illustrated by the examples throughout the document. These

guidelines are best understood after developing a preliminary example (Clause 8). They explain how

to use the metric model with the tabular templates for metric use cases posing similar challenges or

using similar features.

— Clause 10 develops a more elaborate metric example for cloud service availability. It describes

two variants of the same metric that illustrate two different approaches in using the metric

model elements. Since it comes after the guideline items listed in Clause 9, it is easier to relate the

development of this second example to these guidelines.
6 Motivation
6.1 Preamble

This clause first identifies the audience of this document and for the tabular metric representation

described in this document. This clause then describes some metric usage patterns and then identifies

scenarios and roles for these metric usage patterns. Sharing common guidelines and conventions in

using the metric model improves the ability to reuse and compare metrics. These common guidelines

extend to the aspects of a metric that are part of the metric model but the details of which are out of

scope of the metric model in ISO/IEC 19086-2, such as the use of rule and expression languages and

how these constructs relate to each other. Supportive of the goal of harmonizing the usage of the metric

model across users, this document proposes a tabular representation for metric definitions that is

derived from and augments the tables provided in Annex B of the metric model in ISO/IEC 19086-2:2018,

as explained in 7.2.2.
6.2 Audience and some user categories
6.2.1 General

The audience for this document is expected to be diverse, as the metric representation proposed in

this document is intended for different parties involved in providing or using cloud services. However

not every clause is of interest to all. Those who read, negotiate or create SLA content, such as business

users and administrators, are expected to be interested in Clauses 1 to 7 and in the initial approach

2 © ISO/IEC 2020 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/IEC TR 23951:2020(E)

to the first metric example (see 8.1). In addition to these clauses, metric designers and developers

are expected to be interested in the remaining clauses including more elaborate examples of metrics

(starting from 8.2 and beyond).

The parties interested in this document include representatives of the following roles defined in

ISO/IEC 17788.
6.2.2 Cloud service customer (CSC)

This document helps the CSC to understand the metrics used for service quality and other assurances

described in SLAs. When blended into the narrative of the SLA, metrics are often ambiguous or

incomplete. A structured definition as described in the metric model and made practical with a tabular

representation helps to avoid or at least detect such issues.

Specific types of customers are interested in understanding how a service is measured without having

to read the entire SLA or prior to establishing an SLA. These customers are defined in the CCRA as

a cloud service users (who uses a cloud service to fulfil her/his role), a service administrator (who

oversees all the operational processes relating to the use of cloud services, serving as intermediary

between the user and the provider) and a business manager (who has overall responsibility for the

business aspects of using cloud services, including the purchase of the service under appropriate terms

and possibly the request of audit reports).

The tabular representation in this document is also an analysis tool for the CSC to identify and extract

the metric material found in an SLA in order to get a clearer understanding of how the service is

measured, as illustrated in 8.1.
6.2.3 Cloud service provider (CSP)

This document helps the CSP to describe the service metrics that support his or her SLAs, potentially

avoiding contentious claims afterward that result from CSCs misunderstanding the terms and conditions

of these SLAs. It also helps providers to harmonize their metrics across data-centre operators or world

regions. Among activities expected from CSPs as defined in ISO/IEC 17789, the following are facilitated

by metric definitions and evaluations: monitoring service, administering service security, providing

audit data on request, defining and gathering metrics, managing security and risks, and, finally,

handling support requests, reports and incidents from cloud service customers. For these activities,

this document helps to establish a common and unambiguous representation of metrics used between

parties involved in these activities and distinct from other SLA material.
6.2.4 Cloud service partner (CSN)

The following CSN sub-roles are expected to find value in a metric definition template and guidelines:

— Cloud service developer: this user is responsible for designing, developing, integrating, testing,

and maintaining cloud services. Developers need to understand the measurements used to evaluate

a cloud service. This role includes composing a new cloud service from existing separate cloud

services. By having access to precise metric definitions and their rules, such as those illustrated in

Clauses 8 and 10, developers understand what features are to be monitored, what is the expected

quality of the developed cloud service and its priorities, as well as how to evaluate the quality and

risks when integrating third party cloud services.

— Cloud auditor: the auditor has the responsibility of conducting an audit of the provision and the use

of cloud services. A cloud audit typically covers operations, performance and security and examines

whether a specified set of audit criteria are met. By using metrics, the auditor understands or

communicates clearly the details of the measurements to perform. Such precision and clarity

are provided by a distinct and detailed metric representation, as illustrated in the two examples

developed in Clauses 8 and 10.
© ISO/IEC 2020 – All rights reserved 3
---------------------- Page: 9 ----------------------
ISO/IEC TR 23951:2020(E)
6.2.5 Regulators and policy makers

Several aspects of policy definition and enforcement concern measurable properties both about

the cloud service usage (including cloud service usage duration and times, volume and type of data

involved), and the cloud service performance (such as cloud service quality, elasticity and scalability,

availability and reliability). Other policies (such as those about trust and transparency, security

procedures, privacy) concern the relationship, governance and risk management between parties,

especially CSCs and CSPs. Whether these policies involve automated monitoring or some human

assessment instead, they rely on some form of measurement for tracking their implementation. See

ISO/IEC TR 22678 for more information regarding the development of policies that govern or regulate

cloud service providers (CSPs) and cloud services, and those policies and practices that govern the use

of cloud services in organizations.

The expression of policies and rules sometimes translates into predefined metric elements that are

expected to be used even when defining a customized metric. An example is of a policy that determines

the formula (metric expression) to be used when assessing cloud service uptime percentage, while

leaving other details unconstrained. As another example, if there is agreement for sharing across CSPs

the common definitions of “natural disaster” or “service misuse”, the reuse of such definitions helps to

establish a common understanding of what a valid cloud service downtime means. Creating and sharing

predefined metric material is a usage pattern described in 6.3.5 as sharing a metric foundation.

6.3 Usage patterns
6.3.1 General

A summary of various usage patterns for the tabular metric representation given in 7.2 and a rationale

for doing so are provided in the next subclauses. Some of these usage patterns match usage categories

identified in ISO/IEC 19086-2:2018, 6.4.2.
6.3.2 Extract and clarify an existing metric description from an SLA

Often, the metric(s) information in a cloud service SLA is scattered over the SLA narrative. Parts of

metric material (such as measurement rules, exceptions, underlying quantities and metrics) is mixed

with related information that is not part of the metric definition (such as performance objectives,

remediation measures and penalties).
Distinguishing a metric definition apart from its context
...

TECHNICAL ISO/IEC TR
REPORT 23951
First edition
Information technology — Cloud
computing — Guidance for using the
cloud SLA metric model
PROOF/ÉPREUVE
Reference number
ISO/IEC TR 23951:2020(E)
ISO/IEC 2020
---------------------- Page: 1 ----------------------
ISO/IEC TR 23951:2020(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2020

All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting

on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address

below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii PROOF/ÉPREUVE © ISO/IEC 2020 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC TR 23951:2020(E)
Contents Page

Foreword ..........................................................................................................................................................................................................................................v

Introduction ................................................................................................................................................................................................................................vi

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ...................................................................................................................................................................................... 1

3 Terms and definitions ..................................................................................................................................................................................... 1

4 Symbols and abbreviated terms ........................................................................................................................................................... 1

5 Structure of this document ........................................................................................................................................................................ 2

6 Motivation.................................................................................................................................................................................................................... 2

6.1 Preamble ...................................................................................................................................................................................................... 2

6.2 Audience and some user categories ..................................................................................................................................... 2

6.2.1 General...................................................................................................................................................................................... 2

6.2.2 Cloud service customer (CSC) .............................................................................................................................. 3

6.2.3 Cloud service provider (CSP) ................................................................................................................................ 3

6.2.4 Cloud service partner (CSN) .................................................................................................................................. 3

6.2.5 Regulators and policy makers .............................................................................................................................. 4

6.3 Usage patterns ......................................................................................................................................................................................... 4

6.3.1 General...................................................................................................................................................................................... 4

6.3.2 Extract and clarify an existing metric description from an SLA ............................................. 4

6.3.3 Create and share a metric description .......................................................................................................... 4

6.3.4 Compare metric descriptions ............................................................................................................................... 5

6.3.5 Share a common foundation for a set of metrics ................................................................................. 5

6.3.6 Build a metrics catalogue ......................................................................................................................................... 5

6.4 Examples of scenarios and roles involved in sharing metric definitions ............................................. 5

7 The metric model in practice: templates .................................................................................................................................... 6

7.1 A brief reminder of the metric model ................................................................................................................................. 6

7.2 A tabular representation ................................................................................................................................................................ 7

7.2.1 General...................................................................................................................................................................................... 7

7.2.2 The tabular representation for the Metric element ........................................................................... 8

7.2.3 The tabular representation for the Expression elements ............................................................ 9

7.2.4 The tabular representation for the Rule elements ..........................................................................10

7.2.5 The tabular representation for the Parameter elements ...........................................................11

8 An example of metric definition: the cloud service mean response time metric ..........................11

8.1 The cloud service mean response time metric: informal variant ............................................................11

8.1.1 Extracting metric elements from an SLA narrative .........................................................................11

8.1.2 Using the tabular representation ...................................................................................................................12

8.1.3 Overall structure of the metric .........................................................................................................................14

8.2 The cloud service mean response time metric: more formal variant ...................................................14

8.2.1 A more formal variant of the metric ............................................................................................................14

8.2.2 Adding a parameter ....................................................................................................................................................15

8.2.3 The metric rules ............................................................................................................................................................15

8.2.4 The metric expressions ...........................................................................................................................................15

8.2.5 Overall structure of the metric .........................................................................................................................17

8.2.6 Using constants ..............................................................................................................................................................17

9 Guidelines for using the metric model with the tabular representation ...............................................19

9.1 General ........................................................................................................................................................................................................19

9.2 Guideline 1 about defining expression and rule languages...........................................................................20

9.3 Guideline 2 about associating rules with expressions .......................................................................................20

9.4 Guideline 3 about relating expressions to each other ........................................................................................20

9.5 Guideline 4 about the identifiers of metric elements .........................................................................................21

9.6 Guideline 5 about rules specifically designed to support an expression ...........................................21

9.7 Guideline 6 about the role of parameters .....................................................................................................................21

© ISO/IEC 2020 – All rights reserved PROOF/ÉPREUVE iii
---------------------- Page: 3 ----------------------
ISO/IEC TR 23951:2020(E)

9.8 Guideline 7 about representing constants ...................................................................................................................22

10 The simple cloud service availability metric .......................................................................................................................22

10.1 Measuring cloud service availability .................................................................................................................................22

10.1.1 General...................................................................................................................................................................................22

10.1.2 Overall design approach .........................................................................................................................................23

10.1.3 SLA rules and metric rules ...................................................................................................................................23

10.2 The simple cloud service availability metric variant Simple_SAM_1 ....................................................24

10.2.1 The Metric element .....................................................................................................................................................24

10.2.2 The metric rules ............................................................................................................................................................24

10.2.3 The metric expressions ...........................................................................................................................................25

10.2.4 The metric parameters ............................................................................................................................................27

10.2.5 Overall structure of the metric .........................................................................................................................27

10.3 The simple cloud service availability metric variant Simple_SAM_2 ....................................................28

10.3.1 Differences in metric design and assumptions ...................................................................................28

10.3.2 The Metric element .....................................................................................................................................................29

10.3.3 The metric rules ............................................................................................................................................................29

10.3.4 The metric parameters ............................................................................................................................................30

10.3.5 The metric expressions ...........................................................................................................................................31

10.3.6 Overall structure of the metric .........................................................................................................................32

10.3.7 An alternative metric design using the Configuration element option ..........................32

Bibliography .............................................................................................................................................................................................................................34

iv PROOF/ÉPREUVE © ISO/IEC 2020 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC TR 23951:2020(E)
Foreword

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical

Commission) form the specialized system for worldwide standardization. National bodies that

are members of ISO or IEC participate in the development of International Standards through

technical committees established by the respective organization to deal with particular fields of

technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other

international organizations, governmental and non-governmental, in liaison with ISO and IEC, also

take part in the work.

The procedures used to develop this document and those intended for its further maintenance are

described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for

the different types of document should be noted. This document was drafted in accordance with the

editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).

Attention is drawn to the possibility that some of the elements of this document may be the subject

of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent

rights. Details of any patent rights identified during the development of the document will be in the

Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC

list of patent declarations received (see http:// patents .iec .ch).

Any trade name used in this document is information given for the convenience of users and does not

constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and

expressions related to conformity assessment, as well as information about ISO's adherence to the

World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www .iso .org/

iso/ foreword .html.

This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,

Subcommittee SC 38, Cloud Computing and Distributed Platforms.

Any feedback or questions on this document should be directed to the user’s national standards body. A

complete listing of these bodies can be found at www .iso .org/ members .html.
© ISO/IEC 2020 – All rights reserved PROOF/ÉPREUVE v
---------------------- Page: 5 ----------------------
ISO/IEC TR 23951:2020(E)
Introduction

In most cases, cloud service providers (CSPs) and cloud service customers (CSCs) negotiate service

level agreements (SLAs) which include service level objectives (SLOs) and service qualitative objectives

(SQOs) for which CSPs make commitments. The commitments described in SLAs are expected to be

measured against actual performance of the service to ensure compliance with the SLA. How actual

performance compares against commitments in SLAs is explained in ISO/IEC 19086-2. Cloud SLAs are

covered in ISO/IEC 19086-1 and in ISO/IEC 19086-4.

The metric model in ISO/IEC 19086-2 establishes common terminology, defines a model for specifying

metrics for cloud SLAs, and includes applications of the model with examples. This document provides

guidance and examples on using the metric model to compose the calculation of a cloud service

performance measure in order to compare against an SLA commitment. A few examples from the SLOs

listed in Clause 10 of ISO/IEC 19086-1 are given in the document, such as Cloud Service Mean Response

Time and Simple Cloud Service Availability. As specific, measurable characteristics of a cloud service,

SLOs are the basis for defining the metrics used to evaluate and compare agreements between parties.

In Clauses 8, 9 and 10 of this document, a basic explanation of these examples is provided using a

practical method based on a tabular format that is a refinement of the informative tables provided

in ISO/IEC 19086-2:2018, Annex B. The tabular representation described in this document serves as

templates for designing metrics. Guidance in using the metric model with these templates is provided

while developing metric examples.
vi PROOF/ÉPREUVE © ISO/IEC 2020 – All rights reserved
---------------------- Page: 6 ----------------------
TECHNICAL REPORT ISO/IEC TR 23951:2020(E)
Information technology — Cloud computing — Guidance
for using the cloud SLA metric model
1 Scope

The scope of this technical report is to describe guidance for using the ISO/IEC 19086-2 metric model,

illustrated with examples.
2 Normative references

The following documents are referred to in the text in such a way that some or all of their content

constitutes requirements of this document. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any amendments) applies.

ISO/IEC 17788, Information technology — Cloud computing — Overview and vocabulary

ISO/IEC 17789, Information technology — Cloud computing — Reference architecture

ISO/IEC 19086-1, Information technology — Cloud computing — Service level agreement (SLA)

framework — Part 1: Overview and concepts

ISO/IEC 19086-2, Cloud computing — Service level agreement (SLA) framework — Part 2: Metric model

3 Terms and definitions

For the purposes of this document, the terms and definitions given in ISO/IEC 17788, ISO/IEC 17789,

ISO/IEC 19086-1 and ISO/IEC 19086-2 apply.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
4 Symbols and abbreviated terms
CCRA cloud computing reference architecture
CSC cloud service customer
CSN cloud service partner
CSP cloud service provider
SLA service level agreement
SLO service level objective
SQO service quality objective
© ISO/IEC 2020 – All rights reserved PROOF/ÉPREUVE 1
---------------------- Page: 7 ----------------------
ISO/IEC TR 23951:2020(E)
5 Structure of this document

In supporting the scope presented in Clause 1, this document develops the rationale for a practical

metric representation to complement the metric model in ISO/IEC 19086-2 in the following clauses:

— Clause 6 states the rationale for complementing the metric model as defined in ISO/IEC 19086-2

with a practical representation and for providing related usage guidance as introduced by this

document. It identifies some usage patterns and highlights some usage scenarios where metric

definitions are shared across various parties. The users who benefit from this document include

parties with roles defined in ISO/IEC 17789 (Cloud computing — Reference architecture).

— Clause 7 introduces the tabular metric representation supportive of the metric model and derived

from the informative tables listed in ISO/IEC 19086-2:2018, Annex B. This representation is based on

tables intended to serve as templates for metric definitions. This clause represents initial guidance in

using the metric model, which is then illustrated and discussed throughout the examples developed

later in the document.

— Clause 8 introduces a simple case of metric definition that illustrates the use of the table templates

introduced in Clause 7. This example starts with the description of a metric as it would appear in

the narrative of an existing SLA and illustrates the extraction of this description toward a more

structured and distinct representation using the proposed tabular representation. The example

shows practical aspects when designing and developing metrics, such as how metric rules relate to

expressions, and how to parameterize rules and expressions.

— Clause 9 is a set of guidelines on how to use the metric model with the tabular templates (Clause 7).

This guidance is motivated and illustrated by the examples throughout the document. These

guidelines are best understood after developing a preliminary example (Clause 8). They explain how

to use the metric model with the tabular templates for metric use cases posing similar challenges or

using similar features.

— Clause 10 develops a more elaborate metric example for cloud service availability. It describes

two variants of the same metric that illustrate two different approaches in using the metric

model elements. Since it comes after the guideline items listed in Clause 9, it is easier to relate the

development of this second example to these guidelines.
6 Motivation
6.1 Preamble

This clause first identifies the audience of this document and for the tabular metric representation

described in this document. This clause then describes some metric usage patterns and then identifies

scenarios and roles for these metric usage patterns. Sharing common guidelines and conventions in

using the metric model improves the ability to reuse and compare metrics. These common guidelines

extend to the aspects of a metric that are part of the metric model but the details of which are out of

scope of the metric model in ISO/IEC 19086-2, such as the use of rule and expression languages and

how these constructs relate to each other. Supportive of the goal of harmonizing the usage of the metric

model across users, this document proposes a tabular representation for metric definitions that is

derived from and augments the tables provided in Annex B of the metric model in ISO/IEC 19086-2:2018,

as explained in 7.2.2.
6.2 Audience and some user categories
6.2.1 General

The audience for this document is expected to be diverse, as the metric representation proposed in

this document is intended for different parties involved in providing or using cloud services. However

not every clause is of interest to all. Those who read, negotiate or create SLA content, such as business

users and administrators, are expected to be interested in Clauses 1 to 7 and in the initial approach

2 PROOF/ÉPREUVE © ISO/IEC 2020 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/IEC TR 23951:2020(E)

to the first metric example (see 8.1). In addition to these clauses, metric designers and developers

are expected to be interested in the remaining clauses including more elaborate examples of metrics

(starting from 8.2 and beyond).

The parties interested in this document include representatives of the following roles defined in

ISO/IEC 17788.
6.2.2 Cloud service customer (CSC)

This document helps the CSC to understand the metrics used for service quality and other assurances

described in SLAs. When blended into the narrative of the SLA, metrics are often ambiguous or

incomplete. A structured definition as described in the metric model and made practical with a tabular

representation helps to avoid or at least detect such issues.

Specific types of customers are interested in understanding how a service is measured without having

to read the entire SLA or prior to establishing an SLA. These customers are defined in the CCRA as

a cloud service users (who uses a cloud service to fulfil her/his role), a service administrator (who

oversees all the operational processes relating to the use of cloud services, serving as intermediary

between the user and the provider) and a business manager (who has overall responsibility for the

business aspects of using cloud services, including the purchase of the service under appropriate terms

and possibly the request of audit reports).

The tabular representation in this document is also an analysis tool for the CSC to identify and extract

the metric material found in an SLA in order to get a clearer understanding of how the service is

measured, as illustrated in 8.1.
6.2.3 Cloud service provider (CSP)

This document helps the CSP to describe the service metrics that support his or her SLAs, potentially

avoiding contentious claims afterward that result from CSCs misunderstanding the terms and conditions

of these SLAs. It also helps providers to harmonize their metrics across data-centre operators or world

regions. Among activities expected from CSPs as defined in ISO/IEC 17789, the following are facilitated

by metric definitions and evaluations: monitoring service, administering service security, providing

audit data on request, defining and gathering metrics, managing security and risks, and, finally,

handling support requests, reports and incidents from cloud service customers. For these activities,

this document helps to establish a common and unambiguous representation of metrics used between

parties involved in these activities and distinct from other SLA material.
6.2.4 Cloud service partner (CSN)

The following CSN sub-roles are expected to find value in a metric definition template and guidelines:

— Cloud service developer: this user is responsible for designing, developing, integrating, testing,

and maintaining cloud services. Developers need to understand the measurements used to evaluate

a cloud service. This role includes composing a new cloud service from existing separate cloud

services. By having access to precise metric definitions and their rules, such as those illustrated in

Clauses 8 and 10, developers understand what features are to be monitored, what is the expected

quality of the developed cloud service and its priorities, as well as how to evaluate the quality and

risks when integrating third party cloud services.

— Cloud auditor: the auditor has the responsibility of conducting an audit of the provision and the use

of cloud services. A cloud audit typically covers operations, performance and security and examines

whether a specified set of audit criteria are met. By using metrics, the auditor understands or

communicates clearly the details of the measurements to perform. Such precision and clarity

are provided by a distinct and detailed metric representation, as illustrated in the two examples

developed in Clauses 8 and 10.
© ISO/IEC 2020 – All rights reserved PROOF/ÉPREUVE 3
---------------------- Page: 9 ----------------------
ISO/IEC TR 23951:2020(E)
6.2.5 Regulators and policy makers

Several aspects of policy definition and enforcement concern measurable properties both about

the cloud service usage (including cloud service usage duration and times, volume and type of data

involved), and the cloud service performance (such as cloud service quality, elasticity and scalability,

availability and reliability). Other policies (such as those about trust and transparency, security

procedures, privacy) concern the relationship, governance and risk management between parties,

especially CSCs and CSPs. Whether these policies involve automated monitoring or some human

assessment instead, they rely on some form of measurement for tracking their implementation. See

ISO/IEC TR 22678 for more information regarding the development of policies that govern or regulate

cloud service providers (CSPs) and cloud services, and those policies and practices that govern the use

of cloud services in organizations.

The expression of policies and rules sometimes translates into predefined metric elements that are

expected to be used even when defining a customized metric. An example is of a policy that determines

the formula (metric expression) to be used when assessing cloud service uptime percentage, while

leaving other details unconstrained. As another example, if there is agreement for sharing across CSPs

the common definitions of “natural disaster” or “service misuse”, the reuse of such definitions helps to

establish a common understanding of what a valid cloud service downtime means. Creating and sharing

predefined metric material is a usage pattern described in 6.3.5 as sharing a metric foundation.

6.3 Usage patterns
6.3.1 General

A summary of various usage patterns for the tabular metric representation given in 7.2 and a rationale

for doing so are provided in the next subclauses. Some of these usage patterns match usage categories

identified in ISO/IEC 19086-2:2018, 6.4.2.
6.3.2 Extract and clarify an existing metric description from an SLA

Often, the metric(s) information in a cloud service SLA is scattered over the SLA narrative. Parts of

metric material (such as measurement rules, exceptions, underlying quantities and metrics) is mixed

with related information that is not part of the metric definition (such as performance objectives,

remediation measures and penalties).

Distinguishing a metric definition apart from its context of usage in an SLA and us

...

Questions, Comments and Discussion

Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.