ISO/IEC 19086-2:2018
(Amendment)Cloud computing — Service level agreement (SLA) framework — Part 2: Metric model
Cloud computing — Service level agreement (SLA) framework — Part 2: Metric model
This document establishes common terminology, defines a model for specifying metrics for cloud SLAs, and includes applications of the model with examples. This document establishes a common terminology and approach for specifying metrics. This document is for the benefit of and use for both cloud service providers (CSPs) and cloud service customers (CSCs). This document is intended to complement ISO/IEC 19086-1, ISO/IEC 19086-3 and ISO/IEC 19086-4. This document does not mandate the use of a specific set of metrics for cloud SLAs.
Informatique en nuage — Cadre de travail de l'accord du niveau de service — Partie 2: Modèle métrique
General Information
Standards Content (sample)
INTERNATIONAL ISO/IEC
STANDARD 19086-2
First edition
2018-12
Cloud computing — Service level
agreement (SLA) framework —
Part 2:
Metric model
Informatique en nuage — Cadre de travail de l'accord du niveau de
service —
Partie 2: Modèle métrique
Reference number
ISO/IEC 19086-2:2018(E)
ISO/IEC 2018
---------------------- Page: 1 ----------------------
ISO/IEC 19086-2:2018(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2018
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2018 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC 19086-2:2018(E)
Contents Page
Foreword ........................................................................................................................................................................................................................................iv
Introduction ..................................................................................................................................................................................................................................v
1 Scope ................................................................................................................................................................................................................................. 1
2 Normative references ...................................................................................................................................................................................... 1
3 Terms and definitions ..................................................................................................................................................................................... 1
4 Symbols and abbreviated terms ........................................................................................................................................................... 2
5 Conformance ............................................................................................................................................................................................................. 3
6 Metrics overview .................................................................................................................................................................................................. 3
6.1 General ........................................................................................................................................................................................................... 3
6.2 Background ................................................................................................................................................................................................ 3
6.2.1 Choosing a cloud service ........................................................................................................................................... 3
6.2.2 Convert requirements to agreement .............................................................................................................. 4
6.2.3 Ensure the agreement is being met ................................................................................................................. 5
6.3 Metrics ............................................................................................................................................................................................................ 5
6.4 Cloud service metrics (CSMs) .................................................................................................................................................... 5
6.4.1 Major stakeholders ........................................................................................................................................................ 6
6.4.2 CSM usage categories ................................................................................................................................................... 7
7 Metric model overview ...............................................................................................................................................................................10
7.1 General ........................................................................................................................................................................................................10
7.2 Basic concepts ......................................................................................................................................................................................11
7.2.1 Introduction ......................................................................................................................................................................11
7.2.2 Cloud service level objectives and cloud service qualitative objectives .......................11
7.2.3 Metric data format .......................................................................................................................................................12
8 Metric model ..........................................................................................................................................................................................................12
8.1 Metric model development .......................................................................................................................................................12
8.1.1 Metric model specification ...................................................................................................................................13
8.1.2 Use of UML class diagrams and textual descriptions ....................................................................13
8.1.3 Metric model description ......................................................................................................................................13
8.1.4 Extending the metric .................................................................................................................................................14
8.1.5 Metric model details ..................................................................................................................................................14
Annex A (informative) SO and SO evaluation ............................................................................................................................................18
Annex B (informative) Metric — Table form .............................................................................................................................................20
Annex C (informative) CSM examples ...............................................................................................................................................................23
Annex D (normative) XML schema ......................................................................................................................................................................36
Bibliography .............................................................................................................................................................................................................................39
© ISO/IEC 2018 – All rights reserved iii---------------------- Page: 3 ----------------------
ISO/IEC 19086-2:2018(E)
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
In the field of information technology, ISO and IEC have established a joint technical committee, ISO/
IEC JTC 1The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www .iso .org/patents).Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso
.org/iso/foreword .html.This document was prepared by Technical Committee ISO/IEC JTC1, Information technology,
Subcommittee SC 38, Cloud Computing and Distributed Platforms.A list of all parts in the ISO 19086 series can be found on the ISO website.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/members .html.iv © ISO/IEC 2018 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC 19086-2:2018(E)
Introduction
The measurement of properties of cloud services, especially for the purpose of cloud service level
agreements (SLAs), presents many challenges, which inhibit the uptake of cloud services and inhibit
the overall effectiveness of the cloud services marketplace. Metrics in practice are usually described
using natural languages, typically in ‘plain English’, which is often difficult to understand, compare, and
implement. Such definitions of metrics lead to many problems. Typical concerns include:
— Clarity: The metric definition may be incomplete, ambiguous, illogical, self-contradictory, or not
defined at all. For example, cases exist where ‘availability’ is defined in ways which have little to do
with generally accepted definitions of ‘availability’; where the definition is such that the service can
be unavailable for the majority of the time yet the metric will show 100 % availability; where the
metric requires continuous monitoring, which is actually not possible; or where the provider is able
to determine at its sole discretion what the result is.— Comparability: It may be impractical or effectively impossible to compare different services
in terms of their promised service levels because of the significant inconsistency in how their
respective metrics and SLOs/SQOs are defined.— Implementation: It may be impractical or even impossible to measure the metric in practice, and
to determine whether promised service levels have been met or not.This document has been developed to help address these and similar concerns. It includes technical
content, but the high-level concepts are expected to be understandable by non-technical individuals
who understand the business context for metrics. It provides a metric model that defines the conditions
and rules for performing a measurement and understanding the result.A metric complying with the model defined by this document addresses the concerns above:
— Clarity: A definition of a metric eliminates the ambiguities which currently exist in natural language
descriptions.— Comparability: The structured nature of the metric facilitates the comparison of different metrics
and SLOs/SQOs based on a metric.— Implementation: The structured representation of the information needed to measure a
characteristic facilitates the process of developing measurement tools. Likewise, if the metric is
found not to be implementable, then the metric will need to be revised so that it can be implemented,
and the structure of the technical specification is expected to facilitate this revision process.
The focus of this document is on metrics for cloud SLAs, but it is also usable for cloud service metrics
(CSMs) that are not included in cloud SLAs [such as ones used by cloud service providers (CSPs) for their
internal performance monitoring], and may also be usable for non-CSMs.© ISO/IEC 2018 – All rights reserved v
---------------------- Page: 5 ----------------------
INTERNATIONAL STANDARD ISO/IEC 19086-2:2018(E)
Cloud computing — Service level agreement (SLA)
framework —
Part 2:
Metric model
1 Scope
This document establishes common terminology, defines a model for specifying metrics for cloud
SLAs, and includes applications of the model with examples. This document establishes a common
terminology and approach for specifying metrics.This document is for the benefit of and use for both cloud service providers (CSPs) and cloud service
customers (CSCs). This document is intended to complement ISO/IEC 19086-1, ISO/IEC 19086-3 and
ISO/IEC 19086-4.This document does not mandate the use of a specific set of metrics for cloud SLAs.
2 Normative referencesThe following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 17788, | ITU-T Y.3500, Information technology — Cloud computing — Overview and vocabulary
ISO/IEC 19086-1, Information technology — Cloud computing — Service level agreement (SLA)
framework — Part 1: Overview and conceptsW3C Recommendation 28 October 2004. XML Schema Part 1: Structures Second Edition . ht t p: //w w w .w3
.org/TR/xmlschema -1/W3C Recommendation 28 October 2004. XML Schema Part 2: Datatypes Second Edition. ht t p: //w w w .w3
.org/TR/xmlschema -2/3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC 17788 and the following
definitions apply.ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— IEC Electropedia: available at http: //www .electropedia .org/— ISO Online browsing platform: available at http: //www .iso .org/obp
3.1
cloud service characteristic
qualitative or quantitative property of a cloud service
3.2
cloud service metric
metric (3.6) used to assess a cloud service characteristic (3.1)
© ISO/IEC 2018 – All rights reserved 1
---------------------- Page: 6 ----------------------
ISO/IEC 19086-2:2018(E)
3.3
cloud service objective
commitment a cloud service provider (CSP) makes for a specific characteristic of a cloud service
Note 1 to entry: The set of SOs is the union of the set of cloud service level objectives (SLOs) and the set of cloud
service qualitative objectives (SQOs).3.4
measurement
set of operations having the objective of determining a measurement result (3.5)
Note 1 to entry: Based on the definition of measurement in ISO/IEC/IEEE 15939:2017. Also used here to describe
an actual instance of execution of these operations leading to the production of a measurement result instance.
3.5measurement result
value that expresses a qualitative or quantitative assessment of a cloud service characteristic (3.1)
3.6metric
standard of measurement that defines the conditions and the rules for performing the measurement
(3.4) and for understanding the measurement result (3.5)Note 1 to entry: The metric describes what the result of the measurement means, but not how the measurement
implements the metric.Note 2 to entry: A metric is to be applied in practice within a given context that requires specific properties to be
measured, at a given time(s) for a specific objective.Note 3 to entry: The metrics model proposed in this document supports the definition of composite metrics,
which can be defined in terms of one or more underlying (reusable) metrics.[SOURCE: ISO/IEC 19086-1:2016, 3.10, modified — Notes to entry have been modified.]
3.7unit
real scalar quantity, defined and adopted by convention, with which any other quantity of the same
kind can be compared to express the ratio of the second quantity to the first one as a number
[SOURCE: ISO/IEC 80000-1:2009, modified — NOTES 1 to 5 have not been included.]4 Symbols and abbreviated terms
CSA Cloud Service Agreement
CSC Cloud Service Customer
CSM Cloud Service Metric
CSP Cloud Service Provider
PII Personally Identifiable Information
SLA Service Level Agreement
SLO Cloud Service Level Objective
SO Cloud Service Objective
SQO Cloud Service Qualitative Objective
2 © ISO/IEC 2018 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/IEC 19086-2:2018(E)
5 Conformance
A metric specification is in conformance with this document when the specification of the metric uses
the data-types and relationships described in Clause 8. If the XML namespace at http: //standards .iso
.org/iso -iec/19086/ -2/ed -1/en is used for an XML document representing a metric, that document shall
be valid per the schema in Annex D.6 Metrics overview
6.1 General
This document describes metrics, metrics usage, and defines a model for the consistent specification of
metrics for cloud SLAs. The underlying model and template is important for those defining metrics and
implementing measurement systems based on specific metrics.6.2 Background
Cloud computing is an integral part of IT, yet it is still difficult to define the properties and performance
of a cloud service. A CSC purchases cloud services for one overarching reason – to achieve organization
goals. To this end, it is important for a CSC to understand the properties of a cloud service, to understand
the service capabilities for these properties, and to understand if it will meet the CSC’s requirements
necessary to achieve the CSC’s organization goals. Likewise it is important to a CSP to be able to
communicate the properties and performance of a cloud service so CSCs will be able to determine
whether any given service meets their requirements.Typical categories of cloud service characteristics that the CSC might be interested in include:
performance, availability, information security, accessibility, cloud service support, termination of
service, governance, service changes, service reliability, attestations/certifications, data management,
and PII protection. A description of each along with associated SLOs/SQOs is included in ISO/IEC 19086-
1 and ISO/IEC 19086-4.The cloud procurement process may be split into three basic aspects:
a) selecting a cloud service that meets the CSC’s requirements;
b) agreement (CSA, which includes the cloud SLA) between the CSP and the CSC on the properties and
performance of the cloud service;c) operational management, where the operation of the cloud service is monitored to ensure the service
is operating within the constraints specified within the cloud SLA/CSA.6.2.1 Choosing a cloud service
Currently, CSPs create their own methods to define the representation of a cloud service characteristic,
and therefore influence the understanding of the characteristic itself. This makes comparing cloud
service characteristics across CSPs (and sometimes within a CSP) difficult or impossible. The cloud
service characteristics are often described using ambiguous text descriptions and are not only
difficult to compare, but difficult to understand. This makes transferring a set of requirements into an
agreement between the CSC and CSP difficult; resulting in an agreement that might not meet the needs
of either party.As described in ISO/IEC 19086-1, a commitment written into a cloud SLA takes the form of either an
SLO or an SQO. SLOs are quantitative commitments for representations of cloud service characteristics,
while SQOs are qualitative commitments for representations of cloud service characteristics.
© ISO/IEC 2018 – All rights reserved 3---------------------- Page: 8 ----------------------
ISO/IEC 19086-2:2018(E)
A text description of availability might look something like:
— Commitment
The cloud service will be available 99,9 % of the time in a given billing cycle. If we fail to meet this
commitment, you will be eligible to receive a credit to your account.— Unavailability
Unavailability means: a) The cloud service provides no response, or b) the cloud service returns a
server error response to a valid user request during two or more consecutive one minute intervals,
or c) the cloud service fails to deliver an average download time for a reference document of one
second or less as measured by a third party. Unavailability due to scheduled maintenance is
excluded from these conditions and does not contribute towards unavailability calculations.
This is a good example to consider since it is fundamental to all cloud services (CSCs want the service
to be available for their use). Although a number (percentage) is given for availability it is not clear how
this number is calculated. A definition of unavailability is provided, but it is not clear how the availability
percentage is calculated from unavailability. Another provider could use the concept of availability and
calculate it differently, making comparison of an SO or measured service levels impossible.
Figure 1 — Side by side comparison of availability commitment from two different providers
Figure 1 shows a CSC comparing availability for two services. While the characteristic (availability)
and commitment level (99,99 %) seem identical; the text defining uptime for the two services is
fundamentally different. Cloud service A uses a time based description for availability while cloud
service B uses a transaction based description for availability. The CSC is therefore unable to evaluate
how these two services differ.6.2.2 Convert requirements to agreement
Once a CSC has chosen a service that meets their requirements, the CSP will work with the CSC to come
to an agreement on what exactly will be provided and at what level. The requirements will be codified
into a set of SLOs and SQOs written in a cloud SLA. As with the service capability descriptions used
during the decision making process, the SLOs/SQOs are currently described using natural language
and take different forms depending on the CSP. This not only adds ambiguity to the process, it adds
additional time and complexity to the process as each SO must be thoroughly reviewed and assessed in
each case.4 © ISO/IEC 2018 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/IEC 19086-2:2018(E)
6.2.3 Ensure the agreement is being met
Once an agreement is in place and the cloud service is provisioned for the CSC, the CSC users will start
using the service or services. During operation, both the CSP and the CSC will monitor the services
to ensure that it is operating as expected. The CSP will provide tools to measure the cloud service
characteristics of the service and/or provide the data to the CSC. The CSC will compare these service level
measurement results to the SO specified in the agreement. Due to the ambiguous and inconsistent nature
of how SLOs/SQOs and metrics are currently described, it is difficult for the CSC to have confidence that
these measurement results are calculated in the same manner as defined in the cloud SLA.
6.3 MetricsA metric, or standard of measurement, helps to clarify cloud service characteristics by providing details
about how a cloud service characteristic is measured. A metric provides a definition (e.g. expression,
unit, rules, parameters) of the measurement of the characteristic, and therefore, provides knowledge
about the characteristic itself. It provides the necessary information for repeatability, reproducibility,
and comparability of measurements and measurement results.Figure 2 — Measurement process
The metric describes a cloud service characteristic and the details (parameters, data, rules,
expressions, additional details) necessary to use it. For example, an “availability” metric will define
the practical aspects of how to perform the measurement necessary to calculate availability, how to
measure downtime, exclusion rules, etc. The measurement result is a value that results from making
a measurement that follows a given metric. The measurement result is an estimate of a characteristic
that is being observed. As shown in Figure 2, metric defines the rules so a measurement can be made
in a repeatable, comparable manner. A measurement produces a measurement result that, combined
with the information in the metric, provides knowledge about the characteristic. Characteristics
of cloud services are almost never exactly known. Instead, an approximation of the characteristic is
estimated (based on one or more measurements) with some understanding of the uncertainty between
the approximation and the actual value of the characteristic. This uncertainty is directly tied to the
measurement process used. In other words, a metric is a standard set of rules that allow a measurement
process that follows the rules to generate repeatable, comparable estimates of a characteristic.
Since the metric provides both understanding of the characteristic and the information necessary to
make repeatable and comparable measurements, it can be used not only for the measurement process,
but in the conceptual understanding of a given characteristic.What level of detail is included in a metric is up to the stakeholders. A metric may be only the basic
equations needed to calculate a measurement result or it may include the detailed measurement
process itself.6.4 Cloud service metrics (CSMs)
Example scenarios for cloud services include the application of an availability metric for a SLO
commitment of 99 % in a cloud SLA or the application of an accessibility metric for a service quality
objective of “high” in a decision process scenario. In this manner CSMs help CSPs communicate the
© ISO/IEC 2018 – All rights reserved 5---------------------- Page: 10 ----------------------
ISO/IEC 19086-2:2018(E)
properties of their cloud services, help CSCs and CSPs agree on what will be provided, and allow cloud
service features to be measured to ensure the agreement is met.Cloud services can use metrics in many different ways. Metrics can be used at different layers of a
cloud service (e.g. hardware layers, logic layers, governance layers or service layers). They can also be
used at different stages of the cloud service life cycle (e.g. selection, procurement, operation, audit and
termination).Although the metric model described in Clause 8 is designed for general use, in this document the focus
is on metrics used in cloud SLAs.The definition and usage of appropriate metrics with their underlying measures are essential
components of the cloud SLA and the SLOs/SQOs, which are constituents of the cloud SLA. Within a
cloud SLA, the metrics are used to set the boundaries and margins of errors the provider of the service
commits to deliver, and their limitations. Standardized metrics and metric templates for cloud SLAs
makes it easier and quicker to develop cloud SLAs and the included SLOs/SQOs. Once the cloud SLA is
in place, metrics could be used at runtime to measure the services levels and determine if the service is
meeting the commitments in the cloud SLA.Metrics for cloud services have several uses for CSCs and CSPs including but not limited to:
— Metrics can be used by CSPs to describe the performance of a cloud service.— Metrics help CSCs to compare offerings from different CSPs (when CSPs uses the same metrics to
describe the performance of a cloud service).— Metrics can be used to describe SLOs and SQOs within a cloud SLA.
— Metrics can be used by CSCs to determine if the CSP is meeting their commitments as described in
the cloud SLA and to claim remedies if the commitments are not being met.6.4.1 Major stakeholders
This subclause describes stakeholder types that have an interest in cloud SLA metrics. Each stakeholder
type represents a different set of interests and related concerns. Each stakeholder type uses metrics in
a different way to address those interests.Attorney
Negotiates agreements for cloud services between CSP and CSC. Uses this document as reference for
specifying metrics in cloud SLAs.Auditors
Evaluate cloud services for performance against a set of requirements and commitments. Use ISO/
IEC 19086-2 based metrics to define and measure cloud service characteristics that relate to the
requirements. A specific type of auditor may also use this document to evaluate the measurement
system itself. Needs to understand the metric used to measure a cloud service characteristic.
Certification authorityDevelop certification criteria for cloud services. Use metrics based on this doc
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.