ETSI GS NFV-REL 005 V1.1.1 (2016-01)

Network Functions Virtualisation (NFV); Accountability; Report on Quality Accountability Framework

DGS/NFV-REL005

General Information

• Status: Published
• Publication Date: 03-Jan-2016
• Technical Committee: NFV REL - Reliability & Availability
• Current Stage: 12 - Completion
• Due Date: 28-Dec-2015
• Completion Date: 04-Jan-2016

Standards Content (Sample)

ETSI GS NFV-REL 005 V1.1.1 (2016-01)






GROUP SPECIFICATION
Network Functions Virtualisation (NFV);
Accountability;
Report on Quality Accountability Framework
Disclaimer
This document has been produced and approved by the Network Functions Virtualisation (NFV) ETSI Industry Specification
Group (ISG) and represents the views of those members who participated in this ISG.
It does not necessarily represent the views of the entire ETSI membership.

---------------------- Page: 1 ----------------------
2 ETSI GS NFV-REL 005 V1.1.1 (2016-01)



Reference
DGS/NFV-REL005
Keywords
NFV, quality

ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: [phone]  Fax: [phone]

Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88

Important notice
The present document can be downloaded from:
http://www.etsi.org/standards-search
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the only prevailing document is the
print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
http://portal.etsi.org/tb/status/status.asp
If you find errors in the present document, please send your comment to one of the following services:
https://portal.etsi.org/People/CommiteeSupportStaff.aspx
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying
and microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.

© European Telecommunications Standards Institute 2016.
All rights reserved.

TM TM TM
DECT , PLUGTESTS , UMTS and the ETSI logo are Trade Marks of ETSI registered for the benefit of its Members.
TM
3GPP and LTE™ are Trade Marks of ETSI registered for the benefit of its Members and
of the 3GPP Organizational Partners.
GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.
ETSI

---------------------- Page: 2 ----------------------
3 ETSI GS NFV-REL 005 V1.1.1 (2016-01)
Contents
Intellectual Property Rights . 5
Foreword . 5
Modal verbs terminology . 5
Introduction . 5
1 Scope . 6
2 References . 6
2.1 Normative references . 6
2.2 Informative references . 6
3 Definitions and abbreviations . 7
3.1 Definitions . 7
3.2 Abbreviations . 9
4 Roles in the NFV Ecosystem . 10
4.1 NFV Service Delivery Relationships . 10
4.2 Role: Cloud Service User . 11
4.3 Role: Cloud Service Customer . 11
4.4 Role: Cloud Service Provider . 12
4.5 Role: VNF Supplier . 13
4.6 Role: Service Integrator . 13
4.7 Role: Cloud Auditor . 13
4.8 Role: Cloud Service Broker. 14
4.9 Illustrative Example . 14
5 Responsibilities by Role . 16
5.1 Traditional (PNF) Responsibilities . 16
5.2 VNF Supplier Responsibilities . 16
5.3 Service Integrator Responsibilities . 17
5.4 Cloud Service Provider: NFV Infrastructure . 17
5.5 Cloud Service Provider: NFV Management and Orchestration . 18
5.6 Cloud Service Provider: Functional Component Offered as-a-Service . 19
5.7 Cloud Service Customer . 19
5.8 CSP: Network Provider . 20
6 Responsibilities for Key Cloud Characteristics . 20
6.1 Key Cloud Computing Characteristics . 20
6.2 Broad Network Access . 20
6.3 Measured Service . 20
6.4 Multi-Tenancy . 20
6.5 On-Demand Self Service . 21
6.6 Rapid Elasticity and Scalability . 21
6.7 Resource Pooling . 21
7 Quality Measurement Framework . 22
7.1 Overview . 22
7.1 VNF Software Quality Measurements . 22
7.2 Function Components Offered as-a-Service Quality Measurements . 23
7.3 Automated Lifecycle Management Quality Measurements . 23
7.4 Failure Notification Quality Measurements . 23
7.5 Virtual Infrastructure Quality Measurements . 24
Annex A (informative): Sample Cloud Service Customer SLAs . 25
Annex B (informative): Use Case Scenario: . 28
B.0 Introduction . 28
B.1 Service Outage Downtime . 28
ETSI

---------------------- Page: 3 ----------------------
4 ETSI GS NFV-REL 005 V1.1.1 (2016-01)
B.2 Automated Lifecycle Management Action Failures . 29
B.3 VM Failure Rate . 31
B.4 Virtual Network Impairments . 31
B.5 Virtual Machine Scheduling Latency . 31
B.6 Virtual Machine Stall Time . 32
B.7 Functional Components Offered as-a-Service . 33
B.8 Placement Policy Compliance . 34
B.9 Resource Promise Violations . 34
B.10 VNF Scaling . 34
Annex C (informative): Overview of TMForum and QuEST Forum Roles . 36
Annex D (informative): Administrative Domains and Accountable Parties . 37
Annex E (informative): Authors & contributors . 38
Annex F (informative): Bibliography . 39
History . 40

ETSI

---------------------- Page: 4 ----------------------
5 ETSI GS NFV-REL 005 V1.1.1 (2016-01)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (http://ipr.etsi.org).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This Group Specification (GS) has been produced by ETSI Industry Specification Group (ISG) Network Functions
Virtualisation (NFV).
The present document deals with specific aspects of Service Quality Accountability in the context of Network Function
Virtualisation.
Modal verbs terminology
In the present document "shall", "shall not", "should", "should not", "may", "need not", "will", "will not", "can" and
"cannot" are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of
provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.
Introduction
The NFV Quality Accountability Framework supports the quality management principles of customer focus ([i.7]
principle 0.2.a), mutually beneficial supplier relationships ([i.7] principle 0.2.h) and factual approach to decision
making ([i.7] principle 0.2.g) to enable continual improvement ([i.7] principle 0.2.f). Clearly defining roles,
responsibilities and demarcations is a quality management best practice because it clarifies accountabilities which
permit any quality impairments to be rapidly localized, root causes to be identified and appropriate corrective actions to
be agreed to promptly restore service and drive continuous quality improvement. This informative document defines
key roles including NFV cloud service customer, provider(s) of NFV management, orchestration and/or infrastructure
services, and their VNF suppliers and Integrators. The document lays out the responsibilities for each role based on both
extrapolating traditional responsibilities and considering responsibilities for each of the six essential characteristics of
cloud computing. As objective and quantitative measurement is necessary to enable methodical quality assurance and
management, the present document offers a quality measurement framework that connects standard metrics and
measurements with roles, and an annex that offers sample service quality SLAs for a cloud service customer. A second
annex offers use case scenarios illustrating how the quality accountability framework applies to several quality
impairment scenarios.
This framework uses principles of ISO/IEC 17788 "Cloud computing -- Overview and vocabulary" [i.1] and
ISO/IEC 17789 "Cloud Computing - Reference Architecture" [i.2] to the ETSI NFV architecture [i.12] to enable quality
measurements consistent with both "TL 9000 Quality Management System Measurements Handbook" and "Network
Functions Virtualisation (NFV); Service Quality Metrics" and SLA management consistent with TM Forum's "SLA
Management Guidebook" and "Enabling End-to-End Cloud SLA Management."

ETSI

---------------------- Page: 5 ----------------------
6 ETSI GS NFV-REL 005 V1.1.1 (2016-01)
1 Scope
The present document describes a quality accountability framework for NFV. This release focuses on service quality
management of network services, VNFs, NFV infrastructure, management and orchestration elements.
The present document describes the following aspects of the Quality Accountability Framework:
1) Roles, covered in clause 4 Roles in the NFV Ecosystem.
2) Responsibilities, covered in clauses 5 Responsibilities by Role and 6 Responsibilities for Key Cloud
Characteristics.
3) Service quality measurements and demarcation points, covered in clause 7 Quality Measurement
Framework.
2 References
2.1 Normative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
Referenced documents which are not found to be publicly available in the expected location might be found at
http://docbox.etsi.org/Reference.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are necessary for the application of the present document.
Not applicable.
2.2 Informative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] ISO/IEC 17788 (First edition) (2014-10-15): "Information technology -- Cloud computing --
Overview and vocabulary".
NOTE: Available at http://standards.iso.org/ittf/PubliclyAvailableStandards/c060544_ISO_IEC_17788_2014.zip.
[i.2] ISO/IEC 17789 (First edition) (2014-10-15): "Information Technology -- Cloud Computing --
Reference Architecture".
NOTE: http://standards.iso.org/ittf/PubliclyAvailableStandards/c060545_ISO_IEC_17789_2014.zip.
[i.3] TM Forum, TR 178 (V2.0.2) (October 2014): "Enabling End-to-End Cloud SLA Management",
Framework Release 14.
NOTE: https://www.tmforum.org/resources/technical-report-best-practice/tr178-enabling-end-to-end-cloud-sla-
management-v2-0-2/.
ETSI

---------------------- Page: 6 ----------------------
7 ETSI GS NFV-REL 005 V1.1.1 (2016-01)
[i.4] TM Forum Guidebook GB917 (July 2012): "SLA Management Guidebook, Release 3.1".
NOTE: https://www.tmforum.org/resources/standard/gb917-sla-management-handbook-release-3-1/.
[i.5] QuestForum (Release 5.0, July 2012): "TL 9000 Measurements Handbook".
NOTE: Available at http://www.tl9000.org/handbooks/measurements_handbook.html.
[i.6] ETSI GS NFV-INF 010 (V1.1.1) (12-2014): "Network Functions Virtualisation (NFV); Service
Quality Metrics".
[i.7] ISO 9000 (Third Edition) (September 2005): "Quality Management Systems - Fundamentals and
Vocabulary".
[i.8] "Quality Measurement of Automated Lifecycle Management Actions", 1.0, August 18th, 2015,
QuEST Forum.
NOTE: http://www.tl9000.org/resources/documents/QuEST_Forum_ALMA_Quality_Measurement_150819.pdf.
[i.9] ETSI GS NFV-MAN 001 (V1.1.1) (12-2014): "Network Functions Virtualisation (NFV);
Management and Orchestration".
[i.10] ETSI GS NFV 003 (V1.2.1) (12-2014): "Network Functions Virtualisation (NFV); Terminology
for Main Concepts in NFV".
[i.11] ETSI GS NFV-REL 004 (09-2015): "Network Functions Virtualisation (NFV); Active monitoring
& failure detection report".
[i.12] ETSI GS NFV 002 (V1.2.1) (12-2014): "Network Functions Virtualisation (NFV); Architectural
Framework".
[i.13] ISO 9001:2015: "Quality management systems -- Requirements".
[i.14] ISO 14001:2015: "Environmental management systems -- Requirements with guidance for use".
[i.15] ISO 27729:2012: "Information and documentation -- International standard name identifier
(ISNI)".
[i.16] ISO/IEC 27001:2013: "Information technology -- Security techniques -- Information security
management systems -- Requirements".
3 Definitions and abbreviations
3.1 Definitions
For the purposes of the present document, the terms and definitions given in ETSI GS NFV 003 [i.10] and the following
apply:
audit: systematic, independent and documented process for obtaining audit evidence and evaluating it objectively to
determine the extent to which audit criteria are fulfilled
NOTE 1: Internal audits, sometimes called first-party audits, are conducted by, or on behalf of, the organization
itself for management review and other internal purposes, and may form the basis for an organization's
declaration of conformity. In many cases, particularly in smaller organizations, independence can be
demonstrated by the freedom from responsibility for the activity being audited.
NOTE 2: External audits include those generally termed second- and third-party audits. Second-party audits are
conducted by parties having an interest in the organization, such as customers, or by other persons on
their behalf. Third-party audits are conducted by external, independent auditing organizations, such as
those providing certification/registration of conformity to ISO 9001 [i.13] or ISO 14001 [i.14].
NOTE 3: This definition is from [i.7], clause 3.9.1.
ETSI

---------------------- Page: 7 ----------------------
8 ETSI GS NFV-REL 005 V1.1.1 (2016-01)
cloud auditor: cloud service partner with the responsibility to conduct an audit of the provision and use of cloud
services
NOTE: This definition is from [i.1], clause 3.2.3.
cloud computing: paradigm for enabling network access to a scalable and elastic pool of shareable physical or virtual
resources with self-service provisioning and administration on-demand
NOTE 1: Examples of resources include servers, operating systems, networks, software, applications, and storage
equipment.
NOTE 2: This definition is from [i.1], clause 3.2.5.
cloud service: one or more capabilities offered via cloud computing invoked using a defined interface
NOTE: This definition is from [i.1], clause 3.2.8.
cloud service broker: cloud service partner that negotiates relationships between cloud service customers and cloud
service providers
NOTE: This definition is from [i.1], clause 3.2.9.
cloud service customer: party which is in a business relationship for the purpose of using cloud service
NOTE1: A business relationship does not necessarily imply financial agreements.
NOTE 2: This definition is from [i.1], clause 3.2.11.
cloud service developer: sub-role of cloud service partner which is responsible for designing, developing, testing and
maintaining the implementation of a cloud service
NOTE 1: This can involve composing the service implementation from existing service implementations.
NOTE 2: This definition is from [i.2], clause 8.4.1.1.
cloud service partner: party which is engaged in support of, or auxiliary to, activities of either the cloud service
provider or the cloud service customer, or both
NOTE: This definition is from [i.1], clause 3.2.14.
cloud service product: cloud service, allied to the set of business terms under which the cloud service is offered
NOTE 1: Business terms can include pricing, rating and service levels.
NOTE 2: This definition is from [i.2], clause 3.2.2.
cloud service provider: party which makes cloud services available
NOTE: This definition is from [i.1], clause 3.2.15.
cloud service user: natural person, or entity acting on their behalf, associated with a cloud service customer that uses
cloud services
NOTE 1: Examples of such entities include devices and applications.
NOTE 2: This definition is from [i.1], clause 3.2.17.
functional component: functional building block needed to engage in an activity, backed by an implementation
NOTE: This definition is from [i.2], clause 3.2.3.
party: natural person or legal person, whether or not incorporated, or a group of either
NOTE 1: This definition is from [i.1], clause 3.1.6 and was preceded by "The following term is defined in
ISO 27729".
ETSI

---------------------- Page: 8 ----------------------
9 ETSI GS NFV-REL 005 V1.1.1 (2016-01)
NOTE 2: The term domain used in [i.10] is not necessarily synonymous with party in that the perimeter of a
particular domain is not required to be identical to the accountability perimeter of a particular party. For
example, [i.9] states "Administrative Domains can be mapped to different organizations and therefore
can exist within a single service provider or distributed among several service providers", which is
consistent with "Administrative Domains can be mapped to different organizations and therefore can
exist within a single party or distributed among several parties."
private cloud: cloud deployment model where cloud services are used exclusively by a single cloud service customer
and resources are controlled by that cloud service customer
NOTE: This definition is from [i.1], clause 3.2.32.
product category: recognized grouping of products for calculating TL 9000 measurements
NOTE: This definition is from the glossary of [i.5].
public cloud: cloud deployment model where cloud services are potentially available to any cloud service customer and
resources are controlled by the cloud service provider
NOTE: This definition is from [i.1], clause 3.2.33.
quality: degree to which a set of inherent characteristics fulfils requirements
NOTE 1: The term "quality" can be used with adjectives such as poor, good or excellent.
NOTE 2: "Inherent", as opposed to "assigned", means existing in something, especially as a permanent
characteristic.
NOTE 3: This definition is from [i.7], clause 3.1.1.
Service Level Agreement (SLA): element of a formal, negotiated commercial contract between two Organizations, i.e.
one with a Service Provider (SP) Role and one a Customer Role
NOTE 1: It documents the common understanding of all aspects of the Product and the roles and responsibilities of
both Organizations from product ordering to termination.
NOTE 2: This definition is from [i.3].
3.2 Abbreviations
For the purposes of the present document, the following abbreviations apply:
BRR Basic Return Rate
CSC Cloud Service Customer
CSP Cloud Service Provider
CSP:NP Cloud Service Provider:Network Provider
DOA Dead On Arrival
DPM Defects Per Million
EM Element Manager
EPC Evolved Packet Core
EPO Emergency Power Off
ERI Early Return Index
ESD Electro-Static Discharge
FCAPS Fault Configuration Accounting Performance Security
FRT Fix Response Time
IP Internet Protocol
IP-TV Internet Protocol Television
KQI Key Quality Indicator
LTR Long-term Return Rate
MANO Management and Orchestration
MOP Method of Procedure
NFV Network Function Virtualisation
NFVI NFV Infrastructure
NFVO NFV Orchestrator
NPR Number of Problem Reports
ETSI

---------------------- Page: 9 ----------------------
10 ETSI GS NFV-REL 005 V1.1.1 (2016-01)
OFR Overdue Fix Responsiveness
OSS Operation Support System
OTD On Time Delivery
OTS On-Time Service delivery
PNF Physical Network Function
RTP Real-Time Protocol
SFQ Software Fix Quality
SLA Service Level Agreement
SO Service Outage
SONE Network Element Impact Outage
SP Service Provider
SPR Software Problem Report
SQ Service Quality
SSO Support Service caused Outages
VIM Virtual Infrastructure Management
VM Virtual Machine
VN Virtual Network
VNF Virtualised Network Function
VNFC Virtualised Network Function Component
VNFM VNF Manager
VoLTE Voice over Long Term Evolution
YRR one-Year Return Rate
4 Roles in the NFV Ecosystem
4.1 NFV Service Delivery Relationships
In the present document, an NFV cloud service customer is defined as a party having business relationships for using
NFV infrastructure, management and orchestration services [i.9]. Figure 1 illustrates an example of an application
service delivery relationship of an NFV cloud service customer following the customer/provider style of [i.3] and using
roles from [i.1] and [i.2]. Figure 1 shows NFV Infrastructure, NFV management and orchestration and Functional
component offered as a service being provided by a single cloud service provider (CSP) organization, but different
organizational arrangement are also considered in the present document. Each of the cloud service customer's
providers/suppliers (e.g. NFV infrastructure cloud service provider) may be customers of other suppliers; for example, a
cloud service provider who offers NFV infrastructure-as-a-service to NFV cloud service customers (i.e. an NFV
Infrastructure cloud service provider is a customer of suppliers of physical compute, networking and storage hardware,
various software products and so on).

Figure 1: Example of an NFV Service Delivery Relationship for Cloud Service Customer
ETSI

---------------------- Page: 10 ----------------------
11 ETSI GS NFV-REL 005 V1.1.1 (2016-01)
4.2 Role: Cloud Service User
Cloud Service Users are defined by [i.1] as the end users, or applications operating on their behalf, who use cloud
services. In the context of NFV, a cloud service user refers to a natural person, or system/device acting on their behalf,
that consumes services offered by a cloud service pr
...