ETSI GR NFV-EVE 017 V1.1.1 (2020-08)

Network Functions Virtualisation (NFV); Management and Orchestration; Report on the support of real-time/ultra-low latency aspects in NFV related to service and network handling

DGR/NFV-EVE017

General Information

• Status: Not Published
• Technical Committee: NFV EVE - Evolution and Ecosystem
• Current Stage: 12 - Completion
• Due Date: 31-Aug-2020
• Completion Date: 19-Aug-2020

Standards Content (Sample)

ETSI GR NFV-EVE 017 V1.1.1 (2020-08)






GROUP REPORT
Network Functions Virtualisation (NFV);
Management and Orchestration;
Report on the support of real-time/ultra-low latency aspects
in NFV related to service and network handling
Disclaimer
The present document has been produced and approved by the Network Functions Virtualisation (NFV) ETSI Industry
Specification Group (ISG) and represents the views of those members who participated in this ISG.
It does not necessarily represent the views of the entire ETSI membership.

---------------------- Page: 1 ----------------------
2 ETSI GR NFV-EVE 017 V1.1.1 (2020-08)



Reference
DGR/NFV-EVE017
Keywords
MANO, NFV, real time, service

ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: [phone]  Fax: [phone]

Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88

Important notice
The present document can be downloaded from:
http://www.etsi.org/standards-search
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the prevailing version of an ETSI
deliverable is the one made publicly available in PDF format at www.etsi.org/deliver.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
https://portal.etsi.org/TB/ETSIDeliverableStatus.aspx
If you find errors in the present document, please send your comment to one of the following services:
https://portal.etsi.org/People/CommiteeSupportStaff.aspx
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying
and microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.

© ETSI 2020.
All rights reserved.

DECT™, PLUGTESTS™, UMTS™ and the ETSI logo are trademarks of ETSI registered for the benefit of its Members.

3GPP™ and LTE™ are trademarks of ETSI registered for the benefit of its Members and
of the 3GPP Organizational Partners.
oneM2M™ logo is a trademark of ETSI registered for the benefit of its Members and
of the oneM2M Partners.
®
GSM and the GSM logo are trademarks registered and owned by the GSM Association.
ETSI

---------------------- Page: 2 ----------------------
3 ETSI GR NFV-EVE 017 V1.1.1 (2020-08)
Contents
Intellectual Property Rights . 4
Foreword . 4
Modal verbs terminology . 4
1 Scope . 5
2 References . 5
2.1 Normative references . 5
2.2 Informative references . 5
3 Definition of terms, symbols and abbreviations . 6
3.1 Terms . 6
3.2 Symbols . 6
3.3 Abbreviations . 6
4 Use Cases . 6
4.1 Introduction . 6
4.2 Use Case 1: Re-routing a low latency network service . 7
4.3 Use Case 2: Mobility for a low latency network service . 8
4.4 Use Case 3: Supporting Low latency Application Function Overlaying the Network Service . 9
5 Analysis . 11
5.1 Introduction . 11
5.2 Considerations for low-latency service measurements and recovery . 11
5.3 Delay points for latency measurements . 12
5.4 NFV-MANO Deployment Considerations . 14
5.4.1 Overview . 14
5.4.2 Summary of ETSI GR NFV-IFA 028 . 14
5.4.3 Intra-site NFV-MANO deployment considerations for low-latency services . 17
5.4.4 Inter-site NFV-MANO deployment considerations for low-latency services . 17
6 Recommendations and Conclusions . 19
6.1 General Recommendations . 19
6.2 Conclusion . 20
History . 21


ETSI

---------------------- Page: 3 ----------------------
4 ETSI GR NFV-EVE 017 V1.1.1 (2020-08)
Intellectual Property Rights
Essential patents
IPRs essential or potentially essential to normative deliverables may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (https://ipr.etsi.org/).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Trademarks
The present document may include trademarks and/or tradenames which are asserted and/or registered by their owners.
ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no
right to use or reproduce any trademark and/or tradename. Mention of those trademarks in the present document does
not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks.
Foreword
This Group Report (GR) has been produced by ETSI Industry Specification Group (ISG) Network Functions
Virtualisation (NFV).
Modal verbs terminology
In the present document "should", "should not", "may", "need not", "will", "will not", "can" and "cannot" are to be
interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.

ETSI

---------------------- Page: 4 ----------------------
5 ETSI GR NFV-EVE 017 V1.1.1 (2020-08)
1 Scope
The present document analyses the impact on management and orchestration of Network Service (NS) instance(s)
supporting low latency services from the perspective of the NFV-MANO architectural framework. The following topics
are handled:
• Definition of relevant NFV-MANO use cases.
• Analysis of the use-cases and deriving potential requirements.
• Providing relevant recommendations.
The content of the present document is informative.
2 References
2.1 Normative references
Normative references are not applicable in the present document.
2.2 Informative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] ETSI GR NFV-IFA 012 (V3.1.1): "Network Functions Virtualisation (NFV) Release 3;
Management and Orchestration; Report on Os-Ma-Nfvo reference point - Application and Service
Management Use Cases and Recommendations", October 2018.
[i.2] ETSI GR NFV-IFA 028 (V3.1.1): "Network Functions Virtualisation (NFV) Release 3;
Management and Orchestration; Report on architecture options to support multiple administrative
domains".
[i.3] ETSI GS NFV-IFA 032 (V3.2.1): "Network Functions Virtualisation (NFV) Release 3;
Management and Orchestration; Interface and Information Model Specification for Multi-Site
Connectivity Services".
[i.4] ETSI GS NFV-IFA 010 (V3.3.1): "Network Functions Virtualisation (NFV) Release 3;
Management and Orchestration; Functional requirements specification".
[i.5] ETSI GS NFV-IFA 011 (V3.2.1): "Network Functions Virtualisation (NFV) Release 3;
Management and Orchestration; VNF Descriptor and Packaging Specification".
[i.6] ETSI GS NFV-IFA 014 (V3.3.1): "Network Functions Virtualisation (NFV) Release 3;
Management and Orchestration; Network Service Templates Specification".
[i.7] ETSI GS NFV-IFA 031 (V3.3.1): "Network Functions Virtualisation (NFV) Release 3;
Management and Orchestration; Requirements and interfaces specification for management of
NFV-MANO".
[i.8] ETSI GS NFV-IFA 027 (V2.4.1): "Network Functions Virtualisation (NFV) Release 2;
Management and Orchestration; Performance Measurements Specification".
ETSI

---------------------- Page: 5 ----------------------
6 ETSI GR NFV-EVE 017 V1.1.1 (2020-08)
[i.9] ETSI GS NFV-TST 008 (V3.2.1): "Network Functions Virtualisation (NFV) Release 3; Testing;
NFVI Compute and Network Metrics Specification".
[i.10] ETSI GR NFV 003 (V1.5.1): "Network Functions Virtualisation (NFV); Terminology for Main
Concepts in NFV".
[i.11] ETSI TS 122 261 (V15.8.0): "5G; Service requirements for next generation new services and
markets (3GPP TS 22.261 version 15.8.0 Release 15)".
[i.12] ETSI GS NFV-IFA 013 (V3.3.1): "Network Functions Virtualisation (NFV) Release 3;
Management and Orchestration; Os-Ma-Nfvo reference point - Interface and Information Model
Specification".
3 Definition of terms, symbols and abbreviations
3.1 Terms
For the purposes of the present document, the terms given in ETSI GR NFV 003 [i.10] apply.
3.2 Symbols
Void.
3.3 Abbreviations
For the purposes of the present document, the abbreviations given in ETSI GR NFV 003 [i.10] and the following apply:
AF Application Function
MLPOC Multiple Logical Points Of Contacts
NFVIaaS-C NFVIaaS Consumer
NFVIaaS-P NFVIaaS Provider
SLPOC Single Logical Points Of Contacts
4 Use Cases
4.1 Introduction
In ETSI TS 122 261 [i.11] various services requiring low latency guarantees have been identified. These use cases
include urgent healthcare and emergency services, which require latency guarantees from 1 ms to 10 ms. A second class
of uses cases includes smart factories and tactile interaction applications, which require latency to stay between 0,5 ms
and 1 ms. The requirements towards the network of these use cases are summarized in Table 4.1-1. Thus, it has to be
emphasized that these latency requirements are strictly on an end-to-end service level. Consequently, any operations
impacting the actual service deployment and run-time operation should guarantee that those upper bounds are not
exceeded.
The present document investigates the gaps in NFV-MANO specifications to support low latency services that are
provisioned over NFV Network Services (NS). Thus generic use cases are presented which are later analysed from the
perspective of NFV-MANO system in supporting and managing such services within strict end-to-end delay bound. The
NFV-MANO system will be analysed in order to highlight the various aspects that can potentially impact the latency
bounds of an active service over the NFVI. With respect to the analysis, necessary recommendations will be provided.
ETSI

---------------------- Page: 6 ----------------------
7 ETSI GR NFV-EVE 017 V1.1.1 (2020-08)
Table 4.1-1: Performance requirements for low-latency and high reliability scenarios [i.11]


4.2 Use Case 1: Re-routing a low latency network service
This use case considers the situation when a low-latency service may have to be re-routed e.g. due to a network element
failing, the topology change inside the NFVI, congestion event.
The failing network elements can be located inside a NFVI-PoP or on elements that are used to ensure interconnectivity
between different NFV-PoPs. Apart from network resources the use case also considers the failure of VNFs. This could
potentially involve the relocation of a failed VNF, which then would imply changes to the underlying network
infrastructure connecting the affected VNF to the other elements of the NS.
The investigation will take into account that a VNF itself could be managing a part of the low latency service itself,
e.g. by monitoring redundant links with which it is connected to the other NS elements. The use case should investigate
if and how these kinds of VNF could interact with the NFV-MANO system.
The use case does not assume any specific procedures for re-routing of NS, but the NS rerouting process should take
into account the latency bounds for the low-latency service. If multiples routes are available; those fulfilling the latency
bounds best should be preferred. This would potentially involve monitoring latency bounds for routes (e.g. application
latency monitoring).
ETSI

---------------------- Page: 7 ----------------------
8 ETSI GR NFV-EVE 017 V1.1.1 (2020-08)

Figure 4.2-1: Overview of the use case 1 scenario
Figure 4.2-1 is presenting a network scenario consisting of 3 NFVI-PoPs interconnected over WAN links. There are two
NS instances, a multisite NS and an intra-site NS instance. The multisite NS instance is composed of VNF instances,
which are instantiated in different NFVI-PoPs, whereas the intra-site NS is composed of VNF instances that are
instantiated within the same NFVI-PoP (i.e. VNF-1.a and VNF-1.b in NFVI-PoP #1 in Figure 4.2-1). A more detailed
view on the WAN is showing different routes between NFVI-PoP #1 and NFVI-PoP #2. The following failure
possibilities are assumed:
1) The current route #1 is failing and that there are 2 alternatives, one fulfilling the latency bounds of the
low-latency service and one not fulfilling them.
2) Failure of a VL on a NS, within a single NFVI-PoP or across multiple NFVI-PoPs.
3) Failure of VNF itself that is part of a NS.
Each failure event will require the NFV-MANO system to trigger recovery actions to ensure time integrity of the
affected NS.
4.3 Use Case 2: Mobility for a low latency network service
This use case considers the situation when a low-latency service is established and the client receiving the low-latency
service is mobile. In such a situation, the client could change the access point it is connected to while receiving the
service. The new access point could be connected to the same NFVI-PoP or even in a different NFVI-PoP. Usually the
network connections are preconfigured and the NFV system does not even notice when clients move around. When
dealing with low-latency services this stays basically the same but the network conditions might change when many
clients receiving low-latency services move. When many clients move to the same new access point the network could
get loaded in such a way that the low-latency conditions could no longer be guaranteed. In such a situation, the
NFV-MANO system may have to detect and react to re-enforce the latency bounds given for the low-latency service.
In such a situation it is thus required that the NFV-MANO system is able to detect, or get notification of, degradation of
low-latency services due to mobile users changing access points. When a service degradation is detected, the
NFV-MANO system may have to derive and trigger suitable actions to restore the low-latency characteristics of the
degraded service.
ETSI

---------------------- Page: 8 ----------------------
9 ETSI GR NFV-EVE 017 V1.1.1 (2020-08)

Figure 4.3-1: Overview of the use case 2 scenario
Figure 4.3-1 is presenting a network consisting of 4 NFVI-PoPs hosting different VNFs providing an NS and
interconnecting WAN links. NFVI-PoP#3 and NFVI-PoP#4 are ingress for mobile end-users. Figure 4.3-1 also shows
many end-users currently moving in to the access network connected to NFVI-PoP#4. Different locations are assumed
where low-latency guarantees could be violated due to the mass of end-users moving in.
Each location that causes the violation of these guarantees will require the NFV-MANO system to trigger recovery
actions to ensure low latency characteristics of the affected NS.
The investigation will analyse if the requirements for a low-latency NS will introduce additional requirements on NS
monitoring to detect degradation of NS low-latency guarantees. In addition it will identify which management elements
of the whole NFV-MANO system might be affected and may have to act to restore the low-latency characteristic of the
degraded NS.
4.4 Use Case 3: Supporting Low latency Application Function
Overlaying the Network Service
In ETSI GR NFV-IFA 012 [i.1] Application Functions (AFs) are introduced, where the AFs rely on the
functional/operational characteristics of the NS or one or more of its constituent VNFs that are parts of NS(s) to deliver
application services. On the other hand, the VNF(s) and/or the NS(s) may also utilize the functions provided by the AFs
for their own operational/functional support, e.g. by analysing their KPIs to improve service delivery.
This use case considers NFV-MANO supporting AFs responsible for delivering low latency services, such as live
streaming of multimedia content over NS. This is relevant because the AF instance could be the cause of delay and/or
the underlying NS or one or more of its components can impact the latency bounds of the application service, and thus
it is important for NFV-MANO to be aware of it. The application that is providing/serving the low-latency service could
provide latency information about the end-to-end network path since it usually receives feedback from the clients and
can retrieve traffic information from the underlying NS/VNF instance(s) as laid out in ETSI GR NFV-IFA 012 [i.1].
This latency information could then be provided to the management system of the AF.
ETSI

---------------------- Page: 9 ----------------------
10 ETSI GR NFV-EVE 017 V1.1.1 (2020-08)
ETSI GR NFV-IFA 012 [i.1] foresees relying on the interfaces on the Os-Ma-Nfvo reference point for an AF to interact
with the NFV-MANO system. In this use case, the relevant notifications from the AF can be used by the OSS/BSS to
trigger the NFV-MANO system about degraded conditions to help identify and improve the degraded latency conditions
as far as the network path of an NS is concerned that the AF is using. The AF may have to provide information that
enables the NFV-MANO system to map the application latency requirements to a NS being managed by NFV-MANO.
To fulfil this task the AF might have to use other interfaces apart from the Os-Ma-Nfvo ones. The information provided
by the AF can be manifold: it could be a simple Boolean value informing about bad conditions or might contain more
fine grained information exchanged between the AF and the VNFs that couple it with the NS.
The use case will consider different causes for latency degradation such as load on the AF, load on the NS itself or users
changing access points, etc. In such a situation it is beneficial that the NFV-MANO system gets help in detecting the
degradation of low-latency services through notifications triggered by the AF. When a service degradation is detected
and identified, the NFV-MANO system may have to derive and enforce suitable measures to resurrect the low-latency
guarantees for the degraded service. These measures can be executed by different functional blocks of the NFV-MANO
system depending on the cause of latency degradation.
The investigation of this use case will analyse how and to which functional blocks in NFV-MANO system AF specific
performance and latency information may have to be provided.

Figure 4.4-1: Overview of the use case 3 scenario
The use case scenario is illustrated in Figure 4.4-1 where AF instances are overlaid on a multi-site NS composed of
3 VNFs, namely VNF-1, VNF-2 and VNF-3 hosed in 3 NFVI-PoPs respectively. The 3 NFVI-PoPs are interconnected
over WAN links. The AF instances 1 to 3 are coupled with a single VNF function whereas AF 4 is coupled with the
whole NS. AF 4 could provide a low-latency service to an end-user over the NS while receiving application data from
the NS or from an OTT service provider. The Figure shows the interaction of the AFs with the underlying NS and with
the NFV-MANO system via the OSS/BSS over the Os-Ma-NFVO reference point, as specified in ETSI
GR NFV-IFA 012 [i.1]. As an option, the AFs can interact with the NFV-MANO system over this reference point by
providing application level latency information. This information could then trigger the NFV-MANO system via the
OSS/BSS to take appropriate actions at the NS level to maintain the services' latency bounds.
ETSI

---------------------- Page: 10 ----------------------
11 ETSI GR NFV-EVE 017 V1.1.1 (2020-08)
5 Analysis
5.1 Introduction
The use cases presented in clause 4 describe scenarios in which a low-latency service is provided by a NS and the
established connections get degraded by different events in different locations of the NS. To analyse the use cases this
clause first explains what characterizes a low-latency service and what are the aspired reactions to keep up or re-
establish the service guarantees laid down in the SLA of the NS providing the service.
The characteristics of a low-latency service add some end-to-end latency bounds to the NS that may be guaranteed for
providing the quality of service the service provider wants to deliver to its customers. This is especially useful when a
service provider is delivering real-time video or audio services. As a first step the NFV-MANO system may have to
know the latency that the different NS components introduce when transmitting data. These components include
network links as well as virtual functions that are used to provide the service. With this information, the NFV-MANO
system is able to establish a NS that can provide the end-to-end latency bounds requested by the SLA describing the
NS.
To ensure that the end-to-end latency bounds are obeyed during the whole lifetime of the service, the NFV-MANO
system may have to regularly measure the latency of the NS components and monitor changes of the measurements to
be able to recognize that is violating or even better is soon violating the end-to-end latency bounds defined by the NS
QoS parameters. If such a violation occurs the NFV-MANO system should strive to keep the violation at a minimum
and react on it to avoid that the latency bound are violated to such an extent that the service is disturbed severely. As a
result, this means that NFV-MANO system should react fast on such violations to keep up or re-establish the guaranteed
end-to-end latency bounds.
5.2 Considerations for low-latency service measurements and
recovery
After sketching the overall picture of the important characteristics of a low-latency NS service with the NFV-MANO
system in the previous clause, this clause will give a closer analysis of the different aspects that the characteristic
pointed out.
At the first place, latency measurements are the key feature that is required to support low-latency services within the
NFV-MANO system. The knowledge of the latency of network links and active network components like switches,
VNFs, etc. would facilitate the establishment and maintenance of an end-to-end low-latency bound within a NS
supporting low-latency service(s). Some latency bounds may have to be actively measured (network links) while others
may be derived/included from/in other measurements. As an example, the network link latency can contain the latency
introduced by the network switch, or the latency introduced by a VNF could be derived from the network link latency
where the VNF is located between the endpoint of that link. Nevertheless, the more measurements could be taken at
different locations the easier it would be to detect latency violation early and to locate it within the NS supporting low
latency service. It is thus important for the NFV-MANO system to be aware of the monitoring points of a NS
instance(s) supporting low latency service(s).
As a consequence, the measurement points should be carefully specified. Too few measurements or too few locations
could slow down the detection time of a violation event and make the reestablishment more complicated. Too many
measurement or unsuitable locations can slow down NFV-MANO system processing abilities and impact the
management of services as such or even other NS's. It is noted that ETSI GS NFV-IFA 011 [i.5], ETSI
GS NFV-IFA 014 [i.6] and ETSI GS NFV-IFA 031 [i.7] specify an attribute collection period, which describes the
periodicity at which to collect the performance information. However, it does not specify, neither reflects, the
monitoring frequency at which samples should be collected during the specified collection period. This is important to
specify because two resources with same collection period may have different requirements on the granularity of the
monitoring data, which will depend on the frequency at which samples are collected. Moreover, the sampling frequency
for the same resource can be different for different collection period. Such a provisioning will enable to manage the
monitoring load that the NFV-MANO system has to process. It should be noted that ETSI GS NFV-IFA 027 [i.8] and
ETSI GS NFV-TST 008 [i.9] define the parameter Tick Internal for this purpose.
ETSI

---------------------- Page: 11 ----------------------
12 ETSI GR NFV-EVE 017 V1.1.1 (2020-08)
The location of the measurements can be distributed all over the components of the NS instance(s). To ensure the timely
detection and reaction on latency violations, the measurements can be evaluated e.g. for performance prediction and
analysis, by the local component that does the measurement and analysis or provided from elsewhere. What is important
for a NFV-MANO system is that it should be able to utilize such information related to performance prediction and/or
failures of low latency services supported by NS(s).
In addition to local actions/procedures, defining suitable thresholds that inform the individual components and/or the
central NFV-MANO system early about potential latency violations that can occur in the near future should be taken
into account. This would help in triggering actions even before a latency violation occurs. The source of such triggers
might not be limited to NFV-MANO system itself but might be issued by external instances like an OSS/BSS system. It
is therefore important for the NFV-MANO sy
...