ETSI GS MEC-IEG 006 V1.1.1 (2017-01)

Mobile Edge Computing; Market Acceleration; MEC Metrics Best Practice and Guidelines

DGS/MEC-IEG006Metrics

General Information

• Status: Published
• Publication Date: 30-Jan-2017
• Technical Committee: MEC IEG - Industry Enabling Group
• Current Stage: 12 - Completion
• Due Date: 31-Jan-2017
• Completion Date: 31-Jan-2017

Standards Content (Sample)

GROUP SPECIFICATION
Mobile Edge Computing;
Market Acceleration;
MEC Metrics Best Practice and Guidelines
Disclaimer
The present document has been produced and approved by the Mobile Edge Computing (MEC) ETSI Industry Specification
Group (ISG) and represents the views of those members who participated in this ISG.
It does not necessarily represent the views of the entire ETSI membership.

2 ETSI GS MEC-IEG 006 V1.1.1 (2017-01)

Reference
DGS/MEC-IEG006Metrics
Keywords
MEC, KPI
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: [phone]  Fax: [phone]

Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88

Important notice
The present document can be downloaded from:
http://www.etsi.org/standards-search
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the only prevailing document is the
print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
https://portal.etsi.org/TB/ETSIDeliverableStatus.aspx
If you find errors in the present document, please send your comment to one of the following services:
https://portal.etsi.org/People/CommiteeSupportStaff.aspx
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying
and microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.

© European Telecommunications Standards Institute 2017.
All rights reserved.
TM TM TM
DECT , PLUGTESTS , UMTS and the ETSI logo are Trade Marks of ETSI registered for the benefit of its Members.
TM
3GPP and LTE™ are Trade Marks of ETSI registered for the benefit of its Members and
of the 3GPP Organizational Partners.
GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.
ETSI
3 ETSI GS MEC-IEG 006 V1.1.1 (2017-01)
Contents
Intellectual Property Rights . 5
Foreword . 5
Modal verbs terminology . 5
Introduction . 5
1 Scope . 6
2 References . 6
2.1 Normative references . 6
2.2 Informative references . 6
3 Definitions and abbreviations . 7
3.1 Definitions . 7
3.2 Abbreviations . 8
4 Metrics . 9
4.1 General . 9
4.2 Latency . 10
4.2.1 General . 10
4.2.2 Round-Trip Time . 10
4.2.3 One-Way Delay (OWD) . 11
4.2.4 Set-up Time . 11
4.2.5 Service Processing Time . 11
4.2.6 Context-update time . 12
4.3 Energy efficiency . 12
4.4 Network throughput . 13
4.5 System resource footprint . 14
4.5.1 General . 14
4.5.2 Computational load . 14
4.5.3 Non user data volume exchange . 14
4.6 Quality . 14
4.6.1 General . 14
4.6.2 Objective and service-independent metrics about quality. 14
4.6.3 Objective and service-dependent metrics about quality . 15
4.6.4 Subjective and service-dependent metrics about quality . 15
4.6.5 Objective metrics about user comfort . 15
5 Measurement methodology . 16
5.1 General . 16
5.2 Evaluation of latency . 16
5.2.0 Introduction. 16
5.2.1 Measurement methodology . 17
5.2.1.1 Peak workload test . 17
5.2.1.2 Uniform workload tests . 17
5.2.1.3 Stress tests . 17
5.2.2 Latency measurement setup 1: passive measurements at the terminal . 17
5.2.3 Latency measurement setup 2: passive measurements by probes . 18
5.2.4 Latency measurement setup 3: active measurements . 19
5.3 Evaluation of energy efficiency . 19
5.3.1 Introduction. 19
5.3.2 Measurement methodology . 20
5.3.2.1 General . 20
5.3.2.2 Energy efficiency measurement setup 1 (network side) . 20
5.3.2.2.1 General considerations . 20
5.3.2.2.2 Baseline: measurement without the MEC Server . 21
5.3.2.2.3 Frontline: measurement with the MEC Server . 21
5.3.2.2.4 Computation of EE gains . 21
5.3.2.3 Energy efficiency measurement setup 2 (terminal side) . 22
ETSI
4 ETSI GS MEC-IEG 006 V1.1.1 (2017-01)
5.3.2.3.1 General considerations . 22
5.3.2.3.2 Baseline: measurement without the MEC Server . 22
5.3.2.3.3 Frontline: measurement with the MEC Server . 22
5.3.2.3.4 Computation of EE gains . 23
5.4 Evaluation of network throughput . 23
5.4.1 General . 23
5.4.2 Network throughput measurement setups . 23
5.5 Evaluation of resource footprint . 24
5.5.1 General . 24
5.5.2 Computational load measurement setup 1: isolated execution environment . 24
Annex A (informative): Network Throughput Example . 25
Annex B (informative): Examples of metric value ranges . 26
B.1 5G latency requirements . 26
B.2 5G energy efficiency . 26
Annex C (informative): POC#3 RAVEN - example of latency metric assessment . 27
History . 29

ETSI
5 ETSI GS MEC-IEG 006 V1.1.1 (2017-01)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (https://ipr.etsi.org/).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This Group Specification (GS) has been produced by ETSI Industry Specification Group (ISG) Mobile Edge
Computing (MEC).
Modal verbs terminology
In the present document "shall", "shall not", "should", "should not", "may", "need not", "will", "will not", "can" and
"cannot" are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of
provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.
Introduction
Mobile Edge Computing is a new technology that provides an IT service environment and cloud-computing capabilities
at the edge of the mobile network, in close proximity to mobile subscribers. In order to make MEC a success and
encourage network operators to deploy Mobile Edge (ME) systems as well as to make MEC attractive to application
developers and service providers, it is necessary to demonstrate the benefits of this technology for fulfilling various
requirements. In order to make MEC an attractive proposition for service providers and applications developers to host
their applications on a ME Host instead of in a centralized cloud, it is important to demonstrate a quantifiable
performance increase.
The present document describes a number of performance metrics which can be used to demonstrate the benefits of
deploying services and applications on a ME Host compared to a centralized cloud or server. Examples of how these
metrics can be measured are also described.
Examples of such metrics KPIs are reducing latency, increasing end-to-end energy efficiency and increasing network
throughput.
ETSI
6 ETSI GS MEC-IEG 006 V1.1.1 (2017-01)
1 Scope
The present document describes various metrics which can potentially be improved through deploying a service on a
MEC platform. Example use cases are used to demonstrate where improvements to a number of key performance
indicators can be identified in order to highlight the benefits of deploying MEC for various services and applications.
Furthermore, the present document describes best practices for measuring such performance metrics and these
techniques are further exemplified with use cases.
Metrics described in the present document can be taken from service requirements defined by various organizations
rd
(e.g. 5G service requirements defined by Next Generation Mobile Networks (NGMN) or 3 Generation Partnership
Project (3GPP)). An informative annex is used to document such desired and/or achieved ranges of performance which
could be referenced from the main body of the present document.
2 References
2.1 Normative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
Referenced documents which are not found to be publicly available in the expected location might be found at
https://docbox.etsi.org/Reference.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are necessary for the application of the present document.
[1] ETSI ES 202 706 (V1.4.1): "Environmental Engineering (EE); Measurement method for power
consumption and energy efficiency of wireless access network equipment".
[2] ETSI ES 203 228 (V1.1.1): "Environmental Engineering (EE); Assessment of mobile network
energy efficiency".
[3] ETSI GS MEC 002: "Mobile Edge Computing (MEC); Technical Requirements".
[4] ETSI GS MEC 001: "Mobile Edge Computing (MEC); Terminology".
[5] ETSI ES 202 336-12: "Environmental Engineering (EE); Monitoring and control interface for
infrastructure equipment (power, cooling and building environment systems used in
telecommunication networks); Part 12: ICT equipment power, energy and environmental
parameters monitoring information model".
2.2 Informative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] IETF RFC 4656: "One way active measurement protocol".
[i.2] IETF RFC 5357: "A two-way active measurement protocol".
ETSI
7 ETSI GS MEC-IEG 006 V1.1.1 (2017-01)
[i.3] IETF IP Performance Metrics Working Group: IPPM status pages.
NOTE: Available at https://tools.ietf.org/wg/ippm/.
[i.4] IETF IP Performance Metrics Working Group: Charter.
NOTE: Available at https://tools.ietf.org/wg/ippm/charters.
[i.5] NGMN Alliance 5G White Paper version 1.0 (17 February 2015): "NGMN 5G White Paper".
NOTE: Available at https://www.ngmn.org/uploads/media/NGMN_5G_White_Paper_V1_0.pdf.
[i.6] J. S. Milton, J. Arnold, "Introduction to Probability and Statistics", McGraw-Hill Education,
th
4 Edition.
[i.7] P. Serrano, M. Zink, J. Kurose, "Assessing the fidelity of COTS 802.11 sniffers", IEEE
INFOCOM 2009, Rio de Janeiro, Brazil, April 2009.
[i.8] P. Serrano, A. Garcia-Saavedra, G. Bianchi, A. Banchs, A. Azcorra, "Per-frame Energy
Consumption in 802.11 Devices and its Implication on Modeling and Design," IEEE/ACM
Transactions on Networking, vol.23, no.4, pp.1243-1256, Aug. 2015.
[i.9] N Vallina-Rodriguez, J Crowcroft, "Energy Management Techniques in Modern Mobile
Handsets," IEEE Communications Surveys & Tutorials, 1-20.
[i.10] ETSI MEC PoC#3 RAVEN: "Radio aware video optimization in a fully virtualized network".
NOTE: Available at
http://mecwiki.etsi.org/index.php?title=PoC_3_Radio_aware_video_optimization_in_a_fully_virtualized
_network.
[i.11] ETSI GS MEC 015: "Mobile Edge Computing (MEC) Bandwidth Management API".
3 Definitions and abbreviations
3.1 Definitions
For the purposes of the present document, the terms and definitions given in ETSI GS MEC 001 [4], ETSI
ES 203 228 [2] and the following apply:
NOTE: For some background definitions for network level energy efficiency, see ETSI ES 203 228 [2].
Energy Efficiency (EE): relation between the useful output and energy/power consumption
mobile network coverage Energy Efficiency: ratio between the area covered by the network in the Mobile Network
under investigation and the energy consumption
mobile network data Energy Efficiency: ratio between the performance indicator based on Data Volume and the
energy consumption when assessed during the same time frame
mobile network energy consumption: overall energy consumption of equipment included in the MN under
investigation
system resources: any kinds of entities to be shared to compose services including computing power, processor and
accelerator loads, memory usage, storage, network, database and applications
NOTE: System resources can be considered as a set of coherent functions, network data objects or services,
accessible through a server where such system resources reside on a single host or multiple hosts and are
clearly identifiable.
ETSI
8 ETSI GS MEC-IEG 006 V1.1.1 (2017-01)
3.2 Abbreviations
For the purposes of the present document, the following abbreviations apply:
rd
3GPP 3 Generation Partnership Project
API Application Programming Interface
BER Bit Error Rate
CN Core Network
CPU Central Processing Unit
DC Direct Current
EE Energy Efficiency
eNB eNodeB
GPS Global Positioning System
ICMP Internet Control Message Protocol
IDT Inter Departure Time
IP Internet Protocol
IPPM IP Performance Metrics
KPI Key Performance Indicator
ME Mobile Equipment
MN Mobile Network
MOS Mean Opinion Score
MSL MEC-Specific Latency
MSS Maximum Segment Size
MTU Maximum Transmission Unit
NGMN Next Generation Mobile Networks
NRQA No Reference Quality Assessment
NRT Non Real-Time
NTP Network Time Protocol
OS Operating System
OWD One-Way Delay
PA Power Amplifier
PEAQ Perceptual Evaluation of Audio Quality
PEVQ Perceptual Evaluation of Video Quality
PLR Packet Loss Rate
POC Proof Of Concept
POLQA Perceptual Objective Listening Quality Assessment
PSNR Peak Signal-to-Noise Ratio
PSS Proportional Set Size
PTP Precision Time Protocol
QoS Quality of Service
RAN Radio Access Network
RAVEN Radio Aware Video optimization in a fully virtualized network
RSS Resident Set Size
RT Real-Time
RTT Round-Trip Time
SDT Service Delivery Time
SGW Service GW
SPT Service Processing Time
SUT Set-Up Time
TCP Transmission Control Protocol
UD Update delay
UDP User Datagram Protocol
UE User Equipment
USS Unique Set Size
VSS Virtual Set Size
ETSI
9 ETSI GS MEC-IEG 006 V1.1.1 (2017-01)
4 Metrics
4.1 General
This clause introduces the metrics considered by ETSI ISG MEC for the evaluation of improvements introduced by
Mobile Edge Computing technologies. While clause 4 is describing all the different metrics considered (in separated
clauses), clause 5 is organized similarly (with one clause corresponding to each metric in clause 4) in order to introduce
the related measurement methodologies.
Generally MEC metrics are introduced with different purposes: evaluating the improvement given by MEC (as
perceived by the end user), and assessing the benefits of different MEC deployment options (thus giving insights from a
technologic point of view).
All metrics introduced in the present document can demonstrate the improvements of MEC solutions at least in the two
following ways:
1) comparison between MEC and non-MEC solutions;
2) assessment of MEC deployments: comparison between different ME host positions within the network.
In both cases, the goal is not to compare different vendors or solution providers, but to assess the improvement of MEC
introduction with respect to a traditional system (without MEC), e.g. in order to understand the different deployment
options against the different use cases (e.g. by minimizing costs, maximizing benefits or flexibility).
For this reason, MEC metrics can be classified into two main groups: functional and non-functional metrics. For both
categories (defined here below), metrics can be referred to different MEC use cases, as listed in IETF RFC 4656 [i.1],
and the actual assessment of these metrics can depend on the particular service and/or application utilization:
1) Functional metrics are related to MEC performances impacting on user perception (often called also KPIs, key
performances indicators):
- Examples of functional service performance KPIs include: latency (both end-to-end, and one-way),
energy efficiency, throughput, goodput, loss rate (number of dropped packets), jitter, number of
out-of-order delivery packets, QoS, and MOS. Each of the functional metrics should be defined on per
service basis. Note that the latency in localization (time to fix the position) is different from latency in
content delivery.
2) Non-functional metrics are related to the performance of the service in terms of deployment and management:
- Examples of non-functional metrics include: service lifecycle (instantiation, service deployment, service
provisioning, service update (e.g. service scalability and elasticity), service disposal), service availability
and fault tolerance (aka reliability), service processing/computational load, global ME host load, number
of API request (more generally number of events) processed/second on ME host, delay to process API
request (north and south), number of failed API request. The sum of service instantiation, service
deployment, and service provisioning provide service boot-time.
In both cases, one could measure all the statistics over the above metrics. In fact, all metrics are in principle
time-variable, and could be measured in a defined time interval and described by a profile over time or summarized
through:
• the maximum value;
• mean and minimum value;
• standard deviation;
• the value of a given percentile;
• etc.
All MEC metrics assessments can be done by considering the overall system, or portions of that, according to the
purpose of the measurement itself. An example below (figure 1) shows a mobile network system with ME host, and the
different entities potentially involved in the assessment.
ETSI
10 ETSI GS MEC-IEG 006 V1.1.1 (2017-01)

Figure 1: Measuring MEC metrics
4.2 Latency
4.2.1 General
The concept of latency is wide and encompasses manifolds metrics: in communications, latency refers to a time-interval
whose measurement quantifies the delay elapsed between any event and a consequent target effect. Even more, still in
the communication domain, latency is useful to measure phenomena both in the control plane (e.g. set-up time or
hand-over time) and in the data plane (e.g. transfer delay). The purpose of this clause is not to define all the latency
metrics potentially relevant to the MEC solutions, but rather to highlight what type of latency metrics can be adopted
(or newly defined) and their potential roles.
Referring to all the latency metrics in the clauses 4.2.2 to 4.2.6, it is assumed that an ideal synchronization holds across
the nodes under test for measurements purposes.
Note that different Latency measurements have been specified in IETF RFC 4656 [i.1] and IETF RFC 5357 [i.2].
However, the latency definitions within the subsequent clauses are referring to latency measured on application level.
4.2.2 Round-Trip Time
Round-Trip Time (RTT): by referring to figure 1, it is defined as the time taken for a request (e.g. packet) generated
from a terminal (a) to go to the destination, be updated or replied and travel back to (a), in conditions of ideal service
capabilities (i.e. the server and/or terminal response time is supposed to be fixed and the RTT does not depend on the
server/terminal computational load). Characteristics of RTT include:
1) Depending on the service type, the RTT might include very heterogeneous paths. Referring to figure 1:
- (a)-(c)-(a) in case of MEC client-server applications;
- (a)-(e)-(a) in case of non-MEC client-server applications;
ETSI
11 ETSI GS MEC-IEG 006 V1.1.1 (2017-01)
- (a)-(c)-(a') -(c)-(a) in case of MEC P2P applications;
- (a)-(e)-(a') -(e)-(a) in case of non-MEC P2P applications.
2) RTT is a variable which likely changes over time for the same station and is described by a RTT profile over
time. RTT statistics might be summarized through: the maximum, mean and minimum value of RTT, the
variance, the value of a given percentile, etc.
3) RTT might also vary throughout different stations (e.g. terminal (a) and terminal (a')). A statistical description
of RTT can then be used to describe how much the latency is homogeneous across the terminals (RTT
fairness). As an example, one could measure the maximum difference between the mean latencies achieved by
two different terminals. RTT fairness would be important for applications such as the on-line gaming, where
all the users need to have a relatively similar latency to deliver a fair behaviour.
RTT is meant to describe how efficient the flow of information is; it is useful to evaluate the benefit of MEC
architecture.
4.2.3 One-Way Delay (OWD)
OWD can be defined as a delay of an application request (e.g. data packet) from a source application to a destination
th
application on the user-plane. Formally, the OWD of the i request (e.g. datagram) between two interfaces (source and
destination) can be calculated as: OWD(i) = | t_source(i)) - t_destination(i) |. It is assumed that the timing between the
source (e.g. user terminal) and destination (e.g. the ME host) is synchronized.
True OWD measurements require the capturing and time stamping of data packets at both ends of the connection link.
This most often involves distributed and synchronized measurement nodes. The desired level of accuracy depends on
the application.
4.2.4 Set-up Time
In case of a connection-oriented service from the user to the mobile edge application, also the initial signalling could
initially affect the latency and should be then kept into account. For this purpose, the set-up time should be defined:
• Set-up Time (SUT) is defined as the time elapsed since the service request by the terminal (a) to get the first
packet of information to it (a):
- In case a maximum number of simultaneous connections were set, a metric related to the blocking
probability (or connection refusal) should be jointly considered - even if it is not directly a latency
measurement.
The SUT metric holds wherever the source of information is placed (local cache, remote cache, central server, etc.) and
whatever the signalling architecture is (ME host, signalling proxies, central server); it measures the time for the
successful establishment of the service.
4.2.5 Service Processing Time
The last variable which influences the service latency (and which completes its description) is the service processing
time - it is supposed ideal in RTT, hence neglected so far. The following metrics are meant to complete the description:
• Service Processing Time (SPT) is the time employed by the server (MEC or non-MEC) to process and fulfil a
user request. It depends on manifolds variable (computational load).
• Service Delivery Time (SDT) is the time which is taken to a user request to reach the server, being processed
and reach back the terminal.
• Update delay (UD) is the metric which describes the time (minimum/mean/maximum, etc.) required to have
the servers updated with the relevant information. This is important for MEC services - which is often making
use of local caching of the information - but also for non-MEC solutions.
ETSI
12 ETSI GS MEC-IEG 006 V1.1.1 (2017-01)
It is useful to monitor SPT in order to evaluate the processing and computing capabilities provided by the ME host; SPT
is a non-functional metric. SDT puts together SPT and RTT (SDT=SPT+RTT); SDT is a functional metric since the
RTT is included. UD measures a parameter which cannot be directly be measured at the terminal, but which heavily
influences the quality and consistency of the applications. UD is a non-functional metric, and it is very suitable for
assessing MEC solutions.
In general, considering the specific MEC architecture, additional metrics can be specifically defined for the comparison
between MEC solutions. They can be named MEC-Specific Latency metrics (MSL).
MSL metrics are particularly important to support the optimization of the MEC architecture. MSL can include new
custom metrics definitions which can apply only to a specific MEC architecture.
For example one could study how the SDT or the UD varies depending on the position of the MEC serving nodes.
Additionally, MEC-specific metric could be defined: for example an interesting figure is the number of simultaneous
requests that a central server can manage to update the MEC entities throughout the network architecture.
4.2.6 Context-update time
In order to support an effective provision of context-aware applications, such as e.g. device location, augmented reality,
one key feature is the ability to make available in real time the application with the relevant information about the
general context of the mobile user, including potential information not personally supplied by the user. Some key
context variables are, e.g. the localization or any other information provided by the producer ME application.
To this aim, this metric is defined as the time between a certain key variable is updated at the terminal and the service is
able to provide an updated state of the user based on the global context. This time should be small enough for a
seamless operation of context-aware applications. The context-update time is intended as non-functional metric, since
its impact is on the delay at application level, which is including also other delays, and it is the sum of the one way
delay (OWD) plus the service processing time (SPT).
4.3 Energy efficiency
Energy efficiency (EE) for mobile networks is currently defined in ETSI ES 202 706 [1] and ETSI ES 203 228 [2] and
it expresses the relationship between consumed power (or energy) and the production of a certain selected basic KPI of
interest.
power
η =
(1a)
KPI
As an example considering KPI = Traffic, an EE metric can be expressed in the following manner:
power
η =      [W/bps] or [J/bit] (1b)
traffic
and in this case it expresses the consumed watt of power per transferred traffic (in bps), or equivalently the energy per
number of transferred bits.
Depending on the purpose of the evaluation, other KPIs are also possible.
As a general definition, EE is referred to the ability of a mobile system to perform a certain work (e.g. transmit a
volume of traffic, or satisfying the QoS requirements for a certain service) by minimizing the power consumption. From
this point of view, different kinds of EE metrics can be defined, depending on the scope of the assessment, and thus on
the source of power consumption considered. In fact, EE can be defined:
• at component level (e.g. PA, chip, other components of network equipment or of a terminal, etc.);
• at node level (e.g. terminal, eNB, SGW, etc.);
• at network level (e.g. a set of nodes including or not terminals, only RAN equipment or including CN nodes,
etc.).
ETSI
13 ETSI GS MEC-IEG 006 V1.1.1 (2017-01)
In particular, energy efficiency can be defined from a user perspective (thus measuring the mobile terminals power
consumption) and/or from a network provider perspective (thus by considering parts of the network, or network
equipment, or again the overall network).
By following the previous classifications, the energy efficiency at user equipment is defined as:
P
UE
EE =      [W/bps] or [J/bit] (1c)
UE
T
UE
where P is the power consumption of the mobile terminal(s) and T is the volume of traffic (at user level) transferred
UE UE
within a certain traffic session (assuming that, by definition, during that session all QoS requirements are satisfied).
Similarly, it is defined:
P
NET
(1d)
EE =      [W/bps] or [J/bit]
NET
T
NET
as the energy efficiency at network side, where P is the power consumption of the mobile network (or a portion of
NET
the network) and T is the volume of traffic (at user level) transferred within a certain traffic session (assuming that,
NET
by definition, during that session all QoS requirements are satisfied).
As a consequence, since power consumption of terminals is a KPI directly perceived by end users, the consequent
definition of EEUE is a functional metric, while EENET is a non-functional metric (the latter being related to the system
efficiency, and not necessarily perceivable by the user). While the first one (EE ) has direct impacts on the mobile
UE
terminals battery lifetime, E2E evaluations of the second metric (defined in ETSI ES 202 706 [1] and ETSI
ES 203 228 [2]) are of a particular interest for mobile operators, in order to assess the efficiency sustainability of their
networks from a operation point of view.
4.4 Network throughput
Network has a clear influence on the quality perceived by the users while consuming some applications. Depending on
the kind of application, different parameters and even thresholds could be taken into account.
Commonly, network throughput or bandwidth consumption are determinant in the sense that not having enough
throughput starves the application with the needed payload to be executed. Apart from that, episodes of sporadic
absence of enough throughput could be mitigated by means of buffering, with different impact depending on the
duration of such event.
For instance, video applications present some requirements due to the resolution in which the video content is coded.
Different resolutions imply distinct bit rates for streaming, and then different throughput requirements in the network.
The network throughput is also relevant from a video application perspective to determine the starting time for video
play-out. Typically video applications initially send burst of information to rapidly feeding the application players,
trying to minimize the time the user takes to experience the content.
As mentioned before, events where the throughput level cannot be guaranteed can produce starvation of content in the
players, then experiencing (re-)buffering times that can seriously impact the perception of the user.
Similar situations could happen with some other applications like gaming, software upgrade downloads, etc.
Network throughput is defined as measurement in terms bit rate units (e.g. kbps) at application level, in both upstream
and downstream direction of the communication. Since this is a metric at application level, it is categorized as
functional metric.
Throughput measurements could be performed both at transmitter side and at the receiver side. The latter case could
also be referred as the network goodput.
This metric can serve as a basis for assessing Mobile Edge Computing Bandwidth Management API [i.11].
ETSI
14 ETSI GS MEC-IEG 006 V1.1.1 (2017-01)
4.5 System resource footprint
4.5.1 General
When implementing a service, it is relevant to analyse the amount of system resources consumed, both in terms of a
node's capacity but also in terms of communication requirements. All the metrics considered here are non-functional.
4.5.2 Computational load
The processing/computational time/load measures the amount of CPU processing time or cycles, and memory usage
(VSS, RSS, PSS and USS) a service requires to operate.
A service can also utilize the I/O resources (e.g. Ethernet), in which case, an overall system resource utilization score
(combining compute, memory, and I/O resources) can be used to characterize service requirements in different
conditions (light, medium, and high load).
As described in clause 4.1, computational load related metrics are considered to be non-functional.
4.5.3 Non user data volume exchange
A service deployed with MEC requires the coordination of the modules running across different elements, this including
the exchange of non-user data between entities to e.g. support application and user mobility. This type of system
resource consumption can be accounted for by measuring the non-user-data rate between the following entities:
• Between the ME host and the Radio Network Nodes.
• Between ME hosts.
• Between the ME host and the operational network management.
4.6 Quality
4.6.1 General
Traditionally, QoE measures the global system performance using both subjective and objective measures of customer
satisfaction. Efficiency, ease of use, reliability, customer loyalty are some of the factors the QoE addresses. In addition
to these, other aspects such as service costs, architecture security and user's privacy can be taken into account for a more
comprehensive definition of QoE.
The QoE metrics strongly depend on the service/application under analysis. Since new services are implemented thanks
to the flexibility of MEC, the definition of QoE metrics becomes a relevant aspect.
QoE metrics can be roughly classified into:
1) Objective and service-independent metrics about quality.
2) Objective and service-dependent metrics about quality.
3) Subjective and service-dependent metrics about quality.
4) Objective metrics about user comfort.
4.6.2 Objective and service-independent metrics about quality
In the first class (Objective and service-independent metrics about quality) the following ones can be mentioned:
1) The buffering time (usually expressed in seconds) involves pre-loading data into a certain area of memory so
that the data can be accessed faster when the application needs it. The larger the buffering time, the higher the
delay between the 'live' transmission and the playback. Buffer dimensioning is a critical parameter especially
in media streaming but buffer can also be of great benefit to compensate network fluctuations.
ETSI
15 ETSI GS MEC-IEG 006 V1.1.1 (2017-01)
2) Packet loss rate (PLR) is a traditional metric in packet-based network and it can be referred to as an
application independent metric. A good estimation of this parameter helps the transmitter to better tune
(whenever possible) data encoding to suit channel conditions. A feedback channel from the receiver(s) to the
source is required to collect meaningful information of the perceived received quality. If a TCP transmission is
available, this channel is already present and a proper processing of ACK/NACK messages can help
interpolate the PLR figure.
3) The Bit Error Rate (BER) is the number of bit errors divided by the total number of transferred bits during a
particular time interval. Similar to PLR, it is application independent and it has the great benefit of providing
an insight of the channel status. The BER can be also evaluated using stochastic (Monte Carlo) simulations. If
a simple transmission channel model (e.g. Binary symmetric channel or additive white Gaussian noise), this
parameter can also be calculated analytically.
4.6.3 Objective and service-dependent metrics about quality
Apart from these application-independent metrics, some application-specific metrics can also be devised (Objective and
service-dependent metrics about quality). To define a proper QoE, in this c
...