ETSI GS F5G 005 V1.1.1 (2022-03)

Fifth Generation Fixed Network (F5G) F5G High-Quality Service Experience Factors Release#1

DGS/F5G-005_QoE

General Information

• Status: Not Published
• Technical Committee: F5G - Fifth Generation Fixed Network
• Current Stage: 12 - Completion
• Due Date: 22-Mar-2022
• Completion Date: 15-Mar-2022

Standards Content (Sample)

GROUP SPECIFICATION
Fifth Generation Fixed Network (F5G)
F5G High-Quality Service Experience Factors
Release #1
Disclaimer
The present document has been produced and approved by the Fifth Generation Fixed Network (F5G) ETSI Industry
Specification Group (ISG) and represents the views of those members who participated in this ISG.
It does not necessarily represent the views of the entire ETSI membership.

2 ETSI GS F5G 005 V1.1.1 (2022-03)

Reference
DGS/F5G-005_QoE
Keywords
F5G, QoE, QoS
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: [phone]  Fax: [phone]

Siret N° 348 623 562 00017 - APE 7112B
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° w061004871

Important notice
The present document can be downloaded from:
http://www.etsi.org/standards-search
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the prevailing version of an ETSI
deliverable is the one made publicly available in PDF format at www.etsi.org/deliver.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
https://portal.etsi.org/TB/ETSIDeliverableStatus.aspx
If you find errors in the present document, please send your comment to one of the following services:
https://portal.etsi.org/People/CommiteeSupportStaff.aspx
Notice of disclaimer & limitation of liability
The information provided in the present deliverable is directed solely to professionals who have the appropriate degree of
experience to understand and interpret its content in accordance with generally accepted engineering or
other professional standard and applicable regulations.
No recommendation as to products and services or vendors is made or should be implied.
No representation or warranty is made that this deliverable is technically accurate or sufficient or conforms to any law
and/or governmental rule and/or regulation and further, no representation or warranty is made of merchantability or fitness
for any particular purpose or against infringement of intellectual property rights.
In no event shall ETSI be held liable for loss of profits or any other incidental or consequential damages.

Any software contained in this deliverable is provided "AS IS" with no warranties, express or implied, including but not
limited to, the warranties of merchantability, fitness for a particular purpose and non-infringement of intellectual property
rights and ETSI shall not be held liable in any event for any damages whatsoever (including, without limitation, damages
for loss of profits, business interruption, loss of information, or any other pecuniary loss) arising out of or related to the use
of or inability to use the software.
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and
microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.

© ETSI 2022.
All rights reserved.
ETSI
3 ETSI GS F5G 005 V1.1.1 (2022-03)
Contents
Intellectual Property Rights . 5
Foreword . 5
Modal verbs terminology . 5
1 Scope . 6
2 References . 6
2.1 Normative references . 6
2.2 Informative references . 7
3 Definition of terms, symbols and abbreviations . 7
3.1 Terms . 7
3.2 Symbols . 7
3.3 Abbreviations . 7
4 Introduction . 8
4.1 Overview of High-Quality of Experience (QoE) . 8
4.2 Structure of the present document . 9
5 Application Services and related QoE factors . 10
5.1 QoE of Typical Applications . 10
5.1.1 General Description . 10
5.1.2 QoE Factors of Typical Applications. 10
5.1.2.1 Introduction . 10
5.1.2.2 Voice . 10
5.1.2.3 High-speed Internet . 10
5.1.2.4 Web Browsi ng . 10
5.1.2.5 TV . 10
5.1.3 Generic Measurement Methodology . 10
5.1.4 Generic QoE management . 11
5.2 Cloud VR. 11
5.2.1 General Description . 11
5.2.2 Factors Affecting Cloud VR service . 11
5.2.2.1 Factors Affecting Weak-Interaction Cloud VR Service Experience . 11
5.2.2.2 Factors Affecting Strong-Interaction Cloud VR Service Experience. 13
5.2.3 Measurement Methodology . 13
5.2.4 QoE Management . 14
6 Factors impacting QoE for Networking Services . 14
6.1 Customer Premises Networks. 14
6.1.1 General Description . 14
6.1.2 QoE Factors for CPNs . 14
6.1.3 Measurement Methodology for CPNs . 15
6.1.4 QoE management for CPNs . 16
6.2 Access and Aggregation Network . 16
6.2.1 General Description . 16
6.2.2 QoE Factors for Access and Aggregation Networks . 16
6.2.2.1 Bandwidth . 16
6.2.2.2 Latency . 17
6.2.2.3 Packet Jitter . 17
6.2.2.4 Reliability . 17
6.2.2.5 Security . 17
6.2.3 Measurement Methodology . 17
6.2.3.1 End-to-end Measurements . 17
6.2.3.2 Network Telemetry . 17
6.2.4 QoE management . 17
6.2.4.1 Network Planning . 17
6.2.4.2 QoS management . 17
ETSI
4 ETSI GS F5G 005 V1.1.1 (2022-03)
7 Mechanisms and Approaches for F5G QoE . 18
7.1 General Considerations . 18
7.2 Measurement-based QoE Assessment . 18
7.2.1 Introduction. 18
7.2.2 Measurements with simulated traffic . 19
7.2.3 Measurements of real traffic . 19
7.3 Network slicing to improve QoE . 19
7.3.1 Introduction. 19
7.3.2 F5G Underlay Network Planning . 19
7.3.3 F5G service-oriented slice modelling . 19
7.3.4 Measuring the Quality of a Network Slice . 20
7.4 AI-based QoE Assessment . 20
7.4.1 Based on Network QoS parameters . 20
7.4.2 Based on detected user behaviour . 21
7.5 Service Provider Domain oriented QoS Measurements and QoE Assessment. 21
7.5.1 Introduction. 21
7.5.2 Media application-based QoS Measurements and QoE Assessment . 22
7.5.2.1 General Description . 22
7.5.2.2 Quality Measurement Methods for Media Services over UDP/RTP . 22
7.5.2.3 Quality Measurement Methods for Media Services over TCP . 23
8 Summary of Requirements and Recommendations of F5G QoE . 26
8.1 Requirements . 26
8.2 Recommendations . 27
Annex A (informative): Example of quality monitoring for media services over UDP/RTP . 28
Annex B (informative): Example of quality monitoring for media services over TCP . 30
History . 32

ETSI
5 ETSI GS F5G 005 V1.1.1 (2022-03)
Intellectual Property Rights
Essential patents
IPRs essential or potentially essential to normative deliverables may have been declared to ETSI. The declarations
pertaining to these essential IPRs, if any, are publicly available for ETSI members and non-members, and can be
found in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to
ETSI in respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the
ETSI Web server (https://ipr.etsi.org/).
Pursuant to the ETSI Directives including the ETSI IPR Policy, no investigation regarding the essentiality of IPRs,
including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not
referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become,
essential to the present document.
Trademarks
The present document may include trademarks and/or tradenames which are asserted and/or registered by their owners.
ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no
right to use or reproduce any trademark and/or tradename. Mention of those trademarks in the present document does
not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks.
DECT™, PLUGTESTS™, UMTS™ and the ETSI logo are trademarks of ETSI registered for the benefit of its

Members. 3GPP™ and LTE™ are trademarks of ETSI registered for the benefit of its Members and of the 3GPP
Organizational Partners. oneM2M™ logo is a trademark of ETSI registered for the benefit of its Members and of the ®
oneM2M Partners. GSM and the GSM logo are trademarks registered and owned by the GSM Association.
Foreword
This Group Specification (GS) has been produced by ETSI Industry Specification Group (ISG) Fifth Generation Fixed
Network (F5G).
Modal verbs terminology
In the present document "shall", "shall not", "should", "should not", "may", "need not", "will", "will not", "can" and
"cannot" are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of
provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.

ETSI
6 ETSI GS F5G 005 V1.1.1 (2022-03)
1 Scope
The present document studies the end-to-end Quality of Experience (QoE) factors for services over the broadband
network. High-QoE reflects the overall performance at the service level from the perspective of the end user. The
present document analyses the general factors that impact service performance and identifies the overall high-QoE
dimensions for each service. The key services discussed in the present document are typical Internet applications and
Virtual Reality (VR). Other services and applications QoE are for further study.
2 References
2.1 Normative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
Referenced documents which are not found to be publicly available in the expected location might be found at
https://docbox.etsi.org/Reference.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are necessary for the application of the present document.
[1] ETSI GS F5G 004: "F5G Architecture Release 1".
[2] Recommendation ITU-T Y.1540: "Internet protocol data communication service - IP packet
transfer and availability performance parameters".
[3] Recommendation ITU-T P.863: "Perceptual objective listening quality prediction".
[4] ETSI TS 103 222-1: "Speech and multimedia Transmission Quality (STQ); Reference
benchmarking, background traffic profiles and KPIs; Part 1: Reference benchmarking, background
traffic profiles and KPIs for VoIP and FoIP in fixed networks".
[5] ETSI TS 103 222-2: "Speech and multimedia Transmission Quality (STQ); Reference
benchmarking, background traffic profiles and KPIs; Part 2: Reference benchmarking and KPIs for
High speed internet".
[6] Recommendation ITU-T J.247: "Objective perceptual multimedia video quality measurement in
the presence of a full reference".
[7] IETF RFC 3357: "One-way Loss Pattern Sample Metrics".
[8] IETF RFC 768: "User Datagram Protocol".
[9] IETF RFC 3550: "RTP: A Transport Protocol for Real-Time Applications".
[10] IETF RFC 793: "Transmission Control Protocol".
[11] Recommendation ITU-T P.10: "Vocabulary for performance and quality of service".
[12] Recommendation ITU-T G.988: "ONU management and control interface (OMCI) specification".
ETSI
7 ETSI GS F5G 005 V1.1.1 (2022-03)
2.2 Informative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] ETSI TR 102 505: "Speech and multimedia Transmission Quality (STQ); Development of a
Reference Web page".
[i.2] ETSI EG 202 057 (Part 1 to 4): "Speech Processing, Transmission and Quality Aspects (STQ);
User related QoS parameter definitions and measurements".
[i.3] Broadband Forum (BBF) TR-126: "Triple-play Services Quality of Experience (QoE)
Requirements".
[i.4] ETSI GR F5G 002: "F5G Use Cases Release 1".
[i.5] ETSI White Paper No. 47, "Fibre Development Index: Driving Towards an F5G Gigabit Society",
ISBN No. 979-10-92620-41-1.
3 Definition of terms, symbols and abbreviations
3.1 Terms
For the purposes of the present document, the terms given in F5G Architecture [1] and the following apply:
Key Quality Indicators (KQI): QoS metrics, which are important and have a major impact on the QoE of applications
and networks
Mean Opinion Score (MOS): mean of the values on a predefined scale that users assign to their opinion of the
performance of a system quality
NOTE: See Recommendation ITU-T P.10 [11].
Quality of Experience (QoE): subjective measure of performance of applications or services that relies in human
opinion on the perceived quality
Quality of Service (QoS): description or quantitative measurements of the overall performance of the F5G system at
the network, service, and application domain
3.2 Symbols
Void.
3.3 Abbreviations
For the purposes of the present document, the abbreviations given in F5G Architecture [1] and the following apply: ®
AP Wi-Fi Access Points
Cloud VR Cloud Virtual Reality
DRTT Downstream Round-Trip Time
DSLR Downstream Segment Loss Rate
FoV Field of View
ETSI
8 ETSI GS F5G 005 V1.1.1 (2022-03)
KQI Key Quality Indicators
MOS Mean Opinion Score
NPS Net Promoter Score
USLR Upstream Segment Loss Rate
VoD Video on Demand
4 Introduction
4.1 Overview of High-Quality of Experience (QoE)
This clause provides an introduction to Quality of Experience (QoE) and the distinction between QoE and QoS as used
in the present document.
The QoE (Quality of Experience) and QoS (Quality of Service) terminology (see clause 3.1 for the term) are often used
interchangeably, but are actually two separate concepts.
The QoE is a combination of objective measurable components (as metrics on the conditions in the network and service
platforms that are required for a specific service to work properly) and subjective components (as user expectancy on
the service, user previous experience or user personal preferences).
Mean Opinion Score (MOS) is one often used QoE measurement metric typically used to quantify the perceptual impact
(the users' QoE) for various forms of service degradation.
QoE can also be assessed based on objective QoS metrics. There are different QoS metrics, which can be gathered,
some easy to collect other more difficult. Depending on the specific service different combinations of these metrics may
be needed for QoE assessment. The availability and understanding of these QoS metrics determine QoE assessment in
different levels of detail. This QoE assessment based on QoS metrics is the focus of the present document.
QoS is a measure of the performance of networked services at the network or application level. QoS also refers to a set
of techniques that enable the network administrator to manage the network performance differentiating between
different users. QoS metrics may include network layer measurements such as packet loss, delay or jitter or application
level measurements such as video frame loss, frame freezing, image distortion. The Key Quality Indicators (KQI) are
the QoS metrics, which have the largest impact on QoE.
In general, there is a non-linear relationship between the subjective QoE as measured by the MOS or other metrics and
various parameters used to measure network performance (e.g. encoding bit rate, packet loss, delay, availability,
etc.).Typically there will be multiple service or network level performance (QoS) metrics that will impact overall QoE.
The relationship between QoE and service and network performance (QoS) metrics is typically derived empirically.
Having identified the QoE/QoS relationship, if it is possible, it can be used to predict the expected QoE for a user, given
the QoS parameters, or given a target QoE, the required network and service requirements can be derived.

Figure 4-1: Example QoS/QoE non-linear relationship
ETSI
9 ETSI GS F5G 005 V1.1.1 (2022-03)
The Key Quality Indicators (KQI) are composed by the QoS metrics, which have the largest impact on QoE, namely
user centric and service specific quality patterns that directly influence the user perception for each service category.
The definition of these quality patterns poses a challenge where Artificial Intelligence correlation techniques may play
an important role.
QoE targets are needed for each service and application and should be included from the beginning in system design
and engineering processes where they are translated into objective service level performance metrics.
QoE requirements shall be considered from a complete end-to-end system perspective. All end-systems (client and
servers), application services and networks (nodes, links) that can contribute to the user experience using a service shall
be taken into account. But also several stakeholders are contributing to a high end-to-end QoE. Those include the
network service provider, the application provider, and the server/client device providers among others.
QoE-oriented engineering includes processes to analyses user requirements, derive measurable parameters, having the
different configuration aspects of the components in the end-to-end service delivery chain, and identify the relationship
between the measurable parameters and the subjective user quality of experience.
Quality of Experience is an important factor in the success of F5G services and is expected to be a key differentiator
with respect to competing service offerings. Subscribers to network services and applications are increasingly sensitive
to how well a service meets their expectations for performance, operability, availability, and ease of use.
4.2 Structure of the present document
User assessment of application and service quality has some subjective aspects, however the present document focuses
on QoE assessment based on measurements. These measurements are made at network and at application level.
F5G High Quality Service Experience Factors
(Clause 4)
Overview
(Clause 5) (Clause 6)
KQI on Application level KQI on Network level
Typical applications (voice, HIS, Customers Premises Network
Web browsing, TV)Access and Aggregation
Cloud Virtual Reality
Network
Mechanisms and Approaches for F5G QoE (Clause 7)
Summary of Requirements and Recommendations (Clause 8)

Figure 4-2: Structure of the present document
The present document is structured as follows:
• Clause 5: Specification of the measurable Key Quality Indicators (KQI) on the application level (typical
applications by referencing the appropriate specifications and Cloud VR as a new application with new sets of
KQIs enabled by F5G).
• Clause 6: Specification of the measurable Key Quality Indicators on the network level in the different
segments of the network.
• Clause 7: Specification of different mechanisms and approaches to either measure or improve QoE. Several
measurement approaches for key performance indicators and QoE assessment methods are specified and the
use of novel concepts like network slicing and AI-based QoE assessment are described.
• Clause 8: Finally, the present document is summarizing the requirements and recommendation for a F5G QoE.
ETSI
10 ETSI GS F5G 005 V1.1.1 (2022-03)
5 Application Services and related QoE factors
5.1 QoE of Typical Applications
5.1.1 General Description
In the present document, the term typical application is used for applications that are well established, which have
standardized quality metrics. These applications include Voice, High-speed Internet, Web browsing, and TV.
The Key Quality Indicators for the typical applications considered are not described in the present document, but
existing specifications are referenced.
5.1.2 QoE Factors of Typical Applications
5.1.2.1 Introduction
QoE factors are described by various standardization organizations. In the following clauses only a few specifications
examples are given in order to refer to existing work in the area of QoE for well-known applications.
5.1.2.2 Voice
A list of voice application QoE factors are described in Recommendation ITU-T P.863 (POLQA) [3] and ETSI
EG 202 057 (Parts 1 to 4) [i.2]. The key performance indicators for voice services are described in ETSI
TS 103 222-1 [4].
5.1.2.3 High-speed Internet
ETSI EG 202 057 (Part 4) [i.2] shows key performance indicators for the High-speed Internet Service and ETSI
TS 103 222-2 [5] the QoS parameters for the High-speed Internet service.
5.1.2.4 Web Browsing
For web browsing application many of the parameters for high-speed internet apply, however, for testing purposes there
is also a reference standard for a web page with reference content as defined in ETSI Kepler Reference Web-page [i.1]
5.1.2.5 TV
For perceived video quality, Recommendation ITU-T J.247 [6] shows a set of parameters. For the user interface of TV
services, IETF RFC 3357 [7] defines One-way Loss Pattern Sample Metrics. Finally, the overall triple play service QoE
requirements are shown in BBF TR-126 [i.3].
5.1.3 Generic Measurement Methodology
The measurement of QoE parameters is typically performed through emulation of the full application with client and
servicer running over a network under test. This requires having the test systems, application client and server, which
are located at different positions in the network under test. The QoE is measured at the client application user interface.
The results of these QoE measurements show a level of QoE in a particular situation. However, it is difficult to
generalize these results, since they depend on the location of the clients and application servers and they depend of the
actual traffic in the network under test.
For other applications, the measurements are performed by emulating an application client, using a real implementation
of the application server. Again that runs over a network under test, but also using an application server under test. For
performing these measurements, the location of the application client and application server matters. Furthermore, the
traffic of other users in the network and/or users using a particular application server matters.
Finally, the content of the application might have an impact on the measurement. For some applications there exists
sample content in order to receive comparable measurement results.
ETSI
11 ETSI GS F5G 005 V1.1.1 (2022-03)
5.1.4 Generic QoE management
QoE management can generically have two approaches:
• The network is dimensioned to ensure that the applications quality requirements are fulfilled. QoE depends on
the network characteristics.
• The applications adapt to the QoS of the network to achieve the best possible QoE. This includes managing the
applications in terms of capacity, response times and other parameters.
This is particularly difficult, since different applications are running on the same network and therefore compete for the
same resources, and the services and network resources to be used are difficult to know in advance. In a few cases the
applications can be modelled and network configuration parameters can be derived. But in many cases, the QoE needs
to be assessed and re-configurations of the network maybe required to improve the QoE.
5.2 Cloud VR
5.2.1 General Description
Cloud VR is a new application enabled by F5G networks and therefore the Key Quality Indicators of Cloud VR are
described in the following clauses, addressing different use cases.
Cloud VR is a new cloud computing technology for VR services, which includes VR video, VR gaming, and VR
industry applications, providing an unprecedented level of immersive experience for users. However, these Cloud VR
services require extremely large bandwidth, low latency, and low packet loss rate, which is a huge challenge for the
network. The large-scale deployment of Cloud VR services requires the joint effort from the industry partners to
address E2E quality management and monitoring. Cloud VR is an ever expanding services area and are for further
study.
Local rendering requires expensive high-performance devices to provide acceptable user experience. Fast and stable
transport networks enable VR content to be stored and rendered in the cloud, and video and audio outputs are coded,
compressed, and transmitted to the user terminals. With Cloud VR, users enjoy VR services without having to purchase
expensive hosts or high-end PCs, promoting VR service popularity. Cloud VR services are further classified as having
strong or weak interaction: ®
• Weak-interaction VR services: Full-view video, VR live broadcast, IMAX theatre
• Strong-interaction VR services: VR games, VR home fitness, VR education, and VR social networking
NOTE: In the following, the focus is on a subset of VR services, but many of the key indicators can be
generalized for other VR services and applications of a similar type.
5.2.2 Factors Affecting Cloud VR service
5.2.2.1 Factors Affecting Weak-Interaction Cloud VR Service Experience
Different transmission solutions have different factors affecting user experience. For the weak-interaction Cloud VR
services, Cloud VR video has two transmission solutions.
a) Cloud VR Full-view Transmission solution
The full-view video transmission solution is widely adopted at the initial stage of service development. In this solution,
the streaming media server transmits all 360-degree video content to the user terminal, which is responsible for tracking
the user head motion as well as decoding and displaying locally cached video data.
Table 5-1: Full-view QoE Indicators
Service Experience Indicator Evaluation Indicator
Initial buffering Initial buffering duration
Full-view Video Average percentage/duration of frame freezing
Frame freezing
Number of freeze frame occurrences
ETSI
12 ETSI GS F5G 005 V1.1.1 (2022-03)
Initial buffering
As with traditional online video, after the user clicks the Cloud VR video play button, there is a loading process for
performing CDN scheduling, index downloading, and data caching. For this process, users generally only see the
loading progress bar. The shorter the loading time, the sooner the user sees the video content and the better the
experience.
Key Quality Indicators
• Initial buffering duration: Is the time from when the user clicks the Cloud VR video play button to when the
user sees the normal play screen.
Frame freezing
During full view VR video playing, if the downloaded data is exhausted by the player and it cannot meet the real-time
playing requirements, the terminal will choose to stop playing first and it will wait until the newly buffered video data
reaches a certain level, then restart playing. The phenomenon of buffering and playing after stopping is called a freeze
frame.
Because it will interrupt the user's viewing process, it has a greater impact on the user's experience. In general, the
lower the number of freeze frames and the shorter their duration, the better the user's experience.
Key Quality Indicators
• Average duration of frames freezing: Is the average of multiple freezing time per time window during VR
video playing.
• Average percentage of frames freezing: Is the ratio of the total freezing time to the total playing time per
time window during VR video playing.
• Number of frames freezing: Is the number of frames freezing per second during VR video playing.
b) Cloud VR FoV Transmission solution
In contrast to the full-view transmission solution, the Field of View (FoV) transmission solution only downloads and
plays the high-definition images within the user viewing angle. Although the FoV transmission solution is far less
demanding on the terminal's decoding performance and network transmission bandwidth, it poses new requirements on
service experience.
Table 5-2: FoV Video Indicators
Service Experience Indicator Evaluation Indicator
Initial buffering Initial buffering duration
Average percentage/duration of frame freezing
FoV Video Frame freezing
Number of freeze frame occurrences
Average percentage of the low quality image area
Low quality image display
Percentage of low quality image duration

For the Initial buffering and Frame freezing of FoV video, the experience indicators and evaluations indicators are the
same defined for full-view service.
Low quality image display
In the Cloud VR use case of ETSI GR F5G 002 [i.4], the VR video source file is divided into multiple segments for
storage in the cloud. Each segment corresponds to a different FoV. Based on the head motion of the user, the terminal
locally calculates the current FoV. The terminal requests the corresponding high-definition segment. The cloud server
responds by sending the requested segment and a low-definition full-view background video. The terminal displays the
high-definition segments when available and fills the remaining portion of the screen with background video.
If these dynamic processes suffer network or application delay, the user will only see low-definition content.
ETSI
13 ETSI GS F5G 005 V1.1.1 (2022-03)
Key Quality Indicators
• Average percentage of the low quality image area: is the average value of the low-definition content in the
user's viewing area during the playing process.
• Percentage of low quality image duration: is the proportion of playing time of low-definition content during
the playing process.
5.2.2.2 Factors Affecting Strong-Interaction Cloud VR Service Experience
With strong-interaction Cloud VR services, users experience real-time interaction with cloud applications through
terminal sensors. After performing calculation, rendering, compression, and encoding on an interaction instruction, the
cloud application servers send response images as video streams to the user's terminal for decoding and display. For
strong-interaction Cloud VR services, Cloud VR gaming is currently the most demanding service and is taken as an
example. Other strong interaction services might have similar key indicators.
Table 5-3: Game Indicators
Service Experience Indicator Evaluation Indicator
Average percentage/duration of frame freezing
Frame freezing
Number of freeze frame occurrences
Average percentage of the black edge area
Game Black edge (Head motion)
Percentage of the black edge duration
Operation response
Average response duration
latency
Frame Freezing
The frame freezing indicators are the same as the video frame freezing indicators.
Black edge (Head motion)
Black edge and smearing: to save cloud rendering resources and shorten E2E latency, Cloud VR gaming servers
generally only render and transmit images within the user's view angle. Therefore, the new viewing areas that are not
rendered on time are displayed as black edges or smearing. The faster the head motion, the longer the cloud rendering
and streaming latency, and the more pronounced the black edge and smearing are.
Key Quality Indicators
• Average percentage of the black edge area: is the average value of the black edge / smearing in the user's
viewing area during the game.
• Percentage of the black edge duration: is the proportion of the time duration with black edge effect to the
total time duration of the game.
Operation response latency
In strong-interaction application scenarios, such as Cloud VR gaming, users expect immediate audio-visual responses
when they move horizontally, pull a trigger, or wave a hand. If the response takes longer than they expect, they
experience interactive latency. Operation response latency is caused by the asynchronous collaboration between the
cloud rendering and streaming process and the local playout process.
Key Quality Indicators
• Average response duration: is the average time from the action of the user detected by the terminal to the
corresponding game screen display.
5.2.3 Measurement Methodology
The key indicators described above are measurable on the end-system of the application. QoE is assessed based on that
application QoS measurements. The QoE assessment uses either some heuristics or some subjective tests to assess the
perceived QoE. The detailed assessment of QoE based on the measured key indicators is for further study.
ETSI
14 ETSI GS F5G 005 V1.1.1 (2022-03)
5.2.4 QoE Management
There are various ways of dealing with degraded key indicators. First, the application can trade certain key indicators
against others. For example, the resolution, which might have a lower impact on QoE than freeze frames or black edges,
can be adapted to the available bandwidth. Details are for further study. Second, the network QoS of such application
session can be improved by the collaboration of the application with the network provider's QoS management system.
Finally, the dimensioning of the Cloud VR slice might need to be adapted to have less frequent QoE impacting events.
6 Factors impacting QoE for Networking Services
6.1 Customer Premises Networks
6.1.1 General Description
The following clauses describe the Quality of Experience aspects and related issues for broadband services. The focus is
only on the on-premises part of the end-to-end F5G network that is an essential factor for user experience. The Quality
of Experience factors related to operator network are described in clause 6.2.
With the F5G Access and Aggregation Network enabling higher speeds and lower latency to the customer premise, the
on-premises network shall be able to cope with new demanding services and higher than usual traffic loading. This
clause assumes a scenario of broadband services, where a worse than expected QoE is affecting customer churn rates,
increased operator's touch-point interactions, decreased customer satisfaction, and decreased brand image.
6.1.2 QoE Factors for CPNs
On-premise networks or Customer Premise Networks (CPN) are based on different technologies, different business
models, and different customer requirements. Therefore, only the key factors impacting QoE are described.

Wi-Fi is one of the most popular solutions used in on-premises networks and therefore factors impacting the QoE that
are related to the Wi-Fi and other CPN technologies are considered.
Factor
...