ETSI TR 103 702 V1.1.1 (2020-11)

Speech and multimedia Transmission Quality (STQ); QoS parameters and test scenarios for assessing network capabilities in 5G performance measurements

DTR/STQ-00225m

General Information

• Status: Not Published
• Technical Committee: STQ MOBILE - STQ Mobile
• Current Stage: 12 - Completion
• Due Date: 19-Nov-2020
• Completion Date: 23-Nov-2020

Standards Content (Sample)

ETSI TR 103 702 V1.1.1 (2020-11)






TECHNICAL REPORT
Speech and multimedia Transmission Quality (STQ);
QoS parameters and test scenarios for assessing network
capabilities in 5G performance measurements

---------------------- Page: 1 ----------------------
2 ETSI TR 103 702 V1.1.1 (2020-11)



Reference
DTR/STQ-00225m
Keywords
5G, data, LTE, LTE-Advanced, measurement,
performance, QoE, QoS, service, test
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: [phone]  Fax: [phone]

Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88

Important notice
The present document can be downloaded from:
http://www.etsi.org/standards-search
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the prevailing version of an ETSI
deliverable is the one made publicly available in PDF format at www.etsi.org/deliver.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
https://portal.etsi.org/TB/ETSIDeliverableStatus.aspx
If you find errors in the present document, please send your comment to one of the following services:
https://portal.etsi.org/People/CommiteeSupportStaff.aspx
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying
and microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.

© ETSI 2020.
All rights reserved.

DECT™, PLUGTESTS™, UMTS™ and the ETSI logo are trademarks of ETSI registered for the benefit of its Members.

3GPP™ and LTE™ are trademarks of ETSI registered for the benefit of its Members and
of the 3GPP Organizational Partners.
oneM2M™ logo is a trademark of ETSI registered for the benefit of its Members and
of the oneM2M Partners.
®
GSM and the GSM logo are trademarks registered and owned by the GSM Association.
ETSI

---------------------- Page: 2 ----------------------
3 ETSI TR 103 702 V1.1.1 (2020-11)
Contents
Intellectual Property Rights . 5
Foreword . 5
Modal verbs terminology . 5
Introduction . 5
1 Scope . 7
2 References . 7
2.1 Normative references . 7
2.2 Informative references . 8
3 Definition of terms, symbols and abbreviations . 9
3.1 Terms . 9
3.2 Symbols . 9
3.3 Abbreviations . 9
4 5G Performance measurement criteria . 10
4.1 Overview . 10
4.2 Traffic Scenarios . 11
4.2.1 Overview . 11
4.2.2 Key capability parameters . 11
4.2.3 High data rates and traffic densities . 11
4.2.4 High data rate and low latency. 12
4.3 Service usage scenarios . 13
4.3.1 Overview . 13
4.3.2 Key service usage scenario performance parameters . 13
4.3.3 Existing services & applications . 14
4.3.4 5G enabled services, applications & technologies . 15
4.3.4.1 Overview . 15
4.3.4.2 Enhanced video streaming . 15
4.3.4.3 Enhanced Video Conferencing. 16
4.3.4.4 Messaging & Visual communication . 16
4.3.4.5 Virtual Reality . 16
4.3.4.6 Cloud gaming . 18
4.3.4.7 Augmented Reality. 19
4.4 User type scenarios . 19
5 QoS Parameters . 20
5.1 Technical QoS parameters . 20
5.1.1 Overview . 20
5.1.2 Coverage . 20
5.1.3 Transport . 21
5.1.3.1 Overview . 21
5.1.3.2 Real time considerations . 21
5.1.3.3 Latency & Interactivity . 21
5.1.3.3.1 Overview . 21
5.1.3.3.2 Per packet two-way latency . 21
5.1.3.3.3 Packet delay variation . 22
5.1.3.3.4 Disqualified packets . 22
5.1.3.3.5 Interactivity . 22
5.1.3.4 User perceived peak throughput & data rates. 23
5.2 Service QoS parameters . 24
5.2.1 Overview . 24
5.2.2 Telephony service . 24
5.2.3 Data service QoS parameters . 24
5.2.3.1 Overview . 24
5.2.3.2 Web browsing QoS and HTTPs . 24
5.2.4 Enhanced UHD video QoS . 25
ETSI

---------------------- Page: 3 ----------------------
4 ETSI TR 103 702 V1.1.1 (2020-11)
5.2.5 Virtual Reality QoS . 25
5.2.6 Cloud gaming QoS . 27
6 Test scenarios . 28
6.1 Overview . 28
6.2 Executing network test scenario . 28
6.2.1 Overview . 28
6.2.2 Measuring maximum user perceived throughput and data rates . 28
6.2.3 The TWAMP method to obtain two way latency . 29
6.3 Executing service test scenarios . 30
6.3.1 Overview . 30
6.3.2 Scenario identification . 31
6.3.2.1 Overview . 31
6.3.2.2 Testing methods . 31
6.3.2.2.1 Overview . 31
6.3.2.2.2 Guidelines for testing with real applications . 31
6.3.2.2.3 Guidelines to derive traffic patterns for the emulation of real applications . 31
6.3.2.3 Classification of measurement environment . 32
6.3.3 Impact of 5G features and application intelligence . 32
6.3.4 Test scenarios . 32
6.3.4.1 Overview . 32
6.3.4.2 Telephony testing . 32
6.3.4.3 Video streaming testing . 32
6.3.4.4 Virtual Reality testing . 33
6.3.4.5 Cloud gaming testing . 33
7 Summary . 34
Annex A: Performance recommendations . 35
A.1 Traffic scenario criteria . 35
A.1.1 High data rates and traffic density performance criteria . 35
A.1.2 High data rate and low latency performance criteria . 36
A.2 Service Scenario Criteria . 36
A.2.1 UHD Video performance scenario . 36
A.2.2 Virtual Reality performance scenarios . 37
A.2.3 Cloud gaming performance criteria . 38
A.2.4 Augmented Reality performance criteria . 38
Annex B: Emulation & Interactivity example . 39
B.1 Definition of test cases . 39
B.2 Application emulation and interactivity model parameters . 39
B.2.1 A generic interactivity model approach . 39
B.2.2 Example high-interactive 'e-Gaming real-time' . 40
B.2.3 Example 'Interactive remote meeting' and 'Video chat HD' . 42
History . 45


ETSI

---------------------- Page: 4 ----------------------
5 ETSI TR 103 702 V1.1.1 (2020-11)
Intellectual Property Rights
Essential patents
IPRs essential or potentially essential to normative deliverables may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (https://ipr.etsi.org/).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Trademarks
The present document may include trademarks and/or tradenames which are asserted and/or registered by their owners.
ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no
right to use or reproduce any trademark and/or tradename. Mention of those trademarks in the present document does
not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks.
Foreword
This Technical Report (TR) has been produced by ETSI Technical Committee Speech and multimedia Transmission
Quality (STQ).
Modal verbs terminology
In the present document "should", "should not", "may", "need not", "will", "will not", "can" and "cannot" are to be
interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.
Introduction
As the industry deploys 5G networks and launches 5G plans and services, it is acknowledged that it will facilitate
improved service experience and the enablement of new business and services. This will include a variety of services,
applications and use cases ranging from those requiring high data rates through enhanced mobile broadband (eMBB) to
those requiring ultra-reliable low latency (uRLLC) as well as those supporting massive machine type communication
(mMTC). Many existing applications and services such as voice, data and video will continue to be widely used with
the expectation that they will benefit through superior quality, reduced access times and greater reliability. In addition,
new use cases, applications and service scenarios, which are facilitated by 5G, will have specific performance
requirements that require measurement and evaluation. As operators develop 5G service strategies, establish network
requirements and develop networks to meet those requirements, it is important to be able to quantify and qualify the
capabilities of the network. To achieve this, it is necessary to examine what QoS parameters should be measured to
quantify a networks capability and how the resultant service or application QoS will be assessed. To that end, the
purpose of the present document is to identify those QoS parameters and the test scenarios that can evaluate 5G
performance and measure the network capability and readiness.
At the current stage of network development, with the focus on data rates, the present document will focus on eMBB
and the use cases and services it enables such as ultra-high definition video or virtual reality as primary examples. It
should be noted that while the performance requirements will evolve during the lifecycle of the 5G network to meet
new use cases and customer expectations, the QoS parameters and test scenarios will continue to provide a means to
measure the network capabilities to meet these requirements.
ETSI

---------------------- Page: 5 ----------------------
6 ETSI TR 103 702 V1.1.1 (2020-11)
There are many aspects of advanced technology and 5G features to consider when examining the impact on quality of
service including MEC, where interactivity requirements influence deployment strategies, to network slicing, where
context aware intelligence directs traffic according to application requirements, to radio features such as beam forming
and massive MIMO, which provide intelligent management of the air interface and many others. Given the complexity
of 5G features, the aim of the present document is to focus on the end to end network capability with reference to 5G
feature considerations where required. In this regard, it is necessary to examine performance measurement scenarios
[i.2], to determine the network technical parameters and consider the impact on end user service experience [i.6]. In
addition, to support QoS parameter measurement, the test scenarios that measure the network capability will be
described.

ETSI

---------------------- Page: 6 ----------------------
7 ETSI TR 103 702 V1.1.1 (2020-11)
1 Scope
Given the current stage of 5G deployments and the focus on eMBB, the present document, will primarily concentrate on
QoS parameters in relation to eMBB performance scenarios and the most prevalent eMBB related service scenarios.
Therefore, the focus will be on QoS parameters that reflect the network capabilities in the case of visual applications
such as UHD video and Virtual and Augmented Reality. However, whilst these technologies, are primarily categorized
as requiring high data rates, it is necessary to be mindful and examine the relationship and requirements in high data
rates and low latency scenarios.
In the scope of this analysis, the term QoS relies on service-related characteristics without knowing any details about
the underlying network sections [i.6], the network architecture and the network or application deployment strategies.
The scope will concentrate on measuring the network capability, assessed primarily through network QoS parameters,
such as data rates, capacity, coverage, latency and continuity measurements. The readiness of the network to support the
QoS needs of existing services and applications such as voice, data and video and those newly enabled technologies and
use cases mentioned earlier that benefit from the higher data rates associated with eMBB will also be considered.
The approach therefore will be to assess network capabilities by first identifying the performance scenario
requirements, then discovering the QoS parameters that will measure those requirements and finally defining the test
scenarios to measure those QoS parameters as follows:
1) Identify scenarios in terms of performance, service and user types to determine the performance measurement
requirements and the key performance factors that will satisfy those requirements:
- Performance scenarios [i.2] which are dependent on traffic types, traffic densities and service areas.
- Service scenarios that consider the use cases, technology and applications that place data service
requirements on the network for effective operation.
- User type scenarios that examine various types of users and how they place different requirements on
similar services.
2) QoS parameter discovery to identify and define the parameters that represent the key performance factors and
scenario requirements. The QoS parameters will define how to effectively measure the network technical
performance as well as examining how a use case or application might be affected by those network
conditions. The QoS parameters in as much as possible will refer to existing definitions and best practises.
3) Provide test scenario analysis to detail the types of tests to be executed to verify the network capability. Define
how to represent the measurement scenarios and where to collect data to calculate the QoS parameters. The
test scenarios will reproduce typical service activities to derive quality measures and will identify the
measurement points and the expected data sources.
The aim therefore is to identify the QoS parameters of interest, based on the identified scenarios, referencing existing
specifications and technical reports where available. There are already significant relevant references available from a
number of bodies to identify QoS aspects for a number 5G scenarios, which are at various stages of maturity. This
includes analysis of primary use case scenarios, identification of performance measurement scenarios and definitions of
quality measurement indicators. The expectation is that, the present document, through its analysis will put in place a
means to assess 5G network capabilities and readiness of the network to support those aforementioned prevalent eMBB
applications and use cases.
2 References
2.1 Normative references
Normative references are not applicable in the present document.
ETSI

---------------------- Page: 7 ----------------------
8 ETSI TR 103 702 V1.1.1 (2020-11)
2.2 Informative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] ETSI TS 102 250-2: "Speech and multimedia Transmission Quality (STQ); QoS aspects for
popular services in mobile networks; Part 2: Definition of Quality of Service parameters and their
computation".
[i.2] ETSI TS 122 261: "5G; Service requirements for the 5G system (3GPP TS 22.261)".
[i.3] ETSI TR 101 578: "Speech and multimedia Transmission Quality (STQ); QoS aspects of TCP-
based video services like YouTube™".
[i.4] ETSI TR 126 918: "Universal Mobile Telecommunications System (UMTS); LTE; Virtual Reality
(VR) media services over 3GPP (3GPP TR 26.918)".
[i.5] ETSI TR 126 929: "5G; QoE parameters and metrics relevant to the Virtual Reality (VR) user
experience (3GPP TR 26.929)".
[i.6] ETSI TS 102 250-1: "Speech and multimedia Transmission Quality (STQ); QoS aspects for
popular services in mobile networks; Part 1: Assessment of Quality of Service".
[i.7] Recommendation ITU-T G.QOE-VR: "Influencing Factors on Quality of Experience (QoE) for
Virtual Reality Services".
[i.8] Recommendation ITU-R M-2083-0: "IMT Vision - Framework and overall objectives of the future
development of IMT for 2020 and beyond".
[i.9] ETSI TS 103 222-2: "Speech and multimedia Transmission Quality (STQ); Reference
benchmarking, background traffic profiles and KPIs; Part 2: Reference benchmarking and KPIs for
High speed internet".
[i.10] ETSI TS 102 250-3: "Speech and multimedia Transmission Quality (STQ); QoS aspects for
popular services in mobile networks; Part 3: Typical Procedures for Quality of Service
measurement equipment".
[i.11] ETSI TS 102 250-5: "Speech and multimedia Transmission Quality (STQ); QoS aspects for
popular services in mobile networks; Part 5: Definition of typical measurement profiles".
[i.12] ETSI TR 103 468: "Speech and multimedia Transmission Quality (STQ); Quality of Service
aspects for 5G; Discussion of QoS aspects of services related to the 5G ecosystem".
[i.13] "5G Service Experience-Based Network Planning Criteria", Ovum in partnership with Huawei,
©
Copyright Ovum 2019 .
[i.14] 3GPP TS 26.186: "Enhancement of 3GPP support for V2X scenarios; Stage 1; Release 16".
[i.15] 3GPP TR 29.893: "Study on IETF QUIC Transport for 5GC Service Based Interfaces
(Release 16)".
[i.16] Recommendation ITU-T Y.1540: "Internet protocol data communication service - IP packet
transfer and availability performance parameters".
[i.17] IETF RFC 5357: "A Two-Way Active Measurement Protocol (TWAMP)".
[i.18] Recommendation ITU-T P.1204: "Video quality assessment of streaming services over reliable
transport for resolutions up to 4K".
ETSI

---------------------- Page: 8 ----------------------
9 ETSI TR 103 702 V1.1.1 (2020-11)
[i.19] ETSI
...