ETSI TR 126 909 V17.0.0 (2022-05)

Universal Mobile Telecommunications System (UMTS); LTE; 5G; Study on improved streaming Quality of Experience (QoE) reporting in 3GPP services and networks (3GPP TR 26.909 version 17.0.0 Release 17)

RTR/TSGS-0426909vh00

General Information

• Status: Not Published
• Technical Committee: 3GPP SA 4 - Codec
• Current Stage: 12 - Completion
• Completion Date: 06-May-2022

Standards Content (Sample)

ETSI TR 126 909 V17.0.0 (2022-05)






TECHNICAL REPORT
Universal Mobile Telecommunications System (UMTS);
LTE;
5G;
Study on improved streaming Quality of Experience (QoE)
reporting in 3GPP services and networks
(3GPP TR 26.909 version 17.0.0 Release 17)

---------------------- Page: 1 ----------------------
3GPP TR 26.909 version 17.0.0 Release 17 1 ETSI TR 126 909 V17.0.0 (2022-05)

Reference
RTR/TSGS-0426909vh00
Keywords
5G,LTE,UMTS
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: [phone]  Fax: [phone]

Siret N° 348 623 562 00017 - APE 7112B
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° w061004871

Important notice
The present document can be downloaded from:
http://www.etsi.org/standards-search
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the prevailing version of an ETSI
deliverable is the one made publicly available in PDF format at www.etsi.org/deliver.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
https://portal.etsi.org/TB/ETSIDeliverableStatus.aspx
If you find errors in the present document, please send your comment to one of the following services:
https://portal.etsi.org/People/CommiteeSupportStaff.aspx
If you find a security vulnerability in the present document, please report it through our
Coordinated Vulnerability Disclosure Program:
https://www.etsi.org/standards/coordinated-vulnerability-disclosure
Notice of disclaimer & limitation of liability
The information provided in the present deliverable is directed solely to professionals who have the appropriate degree of
experience to understand and interpret its content in accordance with generally accepted engineering or
other professional standard and applicable regulations.
No recommendation as to products and services or vendors is made or should be implied.
No representation or warranty is made that this deliverable is technically accurate or sufficient or conforms to any law
rule and/or regulation and further, no representation or warranty is made of merchantability or fitness
and/or governmental
for any particular purpose or against infringement of intellectual property rights.
In no event shall ETSI be held liable for loss of profits or any other incidental or consequential damages.

Any software contained in this deliverable is provided "AS IS" with no warranties, express or implied, including but not
limited to, the warranties of merchantability, fitness for a particular purpose and non-infringement of intellectual property
rights and ETSI shall not be held liable in any event for any damages whatsoever (including, without limitation, damages
for loss of profits, business interruption, loss of information, or any other pecuniary loss) arising out of or related to the use
of or inability to use the software.
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and
microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.

© ETSI 2022.
All rights reserved.

ETSI

---------------------- Page: 2 ----------------------
3GPP TR 26.909 version 17.0.0 Release 17 2 ETSI TR 126 909 V17.0.0 (2022-05)
Intellectual Property Rights
Essential patents
IPRs essential or potentially essential to normative deliverables may have been declared to ETSI. The declarations
pertaining to these essential IPRs, if any, are publicly available for ETSI members and non-members, and can be
found in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to
ETSI in respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the
ETSI Web server (https://ipr.etsi.org/).
Pursuant to the ETSI Directives including the ETSI IPR Policy, no investigation regarding the essentiality of IPRs,
including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not
referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become,
essential to the present document.
Trademarks
The present document may include trademarks and/or tradenames which are asserted and/or registered by their owners.
ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no
right to use or reproduce any trademark and/or tradename. Mention of those trademarks in the present document does
not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks.
DECT™, PLUGTESTS™, UMTS™ and the ETSI logo are trademarks of ETSI registered for the benefit of its

Members. 3GPP™ and LTE™ are trademarks of ETSI registered for the benefit of its Members and of the 3GPP
Organizational Partners. oneM2M™ logo is a trademark of ETSI registered for the benefit of its Members and of the
®
oneM2M Partners. GSM and the GSM logo are trademarks registered and owned by the GSM Association.
Legal Notice
This Technical Report (TR) has been produced by ETSI 3rd Generation Partnership Project (3GPP).
The present document may refer to technical specifications or reports using their 3GPP identities. These shall be
interpreted as being references to the corresponding ETSI deliverables.
The cross reference between 3GPP and ETSI identities can be found under http://webapp.etsi.org/key/queryform.asp.
Modal verbs terminology
In the present document "should", "should not", "may", "need not", "will", "will not", "can" and "cannot" are to be
interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.
ETSI

---------------------- Page: 3 ----------------------
3GPP TR 26.909 version 17.0.0 Release 17 3 ETSI TR 126 909 V17.0.0 (2022-05)
Contents
Intellectual Property Rights . 2
Legal Notice . 2
Modal verbs terminology . 2
Foreword . 4
1 Scope . 5
2 References . 5
3 Abbreviations . 5
4 A/V Quality Monitoring Support for 3GPP PSS . 6
4.1 WebTV Quality Monitoring . 6
4.1.1 Use case . 6
4.1.2 Potential Recommended requirements. 7
4.2 ITU-T P.NATS Quality Assessment Model for HTTP Adaptive Streaming . 9
4.3 Gap analysis of PSS QoE metrics for support of ITU-T P.NATS . 11
4.3.1 Supported Mode . 11
4.3.2 Supported Input parameter . 11
4.3.3 Further Parameters for Supporting Video Quality Monitoring . 15
4.3.3.1 Additional Input Parameters. 15
4.3.3.2 Dependency of Video Quality Test Results on Visual Size . 15
4.3.3.3 Relation between Video Quality MOS and QP Parameter . 16
4.4 Calculation of A/V MOS estimation . 17
4.4.0 General . 17
4.4.1 Network Optimization . 17
4.4.2 MOS Models . 17
4.4.3 MOS Windowing . 17
4.4.4 Conclusion . 18
4.5 Options for applying higher P.NATS Modes . 18
5 QoE metrics support for managed streaming service . 20
5.1 Use case: Managed Streaming Service QoE Improvement . 20
5.1.1 Introduction. 20
5.1.2 Use case #1 . 20
5.1.3 Use case #2 . 21
5.2 Managed streaming service deployment model consideration . 21
5.2.1 Introduction. 21
5.2.2 Managed streaming service deployment model . 22
5.3 Recommended requirements . 23
5.4 GAP Analysis and Evaluation . 23
5.4.1 Introduction. 23
5.4.2 Analysis of Activation/Deactivation of QoE reporting . 23
5.5 Assumptions . 24
5.6 Solution . 24
5.6.1 Possible Candidate Options . 24
5.6.2 QoE metrics collection within a certain geographic area . 26
5.6.3 QoE metrics collection of a streaming service of a specific 3rd party . 26
5.7 QoE Handler API Considerations . 26
5.8 Privacy issue on QoE metric collection . 28
5.8.1 Analysis . 28
5.8.2 Solutions . 28
6 Conclusion . 28
Annex A: Change history . 30
History . 31

ETSI

---------------------- Page: 4 ----------------------
3GPP TR 26.909 version 17.0.0 Release 17 4 ETSI TR 126 909 V17.0.0 (2022-05)
Foreword
rd
This Technical Report has been produced by the 3 Generation Partnership Project (3GPP).
The contents of the present document are subject to continuing work within the TSG and may change following formal
TSG approval. Should the TSG modify the contents of the present document, it will be re-released by the TSG with an
identifying change of release date and an increase in version number as follows:
Version x.y.z
where:
x the first digit:
1 presented to TSG for information;
2 presented to TSG for approval;
3 or greater indicates TSG approved document under change control.
y the second digit is incremented for all changes of substance, i.e. technical enhancements, corrections,
updates, etc.
z the third digit is incremented when editorial only changes have been incorporated in the document.
ETSI

---------------------- Page: 5 ----------------------
3GPP TR 26.909 version 17.0.0 Release 17 5 ETSI TR 126 909 V17.0.0 (2022-05)
1 Scope
The present document describes the enhancement of Quality of Experience (QoE) for operator managed streaming
rd
service, 3 party managed streaming service and Over-The-Top (OTT) streaming service.
2 References
The following documents contain provisions which, through reference in this text, constitute provisions of the present
document.
- References are either specific (identified by date of publication, edition number, version number, etc.) or
non-specific.
- For a specific reference, subsequent revisions do not apply.
- For a non-specific reference, the latest version applies. In the case of a reference to a 3GPP document (including
a GSM document), a non-specific reference implicitly refers to the latest version of that document in the same
Release as the present document.
[1] 3GPP TR 21.905: "Vocabulary for 3GPP Specifications".
[2] Yiting Liao, et al. "Achieving high QoE across the compute continuum: How compression,
content, and devices interact." 7th International Workshop on Video Processing and Quality
Metrics for Consumer Electronics, Scottsdale, Arizona, USA. 2013.
[3] ITU-T SG12 "T13-SG12-150505-TD-GEN-0671!R1!MSW-E-P.NATS Terms of Reference
(ToR)".
[4] 3GPP TS 26.247: "Transparent end-to-end Packet-switched Streaming Service (PSS); Progressive
Download and Dynamic Adaptive Streaming over HTTP (3GP-DASH)".
[5] 3GPP TS 26.233: "Transparent end-to-end packet switched streaming service (PSS); General
description".
[6] 3GPP TR 26.938: "Packet-switched Streaming Service (PSS); Improved support for dynamic
adaptive streaming over HTTP in 3GPP".
[7] 3GPP TS 26.244: "Transparent end-to-end packet switched streaming service (PSS); 3GPP file
format (3GP)".
[8] Recommendation ITU-R BT.500: "Methodology for the subjective assessment of the quality of
television pictures".
[9] DASH-IF Position Paper: "Proposed QoE Media Metrics standardization for segmented media
playback": http://dashif.org/wp-
content/uploads/2016/10/ProposedMediaMetricsforSegmentedMediaDelivery-r12.pdf
[10] ISO/IEC 23001-10: "Information technology -- MPEG systems technologies -- Part 10: Carriage
of timed metadata metrics of media in ISO base media file format".
[11] 3GPP TS 32.422: "Telecommunication management; Subscriber and equipment trace; Trace
control and configuration management".
3 Abbreviations
For the purposes of the present document, the abbreviations given in 3GPP TR 21.905 [1] and the following apply.
An abbreviation defined in the present document takes precedence over the definition of the same abbreviation, if any,
in 3GPP TR 21.905 [1].
A/V MOS Audio/Video Mean Opinion Score
ETSI

---------------------- Page: 6 ----------------------
3GPP TR 26.909 version 17.0.0 Release 17 6 ETSI TR 126 909 V17.0.0 (2022-05)
IQoE Improved Quality of Experience
DASH Dynamic Adaptive Streaming over HTTP
KPI Key Performance Indicators
KQI Key Quality Indicator
MDT Minimization of Drive Tests
MNO Mobile Network Operator
MOS Mean Opinion Score
MOS-AVQO Audio-Visual-Quality-Objective
MPD Media Presentation Description
OTT Over The Top
PSS Packet Streaming Server
TCE Trace Collection Entity
VSSQM Video Streaming Service Quality Monitoring
4 A/V Quality Monitoring Support for 3GPP PSS
4.1 WebTV Quality Monitoring
4.1.1 Use case
Over-The-Top video streaming is increasingly dominating the traffic in the networks. An increasing number of services
employing a variety of streaming formats are appearing. In addition, movie services are increasingly causing traffic in
mobile networks.
It is crucial for mobile network operators to manage the video traffic in their networks and services in an optimal
manner. A major objective is to ensure that the customers remain satisfied. Hence monitoring the users QoE with an
appropriate quality indicator is of fundamental importance.
The derivation of quality indicators for the streaming quality may be supported by the service to a lesser or greater
extend. Further dedicated agreements between mobile operator and service provider would help to obtain better quality
indicators. The level of accuracy on the quality indicator would depend on the level of agreement between the mobile
network operator and the service provider.
The main Key Performance Indicators (KPIs) for characterizing insufficient video streaming performance as perceived
by the user are:
- initial stalling of the playout;
- periods of stalling and freezing of the video while playing;
- interruption of the audio while playing;
- low coding quality appearing as blurring, macroblocking or mosquito artefacts;
- varying coding quality while playing.
Beside these KPIs the monitoring system will also provide a Key Quality Indicator (KQI) characterizing the user's
quality experience.
Quality is fundamentally related to the subjective assessment of the considered aspect. The KQI will be related to
corresponding subjective quality assessments. The quality often is rated as an opinion score on a 5-point scale ranging
from "bad" (1), "poor" (2), "fair" (3), "good" (4) to "excellent" (5). The average of these scores calculated from a group
of subjects is the Mean Opinion Score (MOS).
For operational tasks instrumental measurement tools are required. Hence, the subjective test results are used to develop
instrumental methods that replicates the MOS scores obtained in subjective tests. Ideally, the derived quality model will
be established as internationally agreed standard specification, for increasing the confidence that the measured quality
results are reliable and comparable. The estimated MOS values will be labelled as result stemming from an objective
model. For example, the estimated AV-streaming MOS could be named MOS-AVQO (Audio-Visual-Quality-
Objective).
ETSI

---------------------- Page: 7 ----------------------
3GPP TR 26.909 version 17.0.0 Release 17 7 ETSI TR 126 909 V17.0.0 (2022-05)
The use of a KQI, estimating the MOS of a video streaming session has the major advantage of combining the quality
impact of occurring stalling and coding effects in a single number. By this means the service can be monitored
efficiently.
In addition to the KPIs and KQI, further the related data such as time, location, service and network will be collected for
each considered streaming session.
The use case "WebTV Quality Monitoring" demands for means that allow collecting accurate KPIs and the audio-visual
KQI for supporting the quality management of the networks and customer services. The collected KPI and KQI data
supports a wide range of utilizations such as the short-term detection of problems and the long term quality monitoring.
With additional other monitoring data sources, the complete delivery chain, from server, via networks, to the user's
terminal can be observed. Data analytics methods can be applied to reveal inherent quality dependencies. The detection
of critical combinations of KPIs with the help of a KQI focuses on the user's quality experience and is therefore very
efficient. In conclusion, the MOS-AVQO is foreseen as mean to increase the usefulness of the quality monitoring
systems significantly.
4.1.2 Potential Recommended requirements
The proposed potential requirements for implementing the WebTV Quality Monitoring use case are based on the
assumption that the user's end device is collecting and aggregating the QoE-related data. The collected QoE-related data
then is further processed and forwarded to a central QoE monitoring server in a next step:
- The monitoring software of the end device can provide a generic applicable subsystem for Video Streaming
Service Quality Monitoring (VSSQM).
- The VSSQM subsystem provides an API for logging QoE related events. The API provides a handler for
receiving and digesting QoE related events and for retrieval of QoE-related data to be reported to a QoE Server
using an appropriate communication channel.
- VSSQM subsystem provides an API for third party QoE monitoring applications running on the end device. This
API allows retrieving the aggregated QoE-related data. The QoE monitoring application may transmit the QoE-
related data to a third party QoE Monitoring Service.
Figure 4.1-1 illustrates the Video Streaming Service Quality Monitoring subsystem with the provided API's for QoE-
related event logging and for QoE data digestion.

Figure 4.1-1: Proposed subsystem for logging and digesting QoE-related data for Video Streaming
Service Quality Monitoring
Detailed proposed requirements for the VSSQM Logging API
1. The VSSQM subsystems provides an interface for registering video streaming service and its player for
monitoring the play-out performance
2. The player of the streaming service is recommended to support the sending of following events:
ETSI

---------------------- Page: 8 ----------------------
3GPP TR 26.909 version 17.0.0 Release 17 8 ETSI TR 126 909 V17.0.0 (2022-05)
- AnouncementOfVideoStreamingSession (ServiceProvider)
- InitialisingPlayer (AudioCodec, VideoCodec, Profile)
- ChangeProfile (Profile)
- LoadingContent (SourceIP)
- Playing
- BitrateReport(Audio,Video)
- Stalling
- Pause
- ScrollForward
- ScrollBackward
3. The mobile clients provides additional related information such as:
- Time
- Location
- Network: 2G, 3G, 4G, Cell-ID, WLAN
- Route: Source IP, destination IP, …
- Device: brand, screen parameters, headphones, processing power, battery usage
The VSSQM subsystem would collect this information via the appropriate API's in that moment when the player event
is processed.
4. The QoE Logging API supports the retrieval of the raw QoE data with different aggregation levels. A QoE
data retrieval module forwards these reports to a QoE server using an appropriate communication channel.
5. Video streaming traffic may be monitored also with a traffic analysis tool operating on the client network
interface. Such a tool would register at the VSSQM in the same manner as streaming services. The tool may
help to observe video streams that do not register and provide QoE related monitoring events.
Detailed proposed requirements for the QoE Monitoring API
6. The QoE Monitoring API allows deriving aggregated QoE reports. A third party monitoring application can
process the data and forward derived QoE reports to a third party QoE server.
7. The QoE Monitoring Application may support to derive the following metrics:
- Initial stalling of the playout
- Periods of stalling and freezing of the video while playing
- Interruption of the audio while playing
- Statistics on coding quality
- Statistics on varying coding quality while playing
- Estimation on overall audio-visual quality (MOS-AVQO)

ETSI

---------------------- Page: 9 ----------------------
3GPP TR 26.909 version 17.0.0 Release 17 9 ETSI TR 126 909 V17.0.0 (2022-05)
4.2 ITU-T P.NATS Quality Assessment Model for HTTP
Adaptive Streaming
ITU-T P.NATS project ([3]) will develop the objective assessment model for progressive download and adaptive type
media streaming. It supports both the OTT and operator managed video service. The supported protocol scope includes
HTTP/TCP/IP, RTMP/TCP/IP, HLS/HTTP/TCP/IP, and DASH/HTTP/TCP/IP. It supports 3GPP, MP4 and other file
format, and the model is agnostic to the type of file format.
It will support sequence duration of 60 sec to 5 min for quality evaluation. The supported video resolution is 240p,
360p, 480p, 720p and 1080p. The supported frame rate range is 8 to 50 fps.
ITU-T P.NATS phase 2 aims at extending the quality model for supporting 2K and 4K.
The current working model agreed in P.NATS project is depicted in figure 4.2-1.

Figure 4.2-1: Building blocks of the P.NATS model
As shown in table 4-1, P.NATS will support 4 modes.
Table 4-1: Different modes defined in P.NATS
Mode Encryption Input
0 Encrypted media payload Meta-data
and media frame headers
1 Encrypted media payload Meta-data and frame header information
2 No encryption Meta-data and up-to 2% of the media
stream
3 No encryption Meta-data and any information from the
video stream

The P.NATS model will receive media information and prior knowledge about the media stream or streams. The model
receives the following input signals regardless of the mode of operation:
- I.GEN: display resolution and device type
- I.11: audio coding information
- I.13: video coding information
ETSI

---------------------- Page: 10 ----------------------
3GPP TR 26.909 version 17.0.0 Release 17 10 ETSI TR 126 909 V17.0.0 (2022-05)
- I.14: Stalling events
The P.NATS model input parameters are provided in table 4-2 below.
Table 4-2: ITU-T P.NATS model input parameters.
ID Description Values Frequency Modes
available
I.GEN
0 The resolution of Number of pixels (WxH) in displayed video Per media session All
the image
displayed to the
user
1 The device type on pc or mobile Per media session All
which the media is
played
I.11
7 Target Audio bit- Bit-rate in kbps. Per media segment All
rate
8 Segment duration Duration in seconds Per media segment All
9 Audio frame Integer, starting with 1 Per media segment 1,2,3
number
10 Audio frame size Size of the frame in bytes Per audio frame 1,2,3
11 Audio frame Duration in seconds Per audio frame 1,2,3
duration
12 Audio codec One of: AAC-LC, AAC-HEv1, AAC-HEv2, AC3 Per media segment All
13 Audio sampling In Hz Per media segment All
frequency
14 Number of audio 2 Per media segment All
channels
15 Audio bit-stream Encoded audio bytes for the frame Per audio frame 2,3
I.13
16 Target Video bit- Bit-rate in kbps. Per media segment All
rate
13 Video frame-rate Frame rate in frames per second. Per media segment All
14 Segment duration Duration in seconds Per media segment All
15 Video encoding Number of pixels (WxH) in transmitted video Per media segment All
resolution
16 Video codec and One of: H264-baseline, H264-high, H264-main Per media segment All
profile
17 Video frame Integer, starting at 1, denoting the frame sequence Per video frame 1,2,3
number number in encoding order.
18 Video frame Duration of the frame in seconds Per video frame 1,2,3
duration
19 Frame presentation The frame presentation timestamp Per video frame 1,2,3
timestamp
20 Frame decoding The frame decoding timestamp Per video frame 1,2,3
timestamp
21 Video frame size
...