ETSI TR 101 578 V1.2.1 (2015-07)

Speech and multimedia Transmission Quality (STQ); QoS aspects of TCP-based video services like YouTubeTM

RTR/STQ-00201m

General Information

• Status: Published
• Publication Date: 27-Jul-2015
• Technical Committee: STQ MOBILE - STQ Mobile
• Current Stage: 12 - Completion
• Due Date: 28-Jul-2015
• Completion Date: 28-Jul-2015

Standards Content (Sample)

ETSI TR 101 578 V1.2.1 (2015-07)






TECHNICAL REPORT
Speech and multimedia Transmission Quality (STQ);
QoS aspects of TCP-based video services like YouTube™

---------------------- Page: 1 ----------------------
2 ETSI TR 101 578 V1.2.1 (2015-07)



Reference
RTR/STQ-00201m
Keywords
measurement, QoS, service, TCP-based video
services
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: [phone]  Fax: [phone]

Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88

Important notice
The present document can be downloaded from:
http://www.etsi.org/standards-search
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the only prevailing document is the
print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
http://portal.etsi.org/tb/status/status.asp
If you find errors in the present document, please send your comment to one of the following services:
https://portal.etsi.org/People/CommiteeSupportStaff.aspx
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying
and microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.

© European Telecommunications Standards Institute 2015.
All rights reserved.

TM TM TM
DECT , PLUGTESTS , UMTS and the ETSI logo are Trade Marks of ETSI registered for the benefit of its Members.
TM
3GPP and LTE™ are Trade Marks of ETSI registered for the benefit of its Members and
of the 3GPP Organizational Partners.
GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.
ETSI

---------------------- Page: 2 ----------------------
3 ETSI TR 101 578 V1.2.1 (2015-07)
Contents
Intellectual Property Rights . 7
Foreword . 7
Modal verbs terminology . 7
Introduction . 7
1 Scope . 9
2 References . 9
2.1 Normative references . 9
2.2 Informative references . 9
3 Abbreviations . 9
4 Quality of Service measurements for TCP-based video services like YouTube™ . 10
4.0 General . 10
4.1 Phases of TCP-based video services . 10
4.2 QoS aspects of TCP-based video services . 12
4.2.0 Scope of aspects . 12
4.2.1 Video freezes and - skips . 12
4.2.2 Downloading with DASH . 12
4.3 QoS parameters for TCP-based video services . 12
4.3.0 Paramater and trigger points . 12
4.3.1 Player IP Service Access Failure Ratio [%] . 15
4.3.2 Player IP Service Access Time [s] . 16
4.3.3 Player Download Cut-off Ratio [%] . 16
4.3.4 Player Download Time [s] . 16
4.3.5 Player Session Failure Ratio [%] . 16
4.3.6 Player Session Time [s] . 16
4.3.7 Video IP Service Access Failure Ratio [%] . 16
4.3.8 Video IP Service Access Time [s] . 17
4.3.9 Video Reproduction Start Failure Ratio [%] . 17
4.3.10 Video Reproduction Start Delay [s] . 17
4.3.11 Video Play Start Failure Ratio [%] . 17
4.3.12 Video Play Start Time [s] . 17
4.3.13 IP Service Access Failure Ratio [%] . 18
4.3.14 IP Service Access Time [s] . 18
4.3.15 Video Session Cut-off Ratio [%] . 18
4.3.16 Video Session Time [s] . 18
4.3.17 Impairment Free Video Session Ratio [%] . 18
4.3.18 Video Expected Size [kbit] . 19
4.3.19 Video Downloaded Size [kbit]. 19
4.3.20 Video Compression Ratio [%] . 19
4.3.21 Video Transfer Cut-off Ratio [%] . 20
4.3.22 Video Transfer Time [s] . 20
4.3.23 Video Mean User Data Rate [kbit/s] . 20
4.3.24 Video Playout Cut-off Ratio [%] . 20
4.3.25 Video Playout Cut-off Time [s] . 20
4.3.26 Video Expected Duration [s] . 21
4.3.27 Video Playout Duration [s] . 21
4.3.28 Video Freeze Occurrences . 21
4.3.29 Accumulated Video Freezing Duration [s] . 21
4.3.30 Video Skip Occurrences . 21
4.3.31 Accumulated Video Skips Duration [s] . 22
4.3.32 Video Maximum Freezing Duration [s] . 22
4.3.33 Video Freezing Impairment Ratio [%]. 22
4.3.34 Video Freezing Time Proportion . 22
4.3.35 End-to-End Session Failure Ratio [%] . 23
4.4 Recommended supplementary information for TCP-based video service measurements . 23
ETSI

---------------------- Page: 3 ----------------------
4 ETSI TR 101 578 V1.2.1 (2015-07)
4.5 Configuration aspects including timeout recommendations for TCP-based video service measurements . 23
4.5.0 Purpose . 23
4.5.1 URL . 23
4.5.2 Timeouts . 24
4.5.2.0 Application of timeouts . 24
4.5.2.1 Player IP Service Access timeout . 24
4.5.2.2 Player Download Time timeout . 24
4.5.2.3 Video IP Service Access timeout . 24
4.5.2.4 Video Reproduction Start Delay timeout . 24
4.5.3 Video Playout Duration . 24
4.5.4 Handling of video freezes . 25
4.5.4.0 Use of freezes . 25
4.5.4.1 Minimum freeze duration . 25
4.5.4.2 Maximum duration of single freeze . 25
4.5.4.3 Maximum duration of all freezes . 25
4.5.4.4 Maximum number of freezes . 25
4.5.5 Timeout and Threshold Frameworks . 25
4.5.6 Hide video during playout . 26
4.5.7 Play until the end . 26
4.5.8 Cache and cookies . 26
4.6 Impacts of measurement hardware for TCP-based video service measurements . 26
Annex A: Measuring YouTube™ QoS with the Smartphone App . 27
A.0. Introduction . 27
A.1. User observable phases of the clip reproduction . 27
A.2. QoS Parameter for You Tube with the Smartphone App . 28
A.2.0 Parameter and trigger points . 28
A.2.1 App Video Access Failure Ratio [%] . 28
A.2.2 App Video Access Time [s] . 28
A.2.3 App Video Playout Cut-off Ratio [%] . 28
A.2.4 App Video Playout Duration [s] . 29
A.2.5 App Impairment Free Video Session Ratio [%] . 29
A.2.6 App Video Freezing Time Ratio [%] . 29
A.3. Configuration aspects . 29
History . 30


ETSI

---------------------- Page: 4 ----------------------
5 ETSI TR 101 578 V1.2.1 (2015-07)
List of figures
Figure 1: Typical phases of TCP-based video services .11
Figure A.1: Typical observable phases of Smartphone App video service .27

ETSI

---------------------- Page: 5 ----------------------
6 ETSI TR 101 578 V1.2.1 (2015-07)
List of tables
Table 1: Overview of QoS parameters and mapping to typical phases of TCP-based video services .13
Table 2: Overview of the trigger points used for the QoS parameter definition .15
Table 3: Observable quantities recommended getting included into the measurement results .23
Table 4: Example settings that do model a standard user .26
Table A.1: Overview of QoS parameters and mapping to typical phases of the video services as experienced by the user
.28
Table A.2: Overview of the trigger points used for the QoS parameter definition .28


ETSI

---------------------- Page: 6 ----------------------
7 ETSI TR 101 578 V1.2.1 (2015-07)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (http://ipr.etsi.org).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This Technical Report (TR) has been produced by ETSI Technical Committee Speech and multimedia Transmission
Quality (STQ).
Modal verbs terminology
In the present document "shall", "shall not", "should", "should not", "may", "need not", "will", "will not", "can" and
"cannot" are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of
provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.
Introduction
There are a variety of popular TCP-based video services available on the internet, on which users can upload, view and
share videos. These services use mainly Adobe® Flash® Video but also RealPlayer® and QuickTime® and lately
HTML5 technology to display a wide variety of video content, including movie clips, TV clips, and music videos, as
well as amateur content such as video blogging and short original videos.
NOTE 1: Adobe® Flash® is the trade name of a product supplied by Adobe. This information is given for the
convenience of users of the present document and does not constitute an endorsement by ETSI of the
product named. Equivalent products may be used if they can be shown to lead to the same results.
NOTE 2: RealPlayer® is the trade name of a product supplied by RealNetworks. This information is given for the
convenience of users of the present document and does not constitute an endorsement by ETSI of the
product named. Equivalent products may be used if they can be shown to lead to the same results.
NOTE 3: QuickTime® is the trade name of a product supplied by Apple. This information is given for the
convenience of users of the present document and does not constitute an endorsement by ETSI of the
product named. Equivalent products may be used if they can be shown to lead to the same results.
These services have become very popular and have a major share of the internet traffic worldwide. Due to its high
popularity in general and its use over mobile internet its availability and quality is of key interest of the provider of
mobile internet access, which makes the services a matter for benchmarking. The down-stream scenario, the probability
to access and see a desired video and the quality of the video is the subject of measurement method laid out in the
present document.
Any video content is accessed via a link that is provided by the service on a web page. The actual linked videos need to
be qualified however e.g. YouTube™ provides different quality profiles of the same video content e.g. a music video
clip.
NOTE 4: YouTube™ is the trade name of a product supplied by Google. This information is given for the
convenience of users of the present document and does not constitute an endorsement by ETSI of the
product named. Equivalent products may be used if they can be shown to lead to the same results.

ETSI

---------------------- Page: 7 ----------------------
8 ETSI TR 101 578 V1.2.1 (2015-07)
The individual quality profiles can differ in resolution (e.g. 240p, 360p, 480p, HD720p, HD1080p), in the data-rate and
other aspects. Since these differences of clips have an impact on their size and thus on the reproduction speed and
quality, a fair comparison can only be provided if actually the same clips are streamed over different networks. On the
other hand the clips not need to come physically from the same server since mobile operators employ proxies in order to
move the content closer to their subscriber and the downlink bandwidth is often controlled primarily by the video
service. Therefore the clips need to be streamed from the actual live network and may not be streamed from a dedicated
server.
For cases in which the video content is compressed during the transfer by a proxy hence the content arriving at the
subscriber is not identical, the compression ratio may be indicated to show that possible advantages in performance are
achieved by reducing the amount of data to be transferred. Whether this enhancement was achieved at the cost of the
general quality of the content could be determined by an objective video quality assessment.
The TCP-based videos can be received either on Smartphone or a PC connected via mobile network to the internet. For
the Smartphone the way the content is provided can differ significantly with the type and the OS the phone is using. In
the present document content delivery for special Apps, RealPlayer® and QuickTime® is not taken into consideration
but only the streaming over TCP as e.g. used by YouTube™ with a Browser on a PC or Smartphone with the respective
player.
ETSI

---------------------- Page: 8 ----------------------
9 ETSI TR 101 578 V1.2.1 (2015-07)
1 Scope
The present document focuses on Quality of Service (QoS) measurements for TCP-based video services where
downloading and viewing takes place in parallel. In principle the presented measurement approach can be used for all
video services, where the video is embedded in a HTML context as of video on demand services like e.g. YouTube™.
Similar applications are also available on social networks.
In the following, QoS parameters to be used for such video service measurements are presented. The underlying
procedure consists of two phases: first requesting a control script containing among other information a link to the
content, and second, requesting this content. In the present document, YouTube™ serves as the default example but the
described QoS parameters can easily be applied to other TCP-based video services.
Furthermore, the present document also offers practical guidance for measurement execution and evaluation of HTTP
streaming QoS measurement.
The present document covers the video request and playout of the video. Other services offered by content providers
such as e.g. uploading video or managing the private account are not covered.
2 References
2.1 Normative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
reference document (including any amendments) applies.
Referenced documents which are not found to be publicly available in the expected location might be found at
http://docbox.etsi.org/Reference.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are necessary for the application of the present document.
Not applicable.
2.2 Informative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
reference document (including any amendments) applies.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] ETSI TS 102 250-2: "Speech and multimedia Transmission Quality (STQ); QoS aspects for
popular services in mobile networks; Part 2: Definition of Quality of Service parameters and their
computation".
[i.2] ETSI TS 102 250-5: "Speech and multimedia Transmission Quality (STQ); QoS aspects for
popular services in mobile networks; Part 5: Definition of typical measurement profiles".
3 Abbreviations
For the purposes of the present document, the following abbreviations apply:
CPU Central Processing Unit
DASH Dynamic Adaptive Streaming over HTTP
ETSI

---------------------- Page: 9 ----------------------
10 ETSI TR 101 578 V1.2.1 (2015-07)
DNS Domain Name System
FLV Flash® Video
FTP File Transfer Protocol
GPU Graphics Processing Unit
HDD Hard Disk Drive
HTML HyperText Markup Language
HTTP HyperText Transfer Protocol
HTTPS Secure HTTP
IP Internet Protocol
LAN Local Area Network
NDIS Network Driver Interface Specification
OS Operating System
PC Personal Computer
PEC Performance Enhancement Client
QoS Quality of Service
RTP Real-time Transport Protocol
RTSP Real Time Streaming Protocol
SYN TCP synchronize flag
TCP Transmission Control Protocol
UDP User Datagram Protocol
URL Uniform Resource Locator
WLAN Wireless Local Area Network
4 Quality of Service measurements for TCP-based
video services like YouTube™
4.0 General
Many TCP-based video services, like e.g. the YouTube™ video service, provide videos in several resolutions and
qualities. For some video services the client can choose the resolution and quality of the video playback manually. On
the other hand, several mobile clients often allow only lower resolutions (delivered in lower bandwidth). Usually,
videos are streamed in proprietary Flash® format (FLV) over TCP. In addition, for very large videos or client devices
not supporting Flash® other formats are supported as well, e.g. 3GP video down-stream via RTP/UDP for RealPlayer®
on Symbian OS™.

Many TCP-based video services, like e.g. the YouTube™ video service, provide videos in several resolutions and
qualities. For some video services the client can choose the resolution and quality of the video playback manually. On
the other hand, several mobile clients often allow only lower resolutions (delivered in lower bandwidth). Usually,
videos are streamed in proprietary Flash® format (FLV) over TCP. In addition, for very large videos or client devices
not supporting Flash® other formats are supported as well, e.g. 3GP video down-stream via RTP/UDP for RealPlayer®
on Symbian OS™.
NOTE: Symbian OS™ is the trade name of a product supplied by Symbian Ltd. This information is given for the
convenience of users of the present document and does not constitute an endorsement by ETSI of the
product named. Equivalent products may be used if they can be shown to lead to the same results.
4.1 Phases of TCP-based video services
Most TCP-based video services, like the YouTube™ video service, are comprised of several phases which are mainly
the set-up of a HTML context including downloading the control script for the multimedia playout entity (in the
following: "player") and the down-stream of the video itself.
Figure 1 shows typical phases of TCP-based video services, like YouTube™.
ETSI

---------------------- Page: 10 ----------------------
11 ETSI TR 101 578 V1.2.1 (2015-07)

Figure 1: Typical phases of TCP-based video services
In principle the video service can be divided into the setup of the context until the player is ready to play and the
download and playout of the video.
The setup of the context until the player is ready to play can be divided into two phases, the "IP service access" phase
and the "player download" phase.
The "IP service access" phase starts when the HTML context and the player configuration download are requested. It
ends upon receipt of the first data byte containing HTML content, starting the pl
...