ETSI TR 101 578 V1.1.1 (2013-12)

Speech and multimedia Transmission Quality (STQ); QoS aspects of TCP-based video services like YouTubeTM

DTR/STQ-00192m

General Information

• Status: Published
• Publication Date: 11-Dec-2013
• Technical Committee: STQ MOBILE - STQ Mobile
• Current Stage: 12 - Completion
• Due Date: 20-Nov-2013
• Completion Date: 12-Dec-2013

Standards Content (Sample)

ETSI TR 101 578 V1.1.1 (2013-12)






Technical Report
Speech and multimedia Transmission Quality (STQ);
QoS aspects of TCP-based video services like YouTube™

---------------------- Page: 1 ----------------------
2 ETSI TR 101 578 V1.1.1 (2013-12)



Reference
DTR/STQ-00192m
Keywords
measurement, QoS, service, TCP-based video
services
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: [phone]  Fax: [phone]

Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88

Important notice
Individual copies of the present document can be downloaded from:
http://www.etsi.org
The present document may be made available in more than one electronic version or in print. In any case of existing or
perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF).
In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive
within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
http://portal.etsi.org/tb/status/status.asp
If you find errors in the present document, please send your comment to one of the following services:
http://portal.etsi.org/chaircor/ETSI_support.asp
Copyright Notification
No part may be reproduced except as authorized by written permission.
The copyright and the foregoing restriction extend to reproduction in all media.

© European Telecommunications Standards Institute 2013.
All rights reserved.

TM TM TM
DECT , PLUGTESTS , UMTS and the ETSI logo are Trade Marks of ETSI registered for the benefit of its Members.
TM
3GPP and LTE™ are Trade Marks of ETSI registered for the benefit of its Members and
of the 3GPP Organizational Partners.
GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.
ETSI

---------------------- Page: 2 ----------------------
3 ETSI TR 101 578 V1.1.1 (2013-12)
Contents
Intellectual Property Rights . 5
Foreword . 5
Introduction . 5
1 Scope . 6
2 References . 6
2.1 Normative references . 6
2.2 Informative references . 6
3 Abbreviations . 6
4 Quality of Service measurements for TCP-based video services like YouTube™ . 7
4.1 Phases of TCP-based video services . 7
4.2 QoS aspects of TCP-based video services . 8
4.2.1 Video freezes and - skips . 9
4.3 QoS parameters for TCP-based video services . 9
4.3.1 Player IP Service Access Failure Ratio [%] . 11
4.3.2 Player IP Service Access Time [s] . 11
4.3.3 Player Download Cut-off Ratio [%] . 11
4.3.4 Player Download Time [s] . 12
4.3.5 Player Session Failure Ratio [%] . 12
4.3.6 Player Session Time [s] . 12
4.3.7 Video IP Service Access Failure Ratio [%] . 12
4.3.8 Video IP Service Access Time [s] . 12
4.3.9 Video Reproduction Start Failure Ratio [%] . 12
4.3.10 Video Reproduction Start Delay [s] . 13
4.3.11 Video Play Start Failure Ratio [%] . 13
4.3.12 Video Play Start Time [s] . 13
4.3.13 IP Service Access Failure Ratio [%] . 13
4.3.14 IP Service Access Time [s] . 13
4.3.15 Video Session Cut-off Ratio [%] . 14
4.3.16 Video Session Time [s] . 14
4.3.17 Impairment Free Video Session Ratio [%] . 14
4.3.18 Video Expected Size [kbit] . 14
4.3.19 Video Downloaded Size [kbit]. 15
4.3.20 Video Compression Ratio [%] . 15
4.3.21 Video Transfer Cut-off Ratio [%] . 15
4.3.22 Video Transfer Time [s] . 15
4.3.23 Video Mean User Data Rate [kbit/s] . 15
4.3.24 Video Playout Cut-off Ratio [%] . 16
4.3.25 Video Playout Cut-off Time [s] . 16
4.3.26 Video Expected Duration [s] . 16
4.3.27 Video Playout Duration [s] . 16
4.3.28 Video Freeze Occurrences . 17
4.3.29 Accumulated Video Freezing Duration [s] . 17
4.3.30 Video Skip Occurrences . 17
4.3.31 Accumulated Video Skips Duration [s] . 17
4.3.32 Video Maximum Freezing Duration [s] . 18
4.3.33 Video Freezing Impairment Ratio [%]. 18
4.3.34 Video Freezing Time Proportion . 18
4.3.35 End-to-End Session Failure Ratio [%] . 19
4.4 Recommended supplementary information for TCP-based video service measurements . 19
4.5 Configuration aspects including timeout recommendations for TCP-based video service measurements . 19
4.5.1 URL . 19
4.5.2 Timeouts . 19
4.5.2.1 Player IP Service Access timeout . 20
ETSI

---------------------- Page: 3 ----------------------
4 ETSI TR 101 578 V1.1.1 (2013-12)
4.5.2.2 Player Download Time timeout . 20
4.5.2.3 Video IP Service Access timeout . 20
4.5.2.4 Video Reproduction Start Delay timeout . 20
4.5.3 Video Playout Duration . 20
4.5.4 Handling of video freezes . 20
4.5.4.1 Minimum freeze duration . 21
4.5.4.2 Maximum duration of single freeze . 21
4.5.4.3 Maximum duration of all freezes . 21
4.5.4.4 Maximum number of freezes . 21
4.5.5 Timeout and Threshold Frameworks . 21
4.5.6 Hide video during playout . 22
4.5.7 Play until the end . 22
4.5.8 Cache and cookies . 22
4.6 Impacts of measurement hardware for TCP-based video service measurements . 22
History . 25

ETSI

---------------------- Page: 4 ----------------------
5 ETSI TR 101 578 V1.1.1 (2013-12)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (http://ipr.etsi.org).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This Technical Report (TR) has been produced by ETSI Technical Committee Speech and multimedia Transmission
Quality (STQ).
Introduction
There are a variety of popular TCP-based video services available on the internet, on which users can upload, view and
share videos. These services use mainly Adobe® Flash® Video but also RealPlayer® and QuickTime® and lately
HTML5 technology to display a wide variety of video content, including movie clips, TV clips, and music videos, as
well as amateur content such as video blogging and short original videos.
These services have become very popular and have a major share of the internet traffic worldwide. Due to its high
popularity in general and its use over mobile internet its availability and quality is of key interest of the provider of
mobile internet access, which makes the services a matter for benchmarking. The down-stream scenario, the probability
to access and see a desired video and the quality of the video is the subject of measurement method laid out in the
present document.
Any video content is accessed via a link that is provided by the service on a web page. The actual linked videos need to
be qualified however e.g. YouTube™ provides different quality profiles of the same video content e.g. a music video
clip. The individual quality profiles can differ in resolution (e.g. 240p, 360p, 480p, HD720p, HD1080p), in the data-rate
and other aspects. Since these differences of clips have an impact on their size and thus on the reproduction speed and
quality, a fair comparison can only be provided if actually the same clips are streamed over different networks. On the
other hand the clips not need to come physically from the same server since mobile operators employ proxies in order to
move the content closer to their subscriber and the downlink bandwidth is often controlled primarily by the video
service. Therefore the clips need to be streamed from the actual live network and may not be streamed from a dedicated
server.
For cases in which the video content is compressed during the transfer by a proxy hence the content arriving at the
subscriber is not identical, the compression ratio may be indicated to show that possible advantages in performance are
achieved by reducing the amount of data to be transferred. Whether this enhancement was achieved at the cost of the
general quality of the content could be determined by an objective video quality assessment.
The TCP-based videos can be received either on Smartphone or a PC connected via mobile network to the internet. For
the Smartphone the way the content is provided can differ significantly with the type and the OS the phone is using. In
the present document content delivery for special Apps, RealPlayer® and QuickTime® is not taken into consideration
but only the streaming over TCP as e.g. used by YouTube™ with a Browser on a PC or Smartphone with the respective
player.
ETSI

---------------------- Page: 5 ----------------------
6 ETSI TR 101 578 V1.1.1 (2013-12)
1 Scope
The present document focuses on Quality of Service (QoS) measurements for TCP-based video services where
downloading and viewing takes place in parallel. In principle the presented measurement approach can be used for all
video services, where the video is embedded in a HTML context as of video on demand services like e.g. YouTube™.
Similar applications are also available on social networks.
In the following, QoS parameters to be used for such video service measurements are presented. The underlying
procedure consists of two phases: first requesting a control script containing among other information a link to the
content, and second, requesting this content. In the present document, YouTube™ serves as the default example but the
described QoS parameters can easily be applied to other TCP-based video services.
Furthermore, this report also offers practical guidance for measurement execution and evaluation of HTTP streaming
QoS measurement.
The present document covers the video request and playout of the video. Other services offered by content providers
such as e.g. uploading video or managing the private account are not covered.
2 References
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
Referenced documents which are not found to be publicly available in the expected location might be found at
http://docbox.etsi.org/Reference.
NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee
their long term validity.
2.1 Normative references
The following referenced documents are necessary for the application of the present document.
Not applicable.
2.2 Informative references
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] ETSI TS 102 250-2: "Speech and multimedia Transmission Quality (STQ); QoS aspects for
popular services in mobile networks; Part 2: Definition of Quality of Service parameters and their
computation".
[i.2] ETSI TS 102 250-5: "Speech and multimedia Transmission Quality (STQ); QoS aspects for
popular services in mobile networks; Part 5: Definition of typical measurement profiles".
3 Abbreviations
For the purposes of the present document, the following abbreviations apply:
CPU Central Processing Unit
DNS Domain Name System
FLV Flash® Video
FTP File Transfer Protocol
ETSI

---------------------- Page: 6 ----------------------
7 ETSI TR 101 578 V1.1.1 (2013-12)
GPU Graphics Processing Unit
HDD Hard Disk Drive
HTML HyperText Markup Language
HTTP HyperText Transfer Protocol
IP Internet Protocol
LAN Local Area Network
NDIS Network Driver Interface Specification
OS Operating System
PC Personal Computer
PEC Performance Enhancement Client
QoS Quality of Service
RTP Real-time Transport Protocol
RTSP Real Time Streaming Protocol
SYN TCP synchronise flag
TCP Transmission Control Protocol
UDP User Datagram Protocol
URL Uniform Resource Locator
WLAN Wireless Local Area Network
4 Quality of Service measurements for TCP-based
video services like YouTube™
Many TCP-based video services, like e.g. the YouTube™ video service, provide videos in several resolutions and
qualities. For some video services the client can choose the resolution and quality of the video playback manually. On
the other hand, several mobile clients often allow only lower resolutions (delivered in lower bandwidth). Usually,
videos are streamed in proprietary Flash® format (FLV) over TCP. In addition, for very large videos or client devices
not supporting Flash® other formats are supported as well, e.g. 3GP video down-stream via RTP/UDP for RealPlayer®
on Symbian OS™.
4.1 Phases of TCP-based video services
Most TCP-based video services, like the YouTube™ video service, are comprised of several phases which are mainly
the set-up of a HTML context including downloading the control script for the multimedia playout entity (in the
following: "player") and the down-stream of the video itself.
Figure 1 shows typical phases of TCP-based video services, like YouTube™.
Start of End of Start of End of
player download player download video transfer video transfer
Start of
Start of video playout (first
First packet of video download End of
frame displayed in player)
player received (pressing “Play”) video playout
IP SERVICE PLAYER STREAMING
BUFFERING PLAYOUT
ACCESS DOWNLOAD ACCESS
PRE-DOWNLOAD DOWNLOAD

Figure 1: Typical phases of TCP-based video services
In principle the video service can be divided into the setup of the context until the player is ready to play and the
download and playout of the video.
ETSI

---------------------- Page: 7 ----------------------
8 ETSI TR 101 578 V1.1.1 (2013-12)
The setup of the context until the player is ready to play can be divided into two phases, the "IP service access" phase
and the "player download" phase.
The "IP service access" phase starts when the HTML context and the player configuration download are requested. It
ends upon receipt of the first data byte containing HTML content, starting the player download phase.
NOTE: The initial DNS request sent when e. g. the YouTube™ URL is opened is not considered to be part of the
"IP service access" phase. Thus, the quality of service for the initial DNS resolution for the HTML
context and the player configuration download is not covered by the QoS parameters defined in the
present document for the "IP service access" phase.
The "IP service access" phase is followed by downloading the HTML context information and the player configuration
script (in case of Flash® Player). It contains potential surrounding HTML based information (YouTube™ site), which
can be the original YouTube™ site with an embedded player (YouTube™ in a browser window), or the player
application without any visible HTML context (YouTube™ App or Flash® Player in an empty HTML context). The
last step of the context setup is the download of the player configuration script. The entire download phase is called
simplified "player download". At the point the context and the configuration is downloaded completely, the player is
"ready to play".
The "streaming access" phase is started by pressing the "Play" button (or in case of "AutoPlay" with the event "Ready
To Play" that is then equivalent to requesting the video) and ends with receiving the first video packet over TCP or
RTP. This phase could be very interesting if there are proxies between the user and the content server, thus making
DNS resolution and other events an influential factor.
In a simple case, during "streaming access" phase there will be only one GET (in case of a requested TCP stream) or a
RTSP DESCRIBE (in case of a requested RTP stream) request for the video, followed immediately by the 200 OK
message and the payload packets for FLV (TCP) or 3GP (RTP) video. In a more complex case, there may either be
several ranged GET requests or it may happen that after the first GET request there will be several redirects (because of
proxies) and several more resolving of DNS, etc. so this phase can be much longer. Basically, QoS parameters related to
this phase allow concluding how close the network is to the content server for the measured video. Even if a preferred
server (URL) is given and firstly requested, the actual approached location of the clip may differ and lead to a
redirection to a closer or more appropriate server and DNS has to be contacted again.
In case of RTSP, an RTSP link is embedded in the HTML context. After receiving the final URL or the video, the video
can be streamed/downloaded (Ready to Play).
Upon receiving the first video packet, the "buffering" phase commences. While the "download" phase will end with the
last received video packet, the "buffering" phase ends in the moment the video playout actually starts. QoS parameters
related to this phase allow estimating the initial buffer size for the measured video and current Internet connection type
(e.g. DialUp, NDIS, WLAN, LAN).
The "IP service access", the "player download" and the "streaming access" phases jointly constitute the "pre-download"
phase. The "buffering" and the remaining video content transfer constitute the "download" phase.
As soon as playout starts, the "playout" phase commences. This phase overlaps with the "download" phase and
represents the full playout time for the video. Depending on the configuration the full video can be displayed or video
playout can be cut short when video download is complete.
During the "playout" phase the "freezes" of the video display and "video skips" are detected.
4.2 QoS aspects of TCP-based video services
When looking at impairments of the video playout, this clause focuses on objectively measurable impairments, such as:
• failures to start;
• video freezes;
• video skips; or
• failures to download completely.
It may be that further subjective impairments exist, which limit comparability of QoS parameters obtained in different
setups. Such subjective impairments can e.g. be downscaling of video image quality or frame-rate.
ETSI

---------------------- Page: 8 ----------------------
9 ETSI TR 101 578 V1.1.1 (2013-12)
4.2.1 Video freezes and - skips
Freezing events are when the video playout stops (freezes), it is mainly caused by a buffer under-run. The video pauses
until the re-buffering is complete. After that, the video continues. There is no video information lost. This is the
common case for YouTube™ using Flash® Player.
Skip events are when playout is not continuous, which means that parts of the video content are not displayed or re-
displayed, i.e. the playout jumps to a future or past point in time. A possible reason for this can be player misbehaviour
or network outage in combination with live streams. When a skip occurs, there is usually no visible re-buffering
information on the screen.
4.3 QoS parameters for TCP-based video services
In this clause, a set of QoS parameters based and expanding on the streaming QoS parameters as defined in [i.1] is
proposed for measuring TCP-based video services.
Table 1 gives an overview of the proposed QoS parameters and provides a mapping of these parameters to the phases
introduced in clause 4.1. Furthermore, a parameter type is assigned for each QoS parameter in order to determine the
calculation method to be used for the respective parameter.
Table 1: Overview of QoS parameters and mapping to typical phases of TCP-based video services
Related Phase(s) QoS parameter name QoS parameter type
IP service access Player IP Service Access Failure Ratio Failure Ratio
IP service access Player IP Service Access Time Duration
Player download Player Download Cut-off Ratio Cut-off Ratio
Player download Player Download Time Duration
IP service access, Player download Player Session Failure Ratio Failure Ratio
IP service access, Player download Player Session Time Duration
Streaming access Video IP Service Access Failure Ratio Failure Ratio
Streaming access Video IP Service Access Time Duration
Buffering Video Reproduction Start Failure Ratio Failure Ratio
Buffering Video Reproduction Start Delay Duration
Streaming access, Buffering Video Play Start Failure Ratio Failure Ratio
Streaming access, Buffering Video Play Start Time Duration
Pre-download IP Service Access Failure Ratio Failure Ratio
Pre-download IP Service Access Time Duration
Streaming access, Buffering, Playout Video Session Cut-off Ratio Cut-off Ratio
Streaming access, Buffering, Playout Video Session Time Duration
Streaming access, Buffering, Playout Impairment Free Video Session Ratio Calculation
Download Video Expected Size Size
Download Video Downloaded Size Size
Download Video Compression Ratio Calculation
Download Video Transfer Cut-off Ratio Cut-off Ratio
Download Video Transfer Time Duration
Download Video Mean User Data Rate Calculation
Playout Video Playout Cut-off Ratio Cut-off Ratio
Playout Video Playout Cut-off Time Duration
Playout Video Expected Duration Duration
Playout Video Playout Duration Duration
Playout Video Freeze Occurrences Count
Playout Accumulated Video Freezing Duration Calculation
Playout Video Skip Occurrences Count
Playout Accumulated Video Skips Duration Calculation
Playout Video Maximum Freezing Duration Calculation
Playout Video Freezing Impairment Ratio Failure Ratio
Playout Video Freezing Time Proportion Calculation
Whole session End-to-End Session Failure Ratio Failure Ratio

ETSI

---------------------- Page: 9 ----------------------
10 ETSI TR 101 578 V1.1.1 (2013-12)
Within table 1, the following QoS parameter types are defined:
• Calculation
• Count
• Duration
• Size
• Cut-off Ratio; and
• Failure Ratio
The type "Calculation" is assigned to QoS parameters gett
...