ETSI TR 102 643 V1.0.2 (2010-01)

Human Factors (HF); Quality of Experience (QoE) requirements for real-time communication services

RTR/HF-00130

General Information

• Status: Published
• Publication Date: 17-Jan-2010
• Technical Committee: HF - Human Factors
• Current Stage: 12 - Completion
• Due Date: 15-Jan-2010
• Completion Date: 18-Jan-2010

Standards Content (Sample)

Technical Report
Human Factors (HF);
Quality of Experience (QoE) requirements for
real-time communication services

2 ETSI TR 102 643 V1.0.2 (2010-01)

Reference
RTR/HF-00130
Keywords
interaction, quality, service
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: [phone]  Fax: [phone]

Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88

Important notice
Individual copies of the present document can be downloaded from:
http://www.etsi.org
The present document may be made available in more than one electronic version or in print. In any case of existing or
perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF).
In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive
within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
http://portal.etsi.org/tb/status/status.asp
If you find errors in the present document, please send your comment to one of the following services:
http://portal.etsi.org/chaircor/ETSI_support.asp
Copyright Notification
No part may be reproduced except as authorized by written permission.
The copyright and the foregoing restriction extend to reproduction in all media.

© European Telecommunications Standards Institute 2010.
All rights reserved.
TM TM TM TM
DECT , PLUGTESTS , UMTS , TIPHON , the TIPHON logo and the ETSI logo are Trade Marks of ETSI registered
for the benefit of its Members.
TM
3GPP is a Trade Mark of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners.
LTE™ is a Trade Mark of ETSI currently being registered
for the benefit of its Members and of the 3GPP Organizational Partners.
GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.
ETSI
3 ETSI TR 102 643 V1.0.2 (2010-01)
Contents
Intellectual Property Rights . 4
Foreword . 4
1 Scope . 5
2 References . 5
2.1 Normative references . 5
2.2 Informative references . 5
3 Definitions and abbreviations . 8
3.1 Definitions . 8
3.2 Abbreviations . 12
4 What is Quality of Experience (QoE)? . 12
4.1 A definition of QoE for the current document. 13
4.2 QoE and QoS . 13
4.3 Usability and user experience . 14
4.3.1 Technology-centred approach: Quality of Service (QoS) . 15
4.3.2 User-centred approaches: QoE and User-perceived QoS . 15
4.3.2.1 Quality of Perception (QoP) . 15
4.3.2.2 Quality of Experience (QoE). 16
4.3.3 The co-existence of QoS and QoE . 17
4.4 The QoE context of the present document . 17
4.4.1 The Choosing situation . 18
4.4.2 The Usage situation . 19
5 For whom are QoE data important? . 20
6 How can QoE data help? . 20
6.1 To prevent churn . 20
6.2 To prevent product or service rejections . 21
6.3 To optimise a product or service . 21
6.4 By expressing QoE expressed in terms of QoS . 21
7 Providing QoE data for real-time communication services. 21
7.1 Real-time communication services . 21
7.2 Guidelines development and dissemination . 27
7.2.1 Guideline development . 27
7.2.2 Guideline dissemination . 29
8 Web-Based Guideline and Tutorial System for improved dissemination and application of QoE
data . 29
8.1 The main facilities of the web-based system . 29
8.1.1 Navigation facility . 30
8.1.2 Education facility . 31
8.1.3 Dissemination facility . 32
8.2 Types of Guidelines. 32
8.3 Maturity and restrictions of the guidelines . 34
8.4 Maintenance of the Web-based system . 34
9 Towards generic QoE guidelines by including non real-time services . 35
10 Conclusions . 35
11 Recommendations for further work . 35
11.1 Extend towards generic QoE guidelines . 35
11.2 Document the user test framework and guideline derivation methodology in a separate ETSI document . 36
11.3 Update the Web-Based Guideline and Tutorial System . 36
History . 37
ETSI
4 ETSI TR 102 643 V1.0.2 (2010-01)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (http://webapp.etsi.org/IPR/home.asp).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This Technical Report (TR) has been produced by ETSI Technical Committee Human Factors (HF).
ETSI
5 ETSI TR 102 643 V1.0.2 (2010-01)
1 Scope
The present document is based on Guidelines produced in Specialist ETSI Task Force 354 and a Web-Based Guideline
access and Tutorial System (WBGTS) (http://portal.etsi.org/stfs/STF_HomePages/STF354/STF354.asp). The main
content of the WBGTS are Quality of Experience (QoE) guidelines for real-time communication services expressed in
Quality of Service (QoS) terms.
The Guidelines and the present document are primarily intended for professionals in network operator, equipment
manufacturer and service provider organisations who are concerned with the user experience of communication
services. Over the last 10 years that include pre-cursor projects to STF 354 about 2000 intended guideline users have
been involved in the development work of the WBGTS. The present document describes an assessment of the current
guidelines and the tutorial system and identifies needs for future work.
2 References
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific.
• For a specific reference, subsequent revisions do not apply.
• Non-specific reference may be made only to a complete document or a part thereof and only in the following
cases:
- if it is accepted that it will be possible to use all future changes of the referenced document for the
purposes of the referring document;
- for informative references.
Referenced documents which are not found to be publicly available in the expected location might be found at
http://docbox.etsi.org/Reference.
NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee
their long term validity.
2.1 Normative references
The following referenced documents are indispensable for the application of the present document. For dated
references, only the edition cited applies. For non-specific references, the latest edition of the referenced document
(including any amendments) applies.
Not applicable.
2.2 Informative references
The following referenced documents are not essential to the use of the present document but they assist the user with
regard to a particular subject area. For non-specific references, the latest version of the referenced document (including
any amendments) applies.
[i.1] ANSI T1.522-2000: "Quality of Service for Business Multimedia Conferencing".
[i.2] Birdwhistle, R. (1970): "Kinesics and Context: Essays in Body Motion Communication",
Harmondsworth: Penguin.
[i.3] Blythe, M.; Overbeeke, K., Monk, A., Wright, P. (Eds) (2004): "Funology: From Usability to
Enjoyment" Kluwer Academic.
ETSI
6 ETSI TR 102 643 V1.0.2 (2010-01)
[i.4] Brooks P, Brundell P, Hamnes K, Heiestad S, Heim J, Hestnes B, Heydari B, O'Malley C,
Schliemann T, Skjetne JH, Ulseth T (1999): "Final Report. ACTS Project AC314 Vis-à-Vis:
Fitness-for-Purpose of Videotelephony in Face-to-Face Situations", CEC Deliverable
A314/NSS/PB/DS/P/005/b1, June 1999.
[i.5] Brooks P, Schliemann T, Hestnes B, Frowein H, Aaby C, O'Malley C, (2003): "Final Report
Project IST-1999-11577 Eye-2-Eye: Fitness-for-Purpose of Person-Person Communication
Technologies", EC Deliverable IST11577/SEF/DIS/DS/Pub/008/b1, June 2003.
[i.6] Brooks, P. & Hestnes, B. (2003): "User-centred technical guidelines for real-time human
communication services: Requirements and derivation", Proceedings of the 19th International
Symposium on Human Factors in Telecommunication, Berlin, Germany, December 1-4 2003,
pp. 11-18.
[i.7] Brooks, P., Hestnes, B., Heiestad, S., Aaby, C. (2006): "Communicating Quality of Experience
data for the development of multimedia services", Proceedings of the 20th International
Symposium on Human Factors in Telecommunication, Sophia Antipolis, France,
March 21-23 2006.
NOTE: Available from http://www.hft.org/HFT06/HFT_06_programme.htm.
[i.8] Bruce, V. (1996): "The role of the face in communication: Implications for videophone design",
Interacting with Computers, 8(2), 166-176.
[i.9] Ekman, P. & Friesen, W. V. (1975): "Unmasking the face. A guide to recognizing emotions from
facial clues", Englewood Cliffs, New Jersey: Prentice-Hall.
[i.10] ETSI ES 202 667: "Speech and multimedia Transmission Quality (STQ); Audiovisual QoS for
communication over IP networks".
[i.11] ETSI ES 202 737: "Speech and multimedia Transmission Quality (STQ); Transmission
requirements for narrowband VoIP terminals (handset and headset) from a QoS perspective as
perceived by the user".
[i.12] ETSI ES 202 738: "Speech and multimedia Transmission Quality (STQ); Transmission
requirements for narrowband VoIP loudspeaking and handsfree terminals from a QoS perspective
as perceived by the user".
[i.13] ETSI ES 202 739: "Speech and multimediaTransmission Quality (STQ); Transmission
requirements for wideband VoIP terminals (handset and headset) from a QoS perspective as
perceived by the user".
[i.14] ETSI ES 202 740: "Speech and multimedia Transmission Quality (STQ); Transmission
requirements for wideband VoIP loudspeaking and handsfree terminals from a QoS perspective as
perceived by the user".
[i.15] ETSI EG 202 534: "Human Factors (HF); guidelines for real-time person-to-person
communication services".
[i.16] ETSI ETR 160: "Human Factors (HF); Human Factors aspects of multimedia
telecommunications".
[i.17] ETSI ETR 297: "Human Factors (HF); Human Factors in Videotelephony".
[i.18] ETSI TR 102 274: "Human Factors (HF); guidelines for real-time person-to-person
communication services".
[i.19] ETSI TR 102 479: "Telecommunications and Internet converged Services and Protocols for
Advanced Networking (TISPAN); Review of available material on QoS requirements of
Multimedia Services".
[i.20] ETSI TR 102 535: "Human Factors (HF); Guidelines for real-time person-to-person
communication services; Future requirements".
ETSI
7 ETSI TR 102 643 V1.0.2 (2010-01)
[i.21] ETSI TS 181 016: "Telecommunications and Internet converged Services and Protocols for
Advanced Networking (TISPAN); Service Layer Requirements to integrate NGN Services and
IPTV".
[i.22] ETSI TS 122 105: "Universal Mobile Telecommunications System (UMTS); Services and service
capabilities (3GPP TS 22.105 version 8.4.0 Release 8)".
[i.23] Heim, J., Asting, T., Brandtzæg, P., Brooks, P., Skjetne, JH., Hestnes, B. et al. (2000): "Initial
Verification of Real-time Communication Requirements", Project IST-1999-11577 Eye-2-Eye:
Fitness-for-Purpose of Person-Person Communication Technologies, EC Deliverable
IST11577/SEF/DIS/DS/5FP/001/b1, July 2000.
[i.24] Hestnes B, Heiestad S, Ulseth T, Schliemann T, Brooks P, Følstad A, Frowein H, Aaby C,
O'Malley C, Brundell P: "Fitness-for-Purpose guidelines for Person-Person Communication's,
Project IST-1999-11577 Eye-2-Eye: Fitness-for-Purpose of Person-Person Communication
Technologies". EC Deliverable IST11577/TEL/RAD/DS/Pub/065/b1, March 2003.
[i.25] Hestnes, B., Brooks, P., Heiestad, S., Ulseth, T., Aaby., C. (2003): "Quality of Experience in real-
time person-person communication - User based QoS expressed in technical network QoS terms",
Proceedings of the 19th International Symposium on Human Factors in Telecommunication,
Berlin, Germany, December 1-4 2003, pp. 3-10.
[i.26] Hestnes, B., Brooks, P., Heiestad, S. (2009): "QoE (Quality of Experience) - measuring QoE for
improving the usage of telecommunication services", Telenor Research & Innovation Research
Report, Telenor Research & Innovation-R21/2009, Telenor, Fornebu, Norway.
[i.27] ISO 9241-11 (1998): "Ergonomic requirements for office work with visual display terminals
(VDTs) Part 11: Guidance on usability".
[i.28] ITU-R Recommendation BS 1534-1: "Method for the Subjective Assessment of Intermediate
Quality Level of Coding Systems".
[i.29] ITU-R Recommendation BT.1359-1 (2003): "Method for the Subjective Assessment of
Intermediate Quality Level of Coding Systems".
[i.30] ITU-T Series H: "Audiovisual and multimedia systems, Supplement 1 (05/99) (1999) Application
profile - Sign language and lip-reading real-time conversation using low bit-rate video
communication".
[i.31] ITU-T Recommendation E.800 (2008): "Definitions of terms related to quality of service".
[i.32] ITU-T Recommendation E.860: "Framework of a service level agreement".
[i.33] ITU-T Recommendation F.700 (2000) Framework Recommendation for multimedia services.
[i.34] ITU-T Recommendation G.108 (1999) "Application of the E-model: A planning guide".
[i.35] ITU-T Recommendation G.114 (2003) "One-way transmission time", International.
[i.36] ITU-T Recommendation G.1010 (2001): "End-user multimedia QoS categories".
[i.37] ITU-T Recommendation P.10/G.100: "Amendment 2: New definitions for inclusion in
Recommendation ITU-T P.10/G.100", International Telecommunication Union, Geneva,
Switzerland, 2008.
[i.38] ITU-T Recommendation Y.1541 (2006): "Network performance objectives for IP-based services",
International Telecommunication Union, Geneva, Switzerland.
[i.39] McGurk, H., & McDonald, J. (1976): "Hearing lips and seeing voices", Nature 264, 746-748.
[i.40] Nokia (2004): "Quality of Experience (QoE) of mobile services: Can it be measured and
improved?" White Paper, Nokia Corporation.
[i.41] Reeves, B. and Nass, C. (1996): "The media equation: How people treat computers, television and
new media like real people and places", Cambridge University Press. Communication
Technologies, EC Deliverable IST11577/TEL/RAD/DS/Pub/065/b1, March 2003.
ETSI
8 ETSI TR 102 643 V1.0.2 (2010-01)
[i.42] Soldani, D., Li, M., Cuny, R. (2006): "QoS and QoE management in UMTS cellular systems",
John Wiley & Sons Ltd: England.
[i.43] Summerfield, Q. (1992): "Lipreading and audio-visual speech perception", Philosophical
Transactions of the Royal Society of London, B335, 71-78.
[i.44] Wikipedia (2009): http://en.wikipedia.org/.
3 Definitions and abbreviations
3.1 Definitions
For the purposes of the present document, the following terms and definitions apply:
asynchrony: when audio and video information that leaves one communicating party at the same time is received by
the other communicating party at different times
NOTE 1: E.g. typically the audio information arrives before the video information in an asynchronous situation.
NOTE 2: It is calculated as audio delay subtracted from video delay (e.g. if audio delay is 50 ms and video delay is
200 ms, then asynchrony is 150 ms; if audio delay is 100 ms and video delay is 50 ms, then asynchrony is
-50 ms).
audio communication: use of a service that transmits voice in real-time over a telecommunication network, such as
ordinary telephony with a handset and loud-speaking audio conferencing
audio conferencing: telephone service that does not rely on amplification of the voice signal in very close proximity to
the recipient's ear
EXAMPLE: Loud-speaking audio communication.
audio delay: time required for an audio signal generated at the talker's mouth to reach the listener's ear
audio protocol: set of rules defining the way audio information is represented in a network
audio telephony: "ordinary" telephone service using a handset as distinct from loud-speaking audio conferencing
avatar telephony: service for transmitting voice signals in real-time over a telecommunication network in combination
with a graphical (human) representation of the speaker
bandwidth: range of frequencies which can safely be conveyed in a communication channel
burst packet loss: loss of two or more packets in sequence
communication media: types of information with which humans communicate
NOTE: Examples are text, audio and moving image (graphics and video). This is consistent with the "Nature of
information" component of the ETSI definition of a representation medium, which has various possible
coded forms (ETR 160 [i.16]).
communication service: service that is provided via a telecommunication network
NOTE: Examples are audio-telephony, email, videoconferencing, avatar-telephony, audio conferencing.
communication situation: combination of task, motive, content and user (group) characteristics
communication task: what the end-users (want to) do with a communication service
NOTE: E.g. social chatting, buying or selling shares, conducting a job interview, etc.
communicative behaviour: end-user behaviour while using a communication service, including turn taking,
interruptions, verbal and non-verbal back-channels and gaze
ETSI
9 ETSI TR 102 643 V1.0.2 (2010-01)
conversational text: See real-time text.
data communication: use of a service that transmits personal computer-based information (e.g. presentation slides)
data conferencing: See data communication.
duration: length of time of the communication task
dyadic communication: (distance) communication between two people
effectiveness: accuracy and completeness with which specified users can achieve specified goals in particular
environments
NOTE: See ISO 9241 [i.27] definition.
efficiency: resources expended in relation to the accuracy and completeness of goals achieved
NOTE: See ISO 9241 [i.27] definition.
end-users: people who use a communication service
fitness-for-purpose: correct balance between technological performance and human performance, such that the
interaction is both sufficient and beneficial for communication and consistent with human expectations
frame-rate: frequency by which a full video frame is updated, sometimes called video temporal resolution or image
frequency
group: (distance) communication between three or more people
NOTE: Either in a point-to-point or a multi-point configuration.
interpersonal perception: extent to which the perception of the other person's attributes (how likeable, intelligent,
friendly, etc.) is positive or negative
media effects: effect a particular communication medium has on an end-users task outcome, communicative behaviour,
attitudes and beliefs
media/medium: See Communication Media/Medium.
monitor size: number in inches of the diagonal of the image screen on a screen
multimedia communication: use of a service that transmits voice, video and data signals in real-time over a
telecommunication network
multimedia conferencing: service for transmitting voice, video and data signals over a telecommunication network
multi-point: distance communication between three or more locations
network quality of service: degree of conformance of the service delivered to a user by a provider with an agreement
between them
NOTE: From ITU-T Recommendation E.860 [i.32].
packet loss: loss of one packet that can be described using a certain statistical model
packet size: magnitude of a data being transmitted over a packet switching network in number of Bytes
personal involvement: extent to which the communication parties are committed to the outcome of the task or perform
the task more on behalf of another party than themselves
point-to-point: communication between two locations
quality of experience (1): measure of user performance based on both objective and subjective psychological measures
of using an ICT service or product
NOTE 1: It takes into account technical parameters (e.g. QoS) and usage context variables (e.g. communication
task) and measures both the process and outcomes of communication (e.g. user effectiveness, efficiency,
satisfaction and enjoyment).
ETSI
10 ETSI TR 102 643 V1.0.2 (2010-01)
NOTE 2: The appropriate psychological measures will be dependent on the communication context. Objective
psychological measures do not rely on the opinion of the user (e.g. task completion time measured in
seconds, task accuracy measured in number of errors). Subjective psychological measures are based on
the opinion of the user (e.g. perceived quality of medium, satisfaction with a service).
EXAMPLE: A service provider may conclude that a service with a certain level of QoS used for a particular
communication situation offers users excellent QoE, whist with a different level of QoS provides
poor QoE.
quality of experience (QoE) (2): overall acceptability of an application or service, as perceived subjectively by the
end-user
NOTE 1: Quality of experience includes the complete end-to-end system effects (client, terminal, network, services
infrastructure, etc.).
NOTE 2: Overall acceptability may be influenced by user expectations and context.
NOTE 3: ITU-T Recommendation P.10 [i.37]/G.100 Amendment 2 definition.
quality of service: totality of characteristics of a telecommunications service that bear on its ability to satisfy stated and
implied needs of the user of the service
NOTE: ITU-T Recommendation E.800 [i.31] definition.
Quality of service delivered/achieved by service provider (QoSD): statement of the level of QoS achieved or
delivered to the customer
NOTE 1: Achieved or delivered QoS is expressed by metrics for the pertinent parameters for a service.
NOTE 2: ITU-T Recommendation E.800 [i.31] definition.
Quality of service experienced/perceived by customer/user (QoSE): statement expressing the level of quality that
customers/users believe they have experienced
NOTE 1: The level of QoS experienced and/or perceived by the customer/user may be expressed by an opinion
rating.
NOTE 2: QoSE has two main man components: quantitative and qualitative. The quantitative component can be
influenced by the complete end-to-end system effects (network infrastructure).
NOTE 3: The qualitative component can be influenced by user expectations, ambient conditions, psychological
factors, application context, etc.
NOTE 4: QoSE may also be considered as QoSD (QoS delivered/achieved by service provider) received and
interpreted by a user with the pertinent qualitative factors influencing his/her perception of the service.
NOTE 5: ITU-T Recommendation E.800 [i.31] definition.
Real-time (1): describes information and communication technologies that are able to generate and deliver information
in a time-frame similar to the real-life process that it is assisting
EXAMPLE 1: Real-time charging and billing information is to be generated, processed, and transported to a
desired conclusion in less than 1 second
EXAMPLE 2: Refers to the generation of network management information in a time frame comparative to the
real life process that it is controlling or monitoring
real time (2): Occurring immediately. The term is used to describe a number of different computer features. For
example, real-time operating systems are systems that respond to input immediately. They are used for such tasks as
navigation, in which the computer reacts to a steady flow of new information without interruption. Most general-
purpose operating systems are not real-time because they can take a few seconds, or even minutes, to react. Real time
can also refer to events simulated by a computer at the same speed that they would occur in real life. In graphics
animation, for example, a real-time program would display objects moving across the screen at the same speed that they
would actually move.
NOTE: Webopedia definition.
ETSI
11 ETSI TR 102 643 V1.0.2 (2010-01)
real-time communication service: service with which users expect to share information instantly and continuously
with one or more other user
NOTE 1: A real-time communication service generates and delivers either text, audio, graphics, video and data or
some combination of these communication media.
NOTE 2: The information sharing process occurs either by: (1) a person interacting via technology directly to
another person (person-to-person) or; (2) a person interacting with a machine (person-to-machine).
EXAMPLE: An example real-time person-to-person communication service is videoconferencing and an
example real-time person-to-machine communication service is Live TV.
real-time computing: study of hardware and software systems that are subject to a "real-time constraint",
i.e. operational deadlines from event to system response
NOTE 1: By contrast, a non-real-time system is one for which there is no deadline, even if fast response or high
performance is desired or preferred. The needs of real-time software are often addressed in the context of
real-time operating systems, and synchronous programming languages, which provide frameworks on
which to build real-time application software.
NOTE 2: Wikipedia definition.
real-time text: service for transmitting alpha-numeric characters in real-time over a telecommunication network
Real-time Transport Protocol: standardized packet format for delivering audio and video over the Internet
NOTE 1: RTP is frequently used in streaming media systems (together with the RTSP) as well as in
videoconferencing and push to talk systems. For these it carries media streams controlled by H.323 or
Session Initiation Protocol (SIP) signalling protocols, making it the technical foundation of the Voice
over IP industry.
NOTE 2: RTP is usually used in conjunction with the Real-time Transport Control Protocol (RTCP). While RTP
carries the media streams (e.g. audio and video) or out-of-band signalling (DTMF), RTCP is used to
monitor transmission statistics and quality of service QoS information. When used in conjunction, RTP is
usually originated and received on even port numbers, whereas RTCP uses the next higher odd port
number.
NOTE 3: Wikipedia definition [i.44].
remote inspection: videoconferencing with video as data (e.g. for a remote person to see an object or environment
rather than the person(s) with whom they are talking) (sometimes also called Tele-inspection and Tele-data)
resolution: term denoting the degree of detail which can be created by a particular visual display system expressed in
pixels in x- and y-directions
satisfaction: comfort and acceptability of the work system to its users and other people affected by its use
NOTE: ISO 9241 [i.27] definition.
situation formality: relative amount of ceremonious or conventional communication versus casual or unconstrained
communication
task outcome: extent to which task performance dependent on the medium
task: what users of communicative technology actually do in order to accomplish some task goal
NOTE: In experiments tasks may be described to the participants or they are embedded in scenarios as a part of a
situation.
telephony: service for transmitting voice signals in real-time over a telecommunication network
text communication: use of a service that transmits alpha-numeric characters in real-time over a telecommunication
network
NOTE: Also known as real-time text and conversational text.
ETSI
12 ETSI TR 102 643 V1.0.2 (2010-01)
urgency: extent to which a task is particularly urgent or under particular time pressure
usability: effectiveness, efficiency, and satisfaction with which specified users achieve specified goals in particular
environments
NOTE: See ISO 9241 [i.27] definition.
user satisfaction: comfort and acceptability of the task performance to the service user
NOTE: Operationalized as the extent to which the service is assessed to be a pleasant communication medium for
the task.
video communication: use of a service that transmits voice and video signals in real-time over a telecommunication
network, i.e. use of videotelephony or videoconferencing
NOTE: For the current report the communication involves a loud-speaking audio system and not a handset.
videoconferencing: service for transmitting voice and video signals in real-time over a telecommunication network for
group communication
NOTE: In the current report the audio system is considered loud-speaking and not with a handset or headset.
video delay: time between the input of the first pixel of a particular picture at the sending end encoder and the output of
the pixel from the decoder at the receiving end
video protocol: set of rules defining the way video information is represented when transferred in a network
videotelephony: service for transmitting voice and video signals in real-time over a telecommunication network for
dyadic communication
3.2 Abbreviations
For the purposes of the present document, the following abbreviations apply:
CIF Common Intermediate Format
GSM Global System for Mobile (telephony)
ICT Information and Communications Technology
IP TV Internet Protocol Television
IPR Industrial Property Rights
IST Information Society Technologies
ITU International Telecommunication Union
LTE Long Term Evolution
MOS Mean Opinion Score
QoE Quality of Experience
QoP Quality of Perception
QoS Quality of Service
QoSD Quality of Service Delivered
QoSE Quality of Service Experienced
SLA Service Level Agreement
STF Specialist Task Force
UMTS Universal Mobile Telecommunications System
WBGTS Web-Based Guidelines and Tutorial System
4 What is Quality of Experience (QoE)?
Definitions of QoE and related concepts and work are introduced and distinguished below. In order to measure QoE
emphasis is given to the importance of performing user tests that are within a particular usage context, for which the
technical parameters of the service are known. It is argued that user experience should be measured by objective
psychological variables in addition to subjective variables.
ETSI
13 ETSI TR 102 643 V1.0.2 (2010-01)
4.1 A definition of QoE for the current document
There are different definitions of Quality of Experience across current ITU, ETSI and other literature (considered
further in clause 4.3.2.2).
In the absence of a suitable harmonised definition, the current document defines Quality of Experience (QoE) as:
"A measure of user performance based on both objective and subjective psychological measures of using an
ICT service or product."
NOTE 1: It takes into account technical parameters (e.g. QoS) and usage context variables (e.g. communication
task) and measures both the process and outcomes of communication (e.g. user effectiveness, efficiency,
satisfaction and enjoyment).
NOTE 2: The appropriate psychological measures will be dependent on the communication context. Objective
psychological measures do not rely on the opinion of the user (e.g. task completion time measured in
seconds, task accuracy measured in number of errors). Subjective psychological measures are based on
the opinion of the user (e.g. perceived quality of medium, satisfaction with a service).
EXAMPLE: A service provider may conclude that a service with a certain level of QoS used for a particular
communication situation offers users excellent QoE, whist with a different level of QoS provides
poor QoE.
Other main definitions of QoE and related concepts are considered below.
4.2 QoE and QoS
The usability of devices and services has been studied for a long time, but attention to user experience and QoE is more
recent. A description of the differences between "usability", "user experience", "user perception" and "quality of
experience" is provided in this clause.
The present document describes an approach that aims to be:
• Understandable by people who are stakeholders in user experience.
• Usable by these stakeholders.
• A reference for these stakeholders when defining "user experience" for a service, routine or product.
A similar term to QoE is QoS (Quality of Service). Among service providers, network operators and equipment
manufacturers the term QoS has been in use for a long time and has reached a high level of common understanding.
QoS work is based on technical performance (i.e. it is mainly technology-centred) whereas QoE is based on end-user
behaviour (it is user-centred) (see figure 1). Subsequent clauses of the present document will explain further the
distinction between QoE and QoS.
It is argued that work on QoS is critical, but not sufficient, for measuring user experience: QoE and QoS are distinct
and both are important and should be related.
The present document presents an approach to measuring and communicating QoE to stakeholders that intentionally
incorporates QoS in order to combine information on users and technology.
In particular, the present document develops results from ETSI STF 284 and ETSI STF 354 to define how QoS metrics
can be described in terms of QoE.
ETSI
14 ETSI TR 102 643 V1.0.2 (2010-01)

Figure 0: QoS is based on technical performance
whereas QoE should be based on end-user behaviour
4.3 Usability and user experience
During the 1980s and 1990s most user-centred work focused on the usability of information and communication
technology (ICT). Since 1998 usability has been defined in ISO 9241, [i.27], p. 2 as the extent to which a product can
be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified
context of use.
In recent years the use of ICT has extended from the workplace to the home and for applications that support leisure and
social activities in addition to work. Consequently, the concerns of human-computer interaction have evolved from a
focus on effectiveness and efficiency to user experience factors such as enjoyment, engagement and the appeal of using
and owning ICT, e.g. [i.3].
Whereas, most of the work on user experience is conducted in relation to computer applications, there is also the need
to address the user-centred development of telecommunication services. Telecommunication services are similar to
computer applications in that they require users to interact with devices and applications with hardware and software
interfaces. In addition, however, with telecommunication services users have the specific intention to communicate with
other people at distance. This communication is either direct to other people through technology (e.g. a telephone, a
videophone) or involves interaction with a machine rather than with a person (e.g. to access web content, to watch video
from a web camera).
During either person-to-person (two-way) communication or person-to-machine (one-way) communication the users
will interact with a service that will have properties that may vary and that may have an effect on users's behaviour. For
example, a delay between the arrival of audio and video information may lead to lack of lip-synchrony of the speaker as
perceived by a listener. Properties such as audio-video asynchrony, transmission delay, video frame-rate and resolution
have the potential to help or hinder communication. These technical properties relate to Quality of Service (QoS).
For communication services it is possible to distinguish three approaches to quality:
• Quality of Service (QoS).
• User-perceived QoS (or Quality of Perception (QoP)).
• Quality of Experience (QoE).
ETSI
15 ETSI TR 102 643 V1.0.2 (2010-01)
As will be described next, whereas QoS is a mainly technology-centred approach to quality, user-perceived QoS (or
QoP) and QoE are complementary user-centred approaches.

Figure 1: The main concepts for user-centred applications and services
4.3.1 Technology-centred approach: Quality of Service (QoS)
The technology-centred approach mainly emphasises the concept of QoS and has its strongest reference from the ITU
(International Telecommunications Union). The ITU Recommendation E.800 [i.31] is the key reference and states that
QoS is the:
"Totality of characteristics of a telecommunications service that bear on its ability to satisfy stated
and implied needs of the user of the service." (ITU-T Recommendation E.800 [i.31], p. 3).
Although the ITU definition refers to user satisfaction, QoS is mainly used by technicians to define technical parameters
of telecommunication applications such as network delay and packet loss.
In addition, a focus on user satisfaction is rather limited because it is only one of many measures of user behaviour with
a communication service. For example, other measures include the time taken to perform a communication task (a
measure of efficiency) and the accuracy with which a task is completed (a measure of effectiveness).
4.3.2 User-centred approaches: QoE and User-perceived QoS
There are two main user-centred approaches:
• Quality of Perception (QoP) or User-perceived QoS.
• Quality of Experience (QoE).
As outlined below, these two approaches are mainly distinguishable by the type of data collected from users.
4.3.2.1 Quality of Perception (QoP)
The QoP approach is primarily concerned with the detectability of a change in quality or the acceptability of a quality
level. Typically the user perception of quality is measured with a subjective rating scale analysed as a "Mean Opinion
Score" (MOS).
ETSI
16 ETSI TR 102 643 V1.0.2 (2010-01)
For example, results of MOS ratings for network performance parameters such as transmission bit rate and packet loss
for audiovisual QoS for communication over IP networks are summarised in ES 202 667 [i.10] and a set of ETSI
standards provide terminal equipment requirements which enable manufacturers and service providers to enable good
quality end-to-end speech performance as perceived by the user [i.11], [i.12], [i.13] and [i.14].
The MOS method is standardised within the ITU (e.g. ITU-R Recommendation BS 1534-1) [i.28].
The ITU also offers a related definition of QoS experienced/perceived by customer/user (QoSE)
(ITU-T Recommendation E.800) [i.31]:
A statement expressing the level of quality that customers/users believe they have experienced.
NOTE 1: QoSE
...