ETSI TR 102 854 V1.1.1 (2011-12)

User Group; The assessment of the overall Quality of Services (QoS) as perceived by the users; Review of practical examples of service QoS assessments

DTR/USER-00037

General Information

• Status: Published
• Publication Date: 04-Dec-2011
• Technical Committee: USER - User Group
• Current Stage: 12 - Completion
• Due Date: 25-Nov-2011
• Completion Date: 05-Dec-2011

Standards Content (Sample)

Technical Report
User Group;
The assessment of the overall Quality of Services (QoS)
as perceived by the users;
Review of practical examples of service QoS assessments

�
2 ETSI TR 102 854 V1.1.1 (2011-12)

ReRefeferreencncee
DDTTRR//UUSERSER-00037 -00037
KKeeyywwords ords
QQooS, interfS, interfacaceses

ETETSISI
650 R650 Roouutte de dees s LucLuciolioleess
F-0692F-06921 Sop1 Sophhia Aia Anntiptipoolilis Cs Ceeddeexx - FR - FRANANCCEE

TTeel.: +33 l.: [phone] 422 00 00  Fax  Fax:: +33 4 [phone] 16 16

SiretSiret N N°° 348 348 62 623 3 562562 000 00017 17 - N- NAAFF 7 742 C42 C
AAssssoocciiatatiioon à bn à buutt n noonn l luuccrratatifif eennrregegiissttrrééee à l à laa
SouSouss-Pré-Préffeeccttureure de de G Grraassssee ( (06)06) N N°° 78037803//8888

ImporImportatant noticnt noticee
IndivIndiviidudual al copcopiieses of t of the prhe preseesenntt doc documeumennt t can can be dbe doowwnnloadeloaded fromd from::
http://whttp://wwwww.etsi.org.etsi.org
TThe presehe presennt dot docucumenmentt may may b bee mad madee av availailablable ine in mor moree th than an one eone ellecectrontronic vic veersrsionion or i or inn pri prinnt. In t. In anyany ca case se of exof existinistingg or or
perceperceivived ded diifferefferencnce in e in cocontentntents betws between een susuch vch veersrsionionss, th, the re reeferenferencce ve veerrssionion is is the the Porta Portabbllee D Doocucumenmentt For Formmat (PDat (PDFF)).
In caIn case se of dof diispspute, ute, the rthe reefereferenncce se shalhall bel be the the prinprintingting on ET on ETSI printSI printeers rs of thof the PDe PDF vF veersrsion ion kkept onept on a a spespeccifiificc n neetwtwork drivork drivee
wwiithin ETthin ETSI SecSI Secrretaretariat.iat.
UUssers ers of thof the pree pressenent dot docucumenmentt sho shouuld ld be awbe aware tare that that the he ddooccuumenmentt m maayy be s be subjubject ect to revto reviisision or on or chachange nge of sof sttatuatuss. .
InformInformatiation oon onn the the curr current ent statstatus ofus of thi thiss a and otnd other ETher ETSI doSI docucumentments is iss av avaiailablablle ate at
http://http://portaportal.etl.etsi.si.oorg/trg/tb/stb/statuatuss/s/statustatus.asaspp
If yIf you finou findd err erroors rs in thin the pre preseesenntt doc documeumennt, pt, plleaease se sensendd y your cour coommmment tent too on one of e of the the follfollowowing serving servicices:es:
http://http://portaportal.etl.etsi.si.oorg/rg/cchahaircor/Eircor/ETTSI_supportSI_support.asaspp
CopyCopyrriight Notificght Notificaattionion
NNoo part part maymay be r be reproeproducduced exed except cept as as authauthoorizrized ed byby w wrritten pitten peermirmissission.on.
TThe copyhe copyrighright ant and thed the fore foregoigoinngg restr restriictction ion exextend to retend to reprodproduucctiotionn in in all all me media.dia.

©© Europea European Tn Teeleclecoommmmununicaicattiions Stons Standandards ards InsInsttituitutte 2e 2011.011.
All rigAll righhtsts res reseervrved.ed.

TMTM TMTM TMTM
DECTDECT , , PLUGTESTSPLUGTESTS , , UMUMTSTS and t and the EThe ETSI logSI logoo are are T Trrade Made Maarks ofrks of ET ETSI SI regisregisttereeredd f foor the r the benbenefit efit of itof its Ms Meembersmbers.
TMTM
3G3GPPPP and and LTELTE™ ™ are Tare Trrade ade MMaarks of ETrks of ETSI regisSI registtereeredd for t for thhe bene benefit efit of iof itts Ms Meembermbers ans andd
of the of the 3G3GPP OPP Orrganizganizaattiiononal Pal Paartnerrtners.s.
GSMGSM®® and t and the Ghe GSSMM logo are logo are TTrrade Made Maarks regirks registerstereedd an and od owwned byned by the G the GSSMM Associa Associattionion.
ETSI
3 ETSI TR 102 854 V1.1.1 (2011-12)
Contents
Intellectual Property Rights . 5
Foreword . 5
Introduction . 5
1 Scope . 6
2 References . 6
2.1 Normative references . 6
2.2 Informative references . 6
3 Definitions, symbols and abbreviations . 7
3.1 Definitions . 7
3.2 Symbols . 7
3.3 Abbreviations . 8
4 Overall organization of the QoS information . 9
4.1 Segmentation of the results . 9
4.2 Implementation of the EG 202 934 principles . 9
4.3 Principles for graphical representation . 9
4.3.1 Radar type graphical representation . 10
4.3.2 OVV type graphical representation . 10
4.4 Processing of the results . 11
5 Representation of the results within each CRS . 11
5.1 Sales - Preliminary information (PI) . 11
5.1.1 Reference threshold of PI QoS parameter . 12
5.1.2 Highest QoS boundary of the range of PI QoS parameter . 12
5.1.3 Lowest QoS boundary of the range of PI QoS parameter . 12
5.1.4 Aggregation of the PI QoS assessment results . 13
5.1.4.1 Comparison Table . 13
5.1.4.2 QoS indexes . 13
5.1.4.3 Radar type graphical representation . 14
5.1.4.4 OVV type graphical representation . 15
5.1.4.5 Conclusion . 15
5.2 Sales - Contract Establishment . 15
5.2.1 Reference threshold of each QoS parameter . 16
5.2.2 Highest QoS boundary of the range of each QoS parameter . 16
5.2.3 Lowest QoS boundary of the range of each QoS parameter . 16
5.2.4 Aggregation of the Contract Establishment QoS assessment results . 17
5.2.4.1 Comparison Table . 17
5.2.4.2 QoS indexes . 17
5.2.4.3 Radar type graphical representation . 18
5.2.4.4 OVV type graphical representation . 19
5.2.4.5 Conclusion . 19
5.3 Service management - Service provisioning . 19
5.3.1 Reference threshold of each QoS parameter . 20
5.3.2 Highest QoS boundary of the range of each QoS parameter . 20
5.3.3 Lowest QoS boundary of the range of each QoS parameter . 20
5.3.4 Aggregation of the Provisioning QoS assessment results . 21
5.3.4.1 Comparison Table . 21
5.3.4.2 QoS indexes . 21
5.3.4.3 Radar type graphical representation . 22
5.3.4.4 OVV type graphical representation . 23
5.3.4.5 Conclusion . 23
5.4 Service use (technical QoS). 23
5.4.1 Reference threshold of each QoS parameter . 24
5.4.2 Highest QoS boundary of the range of each QoS parameter . 24
5.4.3 Lowest QoS boundary of the range of each QoS parameter . 24
ETSI
4 ETSI TR 102 854 V1.1.1 (2011-12)
5.4.4 Aggregation of the Service use QoS assessment results . 25
5.4.4.1 Comparison Table . 25
5.4.4.2 QoS indexes . 25
5.4.4.3 Radar type graphical representation . 26
5.4.4.4 OVV type graphical representation . 27
5.4.4.5 Conclusion . 27
5.5 Service management - Customer Support . 27
5.5.1 Reference threshold of each QoS parameter . 28
5.5.2 Highest QoS boundary of the range of each QoS parameter . 29
5.5.3 Lowest QoS boundary of the range of each QoS parameter . 29
5.5.4 Aggregation of the Customer Support QoS assessment results . 29
5.5.4.1 Comparison Table . 29
5.5.4.2 QoS indexes . 29
5.5.4.3 Radar type graphical representation . 31
5.5.4.4 OVV type graphical representation . 32
5.5.4.5 Conclusion . 32
5.6 Service management - Repair services . 32
5.6.1 Reference threshold of each QoS parameter . 33
5.6.2 Highest QoS boundary of the range of each QoS parameter . 33
5.6.3 Lowest QoS boundary of the range of each QoS parameter . 33
5.6.4 Aggregation of the Repair service QoS assessment results . 34
5.6.4.1 Comparison Table . 34
5.6.4.2 QoS indexes . 34
5.6.4.3 Radar type graphical representation . 35
5.6.4.4 OVV type graphical representation . 36
5.6.4.5 Conclusion . 36
5.7 Service management - Metering, Charging and Billing . 36
5.7.1 Reference threshold of each QoS parameter . 37
5.7.2 Highest QoS boundary of the range of each QoS parameter . 37
5.7.3 Lowest QoS boundary of the range of each QoS parameter . 38
5.7.4 Aggregation of the Billing QoS assessment results . 38
5.7.4.1 Comparison Table . 38
5.7.4.2 QoS indexes . 38
5.7.4.3 Radar type graphical representation . 39
5.7.4.4 OVV type graphical representation . 40
5.7.4.5 Conclusion . 40
5.8 Service management - Cessation . 40
5.8.1 Reference threshold of each QoS parameter . 41
5.8.2 Highest QoS boundary of the range of each QoS parameter . 41
5.8.3 Lowest QoS boundary of the range of each QoS parameter . 41
5.8.4 Aggregation of the Cessation QoS assessment results . 41
5.8.4.1 Comparison Table . 41
5.8.4.2 QoS indexes . 42
5.8.4.3 Radar type graphical representation . 43
5.8.4.4 OVV type graphical representation . 44
5.8.4.5 Conclusion . 44
6 Representation of the QoS results for the various CRS of a particular service . 44
6.1 QoS indexes and Comparison Table . 45
6.2 Graphical representation . 46
6.2.1 Radar type chart . 46
6.2.2 OVV type graphical representation . 47
6.2.3 Conclusion . 47
Annex A: Bibliography . 48
History . 49

ETSI
5 ETSI TR 102 854 V1.1.1 (2011-12)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (http://ipr.etsi.org).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This Technical Report (TR) has been produced by ETSI User Group (USER).
Introduction
The present document details some examples of QoS assessments results from available surveys in 2010.
These examples are used to illustrate the principles described in EG 202 934 [i.4].
The present document takes into account the following CRS:
1) Sales - Preliminary information (PI);
2) Service management - Service provisioning;
3) Service use (technical QoS);
4) Customer Support;
5) Repair services;
6) Metering, Charging, Billing and Cessation.
NOTE: To ensure the figures are clearly visible, they can be found, in their original format, in archive
tr_102854v010101p0.zip which accompanies the present document.
ETSI
6 ETSI TR 102 854 V1.1.1 (2011-12)
1 Scope
The present document aims at explaining how the methodology described in EG 202 843 [i.2] can be implemented
using QoS assessments from different sources resulting of various surveys among end-users to compare the QoS of
services provided by different Service Providers (SP). Some of these results, used as examples in EG 202 934 [i.4] are
more detailed in the present document.
The data used for this report have been collected from actual users according to best practices in this area. Nevertheless,
this document should not to be taken as an actual comparison of SP but rather as a tutorial about how such comparison
should be done provided fully comparable data is available.
2 References
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
Referenced documents which are not found to be publicly available in the expected location might be found at
http://docbox.etsi.org/Reference.
NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee
their long term validity.
2.1 Normative references
The following referenced documents are necessary for the application of the present document.
Not applicable.
2.2 Informative references
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] ETSI EG 202 009-1: "User Group; Quality of Telecom Services; Part 1: Methodology for
identification of parameters relevant to the Users".
[i.2] ETSI EG 202 843: "User Group; Quality of ICT Services; Definitions and Methods for Assessing
the QoS parameters of the Customer Relationship Stages other than utilization".
[i.3] ETSI EG 202 057: "Speech and multimedia Transmission Quality (STQ); User related QoS
parameter definitions and measurements".
[i.4] ETSI EG 202 934: "User Group; The assessment of the overall Quality of Services (QoS) as
perceived by the users; Definition of QoS indexes for all the customer relationship stages".
[i.5] ETSI ES 202 765-2: "Speech and multimedia Transmission Quality (STQ); QoS and network
performance metrics and measurement methods; Part 2: Transmission Quality Indicator combining
Voice Quality Metrics".
[i.6] ETSI ES 202 765-4: "Speech and multimedia Transmission Quality (STQ); QoS and network
performance metrics and measurement methods; Part 4: Indicators for supervision of Multiplay
services".
[i.7] ETSI TS 102 852: "User Group; Quality of ICT Services; Assessment process of the QoS
parameters of the customer relationship stages".
[i.8] ITU-T Recommendation P.505: "One-view visualization of speech quality measurement results".
ETSI
7 ETSI TR 102 854 V1.1.1 (2011-12)
3 Definitions, symbols and abbreviations
3.1 Definitions
For the purposes of the present document, the terms and definitions given in EG 202 934 [i.4] and the following apply.
3.2 Symbols
For the purposes of the present document, the symbols given in EG 202 934 [i.4] and the following apply:
P100 Frequency of customer complaints about PI [N/t]: Number of customers' complaints about PI per
million subscribers
P101a Integrity of PI [OR]: Content - Was the relevant information provided as you expected?
P101b Integrity of PI [OR]: Language - Was the information provided clear and understandable without
any ambiguity?
P101c Integrity of PI [OR]: Style - How would you rate the overall style, presentation and
professionalism of the preliminary information provided?
P102 Pricing transparency [OR]: Did you find the pricing information comprehensible?
P103 Availability of PI [%]: Could you retrieve the preliminary information easily?
P200 Frequency of customer complaints about contract establishment [N/t]: Number of customers'
complaints about contract establishment per million subscribers
P201 Integrity of contract information [OR]: How would you rate the integrity of the contractual
document?
P202 Compliance of contractual terms with PI [%]: Was the contract document compliant to the
previously provided preliminary information?
P203 Flexibility for customisation before contract [OR]: How would you rate the flexibility of your
service provider to customise the contract before signature e.g. by applying options?
P204 Ease and flexibility to amend terms after formal contract [OR]: How would you rate the flexibility
of your service provider to further adapt the contract after signature e.g. by applying options?
P300 Frequency of customer complaints about provisioning [N/t]: Number of customers' complaints
about provisioning per million subscribers
P303a Provisioning time [Time & %] - existing subscriber line
P303b Provisioning time [Time & %] - new subscriber line
P309a Successful provisioning within a specified period [%] - existing subscriber line
P309b Successful provisioning within a specified period [%] - new subscriber line
P600 Frequency of customer complaints about service support [N/t]: Number of customers complaints
about service support per million subscribers
P628a Response time of the technical support [Time & %]
P628b Response time of the technical support [Time & %]
P661 Accessibility of the complaint management desk [%]: Concerning your latest attempt to access the
complaint management desk of your service provider: Did you succeed in accessing it?
P662 Recognition of the customer complaints [%]: Concerning your latest attempt to access the
complaint management desk of your service provider: Was your complaint accepted?
P663 Complaint solutions not complete and correct first time [%]: Was the complaint solved to your
satisfaction at the first attempt by the service provider?
P664 Complaint solutions achieved within a specified period [%]: Concerning your latest accepted
complaint: Was the complaint finally solved to your satisfaction by the service provider?
P665 Integrity of complaint resolution [%]: Concerning your latest accepted complaint: Was your
complaint resolved correctly?
P666a Customer perception of the complaint management [OR]: Customer perception of complaint
management (Assurance): How would you rate the service provider's complaint management
related to assurance at all?
P666b Customer perception of the complaint management [OR]: Customer perception of complaint
management (Empathy): How would you rate the service provider's complaint management
related to empathy at all?
P666c Customer perception of the complaint management [OR]: Customer perception of complaint
management (Responsiveness): How would you rate the service provider's complaint management
related to responsiveness at all?
ETSI
8 ETSI TR 102 854 V1.1.1 (2011-12)
P667 Overall quality of the complaint management process [OR]: How would you rate the overall
handling of the complaint management process?
P706a Fault repair time [Time & %] - Time for 95 % fault repair
P706b Fault repair time [Time & %] - % faults repaired within a 48 hours delay.
P707 Frequency of customer complaints related to repair services [N/t]: Number of customers
complaints related to repair services per million subscribers
P800 Frequency of customer complaints about billing [N/t]: Number of customers complaints about
billing per million subscribers
P801 Accessibility of the tariff information [%]: Concerning your latest attempt to access your
provider's tariff information: Were you able to access the tariff information?
P802 Successful notification of exceeding billing budget [%]: If you are using a notification service
when you reach a predefined budget level: Concerning your latest exceeding of budget: Were you
notified accordingly when you exceeded your budget?
P804 Accessibility of the account management [%]: Concerning your latest attempt to access the
account status at your service provider: Did you succeed in accessing it?
P806 Timeliness of bill delivery [%]: Did you receive all the expected bills throughout the last
6 months?
P807 Bill delivery delay [Time]: If you experienced a delay in bill delivery: How many days was the bill
delayed?
P808 Late notification of amount due [%]: Has the bill been received before the direct debit was
executed?
P809 Modes of billing information transfer [Number]: How many ways do you have to access your
accounting information?
P810 Bill correctness complaints [%]: Percentage of bills resulting in a customer complaint per point of
billing per year.
P1004a Contractual cessation achieved within 10 days [%]
P1004b Contractual cessation achieved [%]: time needed (days) to achieved 95 % of cessations requested
P1004c Contractual cessation achieved [%]: time needed (days) to achieved 99 % of cessations requested
P1008 Frequency of customer complaints related to cessation [N/t]: Number of customers' complaints
related to cessation per million subscribers
PT000 Frequency of customer complaints related to use of service [N/t]: Number of customers complaints
related to use of service per million subscribers
PT001a Fault report rate per fixed access lines
PT001b Fault report rate per fixed access lines within 30 days after the delivery.
PT002a unsuccessful call ratio - domestic calls
PT002b unsuccessful call ratio - international calls
PT003a call set up time - domestic calls
PT003b call set up time - international calls
PT004 Speech Quality (MOS)
3.3 Abbreviations
For the purposes of the present document, the abbreviations given in EG 202 934 [i.4] and the following apply:
CRS Customer Relationship Stage
OVV One-View Visualization
QoS Quality of Service
SP Service Provider
ToIP Telephony over IP
ETSI
9 ETSI TR 102 854 V1.1.1 (2011-12)
4 Overall organization of the QoS information
4.1 Segmentation of the results
As explained in EG 202 934 [i.4], a useful means to provide a synthetic overview of the pieces of information related to
users' satisfaction is to segment them with respect to the Customer Relationship Stages, i.e. as detailed in
EG 202 009-1 [i.1]:
Table 4.1.1: Customer Relationship Stage
Customer Relationship Stage CRS detail
Preliminary information, advertisement
Sales
Establishment of the contract (Terms and conditions)
Installation
Service provisioning
Activation and acceptance
Customer initiative
Service alteration / Technical upgrade
Provider initiative
Documentation for service activation and set-up
Service Documentation for service use
management Service support Technical support
Commercial support
Complaint management
Repair/Troubleshooting
Metering/Charging/Billing
Cessation
Network/service management by the customer
Access
Use of Service Bearer service
Service utilization
Service usage
Presentation and user interface

This means to have a single representation (with an indication of the related spreading) for each of these stages.
4.2 Implementation of the EG 202 934 principles
The purpose of the present document is to detail for each QoS parameter of each CRS how the principles of
EG 202 934 [i.4] can be used:
1) to choose the reference thresholds according to clause 6 of EG 202 934 [i.4], in order to make available a table
showing which SP provide a QoS egal or above these thresholds.
2) then to choose a realistic range of variation of the values of the QoS parameters within a given CRS so that the
differences between the results of the various providers rightly represent significant differences from the
perceived QoS viewpoint. This range will be used for the scales of the axis of the graph comparing the results
of the SP within a CRS as well as for the calculation of the indexes used for the aggregation of the results for a
whole service. These ranges are defined according to clauses 8.2.2 and 8.2.3 of EG 202 934 [i.4].
3) and finally to determine the QoS indexes of each CRS according to clause 8.2.4 of EG 202 934 [i.4] in order to
provide a graph displaying the QoS assessed for the selected services of the providers under study.
4.3 Principles for graphical representation
Whatever aggregation scenario is chosen, an appropriate graphical representation of the results is probably the best
means to help the users to identify which services are able to ensure the expected QoS.
ETSI
10 ETSI TR 102 854 V1.1.1 (2011-12)
The graphical representations proposed hereafter (see examples in figure 1) are all based on the principles proposed in
EG 202 843 [i.2] and detailed in EG 202 934 [i.4].
Two types of chart are proposed:
1) a radar type graphical representation; and
2) a "cobweb" type graphical representation based on the "one-view visualization (OVV) methodology"
described in the ITU-T Recommendation P.505 [i.8].
Each type of chart has advantages and drawbacks and of course, other representations can be used depending of the
communication target.
4.3.1 Radar type graphical representation
• The graph is of the radar type.
• A red area drawn in the middle of the chart delineates the best practices quality measure.
• Each QoS parameter is represented by a dot on an axis with a different scale for each QoS parameter.
• The location of this dot depends on the axis scales defined by the values for the border of the red area
(reference threshold) and both ends of the axis.
• The value on the border of the red area is defined by the reference threshold set according to one of the
possible methods described in clause 6.
The value for the top end of the axis (highest QoS) is defined by the highest value set for the agreed range (see
clause 4.2).
• The value for the origin of the axis (lowest QoS) is defined by the lowest value set for the agreed range
(see clause 4.2).
• As a consequence, depending on the type of QoS parameter, the scale can be increasing or decreasing from the
centre to the outside end and the scale of the lower part of the axis can be different from that of the upper part.
• The principle is that the farther the dot from the centre, the better the QoS. Additionally when a dot is outside
the red area, this means the QoS is compliant with the best practices and on the opposite when a dot is within
the red area the QoS is below these best practices.
• Where appropriate, this type of display allows for a representation of the extremes of the distribution of the
assessment results.
Therefore, it is very easy to check the parameters outside the red area, and that they are compliant with the best
practices. ®
This type of graphic display is using a freely available software (Google chart ).
4.3.2 OVV type graphical representation
• Although the ITU-T Recommendation P.505 [i.8] is focusing on the representation of speech quality
measurement results, it was found useful to use the "one-view visualization methodology" described in this
recommendation to represent the CRS quality results as an alternative to that given in clause 4.3.1. This
representation is based on circle segments ("pie diagram", "star plot") according to the following principles
(see figure 2 example).
• Similar to a "cobweb" representation the axes are shown with a common origin.
• By means of a suitable axis scaling, a concentric circle (in red colour) around the origin can be defined which
delineates the best practices quality measure. Falling below this segment size (radius) indicates a non-
compliance with this limit value.
• The value on the border of the red area is defined by the reference threshold set according to one of the
methods described in clause 6.
ETSI
11 ETSI TR 102 854 V1.1.1 (2011-12)
• Each QoS parameter is represented by a circle segment (in green colour) whose radius match the parameter
value with a different scale for each QoS parameter.
• The radius of this segment depends on the axis scales defined by the values for the border of the red area and
both ends of the axis.
• The value for the top end of the axis (highest QoS) is defined by the highest value set for the agreed range.
• The value for the origin of the axis (lowest QoS) is defined by the lowest value set for the agreed range.
• As a consequence, depending on the type of QoS parameter, the scale can be increasing or decreasing from the
centre to the outside end.
• The principle is that the bigger the segment, the better the QoS. Additionally when a segment completely hides
the red area, this means the QoS is compliant with the best practices and on the opposite when a segment let a
part of the red area appear, the QoS is below these best practices.
• If needed various colours can be given to the segments to highlight which ones are most important than the
others.
A tool enabling to draw the chart according to the OVV methodology is expected to be made publicly available by the
ITU-T in the coming year. In the meantime a tentative link to such a tool is provided in annex A.
4.4 Processing of the results
The assessment of the QoS parameters is described in several standards, e.g. EG 202 057 [i.3], EG 202 843 [i.2],
TS 102 852 [i.7], EG 202 009-1 [i.1], ES 202 765-2 [i.5] & 4 [i.6], etc. but to make easier the comparison of different
SP, it is crucial to have a consistent presentation of these results. In this aim, the principle was taken to display the
results with values increasing with the QoS, as customers better understand such a presentation mode. This principle has
led in some cases to a processing of the raw data resulting from the assessments made according to the standards.
Details are provided in clause 5.
5 Representation of the results within each CRS
The present document uses the principles described in EG 202 934 [i.4] for a detailed comparison of different SP using
various available QoS assessments. Nevertheless, as explained in the scope, due to the lack of comparability of the data
used, it should not to be taken as an actual comparison of SP but rather as a tutorial about how such comparison could
be done provided fully comparable data are available.
Even if the results have been obtained from a sample of 7 SP, only 4 are used in the present document.
5.1 Sales - Preliminary information (PI)
The QoS parameters used for this stage are the following:
P100 Frequency of customer complaints about PI [N/t]:
Measure: Number of customers' complaints about PI per million subscribers
P101a Integrity of PI [OR]: Content
Question: Was the relevant information provided as you expected? Measure: % NO
P101b Integrity of PI [OR]: Language
Question: Was the information provided clear and understandable without any ambiguity?
Measure: % NO
P101c Integrity of PI [OR]: Style
Question: How would you rate the overall style, presentation and professionalism of the
preliminary information provided? Measure: % NO
P102 Pricing transparency [OR]:
Question: Did you find the pricing information comprehensible? Measure: % NO
P103 Availability of PI [%]:
Question: Could you retrieve the preliminary information easily? Measure: % NO
ETSI
12 ETSI TR 102 854 V1.1.1 (2011-12)
The values obtained for 4 SP are as follows:
Table 5.1.1: Preliminary Information (PI) results
P100 P101a P101b P101c P102 P103

SP A 2,48 0,0 % 0,0 % 5,6 % 5,9 % 5,9 %
SP B
4,29 20,7 % 17,9 % 16,0 % 19,2 % 26,9 %
SP C 3,30 31,7 % 36,1 % 34,2 % 40,7 % 45,8 %
SP D 3,10 30,4 % 31,8 % 30,2 % 35,4 % 46,9 %
QoS max 2,48 0,0 % 0,0 % 5,6 % 5,9 % 5,9 %
QoS min 4,29 31,7 % 36,1 % 34,2 % 40,7 % 46,9 %

5.1.1 Reference threshold of PI QoS parameter
In this example, the mean values of each QoS parameter in the sample are taken as the reference thresholds.
Table 5.1.2: PI QoS reference thresholds
P100 P101a P101b P101c P102 P103

Threshold 3,3 21 % 21 % 22 % 25 % 31 %

5.1.2 Highest QoS boundary of the range of PI QoS parameter
Although no SP was able to reach the ideal value 0 for all these QoS parameters, this target has been taken as upper
threshold for these QoS parameters since it is not too far from the current practices and figure out an aim to look at.
Table 5.1.3: PI Highest QoS boundaries
P100 P101a P101b P101c P102 P103

Highest QoS
0 0 % 0 % 0 % 0 % 0 %
boundaries
5.1.3 Lowest QoS boundary of the range of PI QoS parameter
For all the QoS parameters in this sample, the ratio between the QoS minimum and the reference threshold is lower
than 2. Therefore higher values will be taken for the lowest QoS boundaries so that a ratio of 2 is obtained.
Table 5.1.4: PI Lowest QoS boundaries
P100 P101a P101b P101c P102 P103

Lowest QoS
6,6 42 % 42 % 44 % 50 % 62 %
boundaries
ETSI
13 ETSI TR 102 854 V1.1.1 (2011-12)
5.1.4 Aggregation of the PI QoS assessment results
5.1.4.1 Comparison Table
Taking into account the values from the previous tables, the results reaching a QoS above or equal to the reference
thresholds are shown in green in the table 5.1.5
Table 5.1.5: PI QoS better than the reference threshold
P100 P101a P101b P101c P102 P103

SP A
2,48 0,0 % 0,0 % 5,6 % 5,9 % 5,9 %
SP B 4,29 20,7 % 17,9 % 16,0 % 19,2 % 26,9 %
SP C 3,30 31,7 % 36,1 % 34,2 % 40,7 % 45,8 %
SP D
3,10 30,4 % 31,8 % 30,2 % 35,4 % 46,9 %
Threshold 3,3 21 % 21 % 22 % 25 % 31 %

This shows clearly that the best PI QoS for this service is provided by SP A.
5.1.4.2 QoS indexes
To determine the QoS indexes for each parameter, a calculation has to be made on the basis of the previous tables
according to the principles given in EG 202 934 [i.4] and summarized in clause 4.2.
The results appear in table 5.1.6:
Table 5.1.6: PI QoS indexes
P100 P101a P101b P101c P101 P102 P103 Overall

SP A 1,2 2,0 2,0 1,7 1,9 1,8 1,8 1,7
SP B
0,7 1,0 1,1 1,3 1,1 1,2 1,1 1,1
SP C 1,0 0,5 0,3 0,4 0,4 0,4 0,5 0,6
SP D 1,1 0,6 0,5 0,6 0,6 0,6 0,5 0,7

In this table the P101 values are the mean values of P101a, P101b and P101c. The overall value is the mean value of
P100, P101, P102 and P103.
These values will be used to draw the graphical representations showing the PI QoS of the service provided by the 4 SP
in clause 5.1.4.3 while the overall QoS indexes will be used for the graphical representations showing the QoS of all the
CRS of a service given in clause 6.
...