SIST-V ETSI/EG 202 765-2 V1.1.1:2009

SLOVENSKI STANDARD
SIST-V ETSI/EG 202 765-2 V1.1.1:2009
01-maj-2009
9LGLNLREGHODYHSUHQRVDLQNDNRYRVWLJRYRUD6740HWRGHPHWULNHLQPHUMHQMD
NDNRYRVWLVWRULWHY4R6LQ]PRJOMLYRVWLRPUHåLMGHO.D]DOQLNSUHQRVQH
NDNRYRVWLYNOMXþQR]PHWULNRNDNRYRVWLJRYRUD
Speech Processing, Transmission and Quality Aspects (STQ) - QoS and network

performance metrics and measurement methods - Part 2: Transmission Quality Indicator

2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

The present document may be made available in more than one electronic version or in print. In any case of existing or

perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF).

In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive

Users of the present document should be aware that the document may be subject to revision or change of status.

Information on the current status of this and other ETSI documents is available at

If you find errors in the present document, please send your comment to one of the following services:

DECT , PLUGTESTS , UMTS , TIPHON , the TIPHON logo and the ETSI logo are Trade Marks of ETSI registered

3GPP is a Trade Mark of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners.

GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.

Intellectual Property Rights ................................................................................................................................ 5

Foreword ............................................................................................................................................................. 5

1 Scope ........................................................................................................................................................ 6

2 References ................................................................................................................................................ 6

2.1 Normative references ......................................................................................................................................... 6

2.2 Informative references ........................................................................................................................................ 6

3 Abbreviations ........................................................................................................................................... 7

4 Introduction .............................................................................................................................................. 8

5 Measurement type .................................................................................................................................... 9

6 Voice quality scale ................................................................................................................................. 10

7 List of indicators ..................................................................................................................................... 10

7.1 Post Dialling Delay .......................................................................................................................................... 10

7.2 Media establishment delay ............................................................................................................................... 10

7.3 Unsuccessful call ratio ...................................................................................................................................... 11

7.4 Premature release probability ........................................................................................................................... 11

7.5 Level of active speech signal at reception ........................................................................................................ 11

7.6 Noise level at reception .................................................................................................................................... 12

7.7 Noise to signal ratio at reception ...................................................................................................................... 12

7.8 Speech signal attenuation (or gain) after transmission ..................................................................................... 13

7.9 Talker echo delay ............................................................................................................................................. 14

7.10 Talker echo attenuation .................................................................................................................................... 15

7.11 Listening speech quality ................................................................................................................................... 15

7.12 Listening speech quality stability ..................................................................................................................... 16

7.13 End to end delay ............................................................................................................................................... 17

7.14 End to end delay variation ................................................................................................................................ 18

7.15 Frequency responses at the reception ............................................................................................................... 18

8 Measurement frequency ......................................................................................................................... 19

9 Duration of test calls............................................................................................................................... 19

10 Measurement configurations .................................................................................................................. 19

10.1 VoIP services .................................................................................................................................................... 19

10.2 VoIP services in triple play context .................................................................................................................. 20

11 Measurement locations and their distribution ........................................................................................ 20

11.1 Measurement location requirements ................................................................................................................. 20

11.2 Method to determine measurement locations ................................................................................................... 21

12 Results presentation ................................................................................................................................ 22

12.1 One-view visualization of performances .......................................................................................................... 22

12.1.1 Pie diagram with all indicators ................................................................................................................... 22

12.1.2 Pie diagram with mandatory indicators ...................................................................................................... 23

12.2 Non-compliant limits for result visualization ................................................................................................... 23

13 Publication of the results ........................................................................................................................ 24

Annex A: Indicator stability formulation ............................................................................................ 25

A.1 Presentation ............................................................................................................................................ 25

A.2 Formulation ............................................................................................................................................ 25

A.3 Graphic illustration of the formulation ................................................................................................... 26

A.4 Some examples of stability indicator calculated on Listening Speech Quality ...................................... 28

Annex B: Calibration to take into account the frequency response of transducers ........................ 30

B.1 Method presentation ............................................................................................................................... 30

B.1.1 Sending ............................................................................................................................................................. 30

B.1.2 Sending ............................................................................................................................................................. 31

B.1.3 Global communication ..................................................................................................................................... 31

B.1.4 Applications ..................................................................................................................................................... 31

Annex C: Echo presentation.................................................................................................................. 32

C.1 Talker echo ............................................................................................................................................. 32

C.2 Listener echo .......................................................................................................................................... 32

Annex D: Examples of measurement point distribution .................................................................... 33

D.1 Example of France.................................................................................................................................. 33

D.2 Example of Switzerland ......................................................................................................................... 34

History .............................................................................................................................................................. 37

IPRs essential or potentially essential to the present document may have been declared to ETSI. The information

pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found

in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in

respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web

Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee

can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web

This ETSI Guide (EG) has been produced by ETSI Technical Committee Speech and multimedia Transmission Quality

The present document aims at identifying and defining indicators and methodologies for a use in a context of end-user

In this context the measurements and metric determinations are perform by analysing signals accessible on user-end

services and not on the network. In order to mirror the reality in terms of access to the services at the user-end

measurements and analysis are perform on electrical signal that exclude the electro-acoustic part of the end equipment

but the probe adaptation to electric interface of the end user equipment much take into account the electro-acoustic

References are either specific (identified by date of publication and/or edition number or version number) or

• Non-specific reference may be made only to a complete document or a part thereof and only in the following

- if it is accepted that it will be possible to use all future changes of the referenced document for the

Referenced documents which are not found to be publicly available in the expected location might be found at

NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee

The following referenced documents are indispensable for the application of the present document. For dated

references, only the edition cited applies. For non-specific references, the latest edition of the referenced document

The following referenced documents are not essential to the use of the present document but they assist the user with

regard to a particular subject area. For non-specific references, the latest version of the referenced document (including

[i.1] ITU-T Recommendation P.800: "Methods for subjective determination of transmission quality".

[i.2] ITU-T Recommendation P.862: "Perceptual evaluation of speech quality (PESQ): An objective

method for end-to-end speech quality assessment of narrow-band telephone networks and speech

[i.3] ITU-T Recommendation P.862.1: "Mapping function for transforming P.862 raw result scores to

[i.4] ITU-T Recommendation P.862.2: "Wideband extension to Recommendation P.862 for the

[i.5] ITU-T Recommendation P.862.3: "Application guide for objective quality measurement based on

[i.7] ITU-T Recommendation E.800: "Terms and definitions related to quality of service and network

[i.8] ITU-T Recommendation E.845: "Connection accessibility objective for the international telephone

[i.9] ETSI EG 201 769: "Speech Processing, Transmission and Quality Aspects (STQ); QoS parameter

defini tions and measurements; Parameters for voice telephony service required under the ONP

[i.10] ITU-T Recommendation P.56: "Objective measurement of active speech level".

[i.11] ITU-T Recommendation O.41: "Psophometer for use on telephone-type circuits".

[i.15] ITU-T Recommendation P.505: "One-view visualization of speech quality measurement results".

[i.16] ETSI EG 201 377 (all parts): "Speech Processing, Transmission and Quality Aspects (STQ);

[i.17] ITU-T Recommendation H.323: "Packet-based multimedia communications systems".

[i.18] ITU-T Recommendation H.225.0: "Call signalling protocols and media stream packetization for

ITU-T International Telecommunication Union - Telecommunication standardization sector

The assessment of transmission quality based on voice quality metrics is already addressed in several standards at ETSI

(e.g. EG 201 377 [i.16] series) and elsewhere (mostly ITU-T recommendations from the P and G series). These

different documents are addressing the measurement methodologies in terms of metrics, threshold, data acquisition or

The objective of the present document is to complement this material with practical requirements of use in the context

of service verification and benchmark on a large and representative scale from the point of view of the end-users or of

the regulatory authorities. This has been made necessary by the current or recent evolutions of the telecommunication

• the competitive environment, in particular in voice services, where public protocols with high quality services

have been replaced by a multitude of service providers with less guarantees, and where clients can very easily

• the development of time varying quality in telecommunications, first in mobile offers (due to mobility and

• the cohabitation, interaction and competition between services based on different technologies.

Voice transmission quality is now recognized as a differentiating factor, but it remains very difficult to quantify.

To achieve the goal mentioned beforehand, there are several existing possibilities, not fully satisfying:

• Customer surveys. This is by far the cheapest way to assess the perception of end users. But the bias

introduced by the other factors like price, as well as the fact that voice quality itself is rarely questioned as

itself or in a satisfactory way (one never knows before a survey what are the problems encountered by end

• Pseudo-subjective tests, with a few human testers assessing the quality of real links in several situations. This

method has the major drawback of its lack of reproducibility, and is often applied without using the standard

metrics and quality scales that can be found in standards like ITU-T Recommendation P.800 [i.1]. It is also

very long to run and not really cheap in the current competitive context where so many offers have to be

assessed. And it is not easily applicable in a context of quality changing over time.

• Objective tests. This is the most reliable way, although it is also based on sampling and can cost a lot of money

The present document assumes that this last family of methodology answers the needs of a reliable comparison of

What definitely matters is the point of view of the end-users. What they perceive is not only the result of the

transmission of a signal across a network; the processing of this signal at the sending and at the receiving sides has also

a big importance. Therefore, it seems obvious not to use passive network monitoring systems to assess end-to-end voice

quality, but rather active systems simulating the behaviour of the end users, including the terminal. A big advantage of

such an approach is that it is highly technical and protocol agnostic, and therefore compliant with the expectations of

users, which are not judging voice quality of PSTN, GSM or VoIP services following different criteria.

Last important aspect that is addressed in the present document is the practical organization of measurement campaigns

in order to get a realistic and reliable vision of the services as perceived by the end-users. In particular, the questions of

the periodicity of measurement and of the geographical coverage (i.e. more generally the sampling approach).

In order to mirror the reality in terms of access to the services, a reliable measurement or supervision system should

provide the possibility to collect information from probes or robots adapted to the most common interfaces available.

• analogue access (for the simulation of PSTN or of analogue phones behind an ATA box or an ADSL modem);

• ethernet with IP phone termination (SIP, ITU-T Recommendation H.323 [i.17], MGCP, etc.).

Any combination of end-to-end connection between the types of access mentioned here have to be considered when a

measurement campaign is scheduled. Nevertheless, of course, there are practical limitations:

• the number of measurements for a given type of access should be in proportion with its level of use in the real

• the number of probes and of measurement results available will be adapted to the real needs as well as to the

capacity (mostly in terms of cost and of processing capability) of the entity running these measurements.

Figure 4.1: Possible configurations and interfaces in context of user characterization

To perform quality services assessments, there are two different methods: intrusive and non intrusive measurements.

The non intrusive measurements are not really adapted to end user surveys because it requires to install probes at the

The intrusive measurements are more adapted to end user surveys because probe connection with end user terminals is

easier. Compared to non intrusive measurements, the intrusive methods have an advantage: the opportunity for voice

quality assessment to use models with references such as ITU-T Recommendation P.862[ i.2]

(see also ITU-T recommendations P.862.1 [i.3], P.862.2 [i.4] and P.862.3 [i.5] concerning mapping functions and

application guide) which give results close to subjective perception of the speech quality.

In this context, the intrusive measurements using models working with references for speech quality assessment will be

It is important to consider that nowadays telephony has entered an era where traditional narrowband services will

cohabit with new services offering wideband audio capacities. For end-users, these are not separated kinds of services.

Therefore, the assessment of transmission quality of voice should now be based on common metrics and objective

quality levels and scales, in replacement of the existing narrow-band only ones. In this context, it is appropriate to use

the MOS-LQOM scale to characterize voice quality of narrow-band services and wideband services.

The indicators proposed for the context of end-user quality survey of voice services are:

Definition Post Dialling Delay (PDD) evaluates service availability to set up calls in an acceptable

delay. It is linked to the service architecture complexity, and to the performance of the

Post Dialling Delay is the time interval between the end of dialling by the caller and the

Assessment method Indicator determines sequentially from the two access of call configuration. This indicator

Comment This indicator has to be separated between call types (IP to IP, IP to PSTN, IP to mobile,

The objective set up in for universal telephony service has been set up to 2 900 ms in the

Definition Time determines on one of the two access of the communication, between off hock of the

Assessment method Indicator determines sequentially from the two access of call configuration.

On an IP access this indicator may be assessed by using a non-intrusive probe, such as a

protocol analyser. Media establishment delay may be evaluated through the analysis of

media flows and signalling. For ITU-T Recommendation H.323 protocol [i.17] the flow

ITU-T Recommendation H.225.0 [i.18] "CONNECT" message and the arrival of the first

Comment This indicator has to be separated between caller and called site for a detailed analysis.

Definition Ratio of unsuccessful calls to the total number of call attempts in a specified time period.

An unsuccessful call is a call attempt to a valid number, properly dialled following dial

tone, where neither called party busy tone, nor ringing tone, nor answer signal, is

recognized on the access line of the calling user within 30 seconds from the instant when

the address information required for setting up a call is received by the network.

Assessment method Indicator determines sequentially from the two access of call configuration.

Standardization ITU-T Recommendation E.800 [i.7], ITU-T Recommendation E.845 [i.8],