SIST-TP TR 101 329-6 V2.1.1:2004
(Main)Telecommunications and Internet Protocol Harmonization Over Networks (TIPHON) Release 3; End-to-end Quality of Service in TIPHON systems; Part 6: Actual measurements of network and terminal characteristics and performance parameters in TIPHON networks and their influence on voice quality
Telecommunications and Internet Protocol Harmonization Over Networks (TIPHON) Release 3; End-to-end Quality of Service in TIPHON systems; Part 6: Actual measurements of network and terminal characteristics and performance parameters in TIPHON networks and their influence on voice quality
To include further measurement results
Harmonizacija telekomunikacij in internetnega protokola prek omrežij (TIPHON), 3. izdaja - Kakovost storitve od konca do konca v sistemih TIPHON - 6. del: Dejanske meritve parametrov karakteristik in zmogljivosti omrežja in terminalov v omrežjih TIPHON ter njihov vpliv na kakovost govora
General Information
Standards Content (Sample)
SLOVENSKI STANDARD
SIST-TP TR 101 329-6 V2.1.1:2004
01-april-2004
Harmonizacija telekomunikacij in internetnega protokola prek omrežij (TIPHON), 3.
izdaja - Kakovost storitve od konca do konca v sistemih TIPHON - 6. del: Dejanske
meritve parametrov karakteristik in zmogljivosti omrežja in terminalov v omrežjih
TIPHON ter njihov vpliv na kakovost govora
Telecommunications and Internet Protocol Harmonization Over Networks (TIPHON)
Release 3; End-to-end Quality of Service in TIPHON systems; Part 6: Actual
measurements of network and terminal characteristics and performance parameters in
TIPHON networks and their influence on voice quality
Ta slovenski standard je istoveten z: TR 101 329-6 Version 2.1.1
ICS:
33.020 Telekomunikacije na splošno Telecommunications in
general
SIST-TP TR 101 329-6 V2.1.1:2004 en
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.
---------------------- Page: 1 ----------------------
SIST-TP TR 101 329-6 V2.1.1:2004
---------------------- Page: 2 ----------------------
SIST-TP TR 101 329-6 V2.1.1:2004
ETSI TR 101 329-6 V2.1.1 (2002-02)
Technical Report
Telecommunications and Internet Protocol
Harmonization Over Networks (TIPHON) Release 3;
End-to-end Quality of Service in TIPHON systems;
Part 6: Actual measurements of network and terminal
characteristics and performance parameters in
TIPHON networks and their influence on voice quality
---------------------- Page: 3 ----------------------
SIST-TP TR 101 329-6 V2.1.1:2004
2 ETSI TR 101 329-6 V2.1.1 (2002-02)
Reference
RTR/TIPHON-05013
Keywords
IP, network, performance, QoS, quality, speech,
terminal, voice
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE
Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16
Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88
Important notice
Individual copies of the present document can be downloaded from:
http://www.etsi.org
The present document may be made available in more than one electronic version or in print. In any case of existing or
perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF).
In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive
within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
http://portal.etsi.org/tb/status/status.asp
If you find errors in the present document, send your comment to:
editor@etsi.fr
Copyright Notification
No part may be reproduced except as authorized by written permission.
The copyright and the foregoing restriction extend to reproduction in all media.
© European Telecommunications Standards Institute 2002.
All rights reserved.
ETSI
---------------------- Page: 4 ----------------------
SIST-TP TR 101 329-6 V2.1.1:2004
3 ETSI TR 101 329-6 V2.1.1 (2002-02)
Contents
Intellectual Property Rights.5
Foreword.5
Introduction .6
1 Scope.7
2 References.7
3 Abbreviations.9
4 List of Measurement Results.10
5 General Measurement Results.10
5.1 Subjective Testing.10
5.1.1 Simulation Results of VoIP Scenarios.10
5.1.1.1 Introduction.10
5.1.1.2 Measurement Set Up.11
5.1.1.2.1 Basics.11
5.1.1.2.2 Test Cases.11
5.1.1.3 Results.13
5.1.1.3.1 G.711 + Codec + G.711.13
5.1.1.3.2 Tandem Conditions with GSM x FR .13
5.1.2 Subjective Results on impairment effects of IP packet loss.14
5.1.2.1 Introduction.14
5.1.2.2 Measurement Set Up.14
5.1.2.2.1 Subjects.14
5.1.2.2.2 Speech Processing.14
5.1.2.2.3 Procedure.15
5.1.2.2.4 MOS Data and MNRU Conditions.15
5.1.2.3 Results.15
5.1.2.3.1 Reference Codec Conditions .15
5.1.2.3.2 Packet Loss Conditions.16
5.1.2.3.3 Tables of Results .17
5.2 Objective Testing.19
5.2.1 Objective Results of SuperOp event 1999 in Hawaii.19
5.2.1.1 Introduction.19
5.2.1.2 Measurement Set Up.19
5.2.1.3 Quality Assessment.20
5.2.1.4 Results.20
5.2.2 Objective Results of ETSI Speech Quality Test Event 2000 .20
5.2.2.1 Introduction.20
5.2.2.2 Measurement Set Up.21
5.2.2.3 Quality Assessment.22
5.2.2.4 TOSQA Results.24
5.2.2.4.1 G.711 codec.24
5.2.2.4.2 G.723 codec.24
5.2.2.4.3 G.729 codec.25
5.2.2.4.4 Summary and conclusion.25
5.3 Delay.25
5.3.1 Delay between two analogue PBX subscribers.25
5.3.1.1 Introduction.25
5.3.1.2 Measurement Set Up.26
5.3.1.3 Results.26
5.3.1.4 Conclusion.27
5.3.2 Delay between two PABX systems with analogue subscribers .27
5.3.2.1 Measurement Set Up.27
5.3.2.2 Result.28
5.3.3 Delay between two PC SW clients .28
ETSI
---------------------- Page: 5 ----------------------
SIST-TP TR 101 329-6 V2.1.1:2004
4 ETSI TR 101 329-6 V2.1.1 (2002-02)
5.3.3.1 Measurement Set Up.28
5.3.3.2 Results.28
5.3.4 Delay between an analogue PBX subscriber and a PC client .28
5.3.4.1 Measurement Set Up.28
5.3.4.2 Results.29
5.3.4.3 Conclusion.29
5.4 Echo.29
5.4.1 Echo between two analogue PBX subscribers.29
5.4.1.1 Measurement Set Up.30
5.4.1.2 Results.30
5.4.1.3 Conclusion.30
5.5 Impairment Factors.31
5.5.1 Transmission Impairments according to ITU-T G.113.31
5.6 R-Values.33
5.6.1 Analogue PBX- and SW Client scenarios.33
5.6.1.1 Introduction.33
5.6.1.2 Results.33
5.6.1.2.1 Two Gateways type A with analogue telephone sets.33
5.6.1.2.2 Gateway type A with analogue subscriber behind PBX.34
5.6.1.2.3 Gateway type B with analogue telephone sets on both sides.35
5.6.1.3 Conclusion.36
5.7 Advanced Measurement Techniques.36
5.7.1 QoS Measurements of IP-Configurations .36
5.7.1.1 Configuration and Measurement Set Up .36
5.7.1.2 Results.37
5.7.1.2.1 Parameters in Single Talk Conditions .37
5.7.1.2.2 Level dependant input-output characteristics .38
5.7.1.2.3 Echo loss and convergence.40
5.7.1.2.4 Performance in sending direction in the presence of background noise.41
5.7.1.2.5 Background noise performance in receiving direction .42
5.7.1.2.6 Evaluation of Double Talk Conditions .42
5.7.1.2.7 Relationship to subjective tests.44
5.7.1.3 Conclusion.44
5.7.2 QoS Measurements of ETSI Speech Quality Test Event 2000.44
5.7.2.1 Introduction.44
5.7.2.2 Measurement Set Up.44
5.7.2.3 Results.47
5.7.2.3.1 Packet loss.47
5.7.2.3.2 Transmission characteristics for background noise .48
5.7.2.3.3 Transmission performance under double talk performance.48
5.7.2.3.4 Detailed analysis of echo during double talk.50
5.8 Jitter Buffer.52
5.8.1 Simulation.52
5.8.1.1 Static Jitter Buffer .52
5.8.1.2 Dynamic Jitter Buffer.53
5.8.1.3 Comparison between Static and Dynamic Jitter Buffer Simulations .54
5.8.2 Practical Experiments.54
5.8.2.1 Introduction.54
5.8.2.2 Results.55
5.8.3 Conclusions.55
Annex A: Bibliography.56
History .57
ETSI
---------------------- Page: 6 ----------------------
SIST-TP TR 101 329-6 V2.1.1:2004
5 ETSI TR 101 329-6 V2.1.1 (2002-02)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (http://webapp.etsi.org/IPR/home.asp).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This Technical Report (TR) has been produced by ETSI Project Telecommunications and Internet Protocol
Harmonization Over Networks (TIPHON).
The present document is part 6 of a multi-part deliverable covering End-to-end Quality of Service in TIPHON systems,
as identified below:
TR 101 329-1: "General aspects of Quality of Service (QoS)";
TS 101 329-2: "Definition of speech Quality of Service (QoS) Classes";
TS 101 329-3: "Signalling and control of end-to-end Quality of Service (QoS)";
TS 101 329-5: "Quality of Service (QoS) measurement methodologies";
TR 101 329-6: "Actual measurements of network and terminal characteristics and performance parameters
in TIPHON networks and their influence on voice quality";
TR 101 329-7: "Design guide for elements of a TIPHON connection from an end-to-end speech transmission
performance point of view".
Quality of Service aspects of TIPHON Release 4 and 5 Systems will be covered in TS 102 024 and TS 102 025
respectively, and more comprehensive versions of the Release 3 documents listed above will be published as part of
Release 4 and 5 as work progresses.
ETSI
---------------------- Page: 7 ----------------------
SIST-TP TR 101 329-6 V2.1.1:2004
6 ETSI TR 101 329-6 V2.1.1 (2002-02)
Introduction
The present document forms one of a series of technical specifications and technical reports produced by TIPHON
Working Group 5 addressing Quality of Service (QoS) in TIPHON Systems. The structure of this work is illustrated in
Figure 1.
Introduction Definition of 5
Speech Classes
TR 101 329-1 TS 101 329-2
General Speech
QoS
Aspects
of QoS Classes
SPEC
REPORT
Generic QoS
Specific Aspects of QoS
TS 101 329-5 TR 101 329-6 TR 101 329-7
TS 101 329-3
QoS Measure- Actual Design
Control ment Test Guidelines
Methods Results
SPEC REPORT REPORT
SPEC
QoS signalling Measurement Repository of Useful info for
requirements methodologies real test results designers
Figure 1: Structure of TIPHON QoS Documentation for Release 3
ETSI
---------------------- Page: 8 ----------------------
SIST-TP TR 101 329-6 V2.1.1:2004
7 ETSI TR 101 329-6 V2.1.1 (2002-02)
1 Scope
The present document applies to IP networks that provide voice telephony in accordance with any of the TIPHON
Scenarios.
The objective with the present document is to collect all results of various VoIP speech transmission quality tests and
related information. This collection should be used for information and to review and discuss the values of the TIPHON
QoS classes which are described in WG5 documents TR 101 329-1 [3] and TR 101 329-7 [6].
The separate measurements should give a very good opportunity to understand the goal of the measurement itself and
the exact measurement Set Up conditions to understand under which framework the measurements were done.
The present document covers measurement results provided to TIPHON during the years 1999 to 2001, which have
contributed to the measurement methodologies in 101 329-5 [5] as well as providing design parameters in
101 329-7 [6].
2 References
For the purposes of this Technical Report (TR) the following references apply:
[1] ETSI ETR 275 (1996): "Transmission and Multiplexing (TM); Considerations on transmission
delay and transmission delay values for components on connections supporting speech
communication over evolving digital networks".
[2] ETSI TR 101 329 (V2.1.1): "Telecommunications and Internet Protocol Harmonization Over
Networks (TIPHON); General aspects of Quality of Service (QoS)".
[3] ETSI TR 101 329-1: "Telecommunications and Internet Protocol Harmonization Over Networks
(TIPHON) Release 3; End-to-end Quality of Service in TIPHON systems; Part 1: General aspects
of Quality of Service (QoS)".
[4] ETSI TR 101 329-2: "Telecommunications and Internet Protocol Harmonization Over Networks
(TIPHON) Release 3; End-to-end Quality of Service in TIPHON systems; Part 2: Definition of
speech Quality of Service (QoS) classes".
[5] ETSI TR 101 329-5: "Telecommunications and Internet Protocol Harmonization Over Networks
(TIPHON) Release 3; End-to-end Quality of Service in TIPHON systems; Part 5: Quality of
Service (QoS) measurement methodologies".
[6] ETSI TR 101 329-7: "Telecommunications and Internet Protocol Harmonization Over Networks
(TIPHON) Release 3; End-to-end Quality of Service in TIPHON systems; Part 7: Design guide for
elements of a TIPHON connection from an end-to-end speech transmission performance point of
view".
[7] ETSI ES 201 168 (V1.1.1): "Corporate Telecommunication Networks (CN); Transmission
characteristics of digital Private Branch eXchanges (PBXs)".
[8] ETSI GTS 06.10 (V3.2.0): "European digital cellular telecommunications system (Phase 1); GSM
Full Rate Speech Transcoding (GSM 06.10)".
[9] ITU–T Recommendation G.107: "The E-Model, a computational model for use in transmission
planning".
[10] ITU–T Recommendation G.108: "Application of the E-model: A planning guide".
[11] ITU–T Recommendation G.109 (1999): "Definition of categories of speech transmission quality".
[12] ITU–T Recommendation G.113 (2001): "Transmission impairments due to speech processing".
[13] ITU–T Recommendation G.131 (1996): "Control of talker echo".
[14] ITU–T Recommendation G.165 (1993): "Echo cancellers".
ETSI
---------------------- Page: 9 ----------------------
SIST-TP TR 101 329-6 V2.1.1:2004
8 ETSI TR 101 329-6 V2.1.1 (2002-02)
[15] ITU–T Recommendation G.168 (2000): "Digital network echo cancellers".
[16] ITU–T Recommendation G.711 (1988): "Pulse code modulation (PCM) of voice frequencies".
[17] ITU–T Recommendation G.721 (1988): "32 kbit/s adaptive differential pulse code modulation
(ADPCM)".
[18] ITU–T Recommendation G.723.1 (1996): "Dual rate speech coder for multimedia communications
transmitting at 5.3 and 6.3 kbit/s".
[19] ITU–T Recommendation G.726 (1990): "40, 32, 24, 16 kbit/s adaptive differential pulse code
modulation (ADPCM)".
[20] ITU–T Recommendation G.727 (1990): "5-, 4-, 3- and 2-bit/ sample embedded adaptive
differential pulse code modulation (ADPCM)".
[21] ITU–T Recommendation G.728 (1992): "Coding of speech at 16 kbit/s using low-delay code
excited linear prediction".
[22] ITU–T Recommendation G.729 (1996): "Coding of speech at 8 kbit/s using conjugate-structure
algebraic-code-excited linear-prediction (CS-ACELP)".
[23] ITU–T Recommendation G.729A (Annex A - 1996): "Reduced complexity 8 kbit/s CS-ACELP
speech codec".
[24] ITU–T Recommendation G.729B (Annex B - 1996): "A silence compression scheme for G.729
optimized for terminals conforming to Recommendation V.70".
[25] ITU–T Recommendation H.323 (2000): "Packet-based multimedia communications systems".
[26] ITU-T Recommendation P.57: "Artificial ears".
[27] ITU-T Recommendation P.58: "Head and torso simulator for telephonometry".
[28] ITU–T Recommendation P.64 (1999): "Determination of sensitivity/frequency characteristics of
local telephone systems".
[29] ITU-T Recommendation P.501: "Test signals for use in telephonometry".
[30] ITU-T Recommendation P.502: "Objective test methods for speech communication systems, using
complex test signals".
[31] ITU-T Recommendation P.581: "Use of head and torso simulator (HATS) for hands-free terminal
testing".
[32] ITU–T Recommendation P.800 (1996): "Methods for subjective determination of transmission
quality".
[33] ITU–T Recommendation P.861 (1998): "Objective quality measurement of telephone-band
(300-3 400 Hz) speech codecs".
[34] ITU–T Recommendation Q.13/12: "Rapporteur of Question 13 inside ITU-T Study Group 12;
ETSI TIPHON 17TD135".
[35] SG 16, Santiago, Chile, 17-28 May 1999; D.249 (WP 3/16): "A High Quality Low-Complexity
Algorithm For Frame Erasure Concealment (FEC) With G.711" (Source: AT&T).
[36] T1A1.7/99-012r3; Jul-28-1999: "Draft Proposed American National Standard - A Packet Loss
Concealment Technique for Use with ITU-T Recommendation G.711" (Source: AT&T).
[37] ETSI ETS 300 581-2: "Digital cellular telecommunications system (Phase 2) (GSM); Half rate
speech; Part 2: Half rate speech transcoding (GSM 06.20 version 4.3.1)".
[38] ETSI EN 300 726: "Digital cellular telecommunications system (Phase 2+) (GSM); Enhanced Full
Rate (EFR) speech transcoding (GSM 06.60 version 8.0.1 Release 1999)".
ETSI
---------------------- Page: 10 ----------------------
SIST-TP TR 101 329-6 V2.1.1:2004
9 ETSI TR 101 329-6 V2.1.1 (2002-02)
[39] ITU-T Recommendation P.862 (2001): "Perceptual evaluation of speech quality (PESQ), an
objective method for end-to-end speech quality assessment of narrowband telephone networks and
speech codecs".
[40] ITU-T Recommendation G.723: "Extensions of Recommendation G.721 adaptive differential
pulse code modulation to 24 and 40 kbit/s for digital circuit multiplication equipment application".
[41] ITU-T Recommendation G.723.1-A: "Speech coders : Silence compression scheme".
[42] ITU-T Recommendation P.50: "Artificial voices".
[43] ITU-T Recommendation G.114: "One-way transmission time".
3 Abbreviations
For the purposes of the present document, the following abbreviations apply:
ACR Absolute Category Rating
ASL Active Speech Level
CAS Communication Analysis System (HEAD acoustics test system)
CSS Composite Source Signal
EC Echo Canceller
EP Error Pattern
GSM Global System for Mobile communications
GSM EFR GSM Enhanced Full Rate Speech Coder
GSM FR GSM Ful
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.