ETSI EG 202 765-3 V1.1.1 (2009-12)
Speech and multimediaTransmission Quality (STQ); QoS and network performance metrics and measurement methods; Part 3: Network performance metrics and measurement methods in IP networks
Speech and multimediaTransmission Quality (STQ); QoS and network performance metrics and measurement methods; Part 3: Network performance metrics and measurement methods in IP networks
DEG/STQ-00104-3
Kakovost prenosa govora in večpredstavnih vsebin (STQ) - Metrika kakovosti storitev (QoS) in zmogljivosti omrežja ter merilne metode - 3. del: Metode metrike in merjenja zmogljivosti omrežij v omrežjih IP
Ta dokument zagotavlja pregled splošnih definicij metrik in specifikacij metod merjenja, na katerih temelji skupna uporabnost merjenja zmogljivosti omrežja (z drugim imenom merjenje kakovosti storitev (QoS)). To težavo obravnavata dva različna organa za standardizacijo, Delovna skupina za internetno inženirstvo (IETF) in Mednarodna telekomunikacijska zveza – Sektor za standardizacijo telekomunikacij (ITU-T). Ta dokument obravnava naslednje točke:
• Raziskava obstoječih standardov IETF glede zmogljivosti omrežja in kako jih je mogoče uporabiti za celovito merjenje zmogljivosti omrežja. Področje tega dela je tudi razprava o razmerju teh standardov ter standardov ITU-T in ETSI.
• Predstavitev in primerjava definicij metrik, ki se uporabljajo za določanje in ocenjevanje zmogljivosti v omrežjih IP. Metrike, obravnavane v tem dokumentu, določata delovna skupina IETF IPPM in študijska skupina ITU-T 12. Poleg primerjave različnih definicij ta dokument podaja tudi smernice o tem, katera metrika je ustreznejša za določeno vrsto uporabe, konfiguracijo ali scenarij.
• Opredelitev metod merjenja za izbrane metrike zmogljivosti v omrežjih IP, pri čemer so zajete tako aktivne kot pasivne metode. Podane so pojasnjevalne smernice.
OPOMBA: Vsi razdelki besedila v preostanku tega dokumenta, ki so v narekovajih (»«) in oblikovani ležeče, označujejo citate, vzete neposredno iz referenčnih dokumentov.
General Information
Buy Standard
Standards Content (Sample)
ETSI EG 202 765-3 V1.1.1 (2009-12)
ETSI Guide
Speech and multimedia Transmission Quality (STQ);
QoS and network performance metrics and
measurement methods;
Part 3: Network performance metrics and
measurement methods in IP networks
---------------------- Page: 1 ----------------------
2 ETSI EG 202 765-3 V1.1.1 (2009-12)
Reference
DEG/STQ-00104-3
Keywords
performance, QoS
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE
Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16
Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88
Important notice
Individual copies of the present document can be downloaded from:
http://www.etsi.org
The present document may be made available in more than one electronic version or in print. In any case of existing or
perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF).
In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive
within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
http://portal.etsi.org/tb/status/status.asp
If you find errors in the present document, please send your comment to one of the following services:
http://portal.etsi.org/chaircor/ETSI_support.asp
Copyright Notification
No part may be reproduced except as authorized by written permission.
The copyright and the foregoing restriction extend to reproduction in all media.
© European Telecommunications Standards Institute 2009.
All rights reserved.
TM TM TM TM
DECT , PLUGTESTS , UMTS , TIPHON , the TIPHON logo and the ETSI logo are Trade Marks of ETSI registered
for the benefit of its Members.
TM
3GPP is a Trade Mark of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners.
LTE™ is a Trade Mark of ETSI currently being registered
for the benefit of its Members and of the 3GPP Organizational Partners.
GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.
ETSI
---------------------- Page: 2 ----------------------
3 ETSI EG 202 765-3 V1.1.1 (2009-12)
Contents
Intellectual Property Rights . 5
Foreword . 5
Introduction . 5
1 Scope . 7
2 References . 7
2.1 Normative references . 7
2.2 Informative references . 8
3 Definitions, symbols and abbreviations . 9
3.1 Definitions . 9
3.2 Symbols . 9
3.3 Abbreviations . 10
4 Performance Metrics Definitions and Measurement Methods . 10
4.1 One Way Delay vs. IP Packet Transfer Delay . 11
4.1.1 IETF Definition . 11
4.1.2 ITU-T Definition . 12
4.1.3 Comparison and Recommendations . 12
4.1.4 Active Measurement Method . 13
4.1.5 Passive Measurement Method . 14
4.2 Round Trip Delay . 15
4.2.1 IETF Definition . 15
4.2.2 ITU-T Definition . 16
4.2.3 Comparison and Recommendations . 16
4.2.4 Active Measurement Method . 16
4.2.5 Passive Measurement Method . 17
4.3 IP Packet Delay Variation vs. End-to-end 2-point IP Packet Delay Variation . 19
4.3.1 IETF Definition . 19
4.3.2 ITU-T Definition . 20
4.3.3 Comparison and Recommendations . 21
4.3.4 Active Measurement Method . 22
4.3.5 Passive Measurement Method . 22
4.4 One Way Packet Loss vs. IP Packet Loss Ratio . 22
4.4.1 IETF Definition . 23
4.4.2 ITU-T Definition . 23
4.4.3 Comparison and Recommendations . 23
4.4.4 Active Measurement Method . 23
4.4.5 Passive Measurement Method . 24
4.5 Connectivity vs. IP Service Availability . 24
4.5.1 IETF Definition . 24
4.5.2 ITU-T Definition . 25
4.5.3 Comparison and Recommendations . 25
4.5.4 Active Measurement Method . 26
4.5.5 Passive Measurement Method . 26
5 Other Metrics . 26
5.1 Data and Packet Volume . 26
5.2 Packet Reordering . 27
5.3 Bandwidth Capacity, Available Bandwidth, and Utilization . 27
5.4 Bulk Transport Capacity . 27
5.5 Loss Patterns . 27
5.6 RTCP reported metrics . 28
6 Overview of Network Performance Relevant Standard Bodies and Working Groups . 29
6.1 IETF . 29
6.1.1 IPPM (IP Performance Metrics) Working Group . 29
ETSI
---------------------- Page: 3 ----------------------
4 ETSI EG 202 765-3 V1.1.1 (2009-12)
6.1.2 IPFIX (IP Flow Information eXport) Working Group . 29
6.1.3 PSAMP (Packet SAMPling) Working Group . 30
6.2 ITU-T . 30
6.2.1 Study Group 12 (Performance and quality of service) . 30
6.2.2 Study Group 15 (Optical and other transport network infrastructures) . 30
Annex A: Bibliography . 32
History . 33
ETSI
---------------------- Page: 4 ----------------------
5 ETSI EG 202 765-3 V1.1.1 (2009-12)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (http://webapp.etsi.org/IPR/home.asp).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This ETSI Guide (EG) has been produced by ETSI Technical Committee Speech and multimedia Transmission Quality
(STQ).
The present document is part 3 of a multi-part deliverable covering the QoS and network performance metrics and
measurement methods, as identified below:
Part 1: "General considerations";
Part 2: "Transmission Quality Indicator combining Voice Quality Metrics";
Part 3: "Network performance metrics and measurement methods in IP networks";
Part 4: "Indicators for supervision of Multiplay services".
Introduction
The need to define Internet performance metrics and measurement methodologies stems from the need to compare
different measurements and to measure performance with a reproducible and unambiguous methodology, independent
from transmission technology and implementation details. Both the ITU-T Study Group 12 and the IETF IPPM
Working Group have produced such definitions (see table 1), although each with a different emphasis closely linked to
the historical background of both organizations. The ITU has its origins in telephony, while the IETF has a data
networking background. Whereas the ITU emphasizes the evaluation of a service and its quality, the IETF measures the
network and wants to provide the IT-community with an accurate, common understanding and measurement of the
performance and reliability the Internet [i.3].
In most cases this results in different terminology rather than in incompatibilities; most differences in approach and
emphasis serve the different intended use of each metric, but have no operational significance. In some cases the
terminology used by each organisations can be mapped to the other, while in some others there is only approximate
equivalence (e.g. ITU network section versus an IPPM cloud; one focuses on corresponding events while the other
measures the fate of a single packet). Other terms have no correspondence. For example, ITU-T Recommendation
I.380 [i.38] has a notion of an IP packet transfer reference event while IPPM defines "wire time".
Other differences between IETF and ITU-T metrics result from their intended application. ITU-T metrics seek to
provide a common language for providers to communicate about performance, so the ITU-T metrics do not concentrate
on performance within a single network, while the IETF focuses on performance measurement protocols and
implementation. ITU-T seeks to evaluate service and to exclude unfair use, while the IETF seeks to measure network
quantities and avoid biased measurement results. Due to their respective backgrounds, the ITU generally produces
statistical metrics geared towards a quantitative representation of the complete end-to-end user experience while the
IETF IPPM working group mainly focuses more on statistical metrics which provide a detailed technical view of
different aspects of transmission quality along the network path.
ETSI
---------------------- Page: 5 ----------------------
6 ETSI EG 202 765-3 V1.1.1 (2009-12)
Table 1: Overview of Relevant Standards
IETF RFCs ITU-T Recommendations
Framework
RFC 2330 [i.3] Y.1540 [i.1], sections 1 through 5
Loss RFC 2680 [i.6] Y.1540 [i.1], section 5.5.6
G.1020 [i.23]
Delay
RFC 2679 [i.5] (One-way) Y.1540 [i.1], section 6.2
RFC 2681 [i.7] (Round Trip) G.1020 [i.23]
G.114 [i.22] (One-way)
Delay Variation RFC 3393 [i.10] Y.1540 [i.1], section 6.2.2
G.1020 [i.23]
Connectivity / Availability RFC 2678 [i.4] Y.1540 [i.1], section 7
Loss Patterns RFC 3357 [i.9] G.1020 [i.23]
Packet Reordering RFC 4737 [i.15] Y.1540 [i.1], sections 5.5.8.1 and 6.6
Packet Duplication Y.1540 [i.1], sections 5.5.8.3, 5.5.8.4,
6.8, and 6.9
Link/Path Bandwidth Capacity, Link
RFC 5136 [i.31]
Utilization, Available Capacity
Bulk Transport Capacity RFC 3148 [i.8], RFC 5136 [i.31]
The goal of the present document is to define network performance metrics for applications sensitive to quality of
service such as Voice over IP, referring to the existing work produced by both IETF and ITU-T. The present document
highlights the differences between the two standards and provides guidelines on resolving these differences, when they
are due to addressing different goals.
The scope of the present document is limited to IP performance metrics relevant for data transmission over IP-based
networks for use in QoS sensitive applications. For each addressed metric, the document recommends one or more
measurement methods. The document only focuses on intrinsic network QoS metrics; perceived QoS metrics applicable
for voice transmission are out of scope of the present document.
The remainder of the present document is organised as follows: Clause 4 describes the definitions of the most important
performance metrics as defined by the standard bodies and methods for measuring them, and discusses the applicability
of the definitions and the differences between them. Clause 5 discusses other metrics applicable to QoS. Finally,
clause 6 gives an overview of relevant QoS measurement standards, which can be used in end to end performance
evaluation.
ETSI
---------------------- Page: 6 ----------------------
7 ETSI EG 202 765-3 V1.1.1 (2009-12)
1 Scope
The present document provides an overview of the common metric definitions and measurement method specifications
upon which the interoperability of network performance measurement (also called QoS measurement) is based. Two
different standardisation bodies, the Internet Engineering Task Force (IETF) and the International Telecommunication
Union - Telecommunication Standardization Sector (ITU - T), have addressed this issue. The present document
addresses the following points:
• Survey the existing network performance related IETF standards and how these standards can be applied to
end-to-end network performance measurements. The scope of this work is also to discuss the relationship of
those standards to those of ITU-T and ETSI.
• Discuss and compare definitions of metrics used to specify and assess performance in IP networks. The
metrics addressed in the present document are those defined by the IETF IPPM working group and ITU-T
Study Group 12. Besides comparing the different definitions, the present document gives applicability
guidelines on which metric is more appropriate for a particular application, configuration or scenario.
• Define measurement methods for selected performance metrics in IP networks, addressing both active and
passive methods. Clarifying guidelines are given.
NOTE: All text sections in the remainder of the present document which are enclosed in quotation marks (") and
formatted in italic style denote citations taken verbatim from referenced documents.
2 References
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific.
• For a specific reference, subsequent revisions do not apply.
• Non-specific reference may be made only to a complete document or a part thereof and only in the following
cases:
- if it is accepted that it will be possible to use all future changes of the referenced document for the
purposes of the referring document;
- for informative references.
Referenced documents which are not found to be publicly available in the expected location might be found at
http://docbox.etsi.org/Reference.
NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee
their long term validity.
2.1 Normative references
The following referenced documents are indispensable for the application of the present document. For dated
references, only the edition cited applies. For non-specific references, the latest edition of the referenced document
(including any amendments) applies.
Not applicable.
ETSI
---------------------- Page: 7 ----------------------
8 ETSI EG 202 765-3 V1.1.1 (2009-12)
2.2 Informative references
The following referenced documents are not essential to the use of the present document but they assist the user with
regard to a particular subject area. For non-specific references, the latest version of the referenced document (including
any amendments) applies.
[i.1] ITU-T Recommendation Y.1540: "Internet protocol data communication service - IP packet
transfer and availability performance parameters".
[i.2] Void.
[i.3] IETF RFC 2330: "Framework for IP Performance Metrics". V. Paxson, G. Almes, J. Mahdavi,
M. Mathis. May 1998.
[i.4] IETF RFC 2678: "IPPM Metrics for Measuring Connectivity". J. Mahdavi, V. Paxson.
September 1999.
[i.5] IETF RFC 2679: "A One-way Delay Metric for IPPM". G. Almes, S. Kalidindi, M. Zekauskas.
September 1999.
[i.6] IETF RFC 2680: "A One-way Packet Loss Metric for IPPM". G. Almes, S. Kalidindi,
M. Zekauskas. September 1999.
[i.7] IETF RFC 2681: "A Round-trip Delay Metric for IPPM". G. Almes, S. Kalidindi, M. Zekauskas.
September 1999.
[i.8] IETF RFC 3148: "A Framework for Defining Empirical Bulk Transfer Capacity Metrics".
M. Mathis, M. Allman. July 2001.
[i.9] IETF RFC 3357: "One-way Loss Pattern Sample Metrics". R. Koodli, R. Ravikanth. August 2002.
[i.10] IETF RFC 3393: "IP Packet Delay Variation Metric for IP Performance Metrics (IPPM)".
C. Demichelis, P. Chimento. November 2002.
[i.11] Void.
[i.12] Void.
[i.13] Void.
[i.14] IETF RFC 4656: "A One-way Active Measurement Protocol (OWAMP)". S. Shalunov,
B. Teitelbaum, A. Karp, J. Boote, M. Zekauskas. September 2006.
[i.15] IETF RFC 4737: "Packet Reordering Metrics". A. Morton, L. Ciavattone, G. Ramachandran,
S. Shalunov, J. Perser. November 2006.
[i.16] Void.
[i.17] Void.
[i.18] IETF RFC 5101: "Specification of the IPFIX Protocol for the Exchange of IP Traffic Flow
Information". B. Claise, S. Bryant, S. Leinen, T. Deitz, B.Trammell. January 2008.
[i.19] IPFIX Architecture. N.Brownlee et Al. Internet-Draft, work in progress.
[i.20] IETF RFC 5102: "IPFIX Information Model". J. Quittek et Al. January 2008.
[i.21] "IPFIX Applicability Statement". T. Zseby, E. Boschi, N.Brownlee, B. Claise. Internet-Draft, work
in progress.
[i.22] ITU-T Recommendation G.114 (05/03): "One-way transmission time".
[i.23] ITU-T Recommendation G.1020 (07/06): "Performance parameter definitions for quality of speech
and other voiceband applications utilizing IP networks".
ETSI
---------------------- Page: 8 ----------------------
9 ETSI EG 202 765-3 V1.1.1 (2009-12)
[i.24] IETF RFC 3917: "Requirements for IP Flow Information Export". J. Quittek, T. Zseby, B. Claise,
S. Zander. October 2004.
[i.25] Void.
[i.26] Void.
[i.27] Void.
[i.28] Void.
[i.29] "Reporting Metrics: Different Points of View", A. Morton, G. Ramachandran, G. Maguluri, work
in progress, draft-morton-ippm-reporting-metrics-02.
NOTE: http://tools.ietf.org/html/draft-morton-ippm-reporting-metrics-02, and the derived presentation "Reporting
Metrics: Different Points of View" presented by Al Morton on IETF66 July 2006,
http://www3.ietf.org/proceedings/06jul/slides/ippm-2.pdf.
[i.30] IETF RFC 3611: "RTP Control Protocol Extended Reports (RTCP XR)", T. Friedman, R. Caceres,
A. Clark. November 2003.
[i.31] IETF RFC 5136: "Defining Network Capacity", P. Chimento, J. Ishac. February 2008.
[i.32] IETF RFC 2581: "TCP Congestion Control", M. Allman, V. Paxson, W. Stevens. April 1999.
[i.33] IETF RFC 5357: "A Two-Way Active Measurement Protocol (TWAMP)", K. Hedayat,
R. Krzanowski, A. Morton, K. Yum, J. Babiarz. October 2008.
[i.34] IETF RFC 1122: "Requirements for Internet Hosts - Communication Layers", R. Braden ed.
October 1989.
[i.35] IETF RFC 3550: "User Accounts for UCSB On-Line System".
[i.36] IETF RFC 1633: "Integrated Services in the Internet Architecture: an Overview".
[i.37] IETF RFC 2216: "Network Element Service Specification Template".
[i.38] ITU-T Recommendation I.380: "Internet protocol data communication service - IP packet transfer
and availability performance parameters".
3 Definitions, symbols and abbreviations
3.1 Definitions
For the purposes of the present document, the terms and definitions given in RFC 2330 [i.3], ITU-T Recommendation
G.1020 [i.23] and RFC 2680 [i.6] apply.
3.2 Symbols
For the purposes of the present document, the following symbols apply:
T, t Time
T Time threshold
max
dT Time difference
ETSI
---------------------- Page: 9 ----------------------
10 ETSI EG 202 765-3 V1.1.1 (2009-12)
3.3 Abbreviations
For the purposes of the present document, the following abbreviations apply:
ASON Automatically Switched Optical Network
ATM Asynchronous Transfer Mode
BTC Bulk transport Capacity
DNS Domain Name System
ESD End System Delay
FTP File Transfer Protocol
HTTP HyperText Transfer Protocol
ICMP Internet Control Message Protocol
IETF Internet Engineering Task Force
IPDV IP Packet Dealy Variation
IPFIX IP Flow Information eXport
IPLR IP Packet Loss Ratio
IPPM IP Performance Metrics
IPTD IP Packet Transfer Delay
ITU-T International Telecommunication Union - Telecommunication standardisation sector
MIB Management Information Base
NSE Network Section Ensemble
OP Observation Point
OWAMP One Way Active Measurement Protocol
OWD One Way Delay
PDV Packet Delay Variation
PIA Percent IP service Availability
PON Passive Optical Network
PSAMP Packet SAMPling
QoS Quality of Service
RFC Request For Comments
RTCP Real Time Control Protocol
RTD Round Trip Delay
RTP Real-Time Transport Protocol
RTT Round Trip Time
SDH Synchronous Digital Hierarchy
SLA Service Level Agreement
TCP Transmission Control Protocol
TWAMP Two-Way Active Measurement Protocol
UTC Coordinated Universal Time
VoIP Voice over IP
4 Performance Metrics Definitions and Measurement
Methods
This clause provides common definitions for network performance metrics. These definitions are based, whenever
possible, on existing definitions proposed by other relevant standard bodies such as IETF or ITU-T. Note that the
different definitions of similar metrics are in most cases compatible, that is, semantically equivalent or easily
convertible into one another.
ETSI
---------------------- Page: 10 ----------------------
11 ETSI EG 202 765-3 V1.1.1 (2009-12)
For each metric, passive and active measurement methods are defined. Note that we chose to focus on commonly used
measurement methods rather than on standards; when a standard exists, a reference is provided as well. Note also that
throughout this text we refer for each metric to active and passive measurements in the following way:
• Active measurements
Active measurement methods inject traffic into the network and compute traffic metrics based on monitoring
the injected traffic or the response to the injected traffic. Active test traffic may perturb other traffic already
present on the network; therefore its scheduling and volume should be carefully configured. One can
distinguish active monitoring systems based on the position of sender and receiver and the observed traffic;
this is specified in detail for the considered metrics in the following text.
• Passive measurements
Passive measurements provide information about traffic in the observed network by capturing all or a selected
subset of the IP packets traversing a monitoring point. Since no test traffic is generated, passive measurements
can only be applied when the traffic of interest is already present on the network. The physical deployment of
monitoring probes in the network can be realised in different ways, depending on the metrics of interest, but
also on the network technology, e.g. via a physical line splitter, via a normal client connection in broadcast
networks, or via a dedicated monitoring port on a switch or router.
4.1 One Way Delay vs. IP Packet Transfer Delay
Delay is used to measure the expected time for an IP packet to traverse the network from one host to another. Delay is
applicable to QoS for latency-sensitive protocols. The IETF and ITU-T metrics for measuring delay are essentially
compatible, though there are minor differences; the details of these metrics are given in this clause.
4.1.1 IETF Definition
RFC 2679 [i.5] distinguishes between a "singleton analytic metric", called Type-P-One-way-Delay, and a "sample",
called Type-P-One-way-Delay-Poisson-Stream. The singleton is introduced to measure a single observation of one-way
delay, while the sample is used to measure a sequence of singleton delays measured at times taken from a Poisson
process. Based on these samples, several statistics are defined, such as Type-P-One-way-Delay-Percentile,
Type-P-One-way-Delay-Median, Type-P-One-way-Delay-Minimum, and Type-P-One-way-Delay-Inverse-Percentile.
Since the value of many of these metrics depends on the type of the IP packet used to perform the measurements, IPPM
metrics definitions include the generic notion of "a packet of type P", which should be further specified when making
actual measurements.
RFC 2679 [i.5] defines:
"For a real number dT, >>the *Type-P-One-way-Delay* from Src to Dst at T is dT<< means that Src sent the first bit
of a Type-P packet to Dst at wire-time* T and that Dst received the last bit of that packet at wire-time T+dT."
The notion of wire time is i
...
Final draft ETSI EG 202 765-3 V1.1.1 (2009-10)
ETSI Guide
Speech and multimedia Transmission Quality (STQ);
QoS and network performance metrics and
measurement methods;
Part 3: Network performance metrics and
measurement methods in IP networks
---------------------- Page: 1 ----------------------
2 Final draft ETSI EG 202 765-3 V1.1.1 (2009-10)
Reference
DEG/STQ-00104-3
Keywords
performance, QoS
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE
Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16
Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88
Important notice
Individual copies of the present document can be downloaded from:
http://www.etsi.org
The present document may be made available in more than one electronic version or in print. In any case of existing or
perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF).
In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive
within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
http://portal.etsi.org/tb/status/status.asp
If you find errors in the present document, please send your comment to one of the following services:
http://portal.etsi.org/chaircor/ETSI_support.asp
Copyright Notification
No part may be reproduced except as authorized by written permission.
The copyright and the foregoing restriction extend to reproduction in all media.
© European Telecommunications Standards Institute 2009.
All rights reserved.
TM TM TM TM
DECT , PLUGTESTS , UMTS , TIPHON , the TIPHON logo and the ETSI logo are Trade Marks of ETSI registered
for the benefit of its Members.
TM
3GPP is a Trade Mark of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners.
LTE™ is a Trade Mark of ETSI currently being registered
for the benefit of its Members and of the 3GPP Organizational Partners.
GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.
ETSI
---------------------- Page: 2 ----------------------
3 Final draft ETSI EG 202 765-3 V1.1.1 (2009-10)
Contents
Intellectual Property Rights . 5
Foreword . 5
Introduction . 5
1 Scope . 7
2 References . 7
2.1 Normative references . 7
2.2 Informative references . 8
3 Definitions, symbols and abbreviations . 9
3.1 Definitions . 9
3.2 Symbols . 9
3.3 Abbreviations . 10
4 Performance Metrics Definitions and Measurement Methods . 10
4.1 One Way Delay vs. IP Packet Transfer Delay . 11
4.1.1 IETF Definition . 11
4.1.2 ITU-T Definition . 12
4.1.3 Comparison and Recommendations . 12
4.1.4 Active Measurement Method . 13
4.1.5 Passive Measurement Method . 14
4.2 Round Trip Delay . 15
4.2.1 IETF Definition . 15
4.2.2 ITU-T Definition . 16
4.2.3 Comparison and Recommendations . 16
4.2.4 Active Measurement Method . 16
4.2.5 Passive Measurement Method . 17
4.3 IP Packet Delay Variation vs. End-to-end 2-point IP Packet Delay Variation . 19
4.3.1 IETF Definition . 19
4.3.2 ITU-T Definition . 20
4.3.3 Comparison and Recommendations . 21
4.3.4 Active Measurement Method . 22
4.3.5 Passive Measurement Method . 22
4.4 One Way Packet Loss vs. IP Packet Loss Ratio . 22
4.4.1 IETF Definition . 23
4.4.2 ITU-T Definition . 23
4.4.3 Comparison and Recommendations . 23
4.4.4 Active Measurement Method . 23
4.4.5 Passive Measurement Method . 24
4.5 Connectivity vs. IP Service Availability . 24
4.5.1 IETF Definition . 24
4.5.2 ITU-T Definition . 25
4.5.3 Comparison and Recommendations . 25
4.5.4 Active Measurement Method . 26
4.5.5 Passive Measurement Method . 26
5 Other Metrics . 26
5.1 Data and Packet Volume . 26
5.2 Packet Reordering . 27
5.3 Bandwidth Capacity, Available Bandwidth, and Utilization . 27
5.4 Bulk Transport Capacity . 27
5.5 Loss Patterns . 27
5.6 RTCP reported metrics . 28
6 Overview of Network Performance Relevant Standard Bodies and Working Groups . 29
6.1 IETF . 29
6.1.1 IPPM (IP Performance Metrics) Working Group . 29
ETSI
---------------------- Page: 3 ----------------------
4 Final draft ETSI EG 202 765-3 V1.1.1 (2009-10)
6.1.2 IPFIX (IP Flow Information eXport) Working Group . 29
6.1.3 PSAMP (Packet SAMPling) Working Group . 30
6.2 ITU-T . 30
6.2.1 Study Group 12 (Performance and quality of service) . 30
6.2.2 Study Group 15 (Optical and other transport network infrastructures) . 30
Annex A: Bibliography . 32
History . 33
ETSI
---------------------- Page: 4 ----------------------
5 Final draft ETSI EG 202 765-3 V1.1.1 (2009-10)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (http://webapp.etsi.org/IPR/home.asp).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This ETSI Guide (EG) has been produced by ETSI Technical Committee Speech and multimedia Transmission Quality
(STQ), and is now submitted for the ETSI standards Membership Approval Procedure.
The present document is part 3 of a multi-part deliverable covering the QoS and network performance metrics and
measurement methods, as identified below:
Part 1: "General considerations";
Part 2: "Transmission Quality Indicator combining Voice Quality Metrics";
Part 3: "Network performance metrics and measurement methods in IP networks";
Part 4: "Indicators for supervision of Multiplay services".
Introduction
The need to define Internet performance metrics and measurement methodologies stems from the need to compare
different measurements and to measure performance with a reproducible and unambiguous methodology, independent
from transmission technology and implementation details. Both the ITU-T Study Group 12 and the IETF IPPM
Working Group have produced such definitions (see table 1), although each with a different emphasis closely linked to
the historical background of both organizations. The ITU has its origins in telephony, while the IETF has a data
networking background. Whereas the ITU emphasizes the evaluation of a service and its quality, the IETF measures the
network and wants to provide the IT-community with an accurate, common understanding and measurement of the
performance and reliability the Internet [i.3].
In most cases this results in different terminology rather than in incompatibilities; most differences in approach and
emphasis serve the different intended use of each metric, but have no operational significance. In some cases the
terminology used by each organisations can be mapped to the other, while in some others there is only approximate
equivalence (e.g. ITU network section versus an IPPM cloud; one focuses on corresponding events while the other
measures the fate of a single packet). Other terms have no correspondence. For example, ITU-T Recommendation
I.380 [i.38] has a notion of an IP packet transfer reference event while IPPM defines "wire time".
Other differences between IETF and ITU-T metrics result from their intended application. ITU-T metrics seek to
provide a common language for providers to communicate about performance, so the ITU-T metrics do not concentrate
on performance within a single network, while the IETF focuses on performance measurement protocols and
implementation. ITU-T seeks to evaluate service and to exclude unfair use, while the IETF seeks to measure network
quantities and avoid biased measurement results. Due to their respective backgrounds, the ITU generally produces
statistical metrics geared towards a quantitative representation of the complete end-to-end user experience while the
IETF IPPM working group mainly focuses more on statistical metrics which provide a detailed technical view of
different aspects of transmission quality along the network path.
ETSI
---------------------- Page: 5 ----------------------
6 Final draft ETSI EG 202 765-3 V1.1.1 (2009-10)
Table 1: Overview of Relevant Standards
IETF RFCs ITU-T Recommendations
Framework
RFC 2330 [i.3] Y.1540 [i.1], sections 1 through 5
Loss RFC 2680 [i.6] Y.1540 [i.1], section 5.5.6
G.1020 [i.23]
Delay
RFC 2679 [i.5] (One-way) Y.1540 [i.1], section 6.2
RFC 2681 [i.7] (Round Trip) G.1020 [i.23]
G.114 [i.22] (One-way)
Delay Variation RFC 3393 [i.10] Y.1540 [i.1], section 6.2.2
G.1020 [i.23]
Connectivity / Availability RFC 2678 [i.4] Y.1540 [i.1], section 7
Loss Patterns RFC 3357 [i.9] G.1020 [i.23]
Packet Reordering RFC 4737 [i.15] Y.1540 [i.1], sections 5.5.8.1 and 6.6
Packet Duplication Y.1540 [i.1], sections 5.5.8.3, 5.5.8.4,
6.8, and 6.9
Link/Path Bandwidth Capacity, Link
RFC 5136 [i.31]
Utilization, Available Capacity
Bulk Transport Capacity RFC 3148 [i.8], RFC 5136 [i.31]
The goal of the present document is to define network performance metrics for applications sensitive to quality of
service such as Voice over IP, referring to the existing work produced by both IETF and ITU-T. The present document
highlights the differences between the two standards and provides guidelines on resolving these differences, when they
are due to addressing different goals.
The scope of the present document is limited to IP performance metrics relevant for data transmission over IP-based
networks for use in QoS sensitive applications. For each addressed metric, the document recommends one or more
measurement methods. The document only focuses on intrinsic network QoS metrics; perceived QoS metrics applicable
for voice transmission are out of scope of the present document.
The remainder of the present document is organised as follows: Clause 4 describes the definitions of the most important
performance metrics as defined by the standard bodies and methods for measuring them, and discusses the applicability
of the definitions and the differences between them. Clause 5 discusses other metrics applicable to QoS. Finally,
clause 6 gives an overview of relevant QoS measurement standards, which can be used in end to end performance
evaluation.
ETSI
---------------------- Page: 6 ----------------------
7 Final draft ETSI EG 202 765-3 V1.1.1 (2009-10)
1 Scope
The present document provides an overview of the common metric definitions and measurement method specifications
upon which the interoperability of network performance measurement (also called QoS measurement) is based. Two
different standardisation bodies, the Internet Engineering Task Force (IETF) and the International Telecommunication
Union - Telecommunication Standardization Sector (ITU - T), have addressed this issue. The present document
addresses the following points:
• Survey the existing network performance related IETF standards and how these standards can be applied to
end-to-end network performance measurements. The scope of this work is also to discuss the relationship of
those standards to those of ITU-T and ETSI.
• Discuss and compare definitions of metrics used to specify and assess performance in IP networks. The
metrics addressed in the present document are those defined by the IETF IPPM working group and ITU-T
Study Group 12. Besides comparing the different definitions, the present document gives applicability
guidelines on which metric is more appropriate for a particular application, configuration or scenario.
• Define measurement methods for selected performance metrics in IP networks, addressing both active and
passive methods. Clarifying guidelines are given.
NOTE: All text sections in the remainder of the present document which are enclosed in quotation marks (") and
formatted in italic style denote citations taken verbatim from referenced documents.
2 References
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific.
• For a specific reference, subsequent revisions do not apply.
• Non-specific reference may be made only to a complete document or a part thereof and only in the following
cases:
- if it is accepted that it will be possible to use all future changes of the referenced document for the
purposes of the referring document;
- for informative references.
Referenced documents which are not found to be publicly available in the expected location might be found at
http://docbox.etsi.org/Reference.
NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee
their long term validity.
2.1 Normative references
The following referenced documents are indispensable for the application of the present document. For dated
references, only the edition cited applies. For non-specific references, the latest edition of the referenced document
(including any amendments) applies.
Not applicable.
ETSI
---------------------- Page: 7 ----------------------
8 Final draft ETSI EG 202 765-3 V1.1.1 (2009-10)
2.2 Informative references
The following referenced documents are not essential to the use of the present document but they assist the user with
regard to a particular subject area. For non-specific references, the latest version of the referenced document (including
any amendments) applies.
[i.1] ITU-T Recommendation Y.1540: "Internet protocol data communication service - IP packet
transfer and availability performance parameters".
[i.2] Void.
[i.3] IETF RFC 2330: "Framework for IP Performance Metrics". V. Paxson, G. Almes, J. Mahdavi,
M. Mathis. May 1998.
[i.4] IETF RFC 2678: "IPPM Metrics for Measuring Connectivity". J. Mahdavi, V. Paxson.
September 1999.
[i.5] IETF RFC 2679: "A One-way Delay Metric for IPPM". G. Almes, S. Kalidindi, M. Zekauskas.
September 1999.
[i.6] IETF RFC 2680: "A One-way Packet Loss Metric for IPPM". G. Almes, S. Kalidindi,
M. Zekauskas. September 1999.
[i.7] IETF RFC 2681: "A Round-trip Delay Metric for IPPM". G. Almes, S. Kalidindi, M. Zekauskas.
September 1999.
[i.8] IETF RFC 3148: "A Framework for Defining Empirical Bulk Transfer Capacity Metrics".
M. Mathis, M. Allman. July 2001.
[i.9] IETF RFC 3357: "One-way Loss Pattern Sample Metrics". R. Koodli, R. Ravikanth. August 2002.
[i.10] IETF RFC 3393: "IP Packet Delay Variation Metric for IP Performance Metrics (IPPM)".
C. Demichelis, P. Chimento. November 2002.
[i.11] Void.
[i.12] Void.
[i.13] Void.
[i.14] IETF RFC 4656: "A One-way Active Measurement Protocol (OWAMP)". S. Shalunov,
B. Teitelbaum, A. Karp, J. Boote, M. Zekauskas. September 2006.
[i.15] IETF RFC 4737: "Packet Reordering Metrics". A. Morton, L. Ciavattone, G. Ramachandran,
S. Shalunov, J. Perser. November 2006.
[i.16] Void.
[i.17] Void.
[i.18] IETF RFC 5101: "Specification of the IPFIX Protocol for the Exchange of IP Traffic Flow
Information". B. Claise, S. Bryant, S. Leinen, T. Deitz, B.Trammell. January 2008.
[i.19] IPFIX Architecture. N.Brownlee et Al. Internet-Draft, work in progress.
[i.20] IETF RFC 5102: "IPFIX Information Model". J. Quittek et Al. January 2008.
[i.21] "IPFIX Applicability Statement". T. Zseby, E. Boschi, N.Brownlee, B. Claise. Internet-Draft, work
in progress.
[i.22] ITU-T Recommendation G.114 (05/03): "One-way transmission time".
[i.23] ITU-T Recommendation G.1020 (07/06): "Performance parameter definitions for quality of speech
and other voiceband applications utilizing IP networks".
ETSI
---------------------- Page: 8 ----------------------
9 Final draft ETSI EG 202 765-3 V1.1.1 (2009-10)
[i.24] IETF RFC 3917: "Requirements for IP Flow Information Export". J. Quittek, T. Zseby, B. Claise,
S. Zander. October 2004.
[i.25] Void.
[i.26] Void.
[i.27] Void.
[i.28] Void.
[i.29] "Reporting Metrics: Different Points of View", A. Morton, G. Ramachandran, G. Maguluri, work
in progress, draft-morton-ippm-reporting-metrics-02.
NOTE: http://tools.ietf.org/html/draft-morton-ippm-reporting-metrics-02, and the derived presentation "Reporting
Metrics: Different Points of View" presented by Al Morton on IETF66 July 2006,
http://www3.ietf.org/proceedings/06jul/slides/ippm-2.pdf .
[i.30] IETF RFC 3611: "RTP Control Protocol Extended Reports (RTCP XR)", T. Friedman, R. Caceres,
A. Clark. November 2003.
[i.31] IETF RFC 5136: "Defining Network Capacity", P. Chimento, J. Ishac. February 2008.
[i.32] IETF RFC 2581: "TCP Congestion Control", M. Allman, V. Paxson, W. Stevens. April 1999.
[i.33] IETF RFC 5357: "A Two-Way Active Measurement Protocol (TWAMP)", K. Hedayat,
R. Krzanowski, A. Morton, K. Yum, J. Babiarz. October 2008.
[i.34] IETF RFC 1122: "Requirements for Internet Hosts - Communication Layers", R. Braden ed.
October 1989.
[i.35] IETF RFC 3550: "User Accounts for UCSB On-Line System".
[i.36] IETF RFC 1633: "Integrated Services in the Internet Architecture: an Overview".
[i.37] IETF RFC 2216: "Network Element Service Specification Template".
[i.38] ITU-T Recommendation I.380: "Internet protocol data communication service - IP packet transfer
and availability performance parameters".
3 Definitions, symbols and abbreviations
3.1 Definitions
For the purposes of the present document, the terms and definitions given in RFC 2330 [i.3], ITU-T Recommendation
G.1020 [i.23] and RFC 2680 [i.6] apply.
3.2 Symbols
For the purposes of the present document, the following symbols apply:
T, t Time
T Time threshold
max
dT Time difference
ETSI
---------------------- Page: 9 ----------------------
10 Final draft ETSI EG 202 765-3 V1.1.1 (2009-10)
3.3 Abbreviations
For the purposes of the present document, the following abbreviations apply:
ASON Automatically Switched Optical Network
ATM Asynchronous Transfer Mode
BTC Bulk transport Capacity
DNS Domain Name System
ESD End System Delay
FTP File Transfer Protocol
HTTP HyperText Transfer Protocol
ICMP Internet Control Message Protocol
IETF Internet Engineering Task Force
IPDV IP Packet Dealy Variation
IPFIX IP Flow Information eXport
IPLR IP Packet Loss Ratio
IPPM IP Performance Metrics
IPTD IP Packet Transfer Delay
ITU-T International Telecommunication Union - Telecommunication standardisation sector
MIB Management Information Base
NSE Network Section Ensemble
OP Observation Point
OWAMP One Way Active Measurement Protocol
OWD One Way Delay
PDV Packet Delay Variation
PIA Percent IP service Availability
PON Passive Optical Network
PSAMP Packet SAMPling
QoS Quality of Service
RFC Request For Comments
RTCP Real Time Control Protocol
RTD Round Trip Delay
RTP Real-Time Transport Protocol
RTT Round Trip Time
SDH Synchronous Digital Hierarchy
SLA Service Level Agreement
TCP Transmission Control Protocol
TWAMP Two-Way Active Measurement Protocol
UTC Coordinated Universal Time
VoIP Voice over IP
4 Performance Metrics Definitions and Measurement
Methods
This clause provides common definitions for network performance metrics. These definitions are based, whenever
possible, on existing definitions proposed by other relevant standard bodies such as IETF or ITU-T. Note that the
different definitions of similar metrics are in most cases compatible, that is, semantically equivalent or easily
convertible into one another.
ETSI
---------------------- Page: 10 ----------------------
11 Final draft ETSI EG 202 765-3 V1.1.1 (2009-10)
For each metric, passive and active measurement methods are defined. Note that we chose to focus on commonly used
measurement methods rather than on standards; when a standard exists, a reference is provided as well. Note also that
throughout this text we refer for each metric to active and passive measurements in the following way:
• Active measurements
Active measurement methods inject traffic into the network and compute traffic metrics based on monitoring
the injected traffic or the response to the injected traffic. Active test traffic may perturb other traffic already
present on the network; therefore its scheduling and volume should be carefully configured. One can
distinguish active monitoring systems based on the position of sender and receiver and the observed traffic;
this is specified in detail for the considered metrics in the following text.
• Passive measurements
Passive measurements provide information about traffic in the observed network by capturing all or a selected
subset of the IP packets traversing a monitoring point. Since no test traffic is generated, passive measurements
can only be applied when the traffic of interest is already present on the network. The physical deployment of
monitoring probes in the network can be realised in different ways, depending on the metrics of interest, but
also on the network technology, e.g. via a physical line splitter, via a normal client connection in broadcast
networks, or via a dedicated monitoring port on a switch or router.
4.1 One Way Delay vs. IP Packet Transfer Delay
Delay is used to measure the expected time for an IP packet to traverse the network from one host to another. Delay is
applicable to QoS for latency-sensitive protocols. The IETF and ITU-T metrics for measuring delay are essentially
compatible, though there are minor differences; the details of these metrics are given in this clause.
4.1.1 IETF Definition
RFC 2679 [i.5] distinguishes between a "singleton analytic metric", called Type-P-One-way-Delay, and a "sample",
called Type-P-One-way-Delay-Poisson-Stream. The singleton is introduced to measure a single observation of one-way
delay, while the sample is used to measure a sequence of singleton delays measured at times taken from a Poisson
process. Based on these samples, several statistics are defined, such as Type-P-One-way-Delay-Percentile, Type-P-One-
way-Delay-Median, Type-P-One-way-Delay-Minimum, and Type-P-One-way-Delay-Inverse-Percentile.
Since the value of many of these metrics depends on the type of the IP packet used to perform the measurements, IPPM
metrics definitions include the generic notion of "a packet of type P", which should be further specified when making
actual measurements.
RFC 2679 [i.5] defines:
"For a real number dT, >>the *Type-P-One-way-
...
SLOVENSKI STANDARD
SIST-V ETSI/EG 202 765-3 V1.1.1:2016
01-oktober-2016
.DNRYRVWSUHQRVDJRYRUDLQYHþSUHGVWDYQLKYVHELQ 674 0HWULNDNDNRYRVWL
VWRULWHY 4R6 LQ]PRJOMLYRVWLRPUHåMDWHUPHULOQHPHWRGHGHO 0HWRGHPHWULNH
LQPHUMHQMD]PRJOMLYRVWLRPUHåLMYRPUHåMLK,3
Speech and multimediaTransmission Quality (STQ) - QoS and network performance
metrics and measurement methods - Part 3: Network performance metrics and
measurement methods in IP networks
Ta slovenski standard je istoveten z: ETSI EG 202 765-3 V1.1.1 (2009-12)
ICS:
33.040.35 Telefonska omrežja Telephone networks
SIST-V ETSI/EG 202 765-3 V1.1.1:2016 en
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.
---------------------- Page: 1 ----------------------
SIST-V ETSI/EG 202 765-3 V1.1.1:2016
---------------------- Page: 2 ----------------------
SIST-V ETSI/EG 202 765-3 V1.1.1:2016
ETSI EG 202 765-3 V1.1.1 (2009-12)
ETSI Guide
Speech and multimedia Transmission Quality (STQ);
QoS and network performance metrics and
measurement methods;
Part 3: Network performance metrics and
measurement methods in IP networks
---------------------- Page: 3 ----------------------
SIST-V ETSI/EG 202 765-3 V1.1.1:2016
2 ETSI EG 202 765-3 V1.1.1 (2009-12)
Reference
DEG/STQ-00104-3
Keywords
performance, QoS
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE
Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16
Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88
Important notice
Individual copies of the present document can be downloaded from:
http://www.etsi.org
The present document may be made available in more than one electronic version or in print. In any case of existing or
perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF).
In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive
within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
http://portal.etsi.org/tb/status/status.asp
If you find errors in the present document, please send your comment to one of the following services:
http://portal.etsi.org/chaircor/ETSI_support.asp
Copyright Notification
No part may be reproduced except as authorized by written permission.
The copyright and the foregoing restriction extend to reproduction in all media.
© European Telecommunications Standards Institute 2009.
All rights reserved.
TM TM TM TM
DECT , PLUGTESTS , UMTS , TIPHON , the TIPHON logo and the ETSI logo are Trade Marks of ETSI registered
for the benefit of its Members.
TM
3GPP is a Trade Mark of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners.
LTE™ is a Trade Mark of ETSI currently being registered
for the benefit of its Members and of the 3GPP Organizational Partners.
GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.
ETSI
---------------------- Page: 4 ----------------------
SIST-V ETSI/EG 202 765-3 V1.1.1:2016
3 ETSI EG 202 765-3 V1.1.1 (2009-12)
Contents
Intellectual Property Rights . 5
Foreword . 5
Introduction . 5
1 Scope . 7
2 References . 7
2.1 Normative references . 7
2.2 Informative references . 8
3 Definitions, symbols and abbreviations . 9
3.1 Definitions . 9
3.2 Symbols . 9
3.3 Abbreviations . 10
4 Performance Metrics Definitions and Measurement Methods . 10
4.1 One Way Delay vs. IP Packet Transfer Delay . 11
4.1.1 IETF Definition . 11
4.1.2 ITU-T Definition . 12
4.1.3 Comparison and Recommendations . 12
4.1.4 Active Measurement Method . 13
4.1.5 Passive Measurement Method . 14
4.2 Round Trip Delay . 15
4.2.1 IETF Definition . 15
4.2.2 ITU-T Definition . 16
4.2.3 Comparison and Recommendations . 16
4.2.4 Active Measurement Method . 16
4.2.5 Passive Measurement Method . 17
4.3 IP Packet Delay Variation vs. End-to-end 2-point IP Packet Delay Variation . 19
4.3.1 IETF Definition . 19
4.3.2 ITU-T Definition . 20
4.3.3 Comparison and Recommendations . 21
4.3.4 Active Measurement Method . 22
4.3.5 Passive Measurement Method . 22
4.4 One Way Packet Loss vs. IP Packet Loss Ratio . 22
4.4.1 IETF Definition . 23
4.4.2 ITU-T Definition . 23
4.4.3 Comparison and Recommendations . 23
4.4.4 Active Measurement Method . 23
4.4.5 Passive Measurement Method . 24
4.5 Connectivity vs. IP Service Availability . 24
4.5.1 IETF Definition . 24
4.5.2 ITU-T Definition . 25
4.5.3 Comparison and Recommendations . 25
4.5.4 Active Measurement Method . 26
4.5.5 Passive Measurement Method . 26
5 Other Metrics . 26
5.1 Data and Packet Volume . 26
5.2 Packet Reordering . 27
5.3 Bandwidth Capacity, Available Bandwidth, and Utilization . 27
5.4 Bulk Transport Capacity . 27
5.5 Loss Patterns . 27
5.6 RTCP reported metrics . 28
6 Overview of Network Performance Relevant Standard Bodies and Working Groups . 29
6.1 IETF . 29
6.1.1 IPPM (IP Performance Metrics) Working Group . 29
ETSI
---------------------- Page: 5 ----------------------
SIST-V ETSI/EG 202 765-3 V1.1.1:2016
4 ETSI EG 202 765-3 V1.1.1 (2009-12)
6.1.2 IPFIX (IP Flow Information eXport) Working Group . 29
6.1.3 PSAMP (Packet SAMPling) Working Group . 30
6.2 ITU-T . 30
6.2.1 Study Group 12 (Performance and quality of service) . 30
6.2.2 Study Group 15 (Optical and other transport network infrastructures) . 30
Annex A: Bibliography . 32
History . 33
ETSI
---------------------- Page: 6 ----------------------
SIST-V ETSI/EG 202 765-3 V1.1.1:2016
5 ETSI EG 202 765-3 V1.1.1 (2009-12)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (http://webapp.etsi.org/IPR/home.asp).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This ETSI Guide (EG) has been produced by ETSI Technical Committee Speech and multimedia Transmission Quality
(STQ).
The present document is part 3 of a multi-part deliverable covering the QoS and network performance metrics and
measurement methods, as identified below:
Part 1: "General considerations";
Part 2: "Transmission Quality Indicator combining Voice Quality Metrics";
Part 3: "Network performance metrics and measurement methods in IP networks";
Part 4: "Indicators for supervision of Multiplay services".
Introduction
The need to define Internet performance metrics and measurement methodologies stems from the need to compare
different measurements and to measure performance with a reproducible and unambiguous methodology, independent
from transmission technology and implementation details. Both the ITU-T Study Group 12 and the IETF IPPM
Working Group have produced such definitions (see table 1), although each with a different emphasis closely linked to
the historical background of both organizations. The ITU has its origins in telephony, while the IETF has a data
networking background. Whereas the ITU emphasizes the evaluation of a service and its quality, the IETF measures the
network and wants to provide the IT-community with an accurate, common understanding and measurement of the
performance and reliability the Internet [i.3].
In most cases this results in different terminology rather than in incompatibilities; most differences in approach and
emphasis serve the different intended use of each metric, but have no operational significance. In some cases the
terminology used by each organisations can be mapped to the other, while in some others there is only approximate
equivalence (e.g. ITU network section versus an IPPM cloud; one focuses on corresponding events while the other
measures the fate of a single packet). Other terms have no correspondence. For example, ITU-T Recommendation
I.380 [i.38] has a notion of an IP packet transfer reference event while IPPM defines "wire time".
Other differences between IETF and ITU-T metrics result from their intended application. ITU-T metrics seek to
provide a common language for providers to communicate about performance, so the ITU-T metrics do not concentrate
on performance within a single network, while the IETF focuses on performance measurement protocols and
implementation. ITU-T seeks to evaluate service and to exclude unfair use, while the IETF seeks to measure network
quantities and avoid biased measurement results. Due to their respective backgrounds, the ITU generally produces
statistical metrics geared towards a quantitative representation of the complete end-to-end user experience while the
IETF IPPM working group mainly focuses more on statistical metrics which provide a detailed technical view of
different aspects of transmission quality along the network path.
ETSI
---------------------- Page: 7 ----------------------
SIST-V ETSI/EG 202 765-3 V1.1.1:2016
6 ETSI EG 202 765-3 V1.1.1 (2009-12)
Table 1: Overview of Relevant Standards
IETF RFCs ITU-T Recommendations
Framework
RFC 2330 [i.3] Y.1540 [i.1], sections 1 through 5
Loss RFC 2680 [i.6] Y.1540 [i.1], section 5.5.6
G.1020 [i.23]
Delay
RFC 2679 [i.5] (One-way) Y.1540 [i.1], section 6.2
RFC 2681 [i.7] (Round Trip) G.1020 [i.23]
G.114 [i.22] (One-way)
Delay Variation RFC 3393 [i.10] Y.1540 [i.1], section 6.2.2
G.1020 [i.23]
Connectivity / Availability RFC 2678 [i.4] Y.1540 [i.1], section 7
Loss Patterns RFC 3357 [i.9] G.1020 [i.23]
Packet Reordering RFC 4737 [i.15] Y.1540 [i.1], sections 5.5.8.1 and 6.6
Packet Duplication Y.1540 [i.1], sections 5.5.8.3, 5.5.8.4,
6.8, and 6.9
Link/Path Bandwidth Capacity, Link
RFC 5136 [i.31]
Utilization, Available Capacity
Bulk Transport Capacity RFC 3148 [i.8], RFC 5136 [i.31]
The goal of the present document is to define network performance metrics for applications sensitive to quality of
service such as Voice over IP, referring to the existing work produced by both IETF and ITU-T. The present document
highlights the differences between the two standards and provides guidelines on resolving these differences, when they
are due to addressing different goals.
The scope of the present document is limited to IP performance metrics relevant for data transmission over IP-based
networks for use in QoS sensitive applications. For each addressed metric, the document recommends one or more
measurement methods. The document only focuses on intrinsic network QoS metrics; perceived QoS metrics applicable
for voice transmission are out of scope of the present document.
The remainder of the present document is organised as follows: Clause 4 describes the definitions of the most important
performance metrics as defined by the standard bodies and methods for measuring them, and discusses the applicability
of the definitions and the differences between them. Clause 5 discusses other metrics applicable to QoS. Finally,
clause 6 gives an overview of relevant QoS measurement standards, which can be used in end to end performance
evaluation.
ETSI
---------------------- Page: 8 ----------------------
SIST-V ETSI/EG 202 765-3 V1.1.1:2016
7 ETSI EG 202 765-3 V1.1.1 (2009-12)
1 Scope
The present document provides an overview of the common metric definitions and measurement method specifications
upon which the interoperability of network performance measurement (also called QoS measurement) is based. Two
different standardisation bodies, the Internet Engineering Task Force (IETF) and the International Telecommunication
Union - Telecommunication Standardization Sector (ITU - T), have addressed this issue. The present document
addresses the following points:
• Survey the existing network performance related IETF standards and how these standards can be applied to
end-to-end network performance measurements. The scope of this work is also to discuss the relationship of
those standards to those of ITU-T and ETSI.
• Discuss and compare definitions of metrics used to specify and assess performance in IP networks. The
metrics addressed in the present document are those defined by the IETF IPPM working group and ITU-T
Study Group 12. Besides comparing the different definitions, the present document gives applicability
guidelines on which metric is more appropriate for a particular application, configuration or scenario.
• Define measurement methods for selected performance metrics in IP networks, addressing both active and
passive methods. Clarifying guidelines are given.
NOTE: All text sections in the remainder of the present document which are enclosed in quotation marks (") and
formatted in italic style denote citations taken verbatim from referenced documents.
2 References
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific.
• For a specific reference, subsequent revisions do not apply.
• Non-specific reference may be made only to a complete document or a part thereof and only in the following
cases:
- if it is accepted that it will be possible to use all future changes of the referenced document for the
purposes of the referring document;
- for informative references.
Referenced documents which are not found to be publicly available in the expected location might be found at
http://docbox.etsi.org/Reference.
NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee
their long term validity.
2.1 Normative references
The following referenced documents are indispensable for the application of the present document. For dated
references, only the edition cited applies. For non-specific references, the latest edition of the referenced document
(including any amendments) applies.
Not applicable.
ETSI
---------------------- Page: 9 ----------------------
SIST-V ETSI/EG 202 765-3 V1.1.1:2016
8 ETSI EG 202 765-3 V1.1.1 (2009-12)
2.2 Informative references
The following referenced documents are not essential to the use of the present document but they assist the user with
regard to a particular subject area. For non-specific references, the latest version of the referenced document (including
any amendments) applies.
[i.1] ITU-T Recommendation Y.1540: "Internet protocol data communication service - IP packet
transfer and availability performance parameters".
[i.2] Void.
[i.3] IETF RFC 2330: "Framework for IP Performance Metrics". V. Paxson, G. Almes, J. Mahdavi,
M. Mathis. May 1998.
[i.4] IETF RFC 2678: "IPPM Metrics for Measuring Connectivity". J. Mahdavi, V. Paxson.
September 1999.
[i.5] IETF RFC 2679: "A One-way Delay Metric for IPPM". G. Almes, S. Kalidindi, M. Zekauskas.
September 1999.
[i.6] IETF RFC 2680: "A One-way Packet Loss Metric for IPPM". G. Almes, S. Kalidindi,
M. Zekauskas. September 1999.
[i.7] IETF RFC 2681: "A Round-trip Delay Metric for IPPM". G. Almes, S. Kalidindi, M. Zekauskas.
September 1999.
[i.8] IETF RFC 3148: "A Framework for Defining Empirical Bulk Transfer Capacity Metrics".
M. Mathis, M. Allman. July 2001.
[i.9] IETF RFC 3357: "One-way Loss Pattern Sample Metrics". R. Koodli, R. Ravikanth. August 2002.
[i.10] IETF RFC 3393: "IP Packet Delay Variation Metric for IP Performance Metrics (IPPM)".
C. Demichelis, P. Chimento. November 2002.
[i.11] Void.
[i.12] Void.
[i.13] Void.
[i.14] IETF RFC 4656: "A One-way Active Measurement Protocol (OWAMP)". S. Shalunov,
B. Teitelbaum, A. Karp, J. Boote, M. Zekauskas. September 2006.
[i.15] IETF RFC 4737: "Packet Reordering Metrics". A. Morton, L. Ciavattone, G. Ramachandran,
S. Shalunov, J. Perser. November 2006.
[i.16] Void.
[i.17] Void.
[i.18] IETF RFC 5101: "Specification of the IPFIX Protocol for the Exchange of IP Traffic Flow
Information". B. Claise, S. Bryant, S. Leinen, T. Deitz, B.Trammell. January 2008.
[i.19] IPFIX Architecture. N.Brownlee et Al. Internet-Draft, work in progress.
[i.20] IETF RFC 5102: "IPFIX Information Model". J. Quittek et Al. January 2008.
[i.21] "IPFIX Applicability Statement". T. Zseby, E. Boschi, N.Brownlee, B. Claise. Internet-Draft, work
in progress.
[i.22] ITU-T Recommendation G.114 (05/03): "One-way transmission time".
[i.23] ITU-T Recommendation G.1020 (07/06): "Performance parameter definitions for quality of speech
and other voiceband applications utilizing IP networks".
ETSI
---------------------- Page: 10 ----------------------
SIST-V ETSI/EG 202 765-3 V1.1.1:2016
9 ETSI EG 202 765-3 V1.1.1 (2009-12)
[i.24] IETF RFC 3917: "Requirements for IP Flow Information Export". J. Quittek, T. Zseby, B. Claise,
S. Zander. October 2004.
[i.25] Void.
[i.26] Void.
[i.27] Void.
[i.28] Void.
[i.29] "Reporting Metrics: Different Points of View", A. Morton, G. Ramachandran, G. Maguluri, work
in progress, draft-morton-ippm-reporting-metrics-02.
NOTE: http://tools.ietf.org/html/draft-morton-ippm-reporting-metrics-02, and the derived presentation "Reporting
Metrics: Different Points of View" presented by Al Morton on IETF66 July 2006,
http://www3.ietf.org/proceedings/06jul/slides/ippm-2.pdf.
[i.30] IETF RFC 3611: "RTP Control Protocol Extended Reports (RTCP XR)", T. Friedman, R. Caceres,
A. Clark. November 2003.
[i.31] IETF RFC 5136: "Defining Network Capacity", P. Chimento, J. Ishac. February 2008.
[i.32] IETF RFC 2581: "TCP Congestion Control", M. Allman, V. Paxson, W. Stevens. April 1999.
[i.33] IETF RFC 5357: "A Two-Way Active Measurement Protocol (TWAMP)", K. Hedayat,
R. Krzanowski, A. Morton, K. Yum, J. Babiarz. October 2008.
[i.34] IETF RFC 1122: "Requirements for Internet Hosts - Communication Layers", R. Braden ed.
October 1989.
[i.35] IETF RFC 3550: "User Accounts for UCSB On-Line System".
[i.36] IETF RFC 1633: "Integrated Services in the Internet Architecture: an Overview".
[i.37] IETF RFC 2216: "Network Element Service Specification Template".
[i.38] ITU-T Recommendation I.380: "Internet protocol data communication service - IP packet transfer
and availability performance parameters".
3 Definitions, symbols and abbreviations
3.1 Definitions
For the purposes of the present document, the terms and definitions given in RFC 2330 [i.3], ITU-T Recommendation
G.1020 [i.23] and RFC 2680 [i.6] apply.
3.2 Symbols
For the purposes of the present document, the following symbols apply:
T, t Time
T Time threshold
max
dT Time difference
ETSI
---------------------- Page: 11 ----------------------
SIST-V ETSI/EG 202 765-3 V1.1.1:2016
10 ETSI EG 202 765-3 V1.1.1 (2009-12)
3.3 Abbreviations
For the purposes of the present document, the following abbreviations apply:
ASON Automatically Switched Optical Network
ATM Asynchronous Transfer Mode
BTC Bulk transport Capacity
DNS Domain Name System
ESD End System Delay
FTP File Transfer Protocol
HTTP HyperText Transfer Protocol
ICMP Internet Control Message Protocol
IETF Internet Engineering Task Force
IPDV IP Packet Dealy Variation
IPFIX IP Flow Information eXport
IPLR IP Packet Loss Ratio
IPPM IP Performance Metrics
IPTD IP Packet Transfer Delay
ITU-T International Telecommunication Union - Telecommunication standardisation sector
MIB Management Information Base
NSE Network Section Ensemble
OP Observation Point
OWAMP One Way Active Measurement Protocol
OWD One Way Delay
PDV Packet Delay Variation
PIA Percent IP service Availability
PON Passive Optical Network
PSAMP Packet SAMPling
QoS Quality of Service
RFC Request For Comments
RTCP Real Time Control Protocol
RTD Round Trip Delay
RTP Real-Time Transport Protocol
RTT Round Trip Time
SDH Synchronous Digital Hierarchy
SLA Service Level Agreement
TCP Transmission Control Protocol
TWAMP Two-Way Active Measurement Protocol
UTC Coordinated Universal Time
VoIP Voice over IP
4 Performance Metrics Definitions and Measurement
Methods
This clause provides common definitions for network performance metrics. These definitions are based, whenever
possible, on existing definitions proposed by other relevant standard bodies such as IETF or ITU-T. Note that the
different definitions of similar metrics are in most cases compatible, that is, semantically equivalent or easily
convertible into one another.
ETSI
---------------------- Page: 12 ----------------------
SIST-V ETSI/EG 202 765-3 V1.1.1:2016
11 ETSI EG 202 765-3 V1.1.1 (2009-12)
For each metric, passive and active measurement methods are defined. Note that we chose to focus on commonly used
measurement methods rather than on standards; when a standard exists, a reference is provided as well. Note also that
throughout this text we refer for each metric to active and passive measurements in the following way:
• Active measurements
Active measurement methods inject traffic into the network and compute traffic metrics based on monitoring
the injected traffic or the response to the injected traffic. Active test traffic may perturb other traffic already
present on the network; therefore its scheduling and volume should be carefully configured. One can
distinguish active monitoring systems based on the position of sender and receiver and the observed traffic;
this is specified in detail for the considered metrics in the following text.
• Passive measurements
Passive measurements provide information about traffic in the observed network by capturing all or a selected
subset of the IP packets traversing a monitoring point. Since no test traffic is generated, passive measurements
can only be applied when the traffic of interest is already present on the network. The physical deployment of
monitoring probes in the network can be realised in different ways, depending on the metrics of interest, but
also on the network technology, e.g. via a physical line splitter, via a normal client connection in broadcast
networks, or via a dedicated monitoring port on a switch or router.
4.1 One Way Delay vs. IP Packet Transfer Delay
Delay is used to measure the expected time for an IP packet to traverse the network from one host to another.
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.