Methods for Testing and Specification (MTS) - Internet Protocol Testing (IPT) - Generic approach to interoperability testing

This document gives general guidance on the specification and execution of interoperability tests for communication systems specifically in the context of product certification. It provides a framework within which interoperability test specifications for a wide range of product types can be developed. The guidelines are expressed as recommendations rather than strict rules and leave enough freedom to allow test specifiers to adopt and adapt processes to suit each particular project while still ensuring that test specifications accurately reflect the requirements of the base standards and can be executed consistently across a range of configurations.

Metode za preskušanje in specificiranje (MTS) - Preskušanje internetnega protokola - Generični pristop k preskušanju medobratovalnosti

Ta dokument podaja splošne smernice glede specifikacije in izvedbe preskusov medobratovalnosti za komunikacijske sisteme zlasti v kontekstu certificiranja proizvodov. Zagotavlja okvir za možnost razvoja specifikacij preskusov medobratovalnosti za številne vrste proizvodov. Smernice so izražene kot priporočila, ne kot stroga pravila, ter puščajo dovolj svobode, da lahko izdajatelji specifikacij preskusov sprejmejo in priredijo postopke tako, da ti ustrezajo posameznemu projektu, obenem pa zagotovijo, da specifikacije preskusov natančno odražajo zahteve osnovnih standardov in jih je mogoče dosledno izvajati v različnih konfiguracijah.

General Information

Status
Published
Publication Date
28-Aug-2016
Current Stage
6060 - National Implementation/Publication (Adopted Project)
Start Date
19-Aug-2016
Due Date
24-Oct-2016
Completion Date
29-Aug-2016

Buy Standard

Standard
ETSI EG 202 237 V1.2.1 (2010-06) - Methods for Testing and Specification (MTS); Internet Protocol Testing (IPT); Generic approach to interoperability testing
English language
41 pages
sale 15% off
Preview
sale 15% off
Preview
Guide
SIST-V ETSI/EG 202 237 V1.2.1:2016
English language
41 pages
sale 10% off
Preview
sale 10% off
Preview

e-Library read for
1 day

Standards Content (sample)

Final draft ETSI EG 202 237 V1.2.1 (2010-06)
ETSI Guide
Methods for Testing and Specification (MTS);
Internet Protocol Testing (IPT);
Generic approach to interoperability testing
---------------------- Page: 1 ----------------------
2 Final draft ETSI EG 202 237 V1.2.1 (2010-06)
Reference
REG/MTS-00129-IOP-TstApproach
Keywords
interoperability, IP, telephony, testing
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE
Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16
Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88
Important notice
Individual copies of the present document can be downloaded from:
http://www.etsi.org

The present document may be made available in more than one electronic version or in print. In any case of existing or

perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF).

In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive

within ETSI Secretariat.

Users of the present document should be aware that the document may be subject to revision or change of status.

Information on the current status of this and other ETSI documents is available at

http://portal.etsi.org/tb/status/status.asp

If you find errors in the present document, please send your comment to one of the following services:

http://portal.etsi.org/chaircor/ETSI_support.asp
Copyright Notification
No part may be reproduced except as authorized by written permission.
The copyright and the foregoing restriction extend to reproduction in all media.
© European Telecommunications Standards Institute 2010.
All rights reserved.
TM TM TM TM

DECT , PLUGTESTS , UMTS , TIPHON , the TIPHON logo and the ETSI logo are Trade Marks of ETSI registered

for the benefit of its Members.

3GPP is a Trade Mark of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners.

LTE™ is a Trade Mark of ETSI currently being registered
for the benefit of its Members and of the 3GPP Organizational Partners.

GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.

ETSI
---------------------- Page: 2 ----------------------
3 Final draft ETSI EG 202 237 V1.2.1 (2010-06)
Contents

Intellectual Property Rights ................................................................................................................................ 6

Foreword ............................................................................................................................................................. 6

1 Scope ........................................................................................................................................................ 7

2 References ................................................................................................................................................ 7

2.1 Normative references ......................................................................................................................................... 7

2.2 Informative references ........................................................................................................................................ 7

3 Definitions and abbreviations ................................................................................................................... 8

3.1 Definitions .......................................................................................................................................................... 8

3.2 Abbreviations ..................................................................................................................................................... 9

4 Types of testing ........................................................................................................................................ 9

4.1 Interoperability testing ....................................................................................................................................... 9

4.2 Conformance testing......................................................................................................................................... 10

4.3 Combining interoperability testing and conformance testing ........................................................................... 11

5 Interoperability testing process overview ............................................................................................... 11

6 Basic concepts ........................................................................................................................................ 12

6.1 Means of Testing .............................................................................................................................................. 12

6.2 Equipment Under Test (EUT) .......................................................................................................................... 13

6.3 Qualified Equipment (QE) ............................................................................................................................... 13

6.3.1 QEs and Devices ......................................................................................................................................... 13

6.3.2 Designating the first QE ............................................................................................................................. 13

6.4 System Under Test (SUT) ................................................................................................................................ 13

6.5 Test interface .................................................................................................................................................... 14

6.6 Test driver ........................................................................................................................................................ 14

6.7 Test coordinator ................................................................................................................................................ 14

6.8 Interoperability test cases ................................................................................................................................. 14

6.9 Means of Communication ................................................................................................................................ 15

7 Generic interoperability test architectures .............................................................................................. 15

7.1 Test architectures with a single QE .................................................................................................................. 15

7.2 Test architectures with multiple QEs................................................................................................................ 16

7.2.1 An example using 3 QEs ............................................................................................................................ 16

7.2.2 Testing IP hosts with multiple QEs ............................................................................................................ 17

8 Developing interoperability tests ............................................................................................................ 18

8.1 Overview .......................................................................................................................................................... 18

8.2 Specify abstract architecture............................................................................................................................. 19

8.3 Prepare draft IFS Proforma .............................................................................................................................. 19

8.4 Specify Test Suite Structure ............................................................................................................................. 20

8.4.1 Identify test groups ..................................................................................................................................... 20

8.4.2 Define test coverage within each test group ............................................................................................... 20

8.5 Write Test Purposes.......................................................................................................................................... 21

8.6 Write test cases ................................................................................................................................................. 21

8.6.1 Pre-test conditions ...................................................................................................................................... 21

8.6.2 Test steps and verdicts ................................................................................................................................ 22

8.6.2.1 Test steps ............................................................................................................................................... 22

8.6.2.2 Verdicts ................................................................................................................................................. 22

8.6.2.3 Specification of test steps and verdicts.................................................................................................. 22

8.6.3 Example ...................................................................................................................................................... 23

8.6.4 Pre-amble and post-amble .......................................................................................................................... 24

8.6.4.1 Alternative test case presentation forms ................................................................................................ 25

8.7 Validate test cases ............................................................................................................................................ 26

8.8 Finalize IFS ...................................................................................................................................................... 26

9 Interoperability testing process .............................................................................................................. 26

ETSI
---------------------- Page: 3 ----------------------
4 Final draft ETSI EG 202 237 V1.2.1 (2010-06)

9.1 Overview .......................................................................................................................................................... 26

9.2 Prepare for testing ............................................................................................................................................ 27

9.2.1 Test arrangement ........................................................................................................................................ 27

9.2.2 Test planning .............................................................................................................................................. 28

9.3 Testing .............................................................................................................................................................. 29

9.3.1 Manual testing ............................................................................................................................................ 29

9.3.2 Automated testing ....................................................................................................................................... 29

9.4 Write test report ................................................................................................................................................ 29

Annex A: Example IFS (Internet Key Exchange protocol, IKEv2) ................................................... 30

A.1 Introduction ............................................................................................................................................ 30

A.2 Instructions for completing the IFS proforma ........................................................................................ 30

A.2.1 General structure of the IFS proforma .............................................................................................................. 30

A.2.2 Additional information ..................................................................................................................................... 30

A.3 IFS proforma .......................................................................................................................................... 31

A.3.1 Implementation identification .......................................................................................................................... 31

A.3.2 Protocol Summary, RFC 4306 ......................................................................................................................... 31

A.4 IKEv2 entities ......................................................................................................................................... 31

A.4.1 Roles ................................................................................................................................................................. 31

A.4.2 IKEv2 Initiator functions .................................................................................................................................. 32

A.4.2.1 IKE exchange types .................................................................................................................................... 32

A.4.2.1.1 IKE SA establishment functions ........................................................................................................... 32

A.4.2.1.2 Child SA establishment functions ......................................................................................................... 32

A.4.2.1.3 INFORMATIONAL exchange functions .............................................................................................. 33

A.4.3 IKEv2 Responder functions ............................................................................................................................. 33

A.4.3.1 IKE exchange types .................................................................................................................................... 33

A.4.3.1.1 IKE SA establishment functions ........................................................................................................... 33

A.4.3.1.2 Child SA establishment functions ......................................................................................................... 34

A.4.3.1.3 INFORMATIONAL exchange functions .............................................................................................. 34

Annex B: Example IFS (TIPHON Profile of SIP, Release 3) ............................................................. 35

B.1 Introduction ............................................................................................................................................ 35

B.2 Instructions for completing the IFS proforma ........................................................................................ 35

B.2.1 General structure of the IFS proforma .............................................................................................................. 35

B.2.2 Additional information ..................................................................................................................................... 35

B.3 IFS proforma .......................................................................................................................................... 36

B.3.1 Implementation identification .......................................................................................................................... 36

B.3.2 Protocol Summary, EN 301 xxx ....................................................................................................................... 36

B.4 SIP entities .............................................................................................................................................. 36

B.4.1 Roles ................................................................................................................................................................. 36

B.4.2 User Agent capabilities .................................................................................................................................... 37

B.4.2.1 Registration ................................................................................................................................................. 37

B.4.2.2 Basic call ..................................................................................................................................................... 37

B.4.3 Registrar capabilities ........................................................................................................................................ 38

B.4.3.1 Registration ................................................................................................................................................. 38

B.4.4 Proxy capabilities ............................................................................................................................................. 38

B.4.4.1 Proxy in the serving and intermediate network........................................................................................... 38

B.4.4.1.1 Registration ........................................................................................................................................... 38

B.4.4.1.2 Basic call ............................................................................................................................................... 38

B.4.4.2 Proxy in the home network ......................................................................................................................... 39

B.4.4.2.1 Registration ........................................................................................................................................... 39

B.4.4.2.2 Basic call ............................................................................................................................................... 39

B.4.5 Gateway capabilities......................................................................................................................................... 39

B.4.5.1 Basic call ..................................................................................................................................................... 39

ETSI
---------------------- Page: 4 ----------------------
5 Final draft ETSI EG 202 237 V1.2.1 (2010-06)

Annex C: Bibliography .......................................................................................................................... 40

History .............................................................................................................................................................. 41

ETSI
---------------------- Page: 5 ----------------------
6 Final draft ETSI EG 202 237 V1.2.1 (2010-06)
Intellectual Property Rights

IPRs essential or potentially essential to the present document may have been declared to ETSI. The information

pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found

in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in

respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web

server (http://webapp.etsi.org/IPR/home.asp).

Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee

can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web

server) which are, or may be, or may become, essential to the present document.
Foreword

This ETSI Guide (EG) has been produced by ETSI Technical Committee Methods for Testing and Specification (MTS),

and is now submitted for the ETSI standards Membership Approval Procedure.
ETSI
---------------------- Page: 6 ----------------------
7 Final draft ETSI EG 202 237 V1.2.1 (2010-06)
1 Scope

The present document, "A generic approach to interoperability testing", gives general guidance on the specification and

execution of interoperability tests for communication systems specifically in the context of product certification. It

provides a framework within which interoperability test specifications for a wide range of product types can be

developed. The guidelines are expressed as recommendations rather than strict rules and leave enough freedom to allow

test specifiers to adopt and adapt processes to suit each particular project while still ensuring that test specifications

accurately reflect the requirements of the base standards and can be executed consistently across a range of

configurations.

Interoperability testing is the structured and formal testing of functions supported remotely by two or more items of

equipment communicating by means of standardized protocols. It is not the detailed verification of protocol

requirements specified in a conformance test suite, neither is it the less formal development testing often associated

with "plug-fest" and "interop" events (frequently referred to as "bake-offs"). A methodology for the latter type of testing

is described in EG 202 810 [i.8].

Although some consideration is given within the methodology to the operating and reporting aspects of interoperability

testing, the primary focus of the present document is on the specification of interoperability testing architectures, test

plans and test suites.
2 References

References are either specific (identified by date of publication and/or edition number or version number) or

non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the

reference document (including any amendments) applies.

Referenced documents which are not found to be publicly available in the expected location might be found at

http://docbox.etsi.org/Reference.

NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee

their long term validity.
2.1 Normative references

The following referenced documents are necessary for the application of the present document.

Not applicable.
2.2 Informative references

The following referenced documents are not necessary for the application of the present document but they assist the

user with regard to a particular subject area.

[i.1] ETSI ES 201 873-1: "Methods for Testing and Specification (MTS); The Testing and Test Control

Notation version 3; Part 1: TTCN-3 Core Language".
[i.2] Void.

[i.3] ETSI TS 101 884: "Telecommunications and Internet Protocol Harmonization Over Networks

(TIPHON) Release 3; Technology Mapping; Implementation of TIPHON architecture using SIP".

[i.4] ETSI EG 202 107: "Methods for Testing and Specification (MTS); Planning for validation and

testing in the standards-making process".

[i.5] ISO/IEC 9646 (parts 1 to 7): "Information technology - Open Systems Interconnection -

Conformance testing methodology and framework".
[i.6] IETF RFC 3261: "SIP: Session Initiation Protocol".
ETSI
---------------------- Page: 7 ----------------------
8 Final draft ETSI EG 202 237 V1.2.1 (2010-06)
[i.7] IETF RFC 4306: "Internet Key Exchange (IKEv2) Protocol".

[i.8] ETSI EG 202 810: "Methods for Testing and Specification (MTS); Automated Interoperability

Testing; Methodology and Framework".

[i.9] ETSI ES 202 553: "Methods for testing and Specification (MTS); TPLan: A notation for

expressing test Purposes".
3 Definitions and abbreviations
3.1 Definitions

For the purposes of the present document, the following terms and definitions apply:

conformance: compliance with requirements specified in applicable standards ISO/IEC 9646 [i.5]

conformance testing: testing the extent to which an Implementation Under Test (IUT) satisfies both static and dynamic

conformance requirements ISO/IEC 9646 [i.5]

NOTE: The purpose of conformance testing is to determine to what extent a single implementation of a particular

standard conforms to the individual requirements of that standard.

device: item of software or hardware which either alone or in combination with other devices implements the

requirements of a standardized specification

Equipment Under Test (EUT): grouping of one or more devices which has not been previously shown to interoperate

with previously Qualified Equipment (QE)

Implementation Under Test (IUT): an implementation of one or more Open Systems Interconnection (OSI) protocols

in an adjacent user/provider relationship, being the part of a real open system which is to be studied by testing

(ISO/IEC 9646-1 [i.5])

interoperability: ability of two systems to interoperate using the same communication protocol

interoperability testing: activity of proving that end-to-end functionality between (at least) two communicating

systems is as required by the base standard(s) on which those systems are based

interoperability test suite: collection of test cases designed to prove the ability of two (or more) systems to

interoperate

InterWorking Function (IWF): translation of one protocol into another one so that two systems using two different

communication protocols are able to interoperate

Qualified Equipment (QE): grouping of one or more devices that has been shown and certified, by rigorous and

well-defined testing, to interoperate with other equipment

NOTE 1: Once an EUT has been successfully tested against a QE, it may be considered to be a QE, itself.

NOTE 2: Once a QE is modified, it loses its status as QE and becomes again an EUT.

System Under Test (SUT): one or more QEs and an EUT

test case: specification of the actions required to achieve a specific test purpose, starting in a stable testing state, ending

in a stable testing state and defined in either natural language for manual operation or in a machine-readable language

(such as TTCN-3) for automatic execution

test purpose: description of a well-defined objective of testing, focussing on a single interoperability requirement or a

set of related interoperability requirements
ETSI
---------------------- Page: 8 ----------------------
9 Final draft ETSI EG 202 237 V1.2.1 (2010-06)
3.2 Abbreviations
For the purposes of the present document, the following abbreviations apply:
API Application Programming Interface
EP End Point
EUT Equipment Under Test
GFT Graphical presentation Format for TTCN-3
IFS Interoperable Features Statement
IUT Implementation Under Test
IWF InterWorking Function
MMI Man-Machine Interface
MoC Means of Communication
MoT Means of Testing
OSI Open Systems Interconnection
PICS Protocol Implementation Conformance Statement
QE Qualified Equipment
SIP Session Initiation Protocol
SUT System Under Test
TP Test Purpose
TSS Test Suite Structure
4 Types of testing

Equipment implementing standardized protocols and services can be formally tested in two related but different ways,

which each have benefits and limitations:

• conformance testing can show that a product correctly implements a particular standardized protocol:

- establishes whether or not the implementation in question meets the requirements specified for the

protocol itself. For example, it will test protocol message contents and format as well as the permitted

sequences of messages.

• interoperability testing can demonstrate that a product will work with other like products:

- assesses the ability of the implementation to support the required trans-network functionality between

itself and another, similar implementation to which it is connected.

Conformance testing in conjunction with interoperability testing provides both the proof of conformance and the

guarantee of interoperation.
4.1 Interoperability testing

The term "interoperability testing" is often used in relation to the semi-formal testing carried out at multi-vendor events

as part of the product development process. While such events, often referred to as "plug-fests", "interops" and

"bake-offs", are valuable sources of information on the ability of similar products to communicate, they generally do

not offer the structured and, therefore, repeatable, testing that is an essential part of a certification scheme. For a

certification (or branding or logo) scheme to be meaningful, it is necessary that interoperability testing is carried out in

accordance with a comprehensive and structured suite of tests. In the context of the present document, it is exactly this

type of testing which is referred to as "interoperability testing". For other types of schemes, such as those arranged

between manufacturers for marketing or other purposes this approach is still valid.

NOTE: It is possible that other organizations within the global standardization community will have

interpretations of this term which differ to a greater or lesser extent.

The purpose of interoperability testing is to prove that end-to-end functionality between (at least) two communicating

systems is as required by the standard(s) on which those systems are based.
...

SLOVENSKI STANDARD
SIST-V ETSI/EG 202 237 V1.2.1:2016
01-oktober-2016
0HWRGH]DSUHVNXãDQMHLQVSHFLILFLUDQMH 076 3UHVNXãDQMHLQWHUQHWQHJD
SURWRNROD*HQHULþQLSULVWRSNSUHVNXãDQMXPHGREUDWRYDOQRVWL

Methods for Testing and Specification (MTS) - Internet Protocol Testing (IPT) - Generic

approach to interoperability testing
Ta slovenski standard je istoveten z: ETSI EG 202 237 V1.2.1 (2010-08)
ICS:
33.040.01 Telekomunikacijski sistemi Telecommunication systems
na splošno in general
SIST-V ETSI/EG 202 237 V1.2.1:2016 en

2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

---------------------- Page: 1 ----------------------
SIST-V ETSI/EG 202 237 V1.2.1:2016
---------------------- Page: 2 ----------------------
SIST-V ETSI/EG 202 237 V1.2.1:2016
---------------------- Page: 3 ----------------------
SIST-V ETSI/EG 202 237 V1.2.1:2016
---------------------- Page: 4 ----------------------
SIST-V ETSI/EG 202 237 V1.2.1:2016
---------------------- Page: 5 ----------------------
SIST-V ETSI/EG 202 237 V1.2.1:2016
---------------------- Page: 6 ----------------------
SIST-V ETSI/EG 202 237 V1.2.1:2016
---------------------- Page: 7 ----------------------
SIST-V ETSI/EG 202 237 V1.2.1:2016
---------------------- Page: 8 ----------------------
SIST-V ETSI/EG 202 237 V1.2.1:2016
---------------------- Page: 9 ----------------------
SIST-V ETSI/EG 202 237 V1.2.1:2016
---------------------- Page: 10 ----------------------
SIST-V ETSI/EG 202 237 V1.2.1:2016
---------------------- Page: 11 ----------------------
SIST-V ETSI/EG 202 237 V1.2.1:2016
---------------------- Page: 12 ----------------------
SIST-V ETSI/EG 202 237 V1.2.1:2016
---------------------- Page: 13 ----------------------
SIST-V ETSI/EG 202 237 V1.2.1:2016
---------------------- Page: 14 ----------------------
SIST-V ETSI/EG 202 237 V1.2.1:2016
---------------------- Page: 15 ----------------------
SIST-V ETSI/EG 202 237 V1.2.1:2016
---------------------- Page: 16 ----------------------
SIST-V ETSI/EG 202 237 V1.2.1:2016
---------------------- Page: 17 ----------------------
SIST-V ETSI/EG 202 237 V1.2.1:2016
---------------------- Page: 18 ----------------------
SIST-V ETSI/EG 202 237 V1.2.1:2016
...

Questions, Comments and Discussion

Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.