ETSI GS NFV-TST 002 V1.1.1 (2016-10)

Network Functions Virtualisation (NFV); Testing Methodology; Report on NFV Interoperability Testing Methodology

DGS/NFV-TST002

General Information

• Status: Published
• Publication Date: 20-Oct-2016
• Technical Committee: NFV TST - Testing, Implementation, and Open Source Working Group
• Current Stage: 12 - Completion
• Due Date: 28-Oct-2016
• Completion Date: 21-Oct-2016

Standards Content (Sample)

GROUP SPECIFICATION
Network Functions Virtualisation (NFV);
Testing Methodology;
Report on NFV Interoperability Testing Methodology
Disclaimer
The present document has been produced and approved by the Network Functions Virtualisation (NFV) ETSI Industry
Specification Group (ISG) and represents the views of those members who participated in this ISG.
It does not necessarily represent the views of the entire ETSI membership.

2 ETSI GS NFV-TST 002 V1.1.1 (2016-10)

Reference
DGS/NFV-TST002
Keywords
interoperability, NFV, testing, methodology

ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: [phone]  Fax: [phone]

Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88

Important notice
The present document can be downloaded from:
http://www.etsi.org/standards-search
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the only prevailing document is the
print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
https://portal.etsi.org/TB/ETSIDeliverableStatus.aspx
If you find errors in the present document, please send your comment to one of the following services:
https://portal.etsi.org/People/CommiteeSupportStaff.aspx
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying
and microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.

© European Telecommunications Standards Institute 2016.
All rights reserved.
TM TM TM
DECT , PLUGTESTS , UMTS and the ETSI logo are Trade Marks of ETSI registered for the benefit of its Members.
TM
3GPP and LTE™ are Trade Marks of ETSI registered for the benefit of its Members and
of the 3GPP Organizational Partners.
GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.
ETSI
3 ETSI GS NFV-TST 002 V1.1.1 (2016-10)
Contents
Intellectual Property Rights . 5
Foreword . 5
Modal verbs terminology . 5
Executive summary . 5
Introduction . 5
1 Scope . 6
2 References . 6
2.1 Normative references . 6
2.2 Informative references . 6
3 Definitions and abbreviations . 7
3.1 Definitions . 7
3.2 Abbreviations . 7
4 Interoperability Testing Methodology Guidelines for NFV . 8
4.1 Introduction . 8
4.2 Basic concepts for interoperability testing . 9
4.2.1 Overview . 9
4.2.2 System Under Test (SUT) . 9
4.2.3 Function Under Test (FUT) . 10
4.2.4 Test interfaces . 10
4.2.5 Test Environment . 10
4.2.6 Test Descriptions . 10
4.2.7 Test drivers . 11
4.3 Interoperability Test Specifications . 11
4.3.1 Overview . 11
4.3.2 Generic SUT Architecture . 11
4.3.3 Interoperable Features Statement (IFS) . 12
4.3.4 SUT Configurations . 13
4.3.5 Test Suite Structure . 13
4.3.6 Test Purposes . 14
4.3.7 Test Descriptions . 14
4.4 Interoperability Testing Process . 17
5 NFV SUT Architecture . 18
5.1 NFV Generic SUT Architecture . 18
5.2 NFV SUT Configuration 1 . 19
5.3 NFV SUT Configuration 2a . 19
5.4 NFV SUT Configuration 2b . 20
5.5 NFV SUT Configuration 3 . 21
5.6 NFV SUT Configuration 4 . 22
6 NFV Interoperability Features . 23
6.1 VNF Package Management . 23
6.1.1 Description . 23
6.1.2 SUT Configuration . 23
6.1.3 Observed Interfaces . 23
6.1.4 Test Interfaces . 24
6.2 Software Image Management . 24
6.2.1 Description . 24
6.2.2 SUT Configuration . 24
6.2.3 Observed Interfaces . 24
6.2.4 Test Interfaces . 25
6.3 VNF Lifecycle Management . 25
6.3.1 Description . 25
6.3.2 SUT Configuration . 25
ETSI
4 ETSI GS NFV-TST 002 V1.1.1 (2016-10)
6.3.3 Observed Interfaces . 25
6.3.4 Test Interfaces . 28
6.4 VNF Configuration Management . 31
6.4.1 Description . 31
6.4.2 SUT Configuration . 31
6.4.3 Observed Interfaces . 31
6.4.4 Test Interfaces . 31
6.5 VNF Fault Management . 33
6.5.1 Description . 33
6.5.2 SUT Configuration . 33
6.5.3 Observed Interfaces . 33
6.5.4 Test Interfaces . 33
6.6 VNF Performance Management . 34
6.6.1 Description . 34
6.6.2 SUT Configuration . 34
6.6.3 Observed Interfaces . 34
6.6.4 Test Interfaces . 35
6.7 Network Service Lifecycle Management . 36
6.7.1 Description . 36
6.7.2 SUT Configuration . 37
6.7.3 Observed Interfaces . 37
6.7.4 Test Interfaces . 37
6.8 Network Service Fault Management . 39
6.8.1 Description . 39
6.8.2 SUT Configuration . 39
6.8.3 Observed Interfaces . 39
6.8.4 Test Interfaces . 39
6.9 Network Service Performance Management . 40
6.9.1 Description . 40
6.9.2 SUT Configuration . 40
6.9.3 Observed Interfaces . 40
6.9.4 Test Interfaces . 41
Annex A (informative): NFV IFS Pro-forma example . 43
Annex B (informative): Authors & contributors . 45
Annex C (informative): Bibliography . 46
History . 47

ETSI
5 ETSI GS NFV-TST 002 V1.1.1 (2016-10)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (https://ipr.etsi.org/).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This Group Specification (GS) has been produced by ETSI Industry Specification Group (ISG) Network Functions
Virtualisation (NFV).
Modal verbs terminology
In the present document "shall", "shall not", "should", "should not", "may", "need not", "will", "will not", "can" and
"cannot" are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of
provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.
Executive summary
The present document studies how interoperability test methodology can be applied to NFV by analysing some of the
core NFV capabilities and the interactions between the functional blocks defined within the NFV architectural
framework required to enable them.
Introduction
The present document provides methodology guidelines for interoperability testing of NFV features, starting from a
review of some basic concepts for interoperability testing and their fit in an NFV environment and a methodology for
the development of interoperability test specifications illustrated with examples of basic NFV operations. A high level
analysis of some core NFV capabilities allows to identify a generic architecture for the associated System Under Test
configurations, and to classify some initial Interoperability Feature areas.
The present document is organized as follows:
• Clause 4 provides an overview of common interoperability concepts and testing methodology guidelines.
• Clause 5 identifies a generic system under test (SUT) architecture and some initial SUT configurations for
interoperability testing of basic NFV capabilities.
• Clause 6 identifies and analyses some initial NFV interoperability feature areas and outlines for each of them
the impacted functional blocks and interfaces, as well as the applicable SUT configurations described in
clause 5.
ETSI
6 ETSI GS NFV-TST 002 V1.1.1 (2016-10)
1 Scope
The present document provides some guidelines for NFV interoperability testing and identifies a generic System Under
Test (SUT) architecture for NFV, some initial SUT configurations, and some interoperability feature areas derived from
core NFV capabilities.
2 References
2.1 Normative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
Referenced documents which are not found to be publicly available in the expected location might be found at
https://docbox.etsi.org/Reference.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are necessary for the application of the present document.
Not applicable.
2.2 Informative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] ISO/IEC 9646 (parts 1 to 7): "Information technology - Open Systems Interconnection -
Conformance testing methodology and framework".
[i.2] ETSI EG 202 237: "Methods for Testing and Specification (MTS); Internet Protocol Testing
(IPT); Generic approach to interoperability testing".
[i.3] ETSI EG 202 568: "Methods for Testing and Specification (MTS); Internet Protocol Testing
(IPT); Testing: Methodology and Framework". .
[i.5] ETSI GS NFV 002: "Network Functions Virtualisation (NFV); Architectural Framework".
[i.6] ETSI GS NFV-MAN 001: "Network Functions Virtualisation (NFV); Management and
Orchestration".
[i.7] ETSI GS NFV-IFA 010 (V2.1.1): "Network Functions Virtualisation (NFV); Management and
Orchestration; Functional requirements specification".
[i.8] ETSI GS NFV-IFA 005 (V2.1.1): "Network Functions Virtualisation (NFV); Management and
Orchestration; Or-Vi reference point - Interface and Information Model Specification".
[i.9] ETSI GS NFV-IFA 006: "Network Functions Virtualisation (NFV); Management and
Orchestration; Vi-Vnfm reference point - Interface and Information Model Specification".
ETSI
7 ETSI GS NFV-TST 002 V1.1.1 (2016-10)
[i.10] ETSI GS NFV-IFA 007: "Network Functions Virtualisation (NFV); Management and
Orchestration; Or-Vnfm reference point - Interface and Information Model Specification".
[i.11] ETSI GS NFV-IFA 008: "Network Functions Virtualisation (NFV); Management and
Orchestration; Ve-Vnfm reference point - Interface and Information Model Specification".
[i.12] ETSI GS NFV-IFA 013: "Network Functions Virtualisation (NFV); Management and
Orchestration; Os-Ma-Nfvo reference point - Interface and Information Model Specification".
[i.13] ETSI GS NFV 003: "Network Functions Virtualisation (NFV); Terminology for main concepts in
NFV".
3 Definitions and abbreviations
3.1 Definitions
For the purposes of the present document, the terms and definitions given in ETSI GS NFV 003 [i.13] apply.
3.2 Abbreviations
For the purposes of the present document, the terms and definitions given in ETSI GS NFV 003 [i.13] and the following
apply:
API Application Programming Interface
CON CONformance
DUT Device Under Test
FUT Function Under Test
IFS Interoperable Features Statement
IOP InterOPerability
IUT Implementation Under Test
LCM Life Cycle Management
MMI Man-Machine Interface
NSD Network Service Descriptor
OSS Operation System Support
PICS Protocol Implementation Conformance Statement
QE Qualified Equipment
QF Qualified Function
SUT System Under Test
TD Test Description
TSS Test Suite Structure
VNFFG Virtual Network Function Forwarding Graph
ETSI
8 ETSI GS NFV-TST 002 V1.1.1 (2016-10)
4 Interoperability Testing Methodology Guidelines for
NFV
4.1 Introduction
Well established test methodology like ETSI EG 202 237 [i.2] and ETSI EG 202 568 [i.3] describe two main and
complementary ways of testing devices implementing standardized services, which each have benefits and limitations:
• Conformance Testing can show that a product correctly implements a particular standard, that is, it
establishes whether or not the Implementation Under Test (IUT) meets the requirements specified by the
standard. For example, it will test protocol message contents and format as well as the permitted sequences of
messages. In this context:
- There is only one Implementation Under Test, which is part of the System Under Test.
- Tests are performed at open standardized interfaces which might not be accessible to an end user, and
executed by a dedicated test system that has full control of the System Under Test and the ability to
observe all incoming and out coming communications.
- The high degree of control of the test system over the sequence and contents of the protocol messages
allows to test both valid and invalid behaviour.

Figure 1: Conformance testing
• Interoperability Testing can demonstrate that a product will work with other like products: it proves that
end-to-end functionality between (at least) two functions is as required by the standard(s) on which those
functions are based. In this context:
- The System Under Test (SUT) is made of the combination of different Functions Under Test (FUT)
coming from different suppliers.
- Interoperability tests are based on functionality as experienced by a user, where the user may be human
or a software application.
- Tests are performed and observed at functional interfaces such as Man-Machine Interfaces (MMIs),
protocol service interfaces and Application Programming Interfaces (APIs).
- Testing at functional interfaces implies that interoperability tests can only describe functional behaviour
and sometimes it might not be possible to trigger or test protocol error behaviour on the interface(s)
among the FUTs.
Figure 2: Interoperability testing
NOTE: The concept of Function Under Test used in the present document corresponds to the concept of Device
Under Test (DUT) introduced in ETSI EG 202 568 [i.3].
ETSI
9 ETSI GS NFV-TST 002 V1.1.1 (2016-10)
Conformance testing in conjunction with interoperability testing provides both the proof of conformance and the
guarantee of interoperation. ETSI EG 202 237 [i.2] and ETSI EG 202 568 [i.3] describe several approaches on how to
combine the two methods, the most common one being Interoperability Testing with Conformance Checks, where
reference points between the FUTs are monitored to verify the appropriate sequence and contents of protocol messages,
API calls, interface operations, etc.
Clauses 4.2 to 4.4 provide an overview of the main concepts and practices associated with interoperability testing. The
intention is to develop simple and pragmatic guidelines that can be used as a "cook-book", rather than a rigid
prescription of how to perform NFV interoperability testing.
The main areas of these guidelines are as follows:
• Definition of basic concepts.
• Instructions for the development of interoperability test specifications, including:
- Definition of a generic System Under Test (SUT) architecture.
- Identification of interoperability features.
- Specification of SUT configurations and Test Descriptions.
• Description of the interoperability testing process.
As their name implies, guidelines are only for guidance and the actual process followed should use and adapt whichever
of these guidelines are most applicable in each particular situation. In some cases this may mean the application of all
aspects.
4.2 Basic concepts for interoperability testing
4.2.1 Overview
There are a number of different terms and concepts that can be used when describing a test methodology. Clauses 4.2 to
4.4 describe the most important concepts used by these guidelines, which can been categorized either as part of the
System Under Test (SUT) or as part of the Test Environment.
Figure 3 provides an overview of these basic concepts, which are described in detail in clauses 4.2.2 to 4.2.7.

Figure 3: Illustration of basic concepts
4.2.2 System Under Test (SUT)
In the context of interoperability testing, the System Under Test (SUT) is made of a number of interacting Functions
Under Test (FUTs) coming from different suppliers.
ETSI
10 ETSI GS NFV-TST 002 V1.1.1 (2016-10)
Depending on the complexity of the end-to-end system, the overall number of FUTs comprising the SUT, and the
interactions among them, it might be advisable to define different SUT configuration addressing specific functional
areas or groups of tests.
The first steps towards defining an Interoperability Tests Specification are identifying the Functions Under Test and
describing a generic architecture where all the required SUT configurations will fit.
4.2.3 Function Under Test (FUT)
In the context of NFV, a Function Under Test is a combination of software and/or hardware items which implement the
functionality of one or several NFV functional blocks and interact with other FUTs via one or more reference points, as
described in ETSI GS NFV 002 [i.5].
NOTE: When using Interoperability Test Specifications in a certification scheme, the notion of Qualified
Equipment (QE) or Qualified Function (QF) applies. A QF is a FUT that has successfully been tested
with other QFs. The usage of interoperability Test Specifications in a certification scheme is out of the
scope of the present document. Further details on this topic can be found at ETSI EG 202 237 [i.2].
4.2.4 Test interfaces
The interfaces that are made available by the SUT to enable the testing are usually known as the test interfaces. These
interfaces are accessed by the test drivers to trigger and verify the test behaviour, as described in clause 4.2.7. Other
(non-test) interfaces offered by the SUT can be used for monitoring, log analysis, etc.
In the simplest case, the test interfaces will be the normal user interfaces offered by some of the FUTs (command line,
GUI, web interface, etc.). FUTs may also offer APIs over which interoperability testing can be performed either
manually using a dedicated application, or automatically using a programmable test function.
In some cases, observing and verifying the functional behaviour or responses of one FUT may require analysing its logs
or records. In that case, it is recommended to pre-define those log messages or records to avoid ambiguity in their
interpretation.
Additionally, while in the context of interoperability testing interfaces between the FUTs are not considered to be test
interfaces, combining interoperability testing with conformance checks may require to monitor those interfaces to assess
the conformance of the exchanged information or messages.
4.2.5 Test Environment
Interoperability testing involves control and observation at the functional (rather than protocol) level. The Test
Environment is the combination of equipment, functions and procedures which allow testing the interoperability of the
FUTs. Entities in the test environment access the different Functions Under Test via the Test Interfaces offered by the
SUT. These entities ensure the selection, interpretation and execution of the test descriptions, coordination and
synchronization of the actions on the test interfaces, and provide mechanisms for logging, reporting, monitoring and
observing the interactions among the FUTs, etc.
4.2.6 Test Descriptions
A test description provides the detailed set of instructions (or steps) that need to be followed in order to perform a test.
Most often, interoperability tests are described in terms of actions that can be performed by the user(s) of the endpoint
device(s).
In the case where the test is executed by a human operator, test will be described in natural language. In the case where
the tests are automated, a programming or test language will be used to implement the test descriptions.
The steps in the test description can be of different nature, depending on the kind of action required: trigger a behaviour
on one FUT, verify the functional response on another FUT, configure the SUT (add/remove a FUT), check a log, etc.
Each step identifies the FUT and/or the interface targeted by the action.
ETSI
11 ETSI GS NFV-TST 002 V1.1.1 (2016-10)
4.2.7 Test drivers
The test driver realizes the steps specified in a test description at one specific test interface. Testing efficiency and
consistency can be improved by implementing the role of the test driver via an automatic device programmed to carry
out the specified test steps. This approach may require standardized test interfaces in the FUTs, or at least
well-documented, open interfaces providing the needed functionality.
In any given instance of testing, there may be more than one test interface over which the tests will be executed. In that
case, coordination among the different test drivers and synchronization of the actions performed by them will be
required. This test coordination role can be played by one of the test drivers, or by and additional entity in the test
environment.
4.3 Interoperability Test Specifications
4.3.1 Overview
The main steps involved in the process of developing an interoperability test specification are as follows:
• describing a generic architecture for the System Under Test;
• collecting interoperable (IOP) features and requirements in the Interoperable Features Statement (IFS);
• identifying the SUT Configurations;
• defining a structure for the Test Specification (TSS);
• writing Test Descriptions (TDs) for each item in the IFS.

Figure 4: Interoperability Test Specification Development process
4.3.2 Generic SUT Architecture
A generic SUT architecture provides an abstract framework within which any specific SUT configuration can fit in. The
starting point for defining a generic SUT architecture is most often the functional architecture described in the base
standards, in combination with pragmatic input on how the industry and open source projects are actually implementing
these functional blocks (grouping, bundling, etc.).
In a complex system, it may be required to define several SUT configurations to cover all the specified groups of tests.
Defining the generic architecture and identifying the SUT configurations at an early stage helps to provide a structure
for the test descriptions later. The generic test architecture is usually specified as a diagram and identifies:
• the Functions Under Test, and the functional blocks implemented by them;
ETSI
12 ETSI GS NFV-TST 002 V1.1.1 (2016-10)
• the interfaces and communications paths between the FUTs;
• if required, the protocols, APIs and/or data models to be used for communication between the FUTs.

Figure 5: Example of Generic SUT Architecture
4.3.3 Interoperable Features Statement (IFS)
The purpose of the Interoperable Features Statement (IFS) is to identify those standardized functions which a FUT
supports, including those which are optional and those which are conditional on the support of other functions.
Although not strictly part of the interoperability test specification, the IFS also helps to provide a structure to the tests
descriptions which will subsequently be developed. In the context of interoperability testing, the IFS provides a similar
functionality than the one provided by the Protocol Implementation Conformance Statement (PICS) for conformance
testing.
The IFS can also be used as a pro-forma for each FUT to identify which standardized functions it will support when
interoperating with peering FUTs from other suppliers.
Standardized functions and IOP Features are compiled by analysing the base standards, use cases, flows, etc.
The IFS provides the means to compile and organize all the following information:
• FUT Identification.
• Supported Functional Blocks: i.e. VNF, VNFM, NFVO, etc.
• Supported Role (when/if applicable): i.e. producer/consumer, source/sink, etc.
• Supported Functional Groups and subgroups (optional): i.e. VNF Life Cycle Management, NS Life Cycle
Management, etc.
• Supported IOP Features: i.e. VNF on-board, VNF update, etc.
• Supported options: i.e. Resource Commitment Model.
• Applicable reference point.
• Supported test interfaces.
• For each identified IOP Feature the following information is provided:
- A unique identifier - the usage of a naming convention allowing to put the Feature into context
(Functional Block, (Role), Functional Group, etc.) is recommended.
ETSI
13 ETSI GS NFV-TST 002 V1.1.1 (2016-10)
- A short description of the feature.
- A reference to the base specification.
- The feature status: Mandatory (M), Optional (O), Conditional (C).
- In the IFS pro-forma, an additional field allows to state whether the implementation in question supports
or not the feature (Y/N).
The IFS can be compiled in different formats, such as a collection of related tables or a database.
Before having completed the development of the Test Specification, the IFS can only be considered a stable draft. As
the test specification matures, it is possible that errors and omissions in the IFS will be identified. Once the test
specification is complete, has been validated, and all the detected errors have been fixed, the IFS can also be considered
complete. An example of an NFV IFS pro-forma is provided in annex A.
4.3.4 SUT Configurations
The Test Specification clearly identifies and eventually provides a diagram for each valid configuration derived from
the generic SUT architecture. A valid configuration is a specific subset of the generic SUT architecture to which a given
group of test descriptions applies. Identifying and describing valid SUT configurations at an early stage in the Test
Specification development process helps to:
• Structure the test specifications in groups.
• Understand the applicability and scope of each test group.
The SUT configurations clearly identifies:
• the required Functions Under Test;
• the observed interfaces exposed/consumed by the FUTs;
• the test interfaces.
Test Interfaces
Test Interface 1
11Interface 1
FUT 1
Descriptor
Test Interface 2
Interface 2
FUT 2
Observed Interface 1
Figure 6: SUT configuration example
4.3.5 Test Suite Structure
The Test Suite Structure is the equivalent of the Table of Contents of the Test Descriptions. The goal of this step is to
facilitate:
• Grouping the TDs together in a logical way.
ETSI
14 ETSI GS NFV-TST 002 V1.1.1 (2016-10)
• Addressing all the targeted IOP features described in the standard(s).
• Providing a minimum coverage of each group.
There is no hard and fast rule that can be used to determine how a test specification is divided up into test groups. In
many cases, the division will be rather arbitrary and based on the preferences of the author(s). However, the following
categorizations can be considered when identifying appropriate test groups within the Test Suite Structure (TSS):
• SUT configuration: A test group for each valid configuration specified.
• Functionality: A test group for each of the major functions supported. For example:
- VNF on-boarding;
- VNF LCM;
- NS LCM;
- etc.
• Success or failure: A test group for normal behaviour and another for exceptional behaviour.
4.3.6 Test Purposes
As recommended in ISO/IEC 9646 [i.1], before writing the individual steps that are required to complete a test
description (how), a full description of the objective of each test case (what) is provided.
These Test Purposes (TPs) are based on the IOP features identified in the relevant standard(s), and compiled in the
Interoperable Feature Statement (IFS).
EXAMPLE: Test Purpose: To verify that a VNF Package can be on-boarded.
In practice, the Test Purposes can be part of the Test Descriptions, as described in clause 4.3.7.
4.3.7 Test Descriptions
For each Test Purpose, one or several Test Descriptions can be specified. Test Descriptions compile all the information
required to execute a test. They describe all the steps required to achieve a test purpose (how). The following
information is provided with each Test Description:
• Identifier: A unique identifier is assigned to each Test Description. The usage of a well-defined naming
convention allowing to put the TD into context (Functional Group, Feature, etc.) is recommended.
• Test Purpose: Description of the objective of the TD (what), see clause 4.3.6.
• Configuration: Reference to the applicable SUT configuration(s), see clause 4.3.4.
• References: Reference to the base specification(s) which describe the feature being tested.
• Applicability: List of items in the IFS that need to be supported by the DUTs in the SUT in order to be able to
execute the test, see 4.3.3. If the list contains an optional item, then the test is optional.
• Pre-test conditions: (Optional) Specific conditions that need to be met by the SUT prior to start executing the
test sequence. It can include information about configuration, and/or initial state of the SUT.
• Test Sequence: Detailed description of the steps that are to be followed in order to achieve the stated test
purpose. These steps are specified in a clear and unambiguous way but without placing unreasonable
restrictions on how the step is performed. Clarity and precision are important to ensure that the step can
followed exactly. The lack of restrictions is necessary to ensure that the test can apply to a range of different
types of implementation.
ETSI
15 ETSI GS NFV-TST 002 V1.1.1 (2016-10)
Table 1: Test Description Template Example
Interoperability Test Description
Identifier
Unique test description ID: TD_AB_XXX_00. Follows a well-defined naming
convention
Test Purpose a concise summary of the test reflecting its purpose and allowing readers to easily
distinguish this test from any other test in the document
Configuration List of all the FUTs required devices for running this test, possibly also including a
(reference to) an illustration of the SUT configuration
References List of references to the base specification clause(s), use case(s), requirement(s),
etc. which are either used in the test or define the functionality being tested
Applicability List of features and capabilities in the IFS which are required to be supported by the
SUT in order to execute this test (e.g. if this list contains an optional feature to be
supported, then the test is optional).

Pre-test conditions List of test specific pre-conditions that need to be met by the SUT including
information about configuration, i.e. precise description of the initial state of the SUT
prior to start executing the test sequence
•
•
Test
Step Type Description Result
Sequence
1 Step description
IOP Verdict
The Steps in the Test Sequence can be of different type, depending on their purpose:
• A stimulus corresponds to an event that triggers a specific action on a FUT, like sending a message for
instance.
• A configure corresponds to an action to modify the FUT or SUT configuration.
• An IOP check consists of observing that one FUT behaves as described in the standard: i.e. resource creation,
update, deletion, etc. For each IOP check in the Test Sequence, a result can be recorded.
• The overall IOP Verdict will be considered OK if all the IOP checks in the sequence are OK.
See here after an example of how an Interoperability Test Description could be specified for the On-board VNF
Package flow described in ETSI GS NFV-MAN 001 [i.6], clause B.2.1.
ETSI
16 ETSI GS NFV-TST 002 V1.1.1 (2016-10)
Figure 7: On-board VNF Package flow
...