ETSI GS NFV-TST 010 V4.4.1 (2024-08)

Network Functions Virtualisation (NFV) Release 4; Testing; API Conformance Testing Specification

RGS/NFV-TST010ed441

General Information

• Status: Not Published
• Technical Committee: NFV TST - Testing, Implementation, and Open Source Working Group
• Current Stage: 12 - Citation in the OJ (auto-insert)
• Due Date: 16-Aug-2024
• Completion Date: 09-Aug-2024

Standards Content (Sample)

GROUP SPECIFICATION
Network Functions Virtualisation (NFV) Release 4;
Testing;
API Conformance Testing Specification
Disclaimer
The present document has been produced and approved by the Network Functions Virtualisation (NFV) ETSI Industry
Specification Group (ISG) and represents the views of those members who participated in this ISG.
It does not necessarily represent the views of the entire ETSI membership.

2 ETSI GS NFV-TST 010 V4.4.1 (2024-08)

Reference
RGS/NFV-TST010ed441
Keywords
API, conformance, NFV, testing

ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: [phone]  Fax: [phone]

Siret N° 348 623 562 00017 - APE 7112B
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° w061004871

Important notice
The present document can be downloaded from the
ETSI Search & Browse Standards application.
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the prevailing version of an ETSI
deliverable is the one made publicly available in PDF format on ETSI deliver.
Users should be aware that the present document may be revised or have its status changed,
this information is available in the Milestones listing.
If you find errors in the present document, please send your comments to
the relevant service listed under Committee Support Staff.
If you find a security vulnerability in the present document, please report it through our
Coordinated Vulnerability Disclosure (CVD) program.
Notice of disclaimer & limitation of liability
The information provided in the present deliverable is directed solely to professionals who have the appropriate degree of
experience to understand and interpret its content in accordance with generally accepted engineering or
other professional standard and applicable regulations.
No recommendation as to products and services or vendors is made or should be implied.
No representation or warranty is made that this deliverable is technically accurate or sufficient or conforms to any law
and/or governmental rule and/or regulation and further, no representation or warranty is made of merchantability or fitness
for any particular purpose or against infringement of intellectual property rights.
In no event shall ETSI be held liable for loss of profits or any other incidental or consequential damages.

Any software contained in this deliverable is provided "AS IS" with no warranties, express or implied, including but not
limited to, the warranties of merchantability, fitness for a particular purpose and non-infringement of intellectual property
rights and ETSI shall not be held liable in any event for any damages whatsoever (including, without limitation, damages
for loss of profits, business interruption, loss of information, or any other pecuniary loss) arising out of or related to the use
of or inability to use the software.
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and
microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.

© ETSI 2024.
All rights reserved.
ETSI
3 ETSI GS NFV-TST 010 V4.4.1 (2024-08)
Contents
Intellectual Property Rights . 5
Foreword . 5
Modal verbs terminology . 5
Introduction . 5
1 Scope . 6
2 References . 6
2.1 Normative references . 6
2.2 Informative references . 7
3 Definition of terms, symbols and abbreviations . 7
3.1 Terms . 7
3.2 Symbols . 7
3.3 Abbreviations . 7
4 Methodology . 8
4.1 General . 8
4.2 System Under Test (SUT) . 8
4.3 Test configurations . 8
4.3.1 General . 8
4.3.2 Confi g_prod_VE . 9
4.3.3 Config_prod_VNFM . 10
4.3.4 Config_prod_NFVO . 10
4.3.5 Config_prod_VNFM_GRANT . 11
4.3.6 Config_prod_Notif_Endpoint . 11
4.3.7 Config_prod_NFV-MANO . 12
4.4 Void . 12
4.5 Generic Test Description . 12
4.5.1 General . 12
4.5.2 Test Description format . 13
4.5.3 Scope of the tests . 14
4.5.3.1 General . 14
4.5.3.2 General characteristics of the reference points . 15
4.5.3.3 Basic behaviours of the API producer/consumer and verification steps . 16
4.5.3.3.0 Introduction . 16
4.5.3.3.1 Producer sends event triggered notifications based on consumer subscriptions . 16
4.5.3.3.2 Producer sends periodic notifications based on the consumer subscriptions . 18
4.5.3.3.3 Producer executes the requested API operation . 19
4.5.3.3.4 Consumer fetches the files/package info . 20
4.5.3.4 Workflow test considerations . 22
4.5.4 Verification . 22
4.5.4.0 Introduction . 22
4.5.4.1 Common verification aspects . 22
4.5.4.2 Verification aspects for individual APIs . 23
4.5.4.3 Verification aspects not considered . 24
5 Void . 24
6 Void . 24
7 Void . 25
Annex A (informative): Known Issues . 26
Annex B (informative): Workflow Test Descriptions . 27
Annex C: Void . 28
ETSI
4 ETSI GS NFV-TST 010 V4.4.1 (2024-08)
Annex D (normative): Word format presentation of the test suite for the Os-Ma-nfvo
Reference Point . 29
Annex E (normative): Word format presentation of the test suite for the Ve-Vnfm
Reference Point . 30
Annex F (normative): Word format presentation of the test suite for the Or-Vnfm
Reference Point . 31
Annex G (normative): Word format presentation of the test suite for NFV-MANO
Management . 32
Annex H (normative): Word format presentation of the test suite for the Or-Or reference
point . 33
H.1 General . 33
H.2 SOL011 NSD Management . 33
H.3 SOL011 NS Lifecycle Management . 33
H.4 SOL011 NS Performance Management . 33
H.5 SOL011 NS Fault Management . 34
Annex I (normative): Word format presentation of the test suite for NFV-MANO Policy
Management . 35
Annex J (informative): Change History . 36
History . 37

ETSI
5 ETSI GS NFV-TST 010 V4.4.1 (2024-08)
Intellectual Property Rights
Essential patents
IPRs essential or potentially essential to normative deliverables may have been declared to ETSI. The declarations
pertaining to these essential IPRs, if any, are publicly available for ETSI members and non-members, and can be
found in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to
ETSI in respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the
ETSI Web server (https://ipr.etsi.org/).
Pursuant to the ETSI Directives including the ETSI IPR Policy, no investigation regarding the essentiality of IPRs,
including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not
referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become,
essential to the present document.
Trademarks
The present document may include trademarks and/or tradenames which are asserted and/or registered by their owners.
ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no
right to use or reproduce any trademark and/or tradename. Mention of those trademarks in the present document does
not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks.
DECT™, PLUGTESTS™, UMTS™ and the ETSI logo are trademarks of ETSI registered for the benefit of its

Members. 3GPP™ and LTE™ are trademarks of ETSI registered for the benefit of its Members and of the 3GPP
Organizational Partners. oneM2M™ logo is a trademark of ETSI registered for the benefit of its Members and of the ®
oneM2M Partners. GSM and the GSM logo are trademarks registered and owned by the GSM Association.
Foreword
This Group Specification (GS) has been produced by ETSI Industry Specification Group (ISG) Network Functions
Virtualisation (NFV).
Modal verbs terminology
In the present document "shall", "shall not", "should", "should not", "may", "need not", "will", "will not", "can" and
"cannot" are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of
provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.
Introduction
The interoperability among the functional entities which supports the requirements and functionalities specified in ETSI
NFV deliverables is one of the key and most important aspect to develop and deploy the NFV environment. In order to
achieve such interoperability, ETSI GR NFV-TST 007 [i.1] specifies the methodologies and test scenario for the testing
of interoperability mainly based on the ETSI NFV IFA specification series which specifies the reference point/interface
and functional requirements. At the same time, the validation of the NFV specification compliance of each functional
entities is also key aspects to be ensured for the interoperability, in particular protocol solution/API level. Therefore, the
present document specifies the API conformance testing.

ETSI
6 ETSI GS NFV-TST 010 V4.4.1 (2024-08)
1 Scope
The present document specifies methodologies of conformance test including Test Descriptions for NFV
implementations with interfaces specified in NFV specifications: ETSI GS NFV-SOL 002 [2] for the Ve-Vnfm reference
point, ETSI GS NFV-SOL 003 [1] for the Or-Vnfm reference point and ETSI GS NFV-SOL 005 [3] for the
Os-Ma-Nfvo reference point. Furthermore, the following specifications are also supported for conformance: ETSI
GS NFV-SOL 009 [7] for the management of NFV-MANO, ETSI GS NFV-SOL 011 [8] for the Or-Or reference point,
and ETSI GS NFV-SOL 012 [i.4] for the policy management interface.
Each ETSI NFV-SOL deliverable specifies a set of interfaces built on the RESTful approach and meant to be used over
the HTTP protocol. The aim of the present document is to define the methodologies and the procedures with Test
Descriptions to test conformance of the exchanged HTTP payloads and the implementation of required actions for one
or more of the available interfaces within a reference point.
Since the targets of the testing are functionality (semantic checks) and the HTTP payloads (syntax checks),
methodologies, including test suite(s) and/or any technologies from any organizations (in particular Open Source
Initiatives) that can improve or help the (automated) test execution are also considered as being in the scope of the
present document.
NOTE: The full set of functionalities supported by ETSI NFV Release 4 (V4.4.1) can be found in ETSI
GS NFV-IFA 010 [i.5].
2 References
2.1 Normative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
Referenced documents which are not found to be publicly available in the expected location might be found at
https://docbox.etsi.org/Reference/.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are necessary for the application of the present document.
[1] ETSI GS NFV-SOL 003 (V4.4.1): "Network Functions Virtualisation (NFV) Release 4; Protocols
and Data Models; RESTful protocols specification for the Or-Vnfm Reference Point".
[2] ETSI GS NFV-SOL 002 (V4.4.1): "Network Functions Virtualisation (NFV) Release 4; Protocols
and Data Models; RESTful protocols specification for the Ve-Vnfm Reference Point".
[3] ETSI GS NFV-SOL 005 (V4.4.1): "Network Functions Virtualisation (NFV) Release 4; Protocols
and Data Models; RESTful protocols specification for the Os-Ma-nfvo Reference Point".
[4] ETSI GS NFV-TST 002 (V1.1.1): "Network Functions Virtualisation (NFV); Testing
Methodology; Report on NFV Interoperability Testing Methodology".
[5] ETSI GS NFV-SOL 013 (V4.4.1): "Network Functions Virtualisation (NFV) Release 4; Protocols
and Data Models; Specification of common aspects for RESTful NFV MANO APIs".
[6] ETSI GS NFV-SOL 001 (V4.4.1): "Network Functions Virtualisation (NFV) Release 4; Protocols
and Data Models; NFV descriptors based on TOSCA specification".
[7] ETSI GS NFV-SOL 009 (V4.4.1): "Network Functions Virtualisation (NFV) Release 4; Protocols
and Data Models; RESTful protocols specification for the management of NFV-MANO".
[8] ETSI GS NFV-SOL 011 (V4.4.1): "Network Functions Virtualisation (NFV) Release 4; Protocols
and Data Models; RESTful protocols specification for the Or-Or Reference Point".
ETSI
7 ETSI GS NFV-TST 010 V4.4.1 (2024-08)
[9] Void.
[10] Void.
2.2 Informative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] ETSI GR NFV-TST 007 (V3.1.1): "Network Functions Virtualisation (NFV) Release 3; Testing;
Guidelines on Interoperability Testing for MANO".
[i.2] Robot Framework.
[i.3] Robot2doc tool.
[i.4] ETSI GS NFV-SOL 012 (V4.4.1): "Network Functions Virtualisation (NFV) Release 4; Protocols
and Data Models; RESTful protocols specification for the Policy Management Interface".
[i.5] ETSI GS NFV-IFA 010 (V4.4.1): "Network Functions Virtualisation (NFV) Release 4;
Management and Orchestration; Functional requirements specification".
3 Definition of terms, symbols and abbreviations
3.1 Terms
For the purposes of the present document, the following terms apply:
Test Description (TD): set of information required to define/run the API conformance test and to realize the verdict for
the API conformance test
3.2 Symbols
Void.
3.3 Abbreviations
For the purposes of the present document, the following abbreviations apply:
API Application Programming Interface
AUT API Under Test
EM Element Manager
FUT Function Under Test
IUT Implementation Under Test
LCM Life Cycle Management
NFVI NFV Infrastructure
NFVO NFV Orchestrator
NS Network Service
NSD Network Service Descriptor
PNFD PNF Descriptor
SUT System Under Test
ETSI
8 ETSI GS NFV-TST 010 V4.4.1 (2024-08)
TD Test Description
TLS Transport Layer Security
VE Virtual Element
VIM Virtualised Infrastructure Manager
VNF Virtual Network Function
VNFD VNF Descriptor
VNFM VNF Manager
4 Methodology
4.1 General
The purpose of general conformance testing is to determine to what extent an implementation of a standard conforms to
the requirements of that standard. Concepts from ETSI GS NFV-TST 002 [4] are used in the present document.
The important factors which characterize conformance testing are as follows:
• the System or Implementation Under Test (SUT or IUT) defines the boundaries (open interfaces) for testing;
• the conformance test system is a specialized tool (system) built for the purpose of testing and on which
specific test scripts can be run;
• the SUT comes from a single supplier (or, at least, a single product line);
• the tests are executed by a dedicated test system that has full control of the SUT and the ability to observe all
communications from the SUT;
• the tests are performed at open standardized interfaces that are not (usually) accessible to a normal user
(i.e. they are specified at the protocol level);
• the tests are specified at the detailed protocol level and are not usually based on functionality as experienced
by a user;
• the tests verify responses or related request operations from SUT and also verify notifications when
appropriate.
4.2 System Under Test (SUT)
The System Under Test (SUT) is identified by an implementation of the Function Under Test (FUT) producing or
consuming the API under test e.g. in the case of the Or-vnfm reference point the Function Under Test (FUT) may be
either a NFVO implementation or a VNFM implementation.
The function shall be tested in isolation with respect to other functional blocks in a NFV platform, to guarantee that the
outcomes of the conformance tests are not the result of interoperability issues with other components.
4.3 Test configurations
4.3.1 General
In accordance with clause 1, the scope of the present document is to define a testing methodology and test suite for both
the conformant protocol exchange (i.e. valid serialization and order of messages) and the initialization or execution of
the functionalities mandated for each protocol operation, including the conformant management of internal state.
ETSI
9 ETSI GS NFV-TST 010 V4.4.1 (2024-08)
In order to enable the FUT to correctly execute the operations mandated, the FUT shall be tested while being executed
in a test environment which provides all the functional elements needed for the correct outcome of the operation.
EXAMPLE: To correctly execute an instantiation a VNFM requires evaluation in a test environment which
provides a VIM and NFVI plus the NFVO to grant the operation.
The test system shall provide the implementation of an API Consumer and a Notification Endpoint for the API Under
Test (AUT). Moreover, the test configuration may contain observation interfaces between the Test System and the FUT
or any other functional block which is part of the test environment. The specification of the mentioned observation
interfaces is out of the scope of the present document.
Stimuli to the FUT shall be injected by the Test System via the AUT only.
Conformance checks on the status and outcome of the operations triggered by the protocol shall be verified by the Test
System by means of:
• read operations issued via the AUT; or
• reception of notifications on the Notification Endpoint exposed by the test system; or
• other test interfaces to support triggers or verifications.
Test system
Test Environment
API
FUT
Consumer
AUT
Notification
Endpoint
API exchanges
API notifications
Test interface
Figure 4.3.1-1: Generic SUT configuration
The test configurations specified in clause 4.3 fulfil the needs of Test Descriptions specified in annexes D, E, F, G, H
and I contained in archive gs_nfv-tst010v040401p0.zip which accompanies the present document for the different FUTs
and AUTs in scope of the present document.
4.3.2 Config_prod_VE
The configuration config_prod_VE shall be implemented to test APIs which are produced by FUTs in a VNF or EM.
The test environment of the VNF/EM is the NFVI where the test is executed.
ETSI
10 ETSI GS NFV-TST 010 V4.4.1 (2024-08)
Test system
Test Environment
API
VNF/EM
Consumer
AUT
Notification
Endpoint
API exchanges
API notifications
Test interface
Figure 4.3.2-1: Configuration for tests of APIs with the FUTs as Producer run in a VNF/EM
4.3.3 Config_prod_VNFM
The configuration config_prod_VNFM shall be implemented to test APIs produced by FUTs which implement a
VNFM. The test environment of the virtual element is the NFVI where the VE is executed.
Test system
Test Environment
API
VNFM
Consumer
AUT
Notification
Endpoint
API exchanges
API notifications
Test interface
Figure 4.3.3-1: Configuration for tests of APIs with VNFM as Producer
4.3.4 Config_prod_NFVO
The configuration config_prod_NFVO shall be implemented to test APIs produced by FUTs which implement a NFVO.
The test environment of the virtual element is a NFV platform providing VNFM, VIM and NFVI.
ETSI
11 ETSI GS NFV-TST 010 V4.4.1 (2024-08)
Test Environment Test system
API
NFVO
Consumer
AUT
Notification
Endpoint
API exchanges
API notifications
Test interface
Figure 4.3.4-1: Configuration for tests of APIs with NFVO as Producer
4.3.5 Config_prod_VNFM_GRANT
The configuration config_prod_VNFM_GRANT shall be implemented to test APIs produced by FUTs which
implement a VNFM for VNF LCM test cases where an Operation Grant is needed. The test environment of the virtual
element is composed by the NFVI where the VE is executed and a NFVO component exposing the VNF Lifecyle
Operation Granting API.
Test Environment
NFVO
GRANT API
Test system
API
VNFM
Consumer
AUT
Notification
Endpoint
API exchanges
API notifications
Test interface
Figure 4.3.5-1: Configuration for tests of APIs with VNFM as Producer,
where the VNFLifecycleOperationGranting API is required
4.3.6 Config_prod_Notif_Endpoint
The configuration config_prod_Notif_Endpoint shall be implemented to test any notification endpoint. The test
environment of the virtual element is a NFV platform providing an API Provider.
ETSI
12 ETSI GS NFV-TST 010 V4.4.1 (2024-08)

Figure 4.3.6-1: Configuration for tests of APIs where the Test System is an API Provider,
and the AUT is a Notification endpoint
4.3.7 Config_prod_NFV-MANO
The configuration config_prod_NFV-MANO shall be implemented to test APIs produced by FUTs which implement
NFV-MANO. The test environment of the virtual element is a NFV platform providing NFVI.

Figure 4.3.7-1: Configuration for tests of APIs with NFV-MANO as Producer
4.4 Void
4.5 Generic Test Description
4.5.1 General
The machine readable language used for the test case implementations is the Robot Framework syntax language [i.2].
The Robot code for all test cases is available at: https://forge.etsi.org/rep/nfv/api-tests/tree/4.4.1 and in the
accompanying annexes, which contain the tabular representations of the Test Descriptions. The primary source for the
Test Descriptions (TDs) is the online repository at ETSI Forge.
Compliant NFV MANO API Conformance Test Systems shall implement the Test Descriptions as documented in the
ETSI Forge repository and in the accompanying annexes which follow the requirements set in the following clauses.
ETSI
13 ETSI GS NFV-TST 010 V4.4.1 (2024-08)
Together with the normative Test Descriptions, informative implementation of the tests steps is made available as well
in the form of "Keywords in Robot Framework" to facilitate adoption of the present document.
Within the Robot Framework files present in the online repository, the Test Descriptions (which are normative) and test
steps implementations (which are informative) can be identified by the heading under which they are grouped, as
follows:
• the normative Tests Descriptions are grouped under the "*** Test Cases ***" heading; while
• the informative test steps implementations are grouped under the "*** Keywords ***" heading.
NOTE: The text for the headings needs to conform to the Robot Framework language syntax.
4.5.2 Test Description format
Test Descriptions specify the information required to run an "API conformance test". Test Descriptions are defined per
test and table 4.5.2-1 defines the elements of the Test Description.
Table 4.5.2-1: Test Description entries
Elements Definition
Test ID The Test ID identifies uniquely the test. Naming convention: the Test ID is the number of the clause
in the present document which includes the Test Descriptions.
Test Objective The test objective indicates clearly which requirement is intended to be tested in the test. This part
eases the understanding of the Test Description behaviour. This also eases the identification of the
requirements, which were used as a basis for the test.
Pre-conditions
The pre-conditions field defines the conditions that the state of the SUT shall verify before
undergoing the actual Test Description. In the process of the test, if the pre-conditions are not met, it
leads to the assignment of an Inconclusive verdict. This field may include identifiers of other tests
within the present document which may be executed to bring the SUT into a state which verifies the
mandated pre-conditions.
Reference In the reference row, the Test Description writer indicates the clauses of NFV-SOL specification
where the tested requirements are expressed. This information is critical, because it justifies the
existence and the behaviour of the Test Description.
The reference row may refer to several clauses, separated by semi-colon.
Config ID
The identifier of the applicable test configuration. A test configuration defines the components that
shall be included in Test System and SUT to execute the tests and their connections.
Applicability Only optional functions which are required for this specific Test Description are listed in this field.
Post-conditions
Definition of the events that the SUT is expected to perform or shall not perform, if any, according to
the NFV-SOL specifications and following the correct execution of the actions in the expected
behaviour below. If the post-conditions concur in the definition of the verdict of the test, a step in the
Test Case shall be explicitly provided.
Multiple post conditions are separated by semi-colon which expresses a logical AND relation.
Test Case A set of steps of the test, carried out by the Test System to verify conformance of the SUT to the
Test Objective.
The steps are written in the form of Keywords in the Robot Framework language [i.2]. The verdict of
the Test shall be se to 'Pass' if all the applicable steps in the Test Case are successfully carried out
and meet expected results criteria, it shall be set to 'Fail' if any of the steps fails. The Inconclusive
verdict applies in all other cases.

The Test Descriptions are formatted as tables, as shown in the example shown in table 4.5.2-2.
ETSI
14 ETSI GS NFV-TST 010 V4.4.1 (2024-08)
Table 4.5.2-2: Test Description entries examples
Test Purpose
Test ID A.B.C.D.E.
Test Objective Example Test Objective.
Pre-conditions Precondition example.
Reference Reference 1; reference 2.
Config ID
Configuration_example_id.
Applicability
Applicability statement example.
Post-conditions
Precondition 1; Precondition 2; Precondition 3.
Test Case
Step Number One
Step Number Two
Steps are described with use of below expressions:
• GET all [resource] [expression]
to query URI of [resource] to SUT by GET operation with [expression].
• GET [resource] [expression]
to query individual [resource] to SUT by GET operation with [expression].
• Send [HTTP method] Request to [purpose]/[HTTP method] [resource]
to send request to SUT by [HTTP method], i.e. POST, PUT, PATCH, DELETE. Same as "Send [HTTP
method] Request to [purpose]".
• Check HTTP Response Status [expression]
to check if the status code in the response which corresponds to the previous step request from SUT
matches [expression].
• Check HTTP Response Header [expression]
to check if the contents of HTTP header in the response which corresponds to the previous step request
from SUT matches [expression].
• Check HTTP Response Body [expression]
to check if the JSON of the body in the payload of the response which corresponds to the previous step
request from SUT matches [expression].
• Check Postcondition [resource] Exists
to check if [resource] of the SUT is accessible by GET operation.
• Check Postcondition [resource] is [expression]
to check if [resource] matches [expression] by GET operation.
• Check [notification name] Http Request Body [expression]
Upon [notification name] received, to check if the JSON of the body in the payload of HTTP post request
from SUT matches [expression].
• Trigger [condition] (external action)
Test environment prepares [condition] for the particular test to run.
• Check [resource] [condition]
to check if the attribute of the [resource] in JSON in the payload of the response from SUT matches
[condition] according to the "Post-Conditions" or "Pre-conditions" by GET operation.

4.5.3 Scope of the tests
4.5.3.1 General
In addition to the material described in clause 4.5.1, the behaviours of the SUT shall be tested in terms of:
• expected handling of the messages received from Test System;
• expected actions inside the SUT.
These two items above shall be checked by examining information retrieved by other operations performed by
Test Systems e.g. GET or appropriate notification messages. These checks of expected handling and internal actions
shall be based on material provided in the referenced specifications including all error cases.
Further checking includes:
• The expected correct response messages sent back to the Test system.
The above item shall be checked by analysing received messages by Test Systems. These checks of response
correctness shall be based on material provided in the referenced specifications, including all error cases.
ETSI
15 ETSI GS NFV-TST 010 V4.4.1 (2024-08)
4.5.3.2 General characteristics of the reference points
This clause describes the basic principles of ETSI GS NFV-SOL 002 [2], ETSI GS NFV-SOL 003 [1], and ETSI
GS NFV-SOL 005 [3] which are considered in API conformance test.
From the API conformance test point of view, the reference points have below characteristics:
• Asynchronous API and State handling by notifications: applicable for Os-Ma-Nfvo, Or-Vnfm:
- Due to the varieties of the reasons, e.g. consuming time of process handling, "VNF Lifecycle
Management interface" defined in ETSI GS NFV-SOL 003 [1], clause 5 and "NS Lifecycle Management
interface" defined in ETSI GS NFV-SOL 005 [3], clause 6 are designed as asynchronous way, i.e. API
designed is based on the combination of the specific methods with task resources and state handling by
related attributes signalled via notification messages.
- For example, the operation "Instantiate VNF" is triggered by POST message with task "Instantiate VNF"
sent from NFVO to VNFM, then VNFM sends "provisional" response "202 accepted" back to NFVO,
which means "accepted request" (ETSI GS NFV-SOL 003 [1], clause 5.4.4). Based on the subscriptions
registered in VNFM, the appropriate notification message will be sent toward NFVO with the state of the
processing in the VNFM at that time. After finishing the process of instantiating VNF in the VNFM, the
notification message with the notification endpoint resource which is set to
"operationState=COMPLETED" shall be sent (ETSI GS NFV-SOL 003 [1], clause 5.4.18), which means
that the VNF has been instantiated. In order to verify such "Asynchronous API" characteristics, the
Robot code implements "workflow test" which includes series of operations required for "VNF Lifecycle
Management interface" and "NS Lifecycle Management interface" operations.
• Linked resource: applicable for Os-Ma-Nfvo, Ve-Vnfm, Or-Vnfm:
- According to ETSI GS NFV-SOL 003 [1], clause 4.3.3.1, ETSI GS NFV-SOL 005 [3], clause 4.3.3.1
and ETSI GS NFV-SOL 002 [2], clause 4.3.3.1, "link" pointing to the resources which may be fetched
by API consumer may be used on the reference point. It is used in order to avoid "big" response
messages and to avoid repeating information request messages. Therefore, the validity of such linked
resources is also verified by test, as same as resources in the response message itself.
• Request and response data values: applicable for Os-Ma-Nfvo, Ve-Vnfm, Or-Vnfm:
- According to ETSI GS NFV-SOL 003 [1] and ETSI GS NFV-SOL 002 [2], clause 5.5, a VNF LCM
request uses the relevant values e.g. set in the valid VNFD, i.e. InstantiateVnfRequest, ScaleVnfRequest,
ScaleVnfToLevelRequest, ChangeVnfFlavourRequest, TerminateVnfRequest HealVnfRequest,
OperateVnfRequest and ChangeExtVnfConnectivityRequest.
- According to ETSI GS NFV-SOL 005 [3], clause 6.5, a NS LCM request uses the relevant values e.g. set
in the valid NSD, i.e. InstantiateNsRequest, ScaleNsRequest, UpdateNsRequest, TerminateNsRequest
and HealNsRequest.
- Test system verifies if each attribute of the response body uses relevant values e.g. set in the valid
VNFD/NSD.
NOTE 1: This is not considered in the present document.
• Grant exchange: applicable for Or-Vnfm:
- According to ETSI GS NFV-SOL 003 [1], clause 9.1, the VNFM executes Grant Request before certain
LCM operations are executed for the direct mode. The test system verifies if each attribute of Grant
request body uses relevant values e.g. set in the valid VNFD.
NOTE 2: The value validation is not considered in the present document.
NOTE 3: The Robot code implements "Individual Grant". The test for "Grant exchange in the VNF LCM
operations" in the workflow test requires specific test configuration "Config_prod_VNFM_GRANT"
specified in clause 4.3, but "Grant exchange" part in the VNF LCM is not considered by the Robot code
in the present document.
ETSI
16 ETSI GS NFV-TST 010 V4.4.1 (2024-08)
• VIM registration: applicable for Or-Vnfm:
- According to ETSI GS NFV-SOL 003 [1] clause 4.4.1.6, the VNFM uses VIM related information
according to ETSI GS NFV-SOL 003 [1], clause C.5, the test system imports the latest VIM registration
before testing.
NOTE 4: The test system imports the values for the VIM related information via test configurations.
4.5.3.3 Basic behaviours of the API producer/consumer and verification steps
4.5.3.3.0 Introduction
Expected behaviours of the API producer specified in the NFV-SOL specifications targeted by the tests (i.e. ETSI
GS NFV-SOL 002 [2], ETSI GS NFV-SOL 003 [1] and ETSI GS NFV-SOL 005 [3]) are categorized as follows:
• Producer sends event triggered notifications based on consumer subscriptions.
• Producer sends periodic notifications based on the consumer subscriptions.
• Producer executes the requested API operation.
• Consumer fetches the files/package info.
These four categories are expanded hereafter.
4.5.3.3.1 Producer sends event triggered notifications based on consumer subscriptions
Typical example for this case is "Fault management". The producer sends a notification to the consumer with
mechanism "filter" and "callback". The Test System checks if the notification aligns to the conditions of the
subscription.
ETSI
17 ETSI GS NFV-TST 010 V4.4.1 (2024-08)

Figure 4.5.3.3.1-1: Verification procedure
The basic method of verification follows the steps as illustrated in figure 4.5.3.3.1-1.
Preconditions: Before sending message#1, required features/functionalities are executed, e.g. API authentication, etc.
Other tests as pre-condition are executed beforehand if required:
1) Test System sends subscription message#1. Test System sets the data and attributes of the request body with
correct/incorrect cardinalities and values.
2) Upon receiving the subscription message#1 from Test System, the SUT shall handle this subscription
according to the requirements defined in SOL specifications, the response message#2 are sent to the
Test System.
3) Test System verifies the received response message#2, in terms of expected response code, expected HTTP
header, HTTP body, cardinality of the responded data, fulfilment of the requirements described in the
description field, and verification aspects shown below, etc.:
a) Verification aspects: 1-C, 2-C, 2-F, 2-G (see clause 4.5.4).
4) When the events which are matched to the subscribed criteria made by message#1 happen in SUT, notification
message#3 sends to the Test System.
NOTE: How to make subscribed event in SUT to match the criteria is out of scope of the present document.
5) Test System verifies the received response message#3, in terms of expected response code, expected HTTP
header, HTTP body, cardinality of the responded data, fulfilment of the requirements described in the
description field, and verification aspects shown below, etc. steps 4 and 5 can be looped:
a) Verification aspects: 1-C, 2-C, 2-F, 2-G (see clause 4.5.4).
ETSI
18 ETSI GS NFV-TST 010 V4.4.1 (2024-08)
6) After expected API messages exchange, the Test System may have some verification aspects which require
retrieval of information from the SUT, e.g. linked resources embedded in the response message. In that case,
the Test System may invoke a query message to th
...