ETSI GR CIM 015 V1.1.1 (2021-06)

Context Information Management (CIM); NGSI-LD Testing Environment Validation

DGR/CIM-0015v111

General Information

• Status: Not Published
• Technical Committee: CIM - cross-cutting Context Information Management
• Current Stage: 12 - Completion
• Due Date: 16-Jun-2021
• Completion Date: 11-Jun-2021

Standards Content (Sample)

ETSI GR CIM 015 V1.1.1 (2021-06)






GROUP REPORT
Context Information Management (CIM);
NGSI-LD Testing Environment Validation
Disclaimer
The present document has been produced and approved by the cross-cutting Context Information Management (CIM) ETSI
Industry Specification Group (ISG) and represents the views of those members who participated in this ISG.
It does not necessarily represent the views of the entire ETSI membership.

---------------------- Page: 1 ----------------------
2 ETSI GR CIM 015 V1.1.1 (2021-06)

Reference
DGR/CIM-0015v111
Keywords
API, IoT, testing

ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: [phone]  Fax: [phone]

Siret N° 348 623 562 00017 - APE 7112B
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° w061004871

Important notice
The present document can be downloaded from:
http://www.etsi.org/standards-search
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the prevailing version of an ETSI
deliverable is the one made publicly available in PDF format at www.etsi.org/deliver.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
https://portal.etsi.org/TB/ETSIDeliverableStatus.aspx
If you find errors in the present document, please send your comment to one of the following services:
https://portal.etsi.org/People/CommiteeSupportStaff.aspx
Notice of disclaimer & limitation of liability
The information provided in the present deliverable is directed solely to professionals who have the appropriate degree of
experience to understand and interpret its content in accordance with generally accepted engineering or
other professional standard and applicable regulations.
No recommendation as to products and services or vendors is made or should be implied.
No representation or warranty is made that this deliverable is technically accurate or sufficient or conforms to any law
and/or governmental rule and/or regulation and further, no representation or warranty is made of merchantability or fitness
for any particular purpose or against infringement of intellectual property rights.
In no event shall ETSI be held liable for loss of profits or any other incidental or consequential damages.

Any software contained in this deliverable is provided "AS IS" with no warranties, express or implied, including but not
limited to, the warranties of merchantability, fitness for a particular purpose and non-infringement of intellectual property
rights and ETSI shall not be held liable in any event for any damages whatsoever (including, without limitation, damages
for loss of profits, business interruption, loss of information, or any other pecuniary loss) arising out of or related to the use
of or inability to use the software.
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and
microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.

© ETSI 2021.
All rights reserved.

ETSI

---------------------- Page: 2 ----------------------
3 ETSI GR CIM 015 V1.1.1 (2021-06)
Contents
Intellectual Property Rights . 4
Foreword . 4
Modal verbs terminology . 4
Executive summary . 4
Introduction . 4
1 Scope . 6
2 References . 6
2.1 Normative references . 6
2.2 Informative references . 6
3 Definition of terms, symbols and abbreviations . 6
3.1 Terms . 6
3.2 Symbols . 6
3.3 Abbreviations . 6
4 Tests Execution Options . 7
4.0 Introduction . 7
4.1 Executing all tests . 7
4.2 Executing tests subsets from a folder . 7
4.3 Executing tests from a single Test Case . 7
4.4 Results output . 7
4.5 Overriding base URL . 8
4.6 Executing tests by specific tag . 8
4.7 Rerun failed tests . 8
5 Permutations . 8
5.1 Introduction . 8
5.2 Data driven approach . 8
5.3 Keyword approach . 8
6 Tags . 9
6.0 Introduction . 9
6.1 Resource and request tags . 9
6.2 Section reference tags . 11
7 Future Work . 11
7.1 Introduction . 11
7.2 Test Suite usability . 11
7.2.1 Exploitation of the results of the Test Suite . 11
7.2.2 Identification of failed checks when comparing results . 12
7.3 Test Suite coverage of the specification . 12
7.4 Test Suite improvements . 12
7.4.1 Severity of the errors. 12
7.4.2 Optional and mandatory requests . 13
7.5 Test Suite maintenance . 13
Annex A: NGSI-LD Implementations . 14
Annex B: Change History . 15
History . 16


ETSI

---------------------- Page: 3 ----------------------
4 ETSI GR CIM 015 V1.1.1 (2021-06)
Intellectual Property Rights
Essential patents
IPRs essential or potentially essential to normative deliverables may have been declared to ETSI. The declarations
pertaining to these essential IPRs, if any, are publicly available for ETSI members and non-members, and can be
found in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to
ETSI in respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the
ETSI Web server (https://ipr.etsi.org/).
Pursuant to the ETSI Directives including the ETSI IPR Policy, no investigation regarding the essentiality of IPRs,
including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not
referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become,
essential to the present document.
Trademarks
The present document may include trademarks and/or tradenames which are asserted and/or registered by their owners.
ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no
right to use or reproduce any trademark and/or tradename. Mention of those trademarks in the present document does
not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks.
DECT™, PLUGTESTS™, UMTS™ and the ETSI logo are trademarks of ETSI registered for the benefit of its

Members. 3GPP™ and LTE™ are trademarks of ETSI registered for the benefit of its Members and of the 3GPP
Organizational Partners. oneM2M™ logo is a trademark of ETSI registered for the benefit of its Members and of the
®
oneM2M Partners. GSM and the GSM logo are trademarks registered and owned by the GSM Association.
Foreword
This Group Report (GR) has been produced by ETSI Industry Specification Group (ISG) cross-cutting Context
Information Management (CIM).
Modal verbs terminology
In the present document "should", "should not", "may", "need not", "will", "will not", "can" and "cannot" are to be
interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.
Executive summary
The present document presents the results of the validation of the NGSI-LD Test Suite and its test environment. Besides
the testing environment validation, it also provides experts who are implementing the NGSI-LD API specification a
description of how to efficiently run the Test Suite and how it can be extended to add more permutations in the Test
Cases. The present document concludes by listing the envisioned future work on the Test Suite to extend it and improve
it as well as to maintain it for future versions of the NGSI-LD API.
Introduction
The ISG CIM group has defined an API for exchange of information contextualized in time, space and relation to other
information using a property graph model with the intent that the associated protocol (called NGSI-LD) becomes the
"glue" between all kinds of applications and databases associated with services for Smart Cities, Smart Agriculture,
Smart Manufacturing, etc.
ETSI

---------------------- Page: 4 ----------------------
5 ETSI GR CIM 015 V1.1.1 (2021-06)
To be successful, the NGSI-LD API specification needs to be well understood and well implemented. The community
of users will not be solely highly professional engineers employed by big companies but will include many small teams
and SMEs and even hobbyists. Therefore, it is essential that the developers have access to not only the standard but also
a test specification and a testing environment to check that their work is (and remains) conform to the ETSI NGSI LD
specification.
The developers will usually write integration tests to validate the behaviour of their NGSI-LD implementation, but it is
important to assert compliance to the specification based on a test suite agreed by the group creating the API
specification, i.e. ETSI ISG CIM. Therefore, it is very important to create a set of ETSI-approved test cases.
What is more, the existence of such a test suite will likely help to increase the adoption of the NGSI-LD specification
by giving developers a ready to use and extensive set of sample tests.

ETSI

---------------------- Page: 5 ----------------------
6 ETSI GR CIM 015 V1.1.1 (2021-06)
1 Scope
The present document presents the experience of running the Test Suite against a set of open-source context brokers
implementing the NGSI-LD specifications.
It also gives some guidelines on how to use the Test Suite and how to expand it for simple use-cases like adding new
permutations in a Test Case.
Finally, it presents some ideas for later improvements of the Test Suite.
2 References
2.1 Normative references
Normative references are not applicable in the present document.
2.2 Informati
...