ETSI GS NFV-SOL 018 V4.4.1 (2023-06)

Network Functions Virtualisation (NFV) Release 4; Protocols and Data Models; Profiling specification of protocol and data model solutions for OS Container management and orchestration

RGS/NFV-SOL018ed441

General Information

• Status: Not Published
• Technical Committee: NFV SOL - Solutions
• Current Stage: 12 - Citation in the OJ (auto-insert)
• Due Date: 13-Jul-2023
• Completion Date: 21-Jun-2023

Standards Content (Sample)

GROUP SPECIFICATION
Network Functions Virtualisation (NFV) Release 4;
Protocols and Data Models;
Profiling specification of protocol and data model solutions for
OS Container management and orchestration
Disclaimer
The present document has been produced and approved by the Network Functions Virtualisation (NFV) ETSI Industry
Specification Group (ISG) and represents the views of those members who participated in this ISG.
It does not necessarily represent the views of the entire ETSI membership.

2 ETSI GS NFV-SOL 018 V4.4.1 (2023-06)

Reference
RGS/NFV-SOL018ed441
Keywords
container, data models, NFV, orchestration,
protocol, service, virtualisation

ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: [phone]  Fax: [phone]

Siret N° 348 623 562 00017 - APE 7112B
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° w061004871

Important notice
The present document can be downloaded from:
https://www.etsi.org/standards-search
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the prevailing version of an ETSI
deliverable is the one made publicly available in PDF format at www.etsi.org/deliver.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
https://portal.etsi.org/TB/ETSIDeliverableStatus.aspx
If you find errors in the present document, please send your comment to one of the following services:
https://portal.etsi.org/People/CommiteeSupportStaff.aspx
If you find a security vulnerability in the present document, please report it through our
Coordinated Vulnerability Disclosure Program:
https://www.etsi.org/standards/coordinated-vulnerability-disclosure
Notice of disclaimer & limitation of liability
The information provided in the present deliverable is directed solely to professionals who have the appropriate degree of
experience to understand and interpret its content in accordance with generally accepted engineering or
other professional standard and applicable regulations.
No recommendation as to products and services or vendors is made or should be implied.
No representation or warranty is made that this deliverable is technically accurate or sufficient or conforms to any law
and/or governmental rule and/or regulation and further, no representation or warranty is made of merchantability or fitness
for any particular purpose or against infringement of intellectual property rights.
In no event shall ETSI be held liable for loss of profits or any other incidental or consequential damages.

Any software contained in this deliverable is provided "AS IS" with no warranties, express or implied, including but not
limited to, the warranties of merchantability, fitness for a particular purpose and non-infringement of intellectual property
rights and ETSI shall not be held liable in any event for any damages whatsoever (including, without limitation, damages
for loss of profits, business interruption, loss of information, or any other pecuniary loss) arising out of or related to the use
of or inability to use the software.
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and
microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.

© ETSI 2023.
All rights reserved.
ETSI
3 ETSI GS NFV-SOL 018 V4.4.1 (2023-06)
Contents
Intellectual Property Rights . 7
Foreword . 7
Modal verbs terminology . 7
1 Scope . 8
2 References . 8
2.1 Normative references . 8
2.2 Informative references . 9
3 Definition of terms, symbols and abbreviations . 9
3.1 Terms . 9
3.2 Symbols . 9
3.3 Abbreviations . 10
4 Overview of protocols and data models for OS Container management and orchestration . 10
4.1 Summary of ETSI GS NFV-IFA 040 and ETSI GS NFV-IFA 036 . 10
4.2 Profiled protocol and data model solutions . 11 ®
4.2.1 Kubernetes API . 11
4.2.1.1 Introduction . 11
4.2.1.2 API structure . 11
4.2.1.3 Data model concepts . 11
TM
4.2.2 Helm CLI . 12
4.2.2.1 Introduction . 12
4.2.2.2 CLI structure . 12
4.2.2.3 Data model concepts . 12
TM
4.2.2.3.1 Helm chart file structure . 12
TM
4.2.2.3.2 Helm release objects . 13
TM
4.2.3 OCI Distribution Specification API . 13
4.2.3.1 Introduction . 13
4.2.3.2 API structure . 13
5 NFV object model mapping to profiled solution objects . 14
5.1 Managed Container Infrastructure Objects . 14
5.1.1 Compute MCIOs . 14
5.1.2 Storage MCIOs . 14
5.1.3 Network MCIOs . 15
5.1.4 MCIO configurations . 15
5.1.5 MCIO policies . 15
5.2 Managed Container Infrastructure Object Packages . 16
5.3 Namespace . 16
5.4 Namespace quota . 16
5.5 OS container image . 16
5.6 Managed CIS Cluster Objects . 16
5.7 CIS Instance . 17
6 Input/Output parameter mapping between NFV data model and profiled solution data models . 17
6.1 Introduction . 17
6.2 Input parameters to CISM APIs . 18
6.2.1 Framework . 18
6.2.1.1 Introduction . 18
6.2.1.2 Mapping to NFV data models . 19
6.2.1.3 Conveying parameters . 19
6.2.2 Workloads . 19
6.2.2.1 Mapping to NFV data models . 19
6.2.3 Discovery and Loadbalancing. 25
6.2.3.1 Mapping to NFV data models . 25
6.2.4 Configuration & Storage . 28
6.2.4.1 Mapping to NFV data models . 28
ETSI
4 ETSI GS NFV-SOL 018 V4.4.1 (2023-06)
6.3 Output parameters from CISM APIs . 33
6.3.1 Framework . 33
6.3.1.1 Introduction . 33
6.3.1.2 Mapping to NFV data models . 33
6.3.2 Workload . 33
6.3.2.1 Mapping to NFV data models . 33
6.3.3 Discovery and Loadbalancing. 36
6.3.3.1 Mapping to NFV data models . 36
6.3.4 Configuration & Storage . 38
6.3.4.1 Mapping to NFV data models . 38
7 OS container workload management service interface . 39
7.1 Description . 39
7.2 CLI version . 40
7.3 Sequence diagrams (informative) . 40
7.3.1 Flow of instantiating a containerized workload based on a MCIOP . 40
7.3.2 Flow of modifying a containerized workload based on a MCIOP via upgrade . 40
7.3.3 Flow of modifying a containerized workload based on a MCIOP via rollback . 41
7.3.4 Flow of terminating a containerized workload based on a MCIOP . 42
7.3.5 Flow of querying information about a containerized workload based on a MCIOP . 42
7.4 Operations . 43
7.4.1 Introduction. 43
TM
7.4.2 Operation: Helm install . 43
TM
7.4.3 Operation: Helm upgrade . 43
TM
7.4.4 Operation: Helm rollback . 44
TM
7.4.5 Operation: Helm uninstall . 44
TM
7.4.6 Operation: Helm status . 44
7.5 Data model . 45
7.6 Additional feature profiling . 45
8 OS container compute management service interface . 45
8.1 Description . 45
8.2 API version . 46
8.3 Resource structure and methods . 46
8.4 Sequence diagrams (informative) . 50
8.4.1 Introduction. 50
8.4.2 Flow of creating a Compute MCIO . 50
8.4.3 Flow of modifying a Compute MCIO. 50
8.4.4 Flow of replacing a Compute MCIO . 51
8.4.5 Flow of deleting a Compute MCIO . 52
8.4.6 Flow of listing/getting Compute MCIO information . 52
8.5 Resources . 53
8.5.1 Introduction. 53
8.5.2 Resource: Pod . 53
8.5.3 Resource: DaemonSet . 54
8.5.4 Resource: Deployment . 54
8.5.5 Resource: ReplicaSet . 55
8.5.6 Resource: StatefulSet . 55
8.5.7 Resource: CronJob . 56
8.5.8 Resource: Job . 56
8.6 Data model . 57
8.7 Additional feature profiling . 57
9 OS container storage management service interface . 57
9.1 Description . 57
9.2 API version . 58
9.3 Resource structure and methods . 58
9.4 Sequence diagrams (informative) . 59
9.5 Resources . 59
9.5.1 Introduction. 59
9.5.2 Resource: PersistentVolumeClaim . 59
9.6 Data model . 60
9.7 Additional feature profiling . 60
ETSI
5 ETSI GS NFV-SOL 018 V4.4.1 (2023-06)
10 OS container network management service interface . 60
10.1 Description . 60
10.2 API version . 61
10.3 Resource structure and methods . 61
10.4 Sequence diagrams (informative) . 63
10.5 Resources . 63
10.5.1 Introduction. 63
10.5.2 Resource: Endpoints . 63
10.5.3 Resource: Service . 64
10.5.4 Resource: EndpointSlice . 64
10.5.5 Resource: Ingress . 65
10.6 Data model . 65
10.7 Additional feature profiling . 65
11 OS container configuration management service interface . 66
11.1 Description . 66
11.2 API version . 67
11.3 Resource structure and methods . 67
11.4 Sequence diagrams (informative) . 70
11.5 Resources . 71
11.5.1 Introduction. 71
11.5.2 Resource: ConfigMap . 71
11.5.3 Resource: Secret . 71
11.5.4 Resource: CustomResourceDefinition . 72
11.5.5 Resource: PodDisruptionBudget . 72
11.5.6 Resource: NetworkPolicy . 73
11.5.7 Resource: Namespace . 74
11.5.8 Resource: NamespaceQuota . 74
11.6 Data model . 74
11.7 Additional feature profiling . 75
12 OS container image management service interface . 75
12.1 Description . 75
12.2 API version . 75
12.3 Resource structure and methods . 75
12.4 Sequence diagrams (informative) . 77
12.4.1 Flow of pushing OS container image constituent blob . 77
12.4.2 Flow of pushing OS container image constituent manifest. 79
12.4.3 Flow of deleting OS container image constituent blob . 80
12.4.4 Flow of deleting OS container image constituent manifest . 80
12.4.5 Flow of deleting OS container image tag . 81
12.4.6 Flow of discovering OS container images . 81
12.5 Resources . 82
12.5.1 Introduction. 82
12.5.2 Resource: Blob . 82
12.5.3 Resource: Manifest . 82
12.5.4 Resource: OS container image tag . 83
12.6 Data model . 83
12.7 Additional feature profiling . 83
13 CIS MCCO management service interface . 84
13.1 Description . 84
13.2 API version . 84
13.3 Resource structure and methods . 84
13.4 Sequence diagrams (informative) . 86
13.4.1 Introduction. 86
13.4.2 Flow of creating an MCCO . 87
13.4.3 Flow of modifying an MCCO . 87
13.4.4 Flow of replacing an MCCO. 88
13.4.5 Flow of deleting an MCCO . 88
13.4.6 Flow of listing/querying an MCCO information . 89
13.5 Resources . 90
13.5.1 Introduction. 90
ETSI
6 ETSI GS NFV-SOL 018 V4.4.1 (2023-06)
13.5.2 Resource: DaemonSet . 90
13.5.3 Resource: CustomResourceDefinition . 91
13.5.4 Resource: PersistentVolume . 91
13.6 Data model . 92
13.7 Additional feature profiling . 92
14 CIS Instance management service interface . 92
14.1 Description . 92
14.2 API version . 93
14.3 Resource structure and methods . 93
14.4 Sequence diagrams (informative) . 93
14.4.1 Introduction. 93
14.4.2 Flow of creating a CIS Instance . 94
14.4.3 Flow of modifying a CIS Instance . 94
14.4.4 Flow of replacing a CIS Instance . 95
14.4.5 Flow of deleting a CIS Instance . 95
14.4.6 Flow of listing/querying an CIS Instance information . 96
14.5 Resources . 97
14.5.1 Introduction. 97
14.5.2 Resource: Node . 97
14.6 Data model . 97
14.7 Additional feature profiling . 98
Annex A (informative): Integration of the profiled solutions into the NFV-MANO framework . 99
A.1 Concepts and evaluation . 99
A.1.1 Releases of ETSI NFV own defined protocol and data modeling . 99
A.1.2 Releases of referenced open source solutions . 99
A.1.3 Versioning correlation between referenced open source solutions, the present document and other NFV-
MANO specifications . 100 ®
Annex B (informative): Mapping between NFV-MANO and Kubernetes . 101
B.1 Overview . 101
B.2 Detail explanation on Input parameter mapping . 101
B.2.1 Images . 101
B.2.2 Affinity and anti-affinity . 101
B.3 Detail explanation on Output parameter mapping . 102
B.3.1 Pod name . 102
Annex C (informative): Change History . 105
History . 109

ETSI
7 ETSI GS NFV-SOL 018 V4.4.1 (2023-06)
Intellectual Property Rights
Essential patents
IPRs essential or potentially essential to normative deliverables may have been declared to ETSI. The declarations
pertaining to these essential IPRs, if any, are publicly available for ETSI members and non-members, and can be
found in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to
ETSI in respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the
ETSI Web server (https://ipr.etsi.org/).
Pursuant to the ETSI Directives including the ETSI IPR Policy, no investigation regarding the essentiality of IPRs,
including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not
referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become,
essential to the present document.
Trademarks
The present document may include trademarks and/or tradenames which are asserted and/or registered by their owners.
ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no
right to use or reproduce any trademark and/or tradename. Mention of those trademarks in the present document does
not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks.
DECT™, PLUGTESTS™, UMTS™ and the ETSI logo are trademarks of ETSI registered for the benefit of its

Members. 3GPP™ and LTE™ are trademarks of ETSI registered for the benefit of its Members and of the 3GPP
Organizational Partners. oneM2M™ logo is a trademark of ETSI registered for the benefit of its Members and of the ®
oneM2M Partners. GSM and the GSM logo are trademarks registered and owned by the GSM Association.
Foreword
This Group Specification (GS) has been produced by ETSI Industry Specification Group (ISG) Network Functions
Virtualisation (NFV).
Modal verbs terminology
In the present document "shall", "shall not", "should", "should not", "may", "need not", "will", "will not", "can" and
"cannot" are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of
provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.

ETSI
8 ETSI GS NFV-SOL 018 V4.4.1 (2023-06)
1 Scope
The present document provides a mapping of the NFV object model for OS container management and orchestration to
® TM ®
managed objects of Kubernetes and Helm as specified by the CNCF along with a specification of a mapping
between a common set of input parameters (e.g. derived from VNFD/NSD and/or NFV-MANO RESTful APIs) and
output parameters associated to the management and orchestration of the referred managed objects. It profiles the ®
reference Kubernetes API as NFV protocol and data model solution for OS container management and orchestration,
TM
for CIS MCCO management and for CIS Instance management. It profiles the reference Helm documentation as
NFV protocol and data model solution for management of OS container workload based on an MCIOP. It profiles the
TM TM
reference OCI Distribution Specification API (which is based on the Docker registry API) as NFV protocol and ®
data model solution for OS container image management. The latest published versions of the reference Kubernetes
TM TM
API, Helm documentation and OCI Distribution Specification API are profiled against the requirements on the
functions and the management service interfaces of the Container Infrastructure Service Management (CISM) and
Container Image Registry (CIR) functions as specified in ETSI GS NFV-IFA 040 [2], ETSI GS NFV-IFA 036 [9] and
ETSI GS NFV-IFA 010 [1].
2 References
2.1 Normative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
Referenced documents which are not found to be publicly available in the expected location might be found at
https://docbox.etsi.org/Reference/.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are necessary for the application of the present document.
[1] ETSI GS NFV-IFA 010: "Network Functions Virtualisation (NFV) Release 4; Management and
Orchestration; Functional requirements specification".
[2] ETSI GS NFV-IFA 040: "Network Functions Virtualisation (NFV) Release 4; Management and
Orchestration; Requirements for service interfaces and object model for OS container management
and orchestration specification". ®
[3] Kubernetes API v1.26.
TM
[4] Helm CLI v3.12.0.
TM
[5] OCI Distribution Specification v1.0.0.
TM
[6] OCI Image Format Specification v1.0.1.
TM
[7] Helm charts v3.12.0. ®
[8] Kubernetes reference documentation, API Access Control.
[9] ETSI GS NFV-IFA 036: "Network Functions Virtualisation (NFV) Release 4; Management and
Orchestration; Requirements for service interfaces and object model for container cluster
management and orchestration specification".
ETSI
9 ETSI GS NFV-SOL 018 V4.4.1 (2023-06)
2.2 Informative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] ETSI GR NFV 003: "Network Functions Virtualisation (NFV); Terminology for Main Concepts in
NFV".
[i.2] ETSI GS NFV-SOL 003: "Network Functions Virtualisation (NFV) Release 4; Protocols and Data
Models; RESTful protocols specification for the Or-Vnfm Reference Point".
[i.3] ETSI GS NFV-SOL 001: "Network Functions Virtualisation (NFV) Release 4; Protocols and Data
Models; NFV descriptors based on TOSCA specification".
[i.4] ETSI NFV Release Documentation.
[i.5] ETSI GS NFV-SOL 013: "Network Functions Virtualisation (NFV) Release 4; Protocols and Data
Models; Specification of common aspects for RESTful NFV MANO APIs".
[i.6] Void.
[i.7] ETSI GS NFV-SOL 002: "Network Functions Virtualisation (NFV) Release 4; Protocols and Data
Models; RESTful protocols specification for the Ve-Vnfm Reference Point".
[i.8] Open Container Initiative Charter v1.3.
[i.9] IETF RFC 9110: "HTTP Semantics".
3 Definition of terms, symbols and abbreviations
3.1 Terms
For the purposes of the present document, the terms given in ETSI GR NFV 003 [i.1] and the following apply:
CIS cluster: set of CIS instances, and one or multiple CISM instances managing them
NOTE: At minimum, the CIS cluster contains one CI
...