ETSI TR 128 908 V18.1.0 (2024-10)

5G; Study on Artificial Intelligence/Machine Learning (AI/ ML) management (3GPP TR 28.908 version 18.1.0 Release 18)

RTR/TSGS-0528908vi10

General Information

• Status: Not Published
• Technical Committee: 3GPP SA 5 - Telecom Management
• Current Stage: 12 - Citation in the OJ (auto-insert)
• Completion Date: 10-Oct-2024

Standards Content (Sample)

TECHNICAL REPORT
5G;
Study on Artificial Intelligence/Machine Learning (AI/ ML)
management
(3GPP TR 28.908 version 18.1.0 Release 18)

3GPP TR 28.908 version 18.1.0 Release 18 1 ETSI TR 128 908 V18.1.0 (2024-10)

Reference
RTR/TSGS-0528908vi10
Keywords
5G
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: [phone]  Fax: [phone]

Siret N° 348 623 562 00017 - APE 7112B
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° w061004871

Important notice
The present document can be downloaded from the
ETSI Search & Browse Standards application.
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the prevailing version of an ETSI
deliverable is the one made publicly available in PDF format on ETSI deliver.
Users should be aware that the present document may be revised or have its status changed,
this information is available in the Milestones listing.
If you find errors in the present document, please send your comments to
the relevant service listed under Committee Support Staff.
If you find a security vulnerability in the present document, please report it through our
Coordinated Vulnerability Disclosure (CVD) program.
Notice of disclaimer & limitation of liability
The information provided in the present deliverable is directed solely to professionals who have the appropriate degree of
experience to understand and interpret its content in accordance with generally accepted engineering or
other professional standard and applicable regulations.
No recommendation as to products and services or vendors is made or should be implied.
No representation or warranty is made that this deliverable is technically accurate or sufficient or conforms to any law
and/or governmental rule and/or regulation and further, no representation or warranty is made of merchantability or fitness
for any particular purpose or against infringement of intellectual property rights.
In no event shall ETSI be held liable for loss of profits or any other incidental or consequential damages.

Any software contained in this deliverable is provided "AS IS" with no warranties, express or implied, including but not
limited to, the warranties of merchantability, fitness for a particular purpose and non-infringement of intellectual property
rights and ETSI shall not be held liable in any event for any damages whatsoever (including, without limitation, damages
for loss of profits, business interruption, loss of information, or any other pecuniary loss) arising out of or related to the use
of or inability to use the software.
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and
microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.

© ETSI 2024.
All rights reserved.
ETSI
3GPP TR 28.908 version 18.1.0 Release 18 2 ETSI TR 128 908 V18.1.0 (2024-10)
Intellectual Property Rights
Essential patents
IPRs essential or potentially essential to normative deliverables may have been declared to ETSI. The declarations
pertaining to these essential IPRs, if any, are publicly available for ETSI members and non-members, and can be
found in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to
ETSI in respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the
ETSI Web server (https://ipr.etsi.org/).
Pursuant to the ETSI Directives including the ETSI IPR Policy, no investigation regarding the essentiality of IPRs,
including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not
referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become,
essential to the present document.
Trademarks
The present document may include trademarks and/or tradenames which are asserted and/or registered by their owners.
ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no
right to use or reproduce any trademark and/or tradename. Mention of those trademarks in the present document does
not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks.
DECT™, PLUGTESTS™, UMTS™ and the ETSI logo are trademarks of ETSI registered for the benefit of its

Members. 3GPP™ and LTE™ are trademarks of ETSI registered for the benefit of its Members and of the 3GPP
Organizational Partners. oneM2M™ logo is a trademark of ETSI registered for the benefit of its Members and of the ®
oneM2M Partners. GSM and the GSM logo are trademarks registered and owned by the GSM Association.
Legal Notice
This Technical Report (TR) has been produced by ETSI 3rd Generation Partnership Project (3GPP).
The present document may refer to technical specifications or reports using their 3GPP identities. These shall be
interpreted as being references to the corresponding ETSI deliverables.
The cross reference between 3GPP and ETSI identities can be found under https://webapp.etsi.org/key/queryform.asp.
Modal verbs terminology
In the present document "should", "should not", "may", "need not", "will", "will not", "can" and "cannot" are to be
interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.
ETSI
3GPP TR 28.908 version 18.1.0 Release 18 3 ETSI TR 128 908 V18.1.0 (2024-10)
Contents
Intellectual Property Rights . 2
Legal Notice . 2
Modal verbs terminology . 2
Foreword . 8
1 Scope . 10
2 References . 10
3 Definitions of terms, symbols and abbreviations . 11
3.1 Terms . 11
3.2 Symbols . 11
3.3 Abbreviations . 11
4 Concepts and overview . 11
4.1 Concepts and terminologies . 11
4.2 Overview . 11
4.3 AI/ML workflow for 5GS . 12
4.3.1 AI/ML operational workflow . 12
4.3.2 AI/ML management capabilities . 13
5 Use cases, potential requirements and possible solutions . 14
5.1 Management Capabilities for ML training phase . 14
5.1.1 Event data for ML training . 14
5.1.1.1 Description . 14
5.1.1.2 Use cases . 14
5.1.1.2.1 Pre-processed event data for ML training. 14
5.1.1.3 Potential requirements . 15
5.1.1.4 Possible solutions . 15
5.1.1.5 Evaluation . 17
5.1.2 ML model validation . 17
5.1.2.1 Description . 17
5.1.2.2 Use cases . 17
5.1.2.2.1 ML model validation performance reporting. 17
5.1.2.3 Potential requirements . 17
5.1.2.4 Possible solutions . 17
5.1.2.4.1 Validation performance reporting by enhancing the existing IOC . 17
5.1.2.5 Evaluation . 18
5.1.3 ML model testing . 18
5.1.3.1 Description . 18
5.1.3.2 Use cases . 18
5.1.3.2.1 Consumer-requested ML model testing . 18
5.1.3.2.2 Control of ML model testing . 19
5.1.3.2.3 Multiple ML entities joint testing . 19
5.1.3.2.4 Model evaluation for ML testing . 19
5.1.3.3 Potential requirements . 19
5.1.3.4 Possible solutions . 20
5.1.3.4.1 NRM based solution . 20
5.1.3.5 Evaluation . 21
5.1.4 ML model re-training. 22
5.1.4.1 Description . 22
5.1.4.2 Use cases . 22
5.1.4.2.1 Producer-initiated threshold-based ML model re-training . 22
5.1.4.2.2 Efficient ML model re-training . 22
5.1.4.3 Potential requirements . 22
5.1.4.4 Possible solutions . 23
5.1.4.4.1 Producer Initiated Retraining . 23
5.1.4.4.2 Efficient ML model re-training . 23
ETSI
3GPP TR 28.908 version 18.1.0 Release 18 4 ETSI TR 128 908 V18.1.0 (2024-10)
5.1.4.5 Evaluation . 24
5.1.5 ML model joint training . 24
5.1.5.1 Description . 24
5.1.5.2 Use cases . 24
5.1.5.2.1 Support for ML model modularity - joint training of ML entities . 24
5.1.5.3 Potential requirements . 25
5.1.5.4 Possible solutions . 25
5.1.5.4.1 Support for ML model modularity - joint training of ML entities . 25
5.1.5.5 Evaluation . 25
5.1.6 Training data effectiveness reporting and analytics . 26
5.1.6.1 Description . 26
5.1.6.2 Use cases . 26
5.1.6.2.1 Training data effectiveness reporting . 26
5.1.6.2.2 Training data effectiveness analytics . 26
5.1.6.2.3 Measurement data correlation analytics for ML training . 26
5.1.6.3 Potential requirements . 27
5.1.6.4 Possible solutions . 27
5.1.6.4.1 Possible solution for training data effectiveness reporting . 27
5.1.6.4.2 Possible solution for training data effectiveness analytics . 28
5.1.6.4.3 Possible solution for measurement data correlation analytics . 28
5.1.6.5 Evaluation . 30
5.1.7 ML context. 30
5.1.7.1 Description . 30
5.1.7.2 Use cases . 30
5.1.7.2.1 ML context monitoring and reporting . 30
5.1.7.2.2 Mobility of ML Context . 31
5.1.7.2.3 Standby mode for ML model . 31
5.1.7.3 Potential requirements . 32
5.1.7.4 Possible solutions . 32
5.1.7.4.1 MLContext <> on MLEntity . 32
5.1.7.4.2 Mobility of MLContext . 32
5.1.7.5 Evaluation . 33
5.1.8 ML model capability discovery and mapping . 33
5.1.8.1 Description . 33
5.1.8.2 Use cases . 34
5.1.8.2.1 Identifying capabilities of ML entities . 34
5.1.8.2.2 Mapping of the capabilities of ML entities . 34
5.1.8.3 Potential requirements . 35
5.1.8.4 Possible solutions . 35
5.1.8.5 Evaluation . 36
5.1.9 AI/ML update management . 36
5.1.9.1 Description . 36
5.1.9.2 Use cases . 36
5.1.9.2.1 ML entities updating initiated by producer. 36
5.1.9.3 Potential requirements . 37
5.1.9.4 Possible solutions . 37
5.1.9.5 Evaluation . 37
5.1.10 Performance evaluation for ML training . 37
5.1.10.1 Description . 37
5.1.10.2 Use cases . 37
5.1.10.2.1 Performance indicator selection for ML model training . 37
5.1.10.2.2 Monitoring and control of AI/ML behavior . 37
5.1.10.2.3 ML model performance indicators query and selection for ML training/testing . 38
5.1.10.2.4 ML model performance indicators selection based on MnS consumer policy for ML
training/testing . 38
5.1.10.3 Potential requirements . 39
5.1.10.4 Possible solutions . 39
5.1.10.4.1 Possible solutions for performance indicator selection for ML model training . 39
5.1.10.4.2 Possible solutions for monitoring and control of AI/ML behavior . 40
5.1.10.4.3 Possible solutions for ML model performance indicators query and selection . 40
5.1.10.4.4 Possible solutions for policy-based performance indicator selection . 41
ETSI
3GPP TR 28.908 version 18.1.0 Release 18 5 ETSI TR 128 908 V18.1.0 (2024-10)
5.1.10.5 Evaluation . 41
5.1.11 Configuration management for ML training phase . 42
5.1.11.1 Description . 42
5.1.11.2 Use cases . 42
5.1.11.2.1 Control of producer-initiated ML training . 42
5.1.11.3 Potential requirements . 42
5.1.11.4 Possible solutions . 42
5.1.11.4.1 ML training policy configuration . 42
5.1.11.4.2 ML training activation and deactivation . 43
5.1.11.4.2.1 General framework for activation and deactivation . 43
5.1.11.4.2.2 Instant activation and deactivation . 43
5.1.11.4.2.3 Schedule based activation and deactivation . 43
5.1.11.5 Evaluation . 43
5.1.12 ML Knowledge Transfer Learning . 44
5.1.12.1 Description . 44
5.1.12.2 Use cases . 44
5.1.12.2.1 Discovering sharable Knowledge . 44
5.1.12.2.2 Knowledge sharing and transfer learning . 45
5.1.12.3 Potential requirements . 46
5.1.12.4 Possible solutions . 47
5.1.12.5 Evaluation . 48
5.2 Management Capabilities for AI/ML inference phase . 48
5.2.1 AI/ML Inference History . 48
5.2.1.1 Description . 48
5.2.1.2 Use cases . 48
5.2.1.2.1 Tracking AI/ML inference decision and context . 48
5.2.1.3 Potential requirements . 49
5.2.1.4 Possible solutions . 49
5.2.1.5 Evaluation . 50
5.2.2 Orchestrating AI/ML Inference . 50
5.2.2.1 Description . 50
5.2.2.2 Use cases . 50
5.2.2.2.1 Knowledge sharing on executed actions . 50
5.2.2.2.2 Knowledge sharing on impacts of executed actions . 50
5.2.2.2.3 Abstract information on impacts of executed actions . 51
5.2.2.2.4 Triggering execution of AI/ML inference functions or ML entities . 52
5.2.2.2.5 Orchestrating decisions of AI/ML inference functions or ML entities . 52
5.2.2.3 Potential requirements . 52
5.2.2.4 Possible solutions . 53
5.2.2.5 Evaluation . 58
5.2.3 Coordination between the ML capabilities . 59
5.2.3.1 Description . 59
5.2.3.2 Use cases . 59
5.2.3.2.1 Alignment of the ML capability between 5GC/RAN and 3GPP management system . 59
5.2.3.3 Potential requirements . 59
5.2.3.4 Possible solutions . 60
5.2.3.4.1 Possible solution #1 . 60
5.2.3.5 Evaluation . 60
5.2.4 ML model loading . 60
5.2.4.1 Description . 60
5.2.4.2 Use cases . 61
5.2.4.2.1 ML model loading control and monitoring. 61
5.2.4.3 Potential requirements . 61
5.2.4.4 Possible solutions . 62
5.2.4.4.1 NRM based solution . 62
5.2.4.5 Evaluation . 63
5.2.5 ML inference emulation . 63
5.2.5.1 Description . 63
5.2.5.2 Use cases . 64
5.2.5.2.1 AI/ML inference emulation . 64
5.2.5.2.2 Managing ML inference emulation . 64
5.2.5.3 Potential requirements . 64
ETSI
3GPP TR 28.908 version 18.1.0 Release 18 6 ETSI TR 128 908 V18.1.0 (2024-10)
5.2.5.4 Possible solutions . 65
5.2.5.5 Evaluation . 66
5.2.6 Performance evaluation for AI/ML inference . 67
5.2.6.1 Description . 67
5.2.6.2 Use cases . 67
5.2.6.2.1 AI/ML performance evaluation in inference phase . 67
5.2.6.2.2 ML model performance indicators query and selection for AI/ML inference . 67
5.2.6.2.3 ML model performance indicators selection based on MnS consumer policy for AI/ML

inference . 68
5.2.6.2.4 AI/ML abstract performance . 68
5.2.6.3 Potential requirements . 68
5.2.6.4 Possible solutions . 69
5.2.6.4.1 Possible solutions for AI/ML performance evaluation in inference phase . 69
5.2.6.4.2 Possible solutions for ML model performance indicators query and selection for AI/ML

inference . 70
5.2.6.4.3 Possible solutions for policy-based performance indicator selection based on MnS consumer
policy for AI/ML inference . 70
5.2.6.4.4 Possible solutions for AI/ML performance abstraction . 70
5.2.6.5 Evaluation . 71
5.2.7 Configuration management for AI/ML inference phase . 72
5.2.7.1 Description . 72
5.2.7.2 Use cases . 72
5.2.7.2.1 ML model configuration for RAN domain ES initiated by consumer . 72
5.2.7.2.2 ML model configuration for RAN domain ES initiated by producer . 73
5.2.7.2.3 Partial activation of AI/ML inference capabilities. 73
5.2.7.2.4 Configuration for AI/ML inference initiated by MnS consumer . 74
5.2.7.2.5 Configuration for AI/ML inference selected by producer . 74
5.2.7.2.6 Enabling policy-based activation of AI/ML capabilities . 74
5.2.7.3 Potential requirements . 74
5.2.7.4 Possible solutions . 75
5.2.7.4.1 AI/ML inference function configuration . 75
5.2.7.4.1.1 Configuration for AI/ML inference initiated by MnS consumer . 75
5.2.7.4.1.2 Configuration for AI/ML inference selected by producer - Context-specific configuration . 75
5.2.7.4.2 AI/ML activation . 76
5.2.7.4.2.1 General framework for activation and deactivation . 76
5.2.7.4.2.2 Instant activation and deactivation . 76
5.2.7.4.2.3 Policy based activation and deactivation . 76
5.2.7.4.2.4 Schedule based activation and deactivation . 76
5.2.7.4.2.5 Gradual activation and deactivation . 77
5.2.7.5 Evaluation . 78
5.2.8 AI/ML update control . 78
5.2.8.1 Description . 78
5.2.8.2 Use cases . 78
5.2.8.2.1 Availability of new capabilities or ML entities . 78
5.2.8.2.2 Triggering ML model update . 78
5.2.8.3 Potential requirements . 79
5.2.8.4 Possible solutions . 79
5.2.8.5 Evaluation . 80
5.3 Common management capabilities for ML training and AI/ML inference phase . 80
5.3.1 Trustworthy Machine Learning . 80
5.3.1.1 Description . 80
5.3.1.2 Use cases . 81
5.3.1.2.1 AI/ML trustworthiness indicators . 81
5.3.1.2.2 AI/ML data trustworthiness . 81
5.3.1.2.3 ML training trustworthiness . 82
5.3.1.2.4 AI/ML inference trustworthiness . 82
5.3.1.2.5 Assessment of AI/ML trustworthiness . 82
5.3.1.3 Potential requirements . 83
5.3.1.4 Possible solutions . 84
5.3.1.4.1 ML trustworthiness indicators . 84
5.3.1.4.2 AI/ML data trustworthiness . 85
5.3.1.4.3 ML training trustworthiness . 86
ETSI
3GPP TR 28.908 version 18.1.0 Release 18 7 ETSI TR 128 908 V18.1.0 (2024-10)
5.3.1.4.4 AI/ML inference trustworthiness . 86
5.3.1.4.5 Assessment of AI/ML trustworthiness . 87
5.3.1.5 Evaluation . 87
6 Deployment scenarios . 88
7 Conclusions and recommendations . 91
Annex A: UML source codes . 92
Annex B: Change history . 94
History . 97

ETSI
3GPP TR 28.908 version 18.1.0 Release 18 8 ETSI TR 128 908 V18.1.0 (2024-10)
Foreword
This Technical Report has been produced by the 3rd Generation Partnership Project (3GPP).
The contents of the present document are subject to continuing work within the TSG and may change following formal
TSG approval. Should the TSG modify the contents of the present document, it will be re-released by the TSG with an
identifying change of release date and an increase in version number as follows:
Version x.y.z
where:
x the first digit:
...