ETSI GR ENI 018 V2.1.1 (2021-08)

Experiential Networked Interlligence (ENI); Introduction to Artificial Intelligence Mechanisms for Modular Systems

DGR/ENI-0028_AI_Mechanisms

General Information

• Status: Not Published
• Technical Committee: ENI - Experiential Networked Intelligence
• Current Stage: 12 - Completion
• Due Date: 13-Aug-2021
• Completion Date: 04-Aug-2021

Standards Content (Sample)

GROUP REPORT
Experiential Networked Intelligence (ENI);
Introduction to Artificial Intelligence Mechanisms for
Modular Systems
Disclaimer
The present document has been produced and approved by the Experiential Networked Intelligence (ENI) ETSI Industry
Specification Group (ISG) and represents the views of those members who participated in this ISG.
It does not necessarily represent the views of the entire ETSI membership.

2 ETSI GR ENI 018 V2.1.1 (2021-08)

Reference
DGR/ENI-0028_AI_Mechanisms
Keywords
artificial intelligence, cognition, design, software

ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: [phone]  Fax: [phone]

Siret N° 348 623 562 00017 - APE 7112B
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° w061004871

Important notice
The present document can be downloaded from:
http://www.etsi.org/standards-search
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the prevailing version of an ETSI
deliverable is the one made publicly available in PDF format at www.etsi.org/deliver.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
https://portal.etsi.org/TB/ETSIDeliverableStatus.aspx
If you find errors in the present document, please send your comment to one of the following services:
https://portal.etsi.org/People/CommiteeSupportStaff.aspx
Notice of disclaimer & limitation of liability
The information provided in the present deliverable is directed solely to professionals who have the appropriate degree of
experience to understand and interpret its content in accordance with generally accepted engineering or
other professional standard and applicable regulations.
No recommendation as to products and services or vendors is made or should be implied.
No representation or warranty is made that this deliverable is technically accurate or sufficient or conforms to any law
and/or governmental rule and/or regulation and further, no representation or warranty is made of merchantability or fitness
for any particular purpose or against infringement of intellectual property rights.
In no event shall ETSI be held liable for loss of profits or any other incidental or consequential damages.

Any software contained in this deliverable is provided "AS IS" with no warranties, express or implied, including but not
limited to, the warranties of merchantability, fitness for a particular purpose and non-infringement of intellectual property
rights and ETSI shall not be held liable in any event for any damages whatsoever (including, without limitation, damages
for loss of profits, business interruption, loss of information, or any other pecuniary loss) arising out of or related to the use
of or inability to use the software.
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and
microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.

© ETSI 2021.
All rights reserved.
ETSI
3 ETSI GR ENI 018 V2.1.1 (2021-08)
Contents
Intellectual Property Rights . 4
Foreword . 4
Modal verbs terminology . 4
Executive summary . 4
1 Scope . 5
2 References . 5
2.1 Normative references . 5
2.2 Informative references . 5
3 Definition of terms, symbols and abbreviations . 5
3.1 Terms . 5
3.2 Symbols . 6
3.3 Abbreviations . 6
4 Artificial Intelligence for Modular Systems . 6
4.1 Introduction . 6
4.2 Types of Learning . 6
4.2.1 Introduction. 6
4.2.2 Supervised. 7
4.2.3 Semi-Supervised . 7
4.2.4 Unsupervised . 7
4.2.5 Reinforcement Learning . 7
4.2.6 Feature Learning . 7
4.2.7 Rule-Based Learning . 7
4.2.8 Explanation-Based Learning . 7
4.2.9 Federated Learning . 7
4.2.10 Active Learning . 8
4.3 Model Training . 8
4.3.1 Online Model Training . 8
4.3.2 Offline Model Training . 8
4.3.3 Comparison . 8
4.4 Bias . 9
4.4.1 Definition . 9
4.4.2 Types of Bias . 9
4.4.2.1 Algorithmic Bias . 9
4.4.2.2 Technical Bias . 9
4.4.2.3 Inductive Bias . 10
4.4.2.4 Emergent Bias . 10
4.5 Ethics and Ethical Decision-Making . 10
4.5.1 Introduction. 10
4.5.2 Definitions . 10
4.5.3 Embedding Ethical Decision-Making into AI Systems . 10
4.5.4 Existing Work in Standards and Fora . 11
4.6 Natural Language Processing using AI: an Overview . 11
4.6.1 Introduction. 11
4.6.2 Embeddings . 12
4.6.3 Long Short-Term Memory Models . 12
4.6.4 Attention . 13
4.6.5 Transformer Models . 13
4.6.5.1 Introduction . 13
4.6.5.2 BERT Models . 13
4.6.5.3 GPT Models . 13
4.6.5.4 Sparse Transformers. 13
4.6.5.5 Mixture of Experts Model . 13
5 Summary and Recommendations . 14
History . 15
ETSI
4 ETSI GR ENI 018 V2.1.1 (2021-08)
Intellectual Property Rights
Essential patents
IPRs essential or potentially essential to normative deliverables may have been declared to ETSI. The declarations
pertaining to these essential IPRs, if any, are publicly available for ETSI members and non-members, and can be
found in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to
ETSI in respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the
ETSI Web server (https://ipr.etsi.org/).
Pursuant to the ETSI Directives including the ETSI IPR Policy, no investigation regarding the essentiality of IPRs,
including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not
referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become,
essential to the present document.
Trademarks
The present document may include trademarks and/or tradenames which are asserted and/or registered by their owners.
ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no
right to use or reproduce any trademark and/or tradename. Mention of those trademarks in the present document does
not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks.
DECT™, PLUGTESTS™, UMTS™ and the ETSI logo are trademarks of ETSI registered for the benefit of its

Members. 3GPP™ and LTE™ are trademarks of ETSI registered for the benefit of its Members and of the 3GPP
Organizational Partners. oneM2M™ logo is a trademark of ETSI registered for the benefit of its Members and of the ®
oneM2M Partners. GSM and the GSM logo are trademarks registered and owned by the GSM Association.
Foreword
This Group Report (GR) has been produced by ETSI Industry Specification Group (ISG) Experiential Networked
Intelligence (ENI).
Modal verbs terminology
In the present document "should", "should not", "may", "need not", "will", "will not", "can" and "cannot" are to be
interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.
Executive summary
The present document specifies a high-level functional abstraction of the ENI System Architecture in terms of
Functional Blocks and External Reference Points. This includes describing how different classes of systems interact
with ENI. Processes, models, and detailed information are beyond the scope of the present document.

ETSI
5 ETSI GR ENI 018 V2.1.1 (2021-08)
1 Scope
The purpose of the present document is to provide information on different types of AI mechanisms that can be used for
cognitive networking and decision making in modern system design. Bias and ethics will also be addressed. This
information can be applied to the ENI reference system architecture (and any other applicable ETSI reports or
standards).
2 References
2.1 Normative references
Normative references are not applicable in the present document.
2.2 Informative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] ETSI GS ENI 005 (V2.1.1): "Experiential Networked Intelligence (ENI); System Architecture".
[i.2] Cointe, N., Bonnel, G. and Boissier, O.: "Ethical judgment of agents' behaviors in multi-agent
systems", AAMAS, pgs 1106-1114, 2016.
[i.3] Anderson, M. and Anderson, S.L.: "GenEth: A general ethical dilemma analyzer", AAAI, pgs 253-
261, 2014.
TM
[i.4] Koene, A., Smith, A.L., Egawa, T., Mandalh, S. and Hatada, Y.: "IEEE P70xx, Establishing
Standards for Ethical Technology", KDD, 2018.
NOTE: Available at http://www.kdd.org/kdd2018/files/project-showcase/KDD18_paper_1743.pdf.
3 Definition of terms, symbols and abbreviations
3.1 Terms
For the purposes of the present document, the terms given in ETSI GS ENI 005 [i.1] and the following apply:
active learning: learning algorithm that can query a user interactively to label data with the desired outputs
NOTE: The algorithm proactively selects the subset of examples to be labeled next from the pool of unlabeled
data. The idea is that an ML algorithm could potentially reach a higher level of accuracy while using a
smaller number of training labels if it were allowed to choose the data it wants to learn from.
batch learning: type of offline learning algorithm that is updated (i.e. retrained) periodically
catastrophic forgetting: tendency of an artificial neural network to forget previously learned information when
learning new information
ETSI
6 ETSI GR ENI 018 V2.1.1 (2021-08)
concept drift: not taking changing data and its meanings into account when training an ML model
one-cold vector: 1 × N matrix (vector) used to distinguish each word in a vocabulary from every other word in the
vocabulary, where the vector consists of 1s in all cells with the exception of a single 0 in a cell used uniquely to identify
the word
one-hot vector: 1 × N matrix (vector) used to distinguish each word in a vocabulary from every other word in the
vocabulary, where the vector consists of 0s in all cells with the exception of a single 1 in a cell used uniquely to identify
the word
3.2 Symbols
Void.
3.3 Abbreviations
For the purposes of the present document, the following abbreviations apply:
AI Artificial Intelligence
ANN Artificial Neural Network
BERT Bidirectional Encoder Representations from Transformers
GPT Generative Pre-trained Transformer
LSTM Long Short-Term Memory
MIT Massachusetts Institute of Technology
ML Machine Learning
NLP Natural Language Processing
4 Artificial Intelligence for Modular Systems
4.1 Introduction
Machine learning algorithms learn a solution to a problem from sample data.
Historically, machine learning has focused on non-incremental, offline learning tasks (i.e. where the training set can be
constructed a priori and training stops once this set has been duly processed). There are, however, a number of areas,
such as agent-based learning and processing sequential data, where learning tasks are inherently incremental.
Machine learning systems that use offline learning do not change their approximation of the target function when the
initial training phase has been completed unless the system can detect that the input data has changed significantly. In
contrast, machine learning systems that use online learning continuously re-evaluate their target function. The
advantage of offline learning is that training is computationally intensive. The advantage of online learning is that if
small changes, such as trends, are expected, online training will perform better than offline variants.
The machine learning process adjusts parameters to minimize observed errors; this does not mean that the error rate
reaches 0. However, it does mean that if the error rate becomes too high, then the Artificial Neural Network (ANN)
needs to be redesigned. This in turn is done by defining a function that minimizes the difference between the observed
and desired values.
4.2 Types of Learning
4.2.1 Introduction
In each of the following clauses, each type of algorithm has a large number of variants available. No single learning
algorithm works in an optimal manner for all problems.
ETSI
7 ETSI GR ENI 018 V2.1.1 (2021-08)
4.2.2 Supervised
Supervised learning defines a function that maps an input to an output based on example pairs of labelled inputs and
outputs. Each input is a tuple that consists of an input object and a desired output value. The learning function analyses
the training data and produces a function that can determine the class labels for new data.
There are a large number of algorithms available. No single supervised learning algorithm works in an optimal manner
for all problems.
4.2.3 Semi-Supervised
Semi-supervised learning is a hybridisation of supervised and unsupervised learning, where the training data consists of
both labelled and unlabelled data.
4.2.4 Unsupervised
Unsupervised learning defines a function that maps an input to an output without the benefit of the data being classified
or labelled. The input data can be modelled as probability densities.
4.2.5 Reinforcement Learning
Reinforcement learning uses software agents to take actions in an environment in order to maximize a cumulative
reward. In this approach, the learning agent is not told which actions to take, but instead is responsible for discovering
which actions yield the highest reward.
4.2.6 Feature Learning
Feature learning analyses raw input data to learn the most important characteristics and behaviour representations of
those data that make it easier to discover information from raw data when building different types of predictors (e.g.
classifiers).
4.2.7 Rule-Based Learning
Rule-based learning uses learned rules to represent the knowledge of a system. This type of system learns rules to make
decisions, instead of using a model. These rules are different than other types of rule-based systems because this set of
rules are learned, while rules in other types of systems are defined. The rules are typically imperative rules.
4.2.8 Explanation-Based Learning
Explanation-based learning uses an explanation-driven approach that enables a search procedure, constrained by general
domain knowledge related to the context of the actual problem, to be used to provide more accurate and efficient
learning in knowledge-intensive systems.
4.2.9 Federated Learning
Federated learning is an approach that trains a centralized model using decentralized data that is distributed across
multiple entities holding local data samples, without exchanging their data samples. Each device trains the model on
their own local data set, and then each client sends a model or model update to a centralized service, which aggregates
each client's contribution into one global
...