ISO/IEC TR 24372:2021
(Main)Information technology — Artificial intelligence (AI) — Overview of computational approaches for AI systems
Information technology — Artificial intelligence (AI) — Overview of computational approaches for AI systems
This document provides an overview of the state of the art of computational approaches for AI systems, by describing: a) main computational characteristics of AI systems; b) main algorithms and approaches used in AI systems, referencing use cases contained in ISO/IEC TR 24030.
Titre manque
General Information
Standards Content (Sample)
TECHNICAL ISO/IEC TR
REPORT 24372
First edition
2021-12
Information technology — Artificial
intelligence (AI) — Overview of
computational approaches for AI
systems
Reference number
ISO/IEC TR 24372:2021(E)
© ISO/IEC 2021
---------------------- Page: 1 ----------------------
ISO/IEC TR 24372:2021(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2021
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on
the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below
or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC TR 24372:2021(E)
Contents Page
Foreword .v
Introduction . vi
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 2
5 General . 3
6 Main characteristics of AI systems . .5
6.1 General . 5
6.2 Typical characteristics of AI systems . 6
6.2.1 Adaptable . 6
6.2.2 Constructive . 6
6.2.3 Coordinated . 6
6.2.4 Dynamic . 6
6.2.5 Explainable . 6
6.2.6 Discriminative or generative . 6
6.2.7 Introspective . . 6
6.2.8 Trained or trainable. 7
6.2.9 Accommodating various data . 7
6.3 Computational characteristics of AI systems . 7
6.3.1 Data-based or knowledge-based . 7
6.3.2 Infrastructure-based. 7
6.3.3 Algorithm-dependent . 8
6.3.4 Multi-step or end-to-end learning-based . 9
7 Types of AI computational approaches . 9
7.1 General . 9
7.2 Knowledge-driven approaches . 10
7.3 Data-driven approaches . 10
8 Selected algorithms and approaches used in AI systems .11
8.1 General . 11
8.2 Knowledge engineering and representation . 11
8.2.1 General . 11
8.2.2 Ontology .12
8.2.3 Knowledge graph .12
8.2.4 Semantic web . . 14
8.3 Logic and reasoning . 14
8.3.1 General . 14
8.3.2 Inductive reasoning . 15
8.3.3 Deductive inference .15
8.3.4 Hypothetical reasoning . 16
8.3.5 Bayesian inference . 17
8.4 Machine learning . 18
8.4.1 General . 18
8.4.2 Decision tree . 18
8.4.3 Random forest . 19
8.4.4 Linear regression .20
8.4.5 Logistic regression . 21
8.4.6 K-nearest neighbour . 21
8.4.7 Naïve Bayes . 22
8.4.8 Feedforward neural network. 22
8.4.9 Recurrent neural network . 23
iii
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 3 ----------------------
ISO/IEC TR 24372:2021(E)
8.4.10 Long short-term memory network. 24
8.4.11 Convolutional neural network . 25
8.4.12 Generative adversarial network . 26
8.4.13 Transfer learning . 27
8.4.14 Bidirectional encoder representations from transformers . 27
8.4.15 XLNet .28
8.5 Metaheuristics .29
8.5.1 General .29
8.5.2 Genetic algorithms .29
Bibliography .31
iv
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC TR 24372:2021(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work.
The procedures used to develop this document and those intended for its further maintenance
are described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria
needed for the different types of document should be noted. This document was drafted in
accordance with the editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives or
www.iec.ch/members_experts/refdocs).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents) or the IEC
list of patent declarations received (see https://patents.iec.ch).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to
the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see
www.iso.org/iso/foreword.html. In the IEC, see www.iec.ch/understanding-standards.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 42, Artificial intelligence.
Any feedback or questions on this document should be directed to the user’s national standards
body. A complete listing of these bodies can be found at www.iso.org/members.html and
www.iec.ch/national-committees.
v
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 5 ----------------------
ISO/IEC TR 24372:2021(E)
Introduction
Artificial intelligence (AI)-related products, systems and solutions have become more common
in recent years thanks to rapid software and hardware improvements that boost computational
performance, data storage capabilities and network bandwidth. The intent of this document is to look at
1) 2)
computational methods and approaches within AI systems. Based on ISO/IEC 22989 , ISO/IEC 23053
and ISO/IEC TR 24030, this document provides a description of the characteristics of an AI system and
its computational approaches. The illustration of computational approaches in AI systems includes
both machine learning and non-machine learning methods. To reflect state-of-the-art methods used in
AI, this document is structured as follows:
— Clause 5 provides an overall description of computational approaches in AI systems;
— Clause 6 discusses the main characteristics of AI systems;
— Clause 7 provides a general taxonomy of computational approaches, including knowledge-driven
and data-driven approaches;
— Clause 8 discusses selected algorithms used in AI systems, including basic theories and techniques,
main characteristics and typical applications.
By giving an overview of different technologies used by AI systems, this document is intended to help
users understand computational characteristics and approaches used in AI.
1) Under preparation. Stage at the time of publication: ISO/IEC DIS 22989:2021.
2) Under preparation. Stage at the time of publication: ISO/IEC DIS 23053:2021.
vi
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 6 ----------------------
TECHNICAL REPORT ISO/IEC TR 24372:2021(E)
Information technology — Artificial intelligence (AI) —
Overview of computational approaches for AI systems
1 Scope
This document provides an overview of the state of the art of computational approaches for AI systems,
by describing: a) main computational characteristics of AI systems; b) main algorithms and approaches
used in AI systems, referencing use cases contained in ISO/IEC TR 24030.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 22989, Information technology — Artificial intelligence — Artificial intelligence concepts and
terminology
ISO/IEC 23053, Framework for artificial intelligence (AI) systems using machine learning (ML)
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC 22989 and ISO/IEC 23053
and the following apply.
ISO and IEC maintain terminology databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at https:// www .electropedia .org/
3.1
heuristic search
search, based on experience and judgment, used to obtain acceptable results without guarantee of
success
[SOURCE: ISO/IEC 2382:2015, 2123854, modified — Notes to entry removed.]
3.2
fuzzy logic
fuzzy-set logic
nonclassical logic in which facts, inference rules and quantifiers are given certainty factors
[SOURCE: ISO/IEC 2382:2015, 2123795, modified — Notes to entry removed.]
3.3
generator
neural network that produces samples usually to be classified by a discriminator
Note 1 to entry: Generators primarily appear in the context of generative adversarial networks.
1
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/IEC TR 24372:2021(E)
3.4
discriminator
neural network that classifies samples usually produced by a generator
Note 1 to entry: Discriminators primarily appear in the context of generative adversarial networks.
3.5
generative adversarial network
GAN
neural network architecture comprised of one or more generators and one or more discriminators that
compete to improve model performance
3.6
platform
combination of an operating system and hardware that makes up the operating environment in which
a program runs
[SOURCE: ISO/IEC/IEEE 26513:2017, 3.30]
3.7
perceptron
neural network consisting of one artificial neuron, with a binary or continuous output value that is
determined by applying a monotonic function to a linear combination of the input values and with
error-correction learning
Note 1 to entry: The perceptron forms two decision regions separated by a hyperplane.
Note 2 to entry: For binary input values, the perceptron cannot implement the non-equivalence operation
(EXCLUSIVE OR, XOR).
[SOURCE: ISO/IEC 2382:2015, 2120656, modified — term revised, “or continuous” added to definition
and Notes 3 and 4 to entry removed.]
4 Abbreviated terms
AI artificial intelligence
ASIC application-specific integrated circuit
BERT bidirectional encoder representations from transformers
BPTT back propagation through time
CNN convolutional neural network
CPU central processing unit
DAG directed acyclic graph
DNN deep neural network
ERM empirical risk minimization
FFNN feedforward neural network
FPGA field programmable gate array
GDM gradient descent method
GPU graphics processing unit
2
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/IEC TR 24372:2021(E)
GPT generative pre-training
IoT internet of things
KG knowledge graph
KNN k-nearest neighbour
LSTM long short-term memory
MFCC Mel-frequency cepstrum coefficient
MLM masked language model
NER named entity recognition
NLP natural language processing
NSP next sentence prediction
OWL web ontology language
QA question answering
RDF resource description framework
RNN recurrent neural network
RTRL real-time recurrent learning
SPARQL SPARQL protocol and RDF query language
SQL structured query language
SRM structure risk minimization
SVM support vector machine
URI uniform resource identifier
XML extensible markup language
5 General
Advances in computational approaches are an important driving force in the maturation of AI to become
capable of processing various tasks. Initial AI methods were primarily rules-based and knowledge-
driven. More recently, data-driven methods such as neural networks have gained prominence.
AI computational approaches continue to evolve in industry and academia and are an important
consideration in AI systems.
Computational approaches for AI systems are often categorized based on various criteria. One such
categorization is by the purpose of the AI system. This purpose-based categorization is adapted from
[1]
studies of AI and includes an exemplary categorization of common types.
a) Search methods. These approaches can be further divided into various types of search: classical,
advanced search algorithms, adversarial search and constraint satisfaction.
1) Classical search algorithms solve problems by a search over some state space and can be
divided into uninformed searches and heuristic searches, which apply a rule of thumb to guide
and speed up the search.
3
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/IEC TR 24372:2021(E)
2) Advanced search algorithms include those that search in a local subspace, those that are
nondeterministic, those that search with partial observation of the search space and online
versions of search algorithms.
3) Adversarial search algorithms search in the presence of an opponent and are generally used in
games. These include notable algorithms such as alpha-beta pruning and also include stochastic
and partially observable variations.
4) Constraint satisfaction problems are solved when each variable in the problem has a value that
satisfies all the constraints.
b) Logics, planning and knowledge. These approaches can be further divided into three cases: logics,
planning and state space search, and knowledge representation.
1) Logics, such as propositional logic and first-order logic, are used in classical AI to represent
knowledge. Problem solution in such computational systems involves inference over the logic
using algorithms such as resolution.
2) Planning in classical AI systems involves search over some state space as well as algorithmic
extensions to deal with planning in the real world. Methods to deal with the complexity of real-
world planning involve time and resource constraints, hierarchical planning where problems
are solved at abstract levels first before fine-grain details, multi-agent systems that handle
uncertainties and dealing with other agents in the system.
3) Knowledge representation is a kind of data structure for describing knowledge using predicate
logic, “if-then” generation and knowledge frame representation.
c) Uncertain knowledge and reasoning. Approaches in this area deal with potentially missing,
uncertain or incomplete knowledge. They generally use either probability or fuzzy logic to represent
concepts. Probabilistic computational systems reason using Bayes rule, Bayesian networks or (in
time-dependent situations) hidden Markov models or Kalman filters. Another set of computational
approaches is used for decision-making, including those based on utility theory and decision
networks.
d) Learning. Computational approaches in this area deal with the problem of making the computer
learn similarly to a human. Approaches can be grouped into learning from examples, knowledge-
based learning, probabilistic learning, reinforcement learning, deep learning approaches, GANs
and other learning approaches.
1) Learning from examples involves supervised learning approaches that learn a machine
learning model from labelled data. It includes methods such as decision trees, linear and
logistic regression approaches, artificial neural networks, non-parametric approaches (e.g. the
KNN), SVMs and ensemble learning methods (e.g. bagging, boosting and variants of random
forest).
2) Knowledge-based learning approaches include logic-based approaches, explanation-based
learning and inductive logic programming.
3) Probabilistic learning involves computational approaches such as Bayesian methods and
expectation-maximization methods.
4) Reinforcement learning involves computational systems that receive feedback, make decisions
and take actions in environments to maximize the overall reward. Notable algorithms include
temporal difference-learning and Q-learning.
5) Deep learning neural approaches involve modern computational approaches with many hidden
layers, including deep feedforward networks, regularisation, modern optimization methods,
CNNs and sequence learning methods such as LSTM networks.
4
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 10 ----------------------
ISO/IEC TR 24372:2021(E)
6) GANs involve two competing networks, a generator and discriminator. The generator produces
samples and the discriminator classifies each sample as real or fake. After this iterative
process, trained generators can be used in applications such as creating artificial images.
7) Other learning approaches include unsupervised learning, which involves identifying the
natural structure of data sets; semi-supervised learning, which deals with partially labelled
data sets; online learning algorithms, which continue to learn as they receive data; networks
and relational learning, ranking and preference learning, representation learning, transfer
learning and active learning.
e) Inference. These approaches embody the application of an AI system in estimating parameters
or aspects of (or classifying new or unobserved data based on) learned, acquired or defined
parameters. Bayesian inference is the act of taking statistical inference from a Bayesian point of
view. Approximate inferences, such as variational inference, solves the inference problem by taking
the best approximation of the statistics. Monte Carlo algorithms generate samples from a known
distribution that is difficult to normalize, then infer statistics from generated samples. Causal
inference involves inferencing the causal connections of the observed data.
f) Dimensionality reduction. These computational approaches involve reducing the number of
dimensions of data by either dimensionality reduction (feature extraction) algorithms, which
identify a new smaller number of attributes to represent data, or feature selection, which chooses a
subset of the most appropriate attributes.
g) Communicating, perceiving and acting. Computation approaches in these areas are associated
with the fields of NLP (including tasks such as language modelling, text classification, information
retrieval, information extraction, parsing, machine translation and speech recognition), computer
vision (including image processing and object recognition) and robotics.
These categories and subcategories are not mutually exclusive. For instance, deep learning approaches
[d)5)] can be either supervised [d)1)] or unsupervised [d)7)], reinforcement learning [d)4)] can be
achieved through deep learning [d)5)], and approaches for machine translation or object recognition
[g)] can be learning approaches [d)].
ISO/IEC 22989 specifies concepts and terminologies relevant to AI computational approaches.
ISO/IEC 23053 provides a framework for AI systems using machine learning, encompassing machine
learning algorithms, optimization algorithms and machine learning methods. ISO/IEC TR 24030
collects and analyses AI use cases.
6 Main characteristics of AI systems
6.1 General
Not all AI systems are based on machine learning or neural networks. To demonstrate the breadth of AI
systems, some frequently encountered characteristics of AI systems are described in 6.2 and 6.3. These
characteristics are broadly conceptual and not tied to a specific methodology or architecture. In the
aggregate these characteristics differentiate AI systems from non-AI systems.
Some characteristics of AI systems are common and apply widely to different use cases. Others
are specific to a small number of use cases within a specific industry. This clause contains a list of
characteristics of AI systems which is not exhaustive but contains attributes intrinsic to many AI
systems. While the list is not limited to a specific base technology (such as AI systems built with neural
networks), it does not encompass every type of dynamic AI system.
5
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/IEC TR 24372:2021(E)
6.2 Typical characteristics of AI systems
6.2.1 Adaptable
Some AI systems adapt
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.