Information technology -- Artificial intelligence (AI) -- Overview of computational approaches for AI systems

This document provides an overview of the state of the art of computational approaches for AI systems, by describing: a) main computational characteristics of AI systems; b) main algorithms and approaches used in AI systems, referencing use cases contained in ISO/IECÂ TRÂ 24030.

Titre manque

General Information

Status
Published
Publication Date
06-Dec-2021
Current Stage
5060 - Close of voting Proof returned by Secretariat
Start Date
08-Nov-2021
Completion Date
08-Nov-2021
Ref Project

Buy Standard

Technical report
ISO/IEC TR 24372:2021 - Information technology -- Artificial intelligence (AI) -- Overview of computational approaches for AI systems
English language
32 pages
sale 15% off
Preview
sale 15% off
Preview

Standards Content (sample)

TECHNICAL ISO/IEC TR
REPORT 24372
First edition
2021-12
Information technology — Artificial
intelligence (AI) — Overview of
computational approaches for AI
systems
Reference number
ISO/IEC TR 24372:2021(E)
© ISO/IEC 2021
---------------------- Page: 1 ----------------------
ISO/IEC TR 24372:2021(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2021

All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on

the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below

or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC TR 24372:2021(E)
Contents Page

Foreword ..........................................................................................................................................................................................................................................v

Introduction .............................................................................................................................................................................................................................. vi

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ..................................................................................................................................................................................... 1

3 Terms and definitions .................................................................................................................................................................................... 1

4 Abbreviated terms ............................................................................................................................................................................................. 2

5 General ........................................................................................................................................................................................................................... 3

6 Main characteristics of AI systems ........................................................................................................................................... ........5

6.1 General ........................................................................................................................................................................................................... 5

6.2 Typical characteristics of AI systems ................................................................................................................................. 6

6.2.1 Adaptable .................................................................................................................................................................................. 6

6.2.2 Constructive ........................................................................................................................................................................... 6

6.2.3 Coordinated ............................................................................................................................................................................ 6

6.2.4 Dynamic ..................................................................................................................................................................................... 6

6.2.5 Explainable .............................................................................................................................................................................. 6

6.2.6 Discriminative or generative ................................................................................................................................... 6

6.2.7 Introspective ...................................................................... .................................................................................................... 6

6.2.8 Trained or trainable......................................................................................................................................................... 7

6.2.9 Accommodating various data ................................................................................................................................. 7

6.3 Computational characteristics of AI systems ............................................................................................................. 7

6.3.1 Data-based or knowledge-based .......................................................................................................................... 7

6.3.2 Infrastructure-based...................................................................................................................................................... 7

6.3.3 Algorithm-dependent ..................................................................................................................................................... 8

6.3.4 Multi-step or end-to-end learning-based ..................................................................................................... 9

7 Types of AI computational approaches ........................................................................................................................................ 9

7.1 General ........................................................................................................................................................................................................... 9

7.2 Knowledge-driven approaches ............................................................................................................................................. 10

7.3 Data-driven approaches .............................................................................................................................................................. 10

8 Selected algorithms and approaches used in AI systems ......................................................................................11

8.1 General ........................................................................................................................................................................................................ 11

8.2 Knowledge engineering and representation ............................................................................................................ 11

8.2.1 General ..................................................................................................................................................................................... 11

8.2.2 Ontology ..................................................................................................................................................................................12

8.2.3 Knowledge graph ............................................................................................................................................................12

8.2.4 Semantic web ...................................................................... ................................................................................................ 14

8.3 Logic and reasoning ........................................................................................................................................................................ 14

8.3.1 General ..................................................................................................................................................................................... 14

8.3.2 Inductive reasoning ...................................................................................................................................................... 15

8.3.3 Deductive inference ......................................................................................................................................................15

8.3.4 Hypothetical reasoning ............................................................................................................................................. 16

8.3.5 Bayesian inference ......................................................................................................................................................... 17

8.4 Machine learning ............................................................................................................................................................................... 18

8.4.1 General ..................................................................................................................................................................................... 18

8.4.2 Decision tree ....................................................................................................................................................................... 18

8.4.3 Random forest ................................................................................................................................................................... 19

8.4.4 Linear regression ............................................................................................................................................................20

8.4.5 Logistic regression ........................................................................................................................................................ 21

8.4.6 K-nearest neighbour ..................................................................................................................................................... 21

8.4.7 Naïve Bayes .......................................................................................................................................................................... 22

8.4.8 Feedforward neural network............................................................................................................................... 22

8.4.9 Recurrent neural network ...................................................................................................................................... 23

iii
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 3 ----------------------
ISO/IEC TR 24372:2021(E)

8.4.10 Long short-term memory network.................................................................................................................. 24

8.4.11 Convolutional neural network ............................................................................................................................ 25

8.4.12 Generative adversarial network ........................................................................................................................ 26

8.4.13 Transfer learning ............................................................................................................................................................ 27

8.4.14 Bidirectional encoder representations from transformers ...................................................... 27

8.4.15 XLNet .........................................................................................................................................................................................28

8.5 Metaheuristics .....................................................................................................................................................................................29

8.5.1 General .....................................................................................................................................................................................29

8.5.2 Genetic algorithms ........................................................................................................................................................29

Bibliography .............................................................................................................................................................................................................................31

© ISO/IEC 2021 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC TR 24372:2021(E)
Foreword

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical

Commission) form the specialized system for worldwide standardization. National bodies that are

members of ISO or IEC participate in the development of International Standards through technical

committees established by the respective organization to deal with particular fields of technical

activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international

organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the

work.

The procedures used to develop this document and those intended for its further maintenance

are described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria

needed for the different types of document should be noted. This document was drafted in

accordance with the editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives or

www.iec.ch/members_experts/refdocs).

Attention is drawn to the possibility that some of the elements of this document may be the subject

of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent

rights. Details of any patent rights identified during the development of the document will be in the

Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents) or the IEC

list of patent declarations received (see https://patents.iec.ch).

Any trade name used in this document is information given for the convenience of users and does not

constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and

expressions related to conformity assessment, as well as information about ISO's adherence to

the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see

www.iso.org/iso/foreword.html. In the IEC, see www.iec.ch/understanding-standards.

This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,

Subcommittee SC 42, Artificial intelligence.

Any feedback or questions on this document should be directed to the user’s national standards

body. A complete listing of these bodies can be found at www.iso.org/members.html and

www.iec.ch/national-committees.
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 5 ----------------------
ISO/IEC TR 24372:2021(E)
Introduction

Artificial intelligence (AI)-related products, systems and solutions have become more common

in recent years thanks to rapid software and hardware improvements that boost computational

performance, data storage capabilities and network bandwidth. The intent of this document is to look at

1) 2)

computational methods and approaches within AI systems. Based on ISO/IEC 22989 , ISO/IEC 23053

and ISO/IEC TR 24030, this document provides a description of the characteristics of an AI system and

its computational approaches. The illustration of computational approaches in AI systems includes

both machine learning and non-machine learning methods. To reflect state-of-the-art methods used in

AI, this document is structured as follows:

— Clause 5 provides an overall description of computational approaches in AI systems;

— Clause 6 discusses the main characteristics of AI systems;

— Clause 7 provides a general taxonomy of computational approaches, including knowledge-driven

and data-driven approaches;

— Clause 8 discusses selected algorithms used in AI systems, including basic theories and techniques,

main characteristics and typical applications.

By giving an overview of different technologies used by AI systems, this document is intended to help

users understand computational characteristics and approaches used in AI.
1) Under preparation. Stage at the time of publication: ISO/IEC DIS 22989:2021.
2) Under preparation. Stage at the time of publication: ISO/IEC DIS 23053:2021.
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 6 ----------------------
TECHNICAL REPORT ISO/IEC TR 24372:2021(E)
Information technology — Artificial intelligence (AI) —
Overview of computational approaches for AI systems
1 Scope

This document provides an overview of the state of the art of computational approaches for AI systems,

by describing: a) main computational characteristics of AI systems; b) main algorithms and approaches

used in AI systems, referencing use cases contained in ISO/IEC TR 24030.
2 Normative references

The following documents are referred to in the text in such a way that some or all of their content

constitutes requirements of this document. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any amendments) applies.

ISO/IEC 22989, Information technology — Artificial intelligence — Artificial intelligence concepts and

terminology

ISO/IEC 23053, Framework for artificial intelligence (AI) systems using machine learning (ML)

3 Terms and definitions

For the purposes of this document, the terms and definitions given in ISO/IEC 22989 and ISO/IEC 23053

and the following apply.

ISO and IEC maintain terminology databases for use in standardization at the following addresses:

— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at https:// www .electropedia .org/
3.1
heuristic search

search, based on experience and judgment, used to obtain acceptable results without guarantee of

success
[SOURCE: ISO/IEC 2382:2015, 2123854, modified — Notes to entry removed.]
3.2
fuzzy logic
fuzzy-set logic

nonclassical logic in which facts, inference rules and quantifiers are given certainty factors

[SOURCE: ISO/IEC 2382:2015, 2123795, modified — Notes to entry removed.]
3.3
generator
neural network that produces samples usually to be classified by a discriminator

Note 1 to entry: Generators primarily appear in the context of generative adversarial networks.

© ISO/IEC 2021 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/IEC TR 24372:2021(E)
3.4
discriminator
neural network that classifies samples usually produced by a generator

Note 1 to entry: Discriminators primarily appear in the context of generative adversarial networks.

3.5
generative adversarial network
GAN

neural network architecture comprised of one or more generators and one or more discriminators that

compete to improve model performance
3.6
platform

combination of an operating system and hardware that makes up the operating environment in which

a program runs
[SOURCE: ISO/IEC/IEEE 26513:2017, 3.30]
3.7
perceptron

neural network consisting of one artificial neuron, with a binary or continuous output value that is

determined by applying a monotonic function to a linear combination of the input values and with

error-correction learning

Note 1 to entry: The perceptron forms two decision regions separated by a hyperplane.

Note 2 to entry: For binary input values, the perceptron cannot implement the non-equivalence operation

(EXCLUSIVE OR, XOR).

[SOURCE: ISO/IEC 2382:2015, 2120656, modified — term revised, “or continuous” added to definition

and Notes 3 and 4 to entry removed.]
4 Abbreviated terms
AI artificial intelligence
ASIC application-specific integrated circuit
BERT bidirectional encoder representations from transformers
BPTT back propagation through time
CNN convolutional neural network
CPU central processing unit
DAG directed acyclic graph
DNN deep neural network
ERM empirical risk minimization
FFNN feedforward neural network
FPGA field programmable gate array
GDM gradient descent method
GPU graphics processing unit
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/IEC TR 24372:2021(E)
GPT generative pre-training
IoT internet of things
KG knowledge graph
KNN k-nearest neighbour
LSTM long short-term memory
MFCC Mel-frequency cepstrum coefficient
MLM masked language model
NER named entity recognition
NLP natural language processing
NSP next sentence prediction
OWL web ontology language
QA question answering
RDF resource description framework
RNN recurrent neural network
RTRL real-time recurrent learning
SPARQL SPARQL protocol and RDF query language
SQL structured query language
SRM structure risk minimization
SVM support vector machine
URI uniform resource identifier
XML extensible markup language
5 General

Advances in computational approaches are an important driving force in the maturation of AI to become

capable of processing various tasks. Initial AI methods were primarily rules-based and knowledge-

driven. More recently, data-driven methods such as neural networks have gained prominence.

AI computational approaches continue to evolve in industry and academia and are an important

consideration in AI systems.

Computational approaches for AI systems are often categorized based on various criteria. One such

categorization is by the purpose of the AI system. This purpose-based categorization is adapted from

[1]
studies of AI and includes an exemplary categorization of common types.

a) Search methods. These approaches can be further divided into various types of search: classical,

advanced search algorithms, adversarial search and constraint satisfaction.

1) Classical search algorithms solve problems by a search over some state space and can be

divided into uninformed searches and heuristic searches, which apply a rule of thumb to guide

and speed up the search.
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/IEC TR 24372:2021(E)

2) Advanced search algorithms include those that search in a local subspace, those that are

nondeterministic, those that search with partial observation of the search space and online

versions of search algorithms.

3) Adversarial search algorithms search in the presence of an opponent and are generally used in

games. These include notable algorithms such as alpha-beta pruning and also include stochastic

and partially observable variations.

4) Constraint satisfaction problems are solved when each variable in the problem has a value that

satisfies all the constraints.

b) Logics, planning and knowledge. These approaches can be further divided into three cases: logics,

planning and state space search, and knowledge representation.

1) Logics, such as propositional logic and first-order logic, are used in classical AI to represent

knowledge. Problem solution in such computational systems involves inference over the logic

using algorithms such as resolution.

2) Planning in classical AI systems involves search over some state space as well as algorithmic

extensions to deal with planning in the real world. Methods to deal with the complexity of real-

world planning involve time and resource constraints, hierarchical planning where problems

are solved at abstract levels first before fine-grain details, multi-agent systems that handle

uncertainties and dealing with other agents in the system.

3) Knowledge representation is a kind of data structure for describing knowledge using predicate

logic, “if-then” generation and knowledge frame representation.

c) Uncertain knowledge and reasoning. Approaches in this area deal with potentially missing,

uncertain or incomplete knowledge. They generally use either probability or fuzzy logic to represent

concepts. Probabilistic computational systems reason using Bayes rule, Bayesian networks or (in

time-dependent situations) hidden Markov models or Kalman filters. Another set of computational

approaches is used for decision-making, including those based on utility theory and decision

networks.

d) Learning. Computational approaches in this area deal with the problem of making the computer

learn similarly to a human. Approaches can be grouped into learning from examples, knowledge-

based learning, probabilistic learning, reinforcement learning, deep learning approaches, GANs

and other learning approaches.

1) Learning from examples involves supervised learning approaches that learn a machine

learning model from labelled data. It includes methods such as decision trees, linear and

logistic regression approaches, artificial neural networks, non-parametric approaches (e.g. the

KNN), SVMs and ensemble learning methods (e.g. bagging, boosting and variants of random

forest).

2) Knowledge-based learning approaches include logic-based approaches, explanation-based

learning and inductive logic programming.

3) Probabilistic learning involves computational approaches such as Bayesian methods and

expectation-maximization methods.

4) Reinforcement learning involves computational systems that receive feedback, make decisions

and take actions in environments to maximize the overall reward. Notable algorithms include

temporal difference-learning and Q-learning.

5) Deep learning neural approaches involve modern computational approaches with many hidden

layers, including deep feedforward networks, regularisation, modern optimization methods,

CNNs and sequence learning methods such as LSTM networks.
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 10 ----------------------
ISO/IEC TR 24372:2021(E)

6) GANs involve two competing networks, a generator and discriminator. The generator produces

samples and the discriminator classifies each sample as real or fake. After this iterative

process, trained generators can be used in applications such as creating artificial images.

7) Other learning approaches include unsupervised learning, which involves identifying the

natural structure of data sets; semi-supervised learning, which deals with partially labelled

data sets; online learning algorithms, which continue to learn as they receive data; networks

and relational learning, ranking and preference learning, representation learning, transfer

learning and active learning.

e) Inference. These approaches embody the application of an AI system in estimating parameters

or aspects of (or classifying new or unobserved data based on) learned, acquired or defined

parameters. Bayesian inference is the act of taking statistical inference from a Bayesian point of

view. Approximate inferences, such as variational inference, solves the inference problem by taking

the best approximation of the statistics. Monte Carlo algorithms generate samples from a known

distribution that is difficult to normalize, then infer statistics from generated samples. Causal

inference involves inferencing the causal connections of the observed data.

f) Dimensionality reduction. These computational approaches involve reducing the number of

dimensions of data by either dimensionality reduction (feature extraction) algorithms, which

identify a new smaller number of attributes to represent data, or feature selection, which chooses a

subset of the most appropriate attributes.

g) Communicating, perceiving and acting. Computation approaches in these areas are associated

with the fields of NLP (including tasks such as language modelling, text classification, information

retrieval, information extraction, parsing, machine translation and speech recognition), computer

vision (including image processing and object recognition) and robotics.

These categories and subcategories are not mutually exclusive. For instance, deep learning approaches

[d)5)] can be either supervised [d)1)] or unsupervised [d)7)], reinforcement learning [d)4)] can be

achieved through deep learning [d)5)], and approaches for machine translation or object recognition

[g)] can be learning approaches [d)].

ISO/IEC 22989 specifies concepts and terminologies relevant to AI computational approaches.

ISO/IEC 23053 provides a framework for AI systems using machine learning, encompassing machine

learning algorithms, optimization algorithms and machine learning methods. ISO/IEC TR 24030

collects and analyses AI use cases.
6 Main characteristics of AI systems
6.1 General

Not all AI systems are based on machine learning or neural networks. To demonstrate the breadth of AI

systems, some frequently encountered characteristics of AI systems are described in 6.2 and 6.3. These

characteristics are broadly conceptual and not tied to a specific methodology or architecture. In the

aggregate these characteristics differentiate AI systems from non-AI systems.

Some characteristics of AI systems are common and apply widely to different use cases. Others

are specific to a small number of use cases within a specific industry. This clause contains a list of

characteristics of AI systems which is not exhaustive but contains attributes intrinsic to many AI

systems. While the list is not limited to a specific base technology (such as AI systems built with neural

networks), it does not encompass every type of dynamic AI system.
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/IEC TR 24372:2021(E)
6.2 Typical characteristics of AI systems
6.2.1 Adaptable
Some AI systems adapt
...

Questions, Comments and Discussion

Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.