ISO 39003:2023
(Main)Road traffic safety (RTS) — Guidance on ethical considerations relating to safety for autonomous vehicles
Road traffic safety (RTS) — Guidance on ethical considerations relating to safety for autonomous vehicles
This document gives guidance on ethical considerations with regards to road traffic safety of autonomous vehicles (AVs). It is applicable to vehicles in level 5 mode according to SAE J3016 in 2022, as part of its report. This document does not apply to the technical method used to control the decision-making process, nor does it give any guidance on the desired outcomes of those decisions; it gives guidance on ethical aspects for consideration in the design of decision-making process. This document does not set requirements for the outcomes of ethical decisions, nor does it offer guidance on methodology. It only details aspects of the behaviour of AVs for which considerations may be made by the designer/manufacturer to ensure that key aspects are not overlooked or disregarded. This document does not offer the technical precision to prescribe the required controls but would, rather, offer a set of “protocol guidelines” that all decision makers regarding automated driving could choose to self-certify against to assure that the desired necessary ethical considerations were addressed during design and effectively controlled.
Sécurité routière — Recommandations relatives aux considérations éthiques en matière de sécurité pour les véhicules autonomes
General Information
Buy Standard
Standards Content (Sample)
INTERNATIONAL ISO
STANDARD 39003
First edition
2023-07
Road traffic safety (RTS) — Guidance
on ethical considerations relating to
safety for autonomous vehicles
Sécurité routière — Recommandations relatives aux considérations
éthiques en matière de sécurité pour les véhicules autonomes
Reference number
ISO 39003:2023(E)
© ISO 2023
---------------------- Page: 1 ----------------------
ISO 39003:2023(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO 2023
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on
the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below
or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii
© ISO 2023 – All rights reserved
---------------------- Page: 2 ----------------------
ISO 39003:2023(E)
Contents Page
Foreword .v
Introduction . vi
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 External factors affecting autonomous vehicle safety . 2
4.1 General . 2
4.2 The road environment . 2
5 Interested parties in AV design and operations . 3
5.1 General . 3
5.2 Producers – Manufacturers, designers and their suppliers . 3
5.3 Distribution chain – Distributors, sellers . 3
5.4 Purchasers, owners and operators . 4
5.5 Government agencies and other interested parties . 4
6 Governance, assessment and evaluation . 4
6.1 General . 4
6.2 Ethical reference for assessment . 4
6.3 Additional standards . 4
6.4 General . 5
6.4.1 Higher organizational level . 5
6.4.2 Development organizational level . 5
6.4.3 Specific development and implementation processes . 6
6.4.4 Post implementation checking against ethical criteria . 7
6.5 Conducting the assessment . 7
6.6 Expression of results and conclusion . 8
7 Operationalization of ethics - discussion on values and ethics to consider .8
7.1 General . 8
7.2 Ethical framework for the design of AVs (driving action policies and ethical design) . 8
7.2.1 Purpose . 9
7.2.2 Values. 10
7.2.3 Principles . 10
7.2.4 Methods for construction and evaluation of maxims .13
7.3 Background of maxims . 13
7.3.1 Maxim design and construction . 13
7.3.2 Evaluations of maxims . 14
7.4 Driving action policies . 15
7.4.1 Need versus desire driving action policy . 16
7.4.2 Once on the road space . 16
7.4.3 In the lane behaviour (includes braking) . 17
7.4.4 Lane switching . 18
7.4.5 In the presence of the other . 19
7.4.6 Road/infrastructure use cases . 19
7.4.7 Resolving conflict .20
7.4.8 Negotiations. 21
7.4.9 AV unable to function as intended . 21
7.4.10 Yielding to first responders and emergency response vehicles .22
7.4.11 Protecting other road users . 22
7.4.12 Unavoidable collision with other road users . 23
7.4.13 Other issues . 24
8 Framework for rule construction and dealing with violations and deviations .24
8.1 General . 24
iii
© ISO 2023 – All rights reserved
---------------------- Page: 3 ----------------------
ISO 39003:2023(E)
8.2 Framework . 24
8.3 Goals . 25
8.4 Primary rules . 25
8.5 Supporting rules . 26
8.6 Precautionary and disabling rules (prevention) . 26
8.7 Reinforcing and enabling rules (performance) . 26
8.8 Counter rules . 26
8.8.1 Exceptions (prevention and performance) . 26
8.8.2 Discretionary and compensatory rules . 27
8.8.3 Misconduct . 27
8.8.4 Violation . 27
8.8.5 Deviations . 27
8.8.6 Breakdowns .28
8.9 Rule strategy .28
8.10 Boundaries (for prevention of unwanted behaviour) .28
8.10.1 Margins .28
8.10.2 Barriers .28
8.10.3 Buffers .28
8.11 Promoters (for performance) .29
8.12 Further considerations . 29
9 External/internal design .29
10 Sustainability .30
11 Review and re-evaluation following controls system updates .31
Annex A (informative) Overview of ethical philosophy related to AV .32
Annex B (informative) Sustainability issues .34
Annex C (informative) Responsibility and accountability in the context of AVs .35
Annex D (informative) Action plan – Example .41
Bibliography .43
iv
© ISO 2023 – All rights reserved
---------------------- Page: 4 ----------------------
ISO 39003:2023(E)
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to
the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see
www.iso.org/iso/foreword.html.
This document was prepared by Technical Committee ISO/TC 241, Road traffic safety management
systems.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www.iso.org/members.html.
v
© ISO 2023 – All rights reserved
---------------------- Page: 5 ----------------------
ISO 39003:2023(E)
Introduction
0.1 General
A long established and commonly held view is that the single most significant factor in road traffic
safety are the actions of the driver. However, the road transport system is a complex socio-technical
system which places high demands on humans to negotiate. Crashes occur since human beings, due
to finite cognitive capacity and physiological limitations, cannot always cope with these demands. To
increase road safety, the road transport system therefore should be designed to support the road user
to cope with this complexity and to mitigate the effects of crashes when this is not possible.
Autonomous vehicles (AVs) have the potential to replace the human driver and to increase road safety
by reducing the risk of human error in daily operations. This will probably take a long time and in
the meantime the capabilities of technology and humans should be combined and integrated in such a
way that the strengths of both are utilized efficiently. Important safety improvements were made over
the last century, but automated driving technology provides new opportunities to improve safety even
further.
AVs are unlikely to gain widespread acceptance until the travelling public feels assured of their
safety and security, not only of passengers but also other vehicles and vulnerable road users. This
includes a behaviour of the AV according to the desires and requirements of society. However, despite
contemporary expectations and a very optimistic view on technology, humans have an extremely
valuable faculty that machines will probably never possess—ethical decision making and judgement.
For real traffic situations the human does not always have the prerequisites to take a rational decision
based on ethics since the time frame is often too narrow. For that reason, many “decisions” are made
instinctively without the possibility of making a well-reasoned and balanced ethical decision. Defining
this element and imparting it on machines is critical for the success of AVs. This can only be achieved by
ensuring that AVs are equipped with driving action policies that align with the general ethical beliefs,
needs, and desires of humanity on a global level, subject to local specific nuances.
To achieve the imprint of global and local ethical considerations in AV design, there is a need for a
framework of ethics involving the necessary stakeholders of different areas. To that objective’s end, it
is important to develop ethical standards for AV behaviour. While there are few standards available,
or under development, that address the engineering and technological aspects of AVs, there are no
International Standards that address aspects concerning the general topics of driving policy and ethical
behaviour, which are also important. Driving policy means a general approach of how an AV makes
a decision and performs manoeuvres. Ethical relevant behaviour represents positive or potentially
negative impact on road users and especially the vulnerable ones as well as the public space at large.
The objective of this document is to lay out a framework for the development of a standard for ethical
and societally accepted driving policy.
0.2 The concept of autonomous vehicles
AVs have the objective to substitute driver, including tasks, decisions, and responsibilities. Hence the
driver behaviour model proposed by Michon (1985) and applied to AVs in Reference [26] (Figure 1)
can be a start to design and operate AVs. The driving task consists of three levels: the strategic level
concerned with the higher-level trip goals (e.g. route choice), the tactical level concerned with the
manoeuvring decisions, including negotiations and interactions with other vehicles, and operational
level concerned with the execution of these tactical and operational behaviours at the level of vehicle
control. There is no strictly hierarchical relationship among these levels.
vi
© ISO 2023 – All rights reserved
---------------------- Page: 6 ----------------------
ISO 39003:2023(E)
Figure 1 — Hierarchical model of driving task (source: Reference [26])
Beside this consideration which has a timely sequence, the AV will ideally be designed thoroughly
to ensure safe and secure operation, and the decision base for the desired behaviour established.
Therefore, all necessary information would be available, and the situation will be captured and
understood while different perspectives will support the final decision.
Figure 1 is a generic decomposition of how an AV works. Figure 2 is a more focused and detailed
description of the decision making module (where the ethical considerations resides) Hence a functional
decomposition into the following six layers (Reference [27]) (Figure 2) can provide the first step into
solution space.
vii
© ISO 2023 – All rights reserved
---------------------- Page: 7 ----------------------
ISO 39003:2023(E)
Figure 2 — Decomposition layers (source: Reference [27])
— Information access
Information available and accessible.
— Information reception
All necessary information can be captured.
— Information processing
Captured information contain all required classification and identification for further process.
— Situational understanding
Based on information, situation is captured and understood.
— Behavioural decision
Based on designed or trained situational awareness, the desired behaviour is chosen.
— Action
Vehicle transforms whished behaviour into action.
Within this framework, technical realization can differ while the focus of the situation decision will be
comprehensible, accountable, and comparable among different designs. A thorough development will
benefit transparency for action and to take decisions. The work described in this document focuses
primarily on Layer 4 – behavioural decision.
viii
© ISO 2023 – All rights reserved
---------------------- Page: 8 ----------------------
ISO 39003:2023(E)
0.3 The concept of ethical considerations for autonomous vehicles
Philosophy helps people question, understand and make sense of the world so they can act properly in
it. This means that decisions and actions are not only intrinsically fair, but that they are also performed
in a way that is balanced with respect with other’s needs, the societal needs and humanistic values,
as well as with respect to the physical world. It is widely accepted that philosophy can be divided into
theoretical philosophy and practical philosophy. Theoretical philosophy consists of subdisciplines such
as ontology, epistemology, logic, and philosophy of mind, amongst others. Practical philosophy consists
of subdisciplines like ethics, political and social philosophy, and aesthetics.
Within ethics there are three main branches: normative ethics, applied ethics as well as meta-ethics.
Normative ethics is the study of ethical action and determining standards for decision making and
conduct (e.g. consequentialism, deontology, virtue ethics, see Annex A). Applied ethics is the application
of standards of ethics to real life situations (e.g. biomedical ethics, AI ethics, political ethics, see
Annex B). In this document, ethics refers to both normative and applied ethics but not to meta-ethics
(which is more concerned with the meaning of ethical concepts such as ‘good’ or ‘bad’ or the nature of
moral judgments. Applied ethics is a flexible and practical way to address ethical considerations in the
development of new technologies because an applied ethics field, such as AV ethics, can borrow from
more than one normative school of ethics. Therefore, the framework offered in this document, although
based on a principle-based approach, can use ideas from deontology and virtue ethics, for example, to
help solve problems. This is a balanced approach; it does not condone one type of normative ethics over
another, it offers a range of perspectives that will help the designer/developer in choosing the best (or
better) decision possible for specific situations.
In summary, ethics is the study of how to choose to act in situations in order to make good and rationally
justifiable decisions. To make decisions, it is necessary to clarify what matters and to understand what
is considered good, bad, right and wrong. Therefore, ethics may be viewed as a tool that helps us create
the difference between a “good” decision and a “bad” one, but also to justify this decision on rational and
intersubjectively acceptable grounds. This is of great importance in the development of AVs because
choices made during the design and development of AV systems determine its “driving behaviour” and
how it caters to its passengers and interacts with other road users (e.g. vulnerable road users). “Driving
behaviour” is what was designed for and programmed into the machine, “conduct” is what actually
transpires as a consequence of applying the driving behaviour to the real world.
This document offers a particular framework for AV ethics which is intended to support the practical
integration of ethics into the AV development process. The framework suggested here builds upon
Socrates conception of the hierarchical nature of philosophy for practical use which has three levels
(see Figure 3). The base level is the conduct that transpires as a consequence of decision and actions
(behaviour), the second level is how these behaviours are governed. Namely what kind of policy and
arrangements are in place to make decisions. The top level is purpose, which behaviour and conduct
will normally align with, in order to achieve the goals and objectives of the entire endeavour. Finally,
knowledge is the “tool” for making sure that the purpose is reasonable and balanced and that the
governance and behaviour/conduct are indeed feasible and appropriate.
ix
© ISO 2023 – All rights reserved
---------------------- Page: 9 ----------------------
ISO 39003:2023(E)
Figure 3 — Socrates’ hierarchical nature of philosophy
Figure 4 is a graphical depiction of the framework advocated here for AV ethics. It shares similarities
with other ethical approaches for emerging technologies, it is also unique as it has been created
specifically for the AV context. Essentially, the AV ethics framework provides guidance for reflective
and critical decision making and is composed of four main elements: purpose, values, principles,
and process. The framework approach provides structure and guidance, yet it is flexible enough to
accommodate multicultural perspectives.
Figure 4 — The AV ethics framework for the integration of ethical considerations into the
design and development of AVs
The purpose element refers to the overall goal of integrating ethics into AV design and development.
The purpose is the “why” aspect of the framework or the reason for the existence of this document.
Purpose offers high level (or meta) guidance when decision making, i.e. will a particular decision lead
to an increase in road traffic safety or not. As this is an international standard, the values element
provides guidance on what is important to the world’s population. Therefore, universal values such
as the UN universal values (e.g. human dignity) are recommended in this framework. The principles
element gives more detailed guidance on decision making by identifying the boundaries of a good
x
© ISO 2023 – All rights reserved
---------------------- Page: 10 ----------------------
ISO 39003:2023(E)
decision and a bad one as well as directing the attention to specific areas (fairness in AV behaviour and
conduct in the context of other vehicles). The process element refers to two levels of process activities,
governance, and daily operations, which are required to integrate ethics into AV development. Table 1
summarises these four elements.
Specific examples for the values, principles, and process elements are provided in the AV ethics
framework section. The purpose is not an example but a strong recommendation as it contributes to
the overall goal of the ISO traffic management safety series of standards. The values, principles a
...
DRAFT INTERNATIONAL STANDARD
ISO/DIS 39003
ISO/TC 241 Secretariat: SIS
Voting begins on: Voting terminates on:
2022-07-11 2022-10-03
Road Traffic Safety (RTS) — Guidance on ethical
considerations relating to safety for autonomous vehicles
ICS: 03.220.20
THIS DOCUMENT IS A DRAFT CIRCULATED
FOR COMMENT AND APPROVAL. IT IS
THEREFORE SUBJECT TO CHANGE AND MAY
This document is circulated as received from the committee secretariat.
NOT BE REFERRED TO AS AN INTERNATIONAL
STANDARD UNTIL PUBLISHED AS SUCH.
IN ADDITION TO THEIR EVALUATION AS
BEING ACCEPTABLE FOR INDUSTRIAL,
TECHNOLOGICAL, COMMERCIAL AND
USER PURPOSES, DRAFT INTERNATIONAL
STANDARDS MAY ON OCCASION HAVE TO
BE CONSIDERED IN THE LIGHT OF THEIR
POTENTIAL TO BECOME STANDARDS TO
WHICH REFERENCE MAY BE MADE IN
Reference number
NATIONAL REGULATIONS.
ISO/DIS 39003:2022(E)
RECIPIENTS OF THIS DRAFT ARE INVITED
TO SUBMIT, WITH THEIR COMMENTS,
NOTIFICATION OF ANY RELEVANT PATENT
RIGHTS OF WHICH THEY ARE AWARE AND TO
PROVIDE SUPPORTING DOCUMENTATION. © ISO 2022
---------------------- Page: 1 ----------------------
ISO/DIS 39003:2022(E)
DRAFT INTERNATIONAL STANDARD
ISO/DIS 39003
ISO/TC 241 Secretariat: SIS
Voting begins on: Voting terminates on:
Road Traffic Safety (RTS) — Guidance on ethical
considerations relating to safety for autonomous vehicles
ICS: 03.220.20
COPYRIGHT PROTECTED DOCUMENT
THIS DOCUMENT IS A DRAFT CIRCULATED
FOR COMMENT AND APPROVAL. IT IS
© ISO 2022
THEREFORE SUBJECT TO CHANGE AND MAY
This document is circulated as received from the committee secretariat.
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
NOT BE REFERRED TO AS AN INTERNATIONAL
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on STANDARD UNTIL PUBLISHED AS SUCH.
the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below
IN ADDITION TO THEIR EVALUATION AS
or ISO’s member body in the country of the requester. BEING ACCEPTABLE FOR INDUSTRIAL,
TECHNOLOGICAL, COMMERCIAL AND
ISO copyright office
USER PURPOSES, DRAFT INTERNATIONAL
CP 401 • Ch. de Blandonnet 8
STANDARDS MAY ON OCCASION HAVE TO
BE CONSIDERED IN THE LIGHT OF THEIR
CH-1214 Vernier, Geneva
POTENTIAL TO BECOME STANDARDS TO
Phone: +41 22 749 01 11
WHICH REFERENCE MAY BE MADE IN
Reference number
Email: copyright@iso.org
NATIONAL REGULATIONS.
Website: www.iso.org ISO/DIS 39003:2022(E)
RECIPIENTS OF THIS DRAFT ARE INVITED
Published in Switzerland
TO SUBMIT, WITH THEIR COMMENTS,
NOTIFICATION OF ANY RELEVANT PATENT
RIGHTS OF WHICH THEY ARE AWARE AND TO
ii
© ISO 2022 – All rights reserved
PROVIDE SUPPORTING DOCUMENTATION. © ISO 2022
---------------------- Page: 2 ----------------------
ISO/DIS 39003:2022(E)
Contents Page
Foreword .v
Introduction . vi
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 External factors affecting autonomous vehicle safety . 2
4.1 General . 2
4.2 The road environment . 2
5 Stakeholders in AV design and operations . 3
5.1 Producers – Manufacturers; designers and their suppliers . 3
5.2 Distribution chain – Distributors; Sellers . 3
5.3 Purchasers, owners, and operators . 4
5.4 Government agencies and interested parties . 4
6 Governance, assessment, and evaluation . 4
6.1 Ethical reference for assessment . 4
6.2 Additional Standards . 4
6.3 General . 5
6.3.1 Higher organizational level . 5
6.3.2 Development Organizational level. 5
6.3.3 Specific Development and implementation processes. 6
6.3.4 Post Implementation Checking Against Ethical Criteria . 7
6.4 Conducting the assessment . 7
6.5 Expression of results and conclusion . 8
7 Operationalization of Ethics - Discussion on values and ethics to consider .8
7.1 Ethical Framework for the design of AVs (driving rules and ethical design) . 8
7.1.1 Purpose . 9
7.1.2 Values . 10
7.1.3 Principles . 10
7.1.4 Methods for Construction and Evaluation of Maxims .13
7.2 The Background of Maxims .13
7.2.1 Maxim Design and Construction . 13
7.2.2 Evaluations of Maxims . 14
7.3 Driving Rules . 15
7.3.1 Need vs. desire Driving Rule . 16
7.3.2 Once on the road space . 16
7.3.3 In the lane behaviour (includes braking). 17
7.3.4 Lane switching . 17
7.3.5 In the presence of the other . 18
7.3.6 Road/Infrastructure Use Cases . 19
7.3.7 Resolving Conflict .20
7.3.8 Negotiations . 20
7.3.9 AV unable to function as intended . 21
7.3.10 Yielding to first responders and emergency response vehicles . 21
7.3.11 Protecting other road users . 22
7.3.12 Unavoidable collision with other road users . 23
7.3.13 Other Issues . 24
8 A Framework for rule construction and dealing with violations and deviations .24
8.1 Framework . 24
8.2 Goals . 25
8.3 Primary rules . 25
8.4 Supporting Rules . 26
iii
© ISO 2022 – All rights reserved
---------------------- Page: 3 ----------------------
ISO/DIS 39003:2022(E)
8.5 Precautionary and Disabling Rules (prevention) . 26
8.6 Reinforcing and Enabling Rules (performance) . 26
8.7 Counter rules . 26
8.8 Exceptions (prevention and performance) . 26
8.9 Discretionary and Compensatory rules . 27
8.10 Misconduct . 27
8.11 Violation . 27
8.12 Deviations . 27
8.13 Breakdowns .28
8.14 RULE STRATEGY .28
8.15 Boundaries (for prevention of unwanted behavior) .28
8.15.1 Margins .28
8.15.2 Barriers .28
8.15.3 Buffers .28
8.16 Promoters (for performance) .29
8.17 Further considerations . 29
9 External/Internal Design .29
10 Sustainability .30
11 Review and Re-evaluation following controls system updates .31
Annex A .32
Annex B Sustainability Issues .34
Annex C .35
Annex D Action plan – an example .41
Bibliography .43
iv
© ISO 2022 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/DIS 39003:2022(E)
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see the following
URL: www.iso.org/iso/foreword.html.
This document was prepared by Technical Committee ISO/TC 241, Road traffic safety management
systems.
A list of all parts in the ISO 39000 series can be found on the ISO website.
v
© ISO 2022 – All rights reserved
---------------------- Page: 5 ----------------------
ISO/DIS 39003:2022(E)
Introduction
0.1 General
A long established and commonly held view is that the single most significant factor in road traffic
safety are the actions of the driver. It must however be emphasized that the road transport system is
a complex socio-technical system which places high demands on humans to negotiate. Crashes occur
since human beings due to finite cognitive capacity and physiological limitations cannot always cope
with these demands. To increase road safety the road transport system therefore has to be designed
to support the road user to cope with this complexity and to mitigate the effects of crashes when this
is not possible. When technology exceeds human capability autonomous vehicles have the potential to
replace the human driver and to further increase road safety
Autonomous vehicles have the potential to replace the human driver and to increase road safety by
reducing the opportunity of poor decision making and improper conduct in daily operations. This will
probably take a long time and in the meantime the capabilities of technology and humans should be
combined and integrated in such a way that the strengths of both are utilized efficiently. Important
safety improvements were made over the last century, but AD technology provides new opportunities
to improve safety even further.
Autonomous vehicles (AV) are unlikely to gain widespread acceptance until the travelling public feels
assured of their safety and security, not only of passengers but also other vehicles and vulnerable
road users. This includes a behaviour of the AV according to the desires and requirements of society.
However, despite contemporary expectations and a very optimistic view on technology, humans have
an extremely valuable faculty that machines will probably never possess -- ethical decision making and
judgement. For real traffic situations the human does not always have the prerequisites to take a rational
decision based on ethics since the time frame is often too narrow. For that reason, many “decisions”
are made instinctively without the possibility of making a well-reasoned and balanced ethical decision.
Defining this element and imparting it on machines is critical for the success of autonomous vehicles.
This can only be achieved by ensuring that autonomous vehicles are equipped with driving rules that
align with the general ethical beliefs, needs, and desires of humanity on a global level, subject to local
specific nuances.
To achieve the imprint of global and local ethical considerations in AV design, there is a need for a
framework of ethics involving the necessary stakeholders of different areas. To that objective’s end,
it is important to develop standards for AV behaviour. The objective is to ensure that these vehicles
are designed to minimize risk and behave in all circumstances in a way that is aligned with universal
expectations of a fair society. While there are few standards available, or under development, that
address the engineering and technological aspects of autonomous vehicles, there are no international
standards that address aspects concerning the general topics of driving policy and ethical behaviour,
which are also important. By driving policy we mean a general approach of how an AV makes a decision
and performs manoeuvres. Ethical relevant behaviour represents positive or potentially negative
impact on road users and especially the vulnerable ones as well as the public space at large.
The objective of this work is to lay out a framework for the development of a standard for ethical and
societally accepted driving policy (e.g., those relating to fairness and equity with respect to other road
users, the public space, and the environment).
0.2 The Concept of Autonomous Vehicles
Autonomous vehicles have the objective to substitute driver, including tasks, decisions, and
responsibilities. Hence the driver behaviour model proposed by Michon (1985) and applied to
autonomous vehicles by Ulbrich and colleagues (2013) (Figure 1) can be a start to design and operate
autonomous vehicles. The driving task consists of three levels: the strategic level concerned with the
higher-level trip goals (e.g., route choice), the tactical level concerned with the maneuvering decisions,
including negotiations and interactions with other vehicles, and operational level concerned with the
execution of these tactical and operational behaviours at the level of vehicle control. There is no strictly
hierarchical relationship among these levels.
vi
© ISO 2022 – All rights reserved
---------------------- Page: 6 ----------------------
ISO/DIS 39003:2022(E)
Figure 1 — Hierarchical model of driving task (source Ulbrich et al., 2013)
Beside this consideration which has a timely sequence, the autonomous vehicle must be designed
thoroughly to ensure safe and secure operation, the decision base for the desired behavior need to be
established. Therefore, all necessary information needs to be available, situation needs to be captured
and understood while different perspectives will support the final decision.
Hence a functional decomposition into six layers (Amersbach, Winner, 2017) (Figure 2) can provide the
first step into solution space.
vii
© ISO 2022 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/DIS 39003:2022(E)
Figure 2 — Decomposition Layers (source Amersbach, Winner, 2017)
— Information access
Information needs to be available and accessible
— Information reception
All necessary information can be captured
— Information processing
Captured information need to contain all required classification and identification for further
process
— Situational understanding
Based on information, situation need to be captured and understood
— Behavioral decision
Based on designed or trained situational awareness, the desired behavior needs to be chosen
— Action
Vehicle transforms whished behavior into action
Within this framework, technical realization can differ while the focus of the situation decision will
be comprehensible, accountable, and comparable among different designs. A thoroughly development
will benefit transparency for action and taken decision. The work described in this standard focuses
primarily on Layer 4 – behavioral decision
0.3 The Concept of Ethical considerations for autonomous vehicles
viii
© ISO 2022 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/DIS 39003:2022(E)
Philosophy helps people question, understand and make sense of the world so they can act properly in
it. This means that decisions and actions are not only intrinsically fair, but that they are also performed
in a way that is balanced with respect with other’s needs, the societal needs and humanistic values,
as well as with respect to the physical world around us. It is widely accepted that philosophy can be
divided into theoretical philosophy and practical philosophy. Theoretical philosophy is concerned with
understanding and making sense of the world and includes topics such as ontology, epistemology, and
meta-ethics amongst others. Practical philosophy focuses on topics such as human agency, practices,
values, as well as ethical dimensions of (human) behaviour and conduct.
Within practical philosophy there are two main branches: Normative and applied ethics. Normative
ethics is the study of ethical action and determining standards for decision making and conduct (e.g.
deontology, virtue ethics, etc. - see Annex B). Applied ethics is the application of standards of ethics to
real life situations (e.g. biomedical ethics, AI ethics, political ethics, etc. - see Annex B). In this document,
ethics refers to both normative and applied ethics but not to meta-ethics (which is more concerned
with the nature and basis of ethical concepts and the underlying reasoning and assumptions behind
moral theories).
Applied ethics is a flexible and practical way to address ethical considerations in the development of
new technologies because an applied ethics field, such as AV ethics, can borrow from more than one
normative school of ethics. Therefore, the framework offered in this document, although based on “a
principle-based approach, can use ideas from deontology and virtue ethics, for example, to help solve
problems. This is a balanced approach; it does not condone one type of normative ethics over another,
it offers a range of perspectives that will help the designer/developer in choosing the best (or better)
decision possible for specific situations.
In summary, ethics is the study of how to choose to act in situations. To make decisions, we need clarity
on what matters and to understand what is considered good, bad, right, and wrong. Therefore, ethics
may be viewed as a tool that helps us create the difference between a “good” decision and a “bad” one.
This is of great importance in the development of autonomous vehicles because choices made during
the design and development of AV systems determine its “driving behaviour” and how it caters for its
passengers and interacts with other road users (e.g., vulnerable road users). “Driving behaviour” is
what was designed for and programmed into the machine, “conduct” is what actually transpires as a
consequence of applying the driving behaviour to the real world.
This document offers a particular framework for AV ethics which is intended to support the practical
integration of ethics into the AV development process. The framework we suggest builds upon
Socrates conception of the hierarchical nature of philosophy for practical use which has three levels
(See Figure 3). The base level is the conduct that transpires as a consequence of decision and actions
(behaviour), the second level is how these behaviours are governed. Namely what kind of policy and
arrangements are in place to make decisions. The top level is Purpose, which Behaviour and Conduct
must align with, in order to achieve the goals and objectives of the entire endeavour. Finally, knowledge
is the “tool” for making sure that the purpose is reasonable and balanced and that the governance and
behaviour/conduct are indeed feasible and appropriate.
Figure 3 — Socrates’ hierarchical nature of philosophy
ix
© ISO 2022 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/DIS 39003:2022(E)
Figure 4 is a graphical depiction of the framework advocated here for AV ethics. It shares similarities
with other ethical approaches for emerging technologies, it is also unique as it has been created
specifically for the AV context. Essentially, the AV ethics framework provides guidance for reflective
and critical decision making and is composed of four main elements: purpose, values, principles,
and process. The framework approach provides structure and guidance, yet it is flexible enough to
a
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.