Road vehicles — Human performance and state in the context of automated driving — Part 2: Considerations in designing experiments to investigate transition processes

This document focuses on system-initiated and human-initiated transitions (Clause 6) from a higher level to a lower level of automated driving. Human factors and system factors that can influence takeover performance are included (Clauses 7 and 8). Although some are still under investigation, there is a need to appropriately set these factors as variables to better understand their effects or to better control/eliminate their influence. This approach will aid research design by ensuring that important factors are considered and support consistency across studies enabling meaningful comparisons of findings. This document also includes information on considerations in test scenario design (Clause 9), common measures for human takeover performance (Clause 10) and considerations in choosing a testing environment (Clause 11) to help readers design experiments comparable to other studies.

Véhicules routiers — Etat et performance humaine dans le contexte de la conduite automatisée — Partie 2: Principes expérimentaux pour etudier les processus de transition

General Information

Status
Published
Publication Date
27-Feb-2020
Current Stage
6060 - International Standard published
Start Date
28-Feb-2020
Completion Date
28-Feb-2020
Ref Project

Buy Standard

Technical report
ISO/TR 21959-2:2020 - Road vehicles -- Human performance and state in the context of automated driving
English language
44 pages
sale 15% off
Preview
sale 15% off
Preview

Standards Content (sample)

TECHNICAL ISO/TR
REPORT 21959-2
First edition
2020-02
Road vehicles — Human performance
and state in the context of automated
driving —
Part 2:
Considerations in designing
experiments to investigate transition
processes
Véhicules routiers — Etat et performance humaine dans le contexte
de la conduite automatisée —
Partie 2: Principes expérimentaux pour etudier les processus de
transition
Reference number
ISO/TR 21959-2:2020(E)
ISO 2020
---------------------- Page: 1 ----------------------
ISO/TR 21959-2:2020(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO 2020

All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting

on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address

below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO 2020 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/TR 21959-2:2020(E)
Contents Page

Foreword ..........................................................................................................................................................................................................................................v

Introduction ................................................................................................................................................................................................................................vi

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ...................................................................................................................................................................................... 1

3 Terms and definitions ..................................................................................................................................................................................... 1

4 List of Acronyms .................................................................................................................................................................................................... 1

5 Purpose .......................................................................................................................................................................................................................... 2

6 Transition process models ......................................................................................................................................................................... 2

6.1 General ........................................................................................................................................................................................................... 2

6.2 Transition process model for system-initiated transitions .............................................................................. 2

6.3 Transition process model for human-initiated transitions .............................................................................. 3

7 Human factors that influence takeover performance ................................................................................................... 4

7.1 General ........................................................................................................................................................................................................... 4

7.2 Driver attributes .................................................................................................................................................................................... 4

7.2.1 Knowledge ............................................................................................................................................................................. 4

7.2.2 Experience and trust .................................................................................................................................................... 5

7.2.3 Demographic attributes ............................................................................................................................................. 5

7.3 Driver readiness/availability ...................................................................................................................................................... 5

7.3.1 Sitting position and posture ................................................................................................................................... 6

7.3.2 Engagement in non-driving related activities ........................................................................................ 6

7.3.3 Drowsiness ............................................................................................................................................................................ 7

7.3.4 Mind wandering ............................................................................................................................................................... 8

7.3.5 Situation awareness ...................................................................................................................................................... 8

7.3.6 Operating state/mode awareness ..................................................................................................................... 9

7.3.7 Attentiveness ....................................................................................................................................................................... 9

7.3.8 Receptivity ............................................................................................................................................................................. 9

8 System factors that influence takeover performance .................................................................................................... 9

8.1 General ........................................................................................................................................................................................................... 9

8.2 System behaviour ..............................................................................................................................................................................10

8.2.1 Type of transition .........................................................................................................................................................10

8.2.2 System behaviour within takeover mode ................................................................................................10

8.2.3 System-initiated risk mitigation strategy ................................................................................................10

8.2.4 System limitations and failures ........................................................................................................................11

8.2.5 Stability and reliability of the system functions ................................................................................11

8.2.6 Level of automated driving to which the system shifts in transition .................. .............12

8.3 Human machine interfaces for RtI ......................................................................................................................................12

8.3.1 Design parameters for HMI .................................................................................................................................12

8.3.2 Total time budget..........................................................................................................................................................13

8.3.3 Other human machine interfaces to improve drivers’ takeover performance ........13

9 Test scenarios .......................................................................................................................................................................................................13

9.1 General ........................................................................................................................................................................................................13

9.2 Parameters for specifying test scenarios ......................................................................................................................14

9.3 Considerations for selecting/designing adequate test scenarios ............................................................16

9.3.1 Investigating driver state transitions during automated driving .......................................16

9.3.2 Investigating takeover performance in non-critical transitions ..........................................16

9.3.3 Assessing takeover performance at system limits ...........................................................................16

10 Takeover performance ................................................................................................................................................................................17

10.1 Introduction ...........................................................................................................................................................................................17

10.2 Taxonomy of human performance measures ............................................................................................................18

10.2.1 Addressed phases of transition ........................................................................................................................18

© ISO 2020 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/TR 21959-2:2020(E)

10.2.2 Suitability for transition type .............................................................................................................................19

10.2.3 Scope of assessment ..................................................................................................................................................20

10.2.4 Data considerations .................. ......................................................................................................................... .........21

10.3 Overview of measures and characteristics ..................................................................................................................22

11 Testing environments ...................................................................................................................................................................................24

11.1 General ........................................................................................................................................................................................................24

11.2 Types.............................................................................................................................................................................................................24

11.2.1 Simulator studies ..........................................................................................................................................................24

11.2.2 Roadway studies ...........................................................................................................................................................26

11.3 Advantages and disadvantages ..............................................................................................................................................27

11.3.1 Realism-to-safety trade-off ..................................................................................................................................27

11.4 Considerations for test environment selections .....................................................................................................29

Annex A (informative) Human machine interfaces/interactions for automated vehicles ........................30

Bibliography .............................................................................................................................................................................................................................34

iv © ISO 2020 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/TR 21959-2:2020(E)
Foreword

ISO (the International Organization for Standardization) is a worldwide federation of national standards

bodies (ISO member bodies). The work of preparing International Standards is normally carried out

through ISO technical committees. Each member body interested in a subject for which a technical

committee has been established has the right to be represented on that committee. International

organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.

ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of

electrotechnical standardization.

The procedures used to develop this document and those intended for its further maintenance are

described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the

different types of ISO documents should be noted. This document was drafted in accordance with the

editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).

Attention is drawn to the possibility that some of the elements of this document may be the subject of

patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of

any patent rights identified during the development of the document will be in the Introduction and/or

on the ISO list of patent declarations received (see www .iso .org/ patents).

Any trade name used in this document is information given for the convenience of users and does not

constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and

expressions related to conformity assessment, as well as information about ISO's adherence to the

World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso .org/

iso/ foreword .html.

This document was prepared by Technical Committee ISO/TC 22, Road vehicles, Subcommittee SC 39,

Ergonomics.
A list of all parts in the ISO 21959 series can be found on the ISO website.

Any feedback or questions on this document should be directed to the user’s national standards body. A

complete listing of these bodies can be found at www .iso .org/ members .html.
© ISO 2020 – All rights reserved v
---------------------- Page: 5 ----------------------
ISO/TR 21959-2:2020(E)
Introduction

Although automation technology is advancing at a rapid pace, the majority of automated driving levels

[1]

(as defined by SAE J3016, 2016 ) still require a human to fulfil specific remaining (driving related)

tasks. The safety-critical human’s task is the takeover task in transition from a higher level to a lower

level of automated driving. Researchers and developers continue to seek system design and human

machine interface improvements for better takeover performance. Researchers face a challenge

in understanding the limitations of a human’s ability to perform the takeover task, which involves

different human factors. Developers work to evaluate systems to see whether the takeover process is

effective at minimum risk in specific scenarios. There are a wide variety of experiments to evaluate

takeover performance in transition for many different purposes. This document contains information

to consider in the takeover scenario, some of which is still under investigation, in order to help readers

design experiments to evaluate takeover performance and design appropriate experiments.

vi © ISO 2020 – All rights reserved
---------------------- Page: 6 ----------------------
TECHNICAL REPORT ISO/TR 21959-2:2020(E)
Road vehicles — Human performance and state in the
context of automated driving —
Part 2:
Considerations in designing experiments to investigate
transition processes
1 Scope

This document focuses on system-initiated and human-initiated transitions (Clause 6) from a higher

level to a lower level of automated driving. Human factors and system factors that can influence

takeover performance are included (Clauses 7 and 8). Although some are still under investigation, there

is a need to appropriately set these factors as variables to better understand their effects or to better

control/eliminate their influence. This approach will aid research design by ensuring that important

factors are considered and support consistency across studies enabling meaningful comparisons of

findings. This document also includes information on considerations in test scenario design (Clause 9),

common measures for human takeover performance (Clause 10) and considerations in choosing a

testing environment (Clause 11) to help readers design experiments comparable to other studies.

2 Normative references
There are no normative references in this document.
3 Terms and definitions
No terms and definitions are listed in this document.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
4 List of Acronyms
DDT Dynamic Driving Task
DMS Driver Monitor System
ECG Electrocardiogram
EEG Electroencephalogram
HMI Human-Machine Interface
KSS Karolinska Sleepiness Scale
MRM Minimal Risk Manoeuvre
NDRT Non-driving Related Task
© ISO 2020 – All rights reserved 1
---------------------- Page: 7 ----------------------
ISO/TR 21959-2:2020(E)
NDRA Non-driving Related Activities
OEDR Object and Event Detection and Response
ODD Operational Design Domain
RtI Request to Intervene
SAGAT Situation Awareness Global Assessment Technique
SDLP Standard Deviation of Lateral Position
SIMS Situational Motivation Scale
SuRT Surrogate Reference Task
TLC Time to Lane Crossing
TTC Time-to-Contact/Collision
5 Purpose

The purpose of this document is to provide considerations in designing experiments to measure human

takeover performance in transition situations in order to better understand human limitations, evaluate

systems, and improve systems, including human machine interfaces. This document is expected to help

users design appropriate experiments for their purposes. This document does not provide any design

principles to restrict or direct the system design.
6 Transition process models
6.1 General

Transition processes included in this document are generally based on the models defined in

[2]

ISO/TR 21959-1 . A human’s safety-critical task is the takeover task in transition from a higher

level to a lower level of automated driving both for system-initiated and human-initiated transitions.

[2]

ISO/TR 21959-1 defines typical transition process models from automated to manual driving (i.e.

level 0). However, the models can be adapted for transitions between different levels (e.g. 4→2 or 3→1).

This clause reminds readers of the relevant transition process models.
6.2 Transition process model for system-initiated transitions

In a system-initiated transition, the system may issue a request to intervene (RtI) when it finds a

dynamic driving task (DDT) performance-relevant system failure or an object/event which cannot be

handled by the system for levels 1–4. The system also may issue an RtI when exiting the operational

design domain (ODD) for which it was designed, (e.g. exiting a motorway, exiting assumed environmental

conditions such as weather and traffic). The driver is expected to take over the DDT in response to an

RtI to continue driving. The system may terminate immediately after issuing an RtI for level 2 while

it shifts to the takeover mode following an RtI before termination for levels 3 and 4 (Figure 1). There

can also be other types of transitions after an RtI, such as transitions from level 2 to level 1 and from

level 3 to level 2. The driver’s task model can be adjusted depending on the level after the RtI (i.e. object

and event detection and response [OEDR] task for transition to level 2, OEDR task +lateral control or

OEDR task +longitudinal control for transition to level 1). When the driver does not initiate intervention

within the takeover mode, the system may shift to the minimal risk manoeuvre (MRM) to stop the

vehicle safely for level 3 and level 4 (see 8.2.2 for details).
2 © ISO 2020 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/TR 21959-2:2020(E)

Figure 1 — Transition process model for system-initiated transitions from automated to

manual driving
6.3 Transition process model for human-initiated transitions

The driver is authorized to take over the DDT at any point during operation of the automated driving

functions, except for some level 4 and level 5 features some or all of the time. The human-initiated

transition may be either optional or mandatory. The optional case is the transition where the user

wishes to drive manually without being in a safety critical situation. The mandatory case is the

transition where the level 2 system fails to avoid an undetected object/event due to the system’s

functional limitations or where the system suddenly terminates without issuing an RtI due to a DDT

performance-relevant system failure. In such mandatory transitions, the driver is expected to detect

the object/event or the failure and initiate transition (Figure 2). This type of transition is mainly from

level 2 to manual driving but can be from level 2 to level 1. The driver’s task model can be adjusted

depending on the level after the initiation.
© ISO 2020 – All rights reserved 3
---------------------- Page: 9 ----------------------
ISO/TR 21959-2:2020(E)

Figure 2 — Transition process model for mandatory human-initiated transitions from

automated (level 2) to manual driving due to driver’s detection of a safety-critical object/event

or a DDT performance-relevant system failure
7 Human factors that influence takeover performance
7.1 General

It is known that a driver’s takeover performance varies with the influences of multiple factors. In

experiments, there is a need to appropriately set factors as variables for investigating their effects or

better controlling/eliminating their influence. This will allow for the design of experiments that are

easier to compare with other studies. This clause presents information about “internal” human factors

that may influence a driver’s takeover performance. Driver’s takeover performance includes time in the

driver state transition phase (i.e. response time of significant driver intervention to RtI) and quality in

the post transition control phase (i.e. how well the driver controls the vehicle right after the significant

driver intervention; see also Clause 10).
7.2 Driver attributes
7.2.1 Knowledge

Drivers’ knowledge about system functions, limitations and the required driver’s role, has been found

[3][4][5]

to influence takeover performance in some studies . Other studies have found that instructions

[6][7]

have limited effects . In general, the sources of a driver’s knowledge are diverse and may include

mass-media, instruction manuals, instructions given at a car-dealership and other various sources.

In experiments, such knowledge can be controlled, to some extent, by screening subjects using

questionnaires investigating their level of a-priori knowledge and by providing them with controlled

information about the functions, limitations and driver’s role for the specific system of study. It is to be

noted that difficulty in forming a detailed picture of subjects’ exact knowledge obtained from various

sources for various systems may lead to some variation in the results of takeover performance. In

some instances, participants may have incorrect knowledge about system function, limitations and the

required driver’s role leading to misbehaviour or misuse.
4 © ISO 2020 – All rights reserved
---------------------- Page: 10 ----------------------
ISO/TR 21959-2:2020(E)
7.2.2 Experience and trust

A driver’s experience with using the system has been found to influence takeover performance. How

the driver has previously interacted with the system may influence performance in different ways as

a result of different levels of understanding and trust. Short term system interaction experiences may

lead to a better understanding of the system’s functions and limitations and better trust calibration,

[8][9][10][11][4][5]

resulting in better driver takeover performance . Longer-term experience with no to a

[12]

few system disengagements may lead to a driver’s over-trust of the system and complacency , which

[6]

can degrade takeover performance . In contrast, longer-term experience with too many system

[13]

disengagements may lead to driver under-trust, which might improve takeover performance but

[14]

which also may lead to disuse of the system . In experiments, such experiences can be controlled, to

some extent, by screening subjects using questionnaires investigating their experiences with specific

systems featured in the study and the frequencies of the interactions they have experienced. After

screening, new subject experience can be introduced by providing subjects specific driving conditions

after providing controlled interactions. It is to be noted that similar systems with the same level of

automated driving can differ in functions and limitations (i.e. detection targets, ODD, reliability and

others) by brand or even by different models within one brand. Also, different users of the same brand

system may use the system in different ways in different traffic environments. Therefore, experience

still may lead to some dispersion in the results.
7.2.3 Demographic attributes
7.2.3.1 Age

A driver’s age-related perceptual, cognitive and physical limitations may influence takeover

performance. Visual impairments of older drivers are diverse and are often accompanied by eye

[15]

diseases . Such impairments may degrade perception of traffic environment in the OEDR task or

in the process of transition. Visual impairments may also cause difficulty in reading system status

[16]

information displayed in the cockpit . Cognitive impairments may degrade understanding of

“complicated” system functions/limitations and the driver’s role. These impairments may also lead to

[17]

problems with divided attention and slow down task switching in transitions. Physical impairment

may degrade speed and accuracy of the response behaviour in transitions. Although, as mentioned

above, there are several hypotheses for older drivers’ degraded takeover performance, the effects of

[18][19]

age are still under discussion. Some researchers have found significant negative effects of age

[5][20][21] [22][23]

, whereas other researchers found only limited effects . In experiments, subjects can

be screened not only based on age but also based on the results of perceptual, cognitive and physical

response tests. However, it is to be noted that the effects of age have large inter/intra-individual

variability and still may lead to some dispersion in the results of takeover performance of subjects who

were screened via tests.
7.2.3.2 Other demographic attributes

There are other driver demographic attributes that may influence takeover performance, such as

experience and skill of manual driving, style of manual driving with individual and cultural differences,

technology-sensitivity and general trust of technology. However, these factors have not yet been well

studied.
7.3 Driver readiness/availability

Conceptually, readiness/availability is a driver’s dynamic state during automated driving, which

influences their takeover performance. Readiness/availability can be continuous; lower readiness/

[24][25][26][27]

availability than a required level may lead to degraded and unsafe takeover performance

[28]

(see also A.2.2). Considering the definitions of the driver’s role for each level of automation, the

required level of readiness/availability generally increases with decreasing levels of automation. The

readiness/availability is considered to include several components related to motoric/physical and

cognitive states (Table 1). Each component can have different metrics and different effects on takeover

performance. The required level for each component of readiness/availability can be experimentally

determined as the level that leads to a successful takeover by comparing the metrics and the takeover

© ISO 2020 – All rights reserved 5
---------------------- Page: 11 ----------------------
ISO/TR 21959-2:2020(E)

performance in time and quality in certain traffic conditions for a specific system transition design (see

also A.2.2).
Table 1 — Components of readiness/availability
Components of readiness/ Motoric/physical state Cognitive state
availability
Sitting position —
Posture —
Hands/arms, feet/legs, trunk
Engagement in NDRAs
Hands/arms, manual operation Visual
Cognitive
Drowsiness —
Mind wandering —
Situation awareness —
Operating state/m
...

Questions, Comments and Discussion

Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.