Road vehicles -- Ergonomic aspects of transport information and control systems -- Calibration tasks for methods which assess driver demand due to the use of in-vehicle systems

This document provides procedures that can be used as a secondary task in a dual task setting to determine whether that evaluation setting is standardized and valid for purposes of assessing driver attentional demand due to the use of an in-vehicle system. This document does not define calibration procedures for other evaluation activities that a laboratory might undertake. This document provides guidance on selecting a calibration task given a specific primary task. The primary tasks of interest include those that would be used in the evaluation of attentional demand. Such primary tasks are defined in other documents. The description of a calibration task includes its application, experimental setup, data collection, and procedures for analysis of results. The purpose of this document is not to define a reference criterion as to whether a given secondary task is suitable for use while driving. Although specific settings of parameters of a calibration task might be used to realize such a predefined pass/fail criterion, this document does not provide such a criterion for a given level of attentional demand.

Véhicules routiers -- Aspects ergonomiques des systèmes d'information et de contrôle du transport -- Tâches de calibration pour méthodes qui évaluent la distraction du conducteur due à l'utilisation des systèmes embarqués

General Information

Status
Published
Publication Date
31-Mar-2019
Current Stage
6060 - International Standard published
Start Date
01-Mar-2019
Completion Date
01-Apr-2019
Ref Project

RELATIONS

Buy Standard

Technical specification
ISO/TS 14198:2019 - Road vehicles -- Ergonomic aspects of transport information and control systems -- Calibration tasks for methods which assess driver demand due to the use of in-vehicle systems
English language
24 pages
sale 15% off
Preview
sale 15% off
Preview

Standards Content (sample)

TECHNICAL ISO/TS
SPECIFICATION 14198
Second edition
2019-04
Road vehicles — Ergonomic aspects
of transport information and control
systems — Calibration tasks for
methods which assess driver demand
due to the use of in-vehicle systems
Véhicules routiers — Aspects ergonomiques des systèmes
d'information et de contrôle du transport — Tâches de calibration
pour méthodes qui évaluent la distraction du conducteur due à
l'utilisation des systèmes embarqués
Reference number
ISO/TS 14198:2019(E)
ISO 2019
---------------------- Page: 1 ----------------------
ISO/TS 14198:2019(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO 2019

All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting

on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address

below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO 2019 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/TS 14198:2019(E)
Contents Page

Foreword ........................................................................................................................................................................................................................................iv

Introduction ..................................................................................................................................................................................................................................v

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ...................................................................................................................................................................................... 1

3 Terms and definitions ..................................................................................................................................................................................... 1

4 Abbreviated terms .............................................................................................................................................................................................. 3

5 Calibration tasks ................................................................................................................................................................................................... 4

5.1 Principle and overview .................................................................................................................................................................... 4

5.2 Types of calibration tasks .............................................................................................................................................................. 4

5.3 Critical Tracking Task (CTT)........................................................................................................................................................ 4

5.3.1 Description .................. .................................................... ...................................................................................................... 4

5.3.2 Operation of the CTT .................................................................................................................................................... 5

5.3.3 Setup for CTT ...................................................................................................................................................................... 6

5.3.4 Test conditions for CTT .............................................................................................................................................. 6

5.3.5 Participants for CTT ...................................................................................................................................................... 6

5.3.6 Participant instruction for CTT ........................................................................................................................... 6

5.3.7 Practice trials ...................................................................................................................................................................... 7

5.3.8 Test metrics .......................................................................................................................................................................... 7

5.4 Surrogate Reference Task (SURT) .......................................................................................................................................... 7

5.4.1 Description .................. .................................................... ...................................................................................................... 7

5.4.2 Setup for SURT ................................................................................................................................................................... 8

5.4.3 Test conditions for SURT ........................................................................................................................................... 9

5.4.4 Participants for SURT ........................................................................................................................................... ........ 9

5.4.5 Participant instructions for SURT ..................................................................................................................... 9

5.4.6 Practice trials ...................................................................................................................................................................10

5.4.7 Test metrics .......................................................................................................................................................................10

5.5 N-back ..........................................................................................................................................................................................................10

5.5.1 Description .................. .................................................... ...................................................................................................10

5.5.2 Operation of the n-back ..........................................................................................................................................10

5.5.3 Setup for n-back ............................................................................................................................................................11

5.5.4 Test conditions for n-back ....................................................................................................................................11

5.5.5 Participants for n-back ............................................................................................................................................11

5.5.6 Participant instruction for n-back .................................................................................................................11

5.5.7 Practice trials ...................................................................................................................................................................12

5.5.8 Test metrics .......................................................................................................................................................................12

6 Calibration criterion ......................................................................................................................................................................................12

6.1 Calibration criterion procedure ............................................................................................................................................12

6.2 General calibration considerations ....................................................................................................................................14

Annex A (informative) Calibration task setup details using the LCT ...............................................................................15

Annex B (informative) Multi-lab reference data for LCT ..............................................................................................................17

Annex C (informative) Calibration task setup details using the DRT ..............................................................................19

Annex D (informative) Multi-lab reference data for DRT .............................................................................................................20

Bibliography .............................................................................................................................................................................................................................22

© ISO 2019 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/TS 14198:2019(E)
Foreword

ISO (the International Organization for Standardization) is a worldwide federation of national standards

bodies (ISO member bodies). The work of preparing International Standards is normally carried out

through ISO technical committees. Each member body interested in a subject for which a technical

committee has been established has the right to be represented on that committee. International

organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.

ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of

electrotechnical standardization.

The procedures used to develop this document and those intended for its further maintenance are

described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the

different types of ISO documents should be noted. This document was drafted in accordance with the

editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/directives).

Attention is drawn to the possibility that some of the elements of this document may be the subject of

patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of

any patent rights identified during the development of the document will be in the Introduction and/or

on the ISO list of patent declarations received (see www .iso .org/patents).

Any trade name used in this document is information given for the convenience of users and does not

constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and

expressions related to conformity assessment, as well as information about ISO's adherence to the

World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso

.org/iso/foreword .html.

This document was prepared by Technical Committee ISO/TC 22, Road vehicles, Subcommittee SC 39,

Ergonomics.

This second edition cancels and replaces the first edition (ISO/TS 14198:2012), which has been

technically revised. The main changes compared to the previous edition are as follows:

— in addition to the Lane Change Test (LCT), the Detection Response Task (DRT) is added as a

primary task;

— in addition to the Critical Tracking Task (CTT) and Surrogate Reference Task (SURT), the n-back

task is added as calibration task.

Any feedback or questions on this document should be directed to the user’s national standards body. A

complete listing of these bodies can be found at www .iso .org/members .html.
iv © ISO 2019 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/TS 14198:2019(E)
Introduction

The number of standardized methods to assess driver attentional demand due to the use of in-vehicle

information and communication devices is continuing to increase. In applying these methodologies, it is

important to understand and document variability in participants’ performance of standard calibration

tasks and procedures across laboratories and/or time.
A suitable calibration task should have the following attributes:

— It should be robust against the variations in cultural background of participants.

— Properly applied, the task should give repeatable quantitative results. It should be sensitive to

inappropriate variations in participants, equipment, location, experimenter and instruction.

— It should use durable and readily available equipment for conducting the task.

— It should apply to the driver population and be usable in a driving-like context.

A standardized calibration task can be used to produce a range of statistically stable, repeatable and

comparable secondary task demands for a participant in an experimental setting. This setting can be

used to assess the effect on driving performance of the attentional demand due to driver interaction

with an information, entertainment, and control or communication system while a vehicle is in motion.

Different calibration tasks are specified in this document to cover calibration manual, visual, and

cognitive aspects of various secondary task characteristics.
© ISO 2019 – All rights reserved v
---------------------- Page: 5 ----------------------
TECHNICAL SPECIFICATION ISO/TS 14198:2019(E)
Road vehicles — Ergonomic aspects of transport
information and control systems — Calibration tasks for
methods which assess driver demand due to the use of in-
vehicle systems
1 Scope

This document provides procedures that can be used as a secondary task in a dual task setting to

determine whether that evaluation setting is standardized and valid for purposes of assessing driver

attentional demand due to the use of an in-vehicle system. This document does not define calibration

procedures for other evaluation activities that a laboratory might undertake.

This document provides guidance on selecting a calibration task given a specific primary task. The

primary tasks of interest include those that would be used in the evaluation of attentional demand.

Such primary tasks are defined in other documents.

The description of a calibration task includes its application, experimental setup, data collection, and

procedures for analysis of results.

The purpose of this document is not to define a reference criterion as to whether a given secondary task

is suitable for use while driving. Although specific settings of parameters of a calibration task might be

used to realize such a predefined pass/fail criterion, this document does not provide such a criterion for

a given level of attentional demand.
2 Normative references

The following documents are referred to in the text in such a way that some or all of their content

constitutes requirements of this document. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any amendments) applies.

ISO 26022:2010, Road vehicles — Ergonomic aspects of transport information and control systems —

Simulated lane change test to assess in-vehicle secondary task demand

ISO 17488, Road vehicles — Transport information and control systems — detection-response-task (DRT)

for assessing attentional effects of cognitive load in driving

ISO/IEC 7498-1, Information technology — Open Systems Interconnection — Basic Reference Model: The

Basic Model
3 Terms and definitions

For the purposes of this document, the terms and definitions given in ISO/IEC 7498-1 and the

following apply.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

— ISO Online browsing platform: available at https: //www .iso .org/obp
— IEC Electropedia: available at http: //www .electropedia .org/
© ISO 2019 – All rights reserved 1
---------------------- Page: 6 ----------------------
ISO/TS 14198:2019(E)
3.1
calibration task

type of reference task used for the purpose of comparing different tests or test results between sites, or

over time at a given site
3.2
criterion
threshold or value of a variable to be met
3.3
demand

total visual, auditory, cognitive, or physical resources required of the driver to accomplish the primary

task and interact with a Transport Information and Control System (TICS) in a dual task setting

3.4
dual task

two tasks concurrently performed, typically the primary task plus the secondary task

3.5
environment

physical surroundings in which data are captured and collected, consequently, the level of control over

the independent variables in a study
EXAMPLE Laboratory, simulator, test track, real road.
3.6
evaluation

procedure in which the effect of a Transport Information and Control System (TICS) or another device

is assessed

Note 1 to entry: It may be undertaken retrospectively after the TICS has been in use for a considerable time as a

product.

Note 2 to entry: The results of the evaluation will depend on the HMI, but also on the equipment reliability and

subsequent behavioural changes which may affect driving performance.
3.7
method

high-level approach to an assessment, based on theory and principles, which implies an underlying

rationale in the choice of assessment techniques

EXAMPLE Behaviour analysis, workload assessment, and analysis of psycho-physiological responses.

3.8
metric

quantitative measure of driver behaviour independent of the tool used to measure it

EXAMPLE Eye glance duration and vehicle speed.
3.9
performance

result of skill application demonstrated by a participant in performing a driving task or Transport

Information and Control System (TICS) related task
3.10
primary task
task to be calibrated

Note 1 to entry: The Lane Change Test in ISO 26022 is one example a task that simulates or approximates a

primary driving task.

Note 2 to entry: This document considers the Detection Response Task (ISO 17488) is the primary task to be

calibrated even if performed with or without driving.
2 © ISO 2019 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/TS 14198:2019(E)
3.11
primary driving task

activities that the driver must undertake while driving including navigating, path following,

manoeuvring, avoiding obstacles, and controlling speed; and which a participant may perform through

the duration of a test (simulated substitute for driving)
3.12
secondary task
non-driving related additional task
Note 1 to entry: A calibration task for the purpose of this document.
3.13
secondary task demand

sum of visual, auditory, cognitive, motor, and speech resource demands required by a non-driving

related task
3.14
system paced secondary task

activity in which the change from the current to the next state in the interaction between user and

system is initiated by the system
Note 1 to entry: The pace can be fixed or variable.
3.15
target bar
moving line on the critical tracking task display which indicates the task error
3.16
task
process of achieving a specific and measureable goal using a prescribed method
3.17
user paced secondary task

activity in which the change from the current to the next state in the interaction between user and

system is initiated by the user
4 Abbreviated terms
CI Confidence Interval
CTT Critical Tracking Task
DRT Detection Response Task
LCT Lane Change Test
MDEV Mean Deviation (According to ISO 26022)
SURT Surrogate Reference Task
TICS Transport Information and Control System
A list of TICS fundamental services has been defined by ISO/TC 204/WG 1.
© ISO 2019 – All rights reserved 3
---------------------- Page: 8 ----------------------
ISO/TS 14198:2019(E)
5 Calibration tasks
5.1 Principle and overview

For calibration purposes, a standardized calibration task shall be used as a secondary task in a dual

task setting in combination with a method to assess attentional demand due to the use of an in-vehicle

system. The dual task setting shall include a primary task and the secondary calibration task.

Examples for driving-like dual task settings may include the operation of a TICS secondary task in the

Lane Change Test (LCT), in a driving simulator environment, or the Detection Response Task (DRT).

The calibration shall be performed in a setting that is intended for the assessment of secondary tasks

and follow the training and experimental procedures of that method for assessing attentional demand.

Development of the calibration tasks and associated procedures have used the ISO 26022 LCT, and

ISO 17488 DRT to represent the primary task. While the calibration tasks described herein are intended

to be applicable to other primary task implementations and dual task settings, care shall be taken

to ensure that the conditions are sufficiently similar to those of ISO 26022 or ISO 17488 considering

equipment and instructions to ensure a valid application of this document and its procedures.

5.2 Types of calibration tasks

There are various possibilities to realize a calibration task. In the following subclauses, three example

alternatives are specified in detail. These alternatives include a system-paced secondary task (critical

tracking task, CTT), a user-paced secondary task (surrogate reference task, SURT), and an auditory-

vocal delayed digit recall task (n-back). Both the CTT and SURT alternatives represent visual-manual

tasks that can be used in a dual task setting and are recommended as calibration tasks while using the

LCT as the primary task. Whereas the n-back is a cognitive task that can be used in a dual task setting

and is recommended as a calibration task while using the DRT as the primary task.

5.3 Critical Tracking Task (CTT)
5.3.1 Description

The CTT is a visual-manual task, which requires continuous control activity by the participant.

The participant controls the position of a vertically moving target bar with respect to a fiducial line

(centreline) within a target area by manipulating up and down arrow keys. The arrow key control gives

discrete commands to the target bar which moves it up and down. The up key moves the target bar up,

and the down key moves it down.

The dynamics of the motion of the target bar are a first order instability. If the participant does nothing,

the target bar moves (divergently) towards the edge of the display. The participant then has to make

suitable corrective arrow key inputs to bring the target bar back towards the centre (the red dashed

line in Figure 1).
A control system block diagram of the CTT is shown in Figure 2.
4 © ISO 2019 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/TS 14198:2019(E)
Key
1 target bar
2 centreline
Figure 1 — Typical screen of the CTT with target bar above the centreline
Key
λ Level of instability or rate of divergence (adjustable).
s Laplace transform variable.
Figure 2 — Control system block diagram of the CTT
5.3.2 Operation of the CTT

When the program is first started, the screen will look like Figure 1. Nothing will happen initially.

The centreline is displayed as a red dashed horizontal line in the centre of the display. The target bar

is displayed as a black line. The target bar will start to move away from the centreline showing an

increasing error. Two short blue reference lines are shown above and below the centre to subdivide the

screen for better orientation. The instruction to the participant is to control the position of the target

bar with the arrow keys (arrow up key and arrow down key) and keep it as close to the centreline as

possible so as to minimize the errors.
© ISO 2019 – All rights reserved 5
---------------------- Page: 10 ----------------------
ISO/TS 14198:2019(E)
5.3.3 Setup for CTT

The setup consists of a screen with SVGA resolution plus keyboard should be as follows:

— The subtended vision angle (width) of the display area relative to participant's eyes shall be 13° ± 1°

horizontally.

— 19 inch (483 mm) screen for a width to height ratio of display area of 4:3. For a display with a width

to height ratio of 16:9 (and equivalent to a 19 inch screen), one should use a setup of with a 23 inch

(584 mm) screen.

— The centre of the secondary display shall be positioned 28° ± 2° horizontally (right or left depending

on the intended display position in the vehicle, LHD or RHD) and 20° ± 2° vertically from participant's

straight ahead line of sight. For further details see Annex A.

— To control the target bar movement, the arrow key of a standard PC keyboard (or an equivalent

arrangement of keys, for example Figure 3) shall be used. Participants are allowed to place the

keyboard in a comfortable position on the same side of the steering wheel as the CTT display on a

table or console aside but not connected to their body.

Figure 3 — Example of key-pad to move the highlight and confirm target location result

5.3.4 Test conditions for CTT

The test in the dual task setting shall include one specified level of λ, 0,5, the CTT easy condition. The λ

level shall be set prior to each run and kept constant during the run. In each run the primary task and

the CTT task shall be performed for at least 2 min.
5.3.5 Participants for CTT

The CTT as a calibration task shall be performed in a dual task setting by at least 16 [n = 16] participants

who are licensed drivers. Participants should be familiar with the primary task as well as operation

of the CTT. The level of participant familiarity shall be documented in the protocol. In the case of

calibration using an LCT setting, it is recommended to select the participants according to ISO 26022,

and following the sample description regarding age, gender and familiarity with primary and secondary

[2]

task in Bengler, K., Mattes, S., Hamm, O., Hensel, M., 2010 . Typically, this was an average age of 32 to

45, gender balanced, and instructed and practiced in both the primary and secondary tasks.

5.3.6 Participant instruction for CTT

The following verbal/written instructions for CTT shall be given to the participant:

“A horizontal black target bar is displayed on the secondary task screen. There is also a red dashed

horizontal line in the centre of the screen. When the task is started, the black target bar will tend to

6 © ISO 2019 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/TS 14198:2019(E)

move away from the red centreline. The motion of the black target bar can be controlled by the arrow-

up- and arrow-down keys in order to bring the target bar back to the centre. These keys give discrete

commands to the target. (The down arrow key moves the target bar down; the up arrow key moves the

target bar up.) The goal is to control the motion of the black target bar to keep it as close as possible to

the red dashed centre line.”

The λ level is set to a fixed value at the beginning of a run and defines the secondary task difficulty.

If the participant does nothing, the target bar moves towards the edge of the display. The further the

black target bar is away from the centre line, the faster it will move, so a larger (longer duration) input

is needed, to bring it back to the centreline.

The difficulty of controlling the black target bar can vary with the size of the error. The participant will

need to work quickly and correctly in order to keep the target bar near the centreline.

5.3.7 Practice trials

Participants shall practice the CTT secondary task alone at λ levels of 0,5 and 1,0. Before proceeding to

the dual task conditions, participants shall be able to perform the CTT alone at a level of λ = 1,0 for one

minute successfully (i.e. without the target bar hitting the upper or lower limit of the display). Only the

λ value of 0,5 is used in the dual task trials.

Primary task and dual task practice trials shall be conducted as prescribed in the corresponding

standard or other procedures.
5.3.8 Test metrics

The primary task measures shall be those prescribed for the primary task. If the Lane Change Test is

used, these are in ISO 26022.
Participants’ performance in the CTT is measured using:

— CTT root mean square deviation of the target bar from the centreline in pixels.

— CTT percentage of time with target bar at the upper or lower limit.
5.4 Surrogate Reference Task (SURT)
5.4.1 Description

The Surrogate Reference Task (SURT) is both visually and manually demanding. The level of demand

can be varied.

The SURT is a visual search paradigm. In a typical visual search paradigm, participants are asked to

report whether or not a pre-specified target is embedded in a multi-item display, usually comprised of

an array of alphanumeric symbols, forms, colours or words. The non-target items are called distractors.

To vary the time the participant will need to identify the target and to answer (reaction time), the

similarity between target and distractors can be manipulated. The more similar the distractors are

to the target the more time is typically needed to identify the target. Unless these distractors have

no unique feature distinguishing them from the target (feature with a visual “pop out” character),

the number of distractors can also increase the time participants will need for search (Treisman and

[36]
Gelade, 1980 ). An example for the task is shown in Figure 4.

For the SURT, participants shall locate a pre-specified target circle which is either

...

Questions, Comments and Discussion

Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.