ISO/TS 14198:2012
(Main)Road vehicles — Ergonomic aspects of transport information and control systems — Calibration tasks for methods which assess driver demand due to the use of in-vehicle systems
Road vehicles — Ergonomic aspects of transport information and control systems — Calibration tasks for methods which assess driver demand due to the use of in-vehicle systems
ISO/TS 14198:2012 provides procedures that can be used as a secondary task in a dual task setting to determine whether that evaluation setting is standardized and valid for purposes of assessing driver attentional demand due to the use of an in-vehicle system. It does not define calibration procedures for other evaluation activities that a laboratory might undertake. ISO/TS 14198:2012 provides advice on the selection of an appropriate candidate calibration task, given an attentional demand evaluation procedure that uses primary driving-like task settings and procedures which are defined outside of ISO/TS 14198:2012.
Véhicules routiers — Aspects ergonomiques des systèmes d'information et de contrôle du transport — Tâches de calibration pour méthodes qui évaluent la distraction du conducteur due à l'utilisation des systèmes embarqués
General Information
Relations
Buy Standard
Standards Content (Sample)
TECHNICAL ISO/TS
SPECIFICATION 14198
First edition
2012-11-15
Road vehicles — Ergonomic aspects
of transport information and control
systems — Calibration tasks for
methods which assess driver demand
due to the use of in-vehicle systems
Véhicules routiers — Aspects ergonomiques des systèmes
d’information et de contrôle du transport — Tâches de calibration
pour méthodes qui évaluent la distraction du conducteur due à
l’utilisation des systèmes embarqués
Reference number
ISO/TS 14198:2012(E)
©
ISO 2012
---------------------- Page: 1 ----------------------
ISO/TS 14198:2012(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO 2012
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any
means, electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the
address below or ISO’s member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO 2012 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/TS 14198:2012(E)
Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 3
5 Calibration tasks . 3
5.1 Principle and overview . 3
5.2 Types of calibration tasks . 4
5.3 Critical Tracking Task (CTT). 4
5.4 Surrogate Reference Task (SURT) . 7
6 Calibration criterion .10
6.1 Calibration criterion procedure .10
6.2 General calibration consideration .11
Annex A (informative) Calibration task setup details .12
Annex B (informative) Multi-lab reference data for LCT .14
Bibliography .16
© ISO 2012 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/TS 14198:2012(E)
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of technical committees is to prepare International Standards. Draft International
Standards adopted by the technical committees are circulated to the member bodies for voting.
Publication as an International Standard requires approval by at least 75 % of the member bodies
casting a vote.
In other circumstances, particularly when there is an urgent market requirement for such documents, a
technical committee may decide to publish other types of normative document:
— an ISO Publicly Available Specification (ISO/PAS) represents an agreement between technical
experts in an ISO working group and is accepted for publication if it is approved by more than 50 %
of the members of the parent committee casting a vote;
— an ISO Technical Specification (ISO/TS) represents an agreement between the members of a
technical committee and is accepted for publication if it is approved by 2/3 of the members of the
committee casting a vote.
An ISO/PAS or ISO/TS is reviewed after three years in order to decide whether it will be confirmed for
a further three years, revised to become an International Standard, or withdrawn. If the ISO/PAS or
ISO/TS is confirmed, it is reviewed again after a further three years, at which time it must either be
transformed into an International Standard or be withdrawn.
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights.
ISO/TS 14198 was prepared by Technical Committee ISO/TC 22, Road vehicles, Subcommittee SC 13,
Ergonomics applicable to road vehicles.
iv © ISO 2012 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/TS 14198:2012(E)
Introduction
The number of standardized methods to assess driver attentional demand due to the use of in-vehicle
information and communication devices is continuing to increase. In applying these methodologies, it is
important to understand and document variability in participants’ performance of standard calibration
tasks and procedures across laboratories and/or time.
A suitable calibration task should have the following attributes:
— It should be robust against the variations in cultural background of participants.
— Properly applied, the task should give repeatable quantitative results. It should be sensitive to
inappropriate variations in participants, equipment, location, experimenter and instruction.
— It should use durable and readily available equipment for conducting the task
— It should apply to the driver population and be usable in a driving-like context.
A standardized calibration task can be used to produce a range of statistically stable, repeatable and
comparable secondary task demands for a participant in an experimental setting. This setting can be
used to assess the effect on driving performance of the attentional demand due to driver interaction
with an information, entertainment, and control or communication system while a vehicle is in motion.
Different calibration tasks are specified in this Technical Specification to cover calibration manual and
visual aspects of various secondary task characteristics.
© ISO 2012 – All rights reserved v
---------------------- Page: 5 ----------------------
TECHNICAL SPECIFICATION ISO/TS 14198:2012(E)
Road vehicles — Ergonomic aspects of transport
information and control systems — Calibration tasks for
methods which assess driver demand due to the use of in-
vehicle systems
1 Scope
This Technical Specification provides procedures that can be used as a secondary task in a dual task
setting to determine whether that evaluation setting is standardized and valid for purposes of assessing
driver attentional demand due to the use of an in-vehicle system. This Technical Specification does not
define calibration procedures for other evaluation activities that a laboratory might undertake.
This Technical Specification provides advice on the selection of an appropriate candidate calibration
task, given an attentional demand evaluation procedure that uses primary driving-like task settings and
procedures which are defined outside of this Technical Specification.
The description of a calibration task includes its application, experimental set-up, data collection, and
procedures for analysis of results.
The purpose of this Technical Specification is not to define a reference criterion as to whether a given
secondary task is suitable for use while driving. Although specific settings of parameters of a calibration
task might be used to realize such a predefined pass/fail criterion, this Technical Specification does not
provide such a criterion for a given level of attentional demand.
2 Normative references
The following referenced documents are indispensable for the application of this document. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced
document (including any amendments) applies.
ISO 15008, Road vehicles — Ergonomic aspects of transport information and control systems — Specifications
and test procedures for in-vehicle visual presentation
ISO 26022, Road vehicles — Ergonomic aspects of transport information and control systems — Simulated
lane change test to assess in-vehicle secondary task demand
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
3.1
calibration task
type of reference task used for the purpose of comparing different tests or test results between sites, or
over time at a given site
3.2
criterion
threshold or value of a variable to be met
3.3
demand
total visual, auditory, cognitive, or physical resources required of the driver to accomplish the primary
driving task and interact with a Transport Information and Control System (TICS) in a dual task setting
© ISO 2012 – All rights reserved 1
---------------------- Page: 6 ----------------------
ISO/TS 14198:2012(E)
3.4
dual task
two tasks concurrently performed, typically the primary driving task plus the secondary task
3.5
environment
physical surroundings in which data are captured and collected, consequently, the level of control over
the independent variables in a study
EXAMPLE Laboratory, simulator, test track, real road.
3.6
evaluation
procedure in which the effect of a Transport Information and Control System (TICS) or other device is
assessed
NOTE 1 It may be undertaken retrospectively after the TICS has been in use for a considerable time as a product.
NOTE 2 The results of the evaluation will depend on the HMI, but also on the equipment reliability and
subsequent behavioural changes which may affect driving performance.
3.7
method
high-level approach to an assessment, based on theory and principles, which implies an underlying
rationale in the choice of assessment techniques
EXAMPLE Behaviour analysis, workload assessment, and analysis of psycho physiological responses.
3.8
metric
quantitative measure of driver behaviour independent of the tool used to measure it
EXAMPLE Eye glance duration and vehicle speed.
3.9
performance
result of skill application demonstrated by a participant in performing a driving task or Transport
Information and Control System (TICS) related task
3.10
primary driving-like task
primary task in a dual task setting that simulates or approximates a primary driving task
NOTE The Lane Change Task in ISO 26022 is one example.
3.11
primary driving task
activities that the driver must undertake while driving including navigating, path following,
manoeuvring, avoiding obstacles, and controlling speed; and which a participant may perform through
the duration of a test (simulated substitute for driving)
3.12
secondary task
non-driving related additional task
NOTE A calibration task for the purpose of this Technical Specification.
3.13
secondary task demand
sum of visual, auditory, cognitive, motor, and speech resource demands required by a non-driving
related task
2 © ISO 2012 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/TS 14198:2012(E)
3.14
system paced secondary task
activity in which the change from the current to the next state in the interaction between user and
system is initiated by the system
NOTE The pace can be fixed or variable.
3.15
target bar
moving line on the critical tracking task display which indicates the task error
3.16
task
process of achieving a specific and measureable goal using a prescribed method
3.17
user paced secondary task
activity in which the change from the current to the next state in the interaction between user and
system is initiated by the user
4 Abbreviated terms
CI Confidence Interval
CTT Critical Tracking Task
LCT Lane Change Test
MDEV Mean Deviation (According to ISO 26022)
SURT Surrogate Reference Task
TICS Transport Information and Control System (A list of TICS fundamental services has been
defined by ISO/TC204/WG1)
5 Calibration tasks
5.1 Principle and overview
For calibration purposes, a standardized calibration task shall be used as a secondary task in a dual
task setting in combination with a method to assess attentional demand due to the use of an in-vehicle
system. The dual task setting shall include a primary driving-like task (primary task) and the secondary
calibration task.
Examples for driving-like dual task settings may include the operation of a TICS secondary task in the
Lane Change Test or in a driving simulator environment.
The calibration shall be performed in a setting that is intended for the assessment of secondary tasks
and follow the training and experimental procedures of that method for assessing attentional demand.
To date, development of the calibration tasks and associated procedures have used the ISO 26022, Lane
Change Test (LCT) to represent the primary task. While the calibration tasks described herein are
intended to be applicable to other primary driving-like task implementations and dual task settings, care
must be taken to ensure that the conditions are sufficiently similar to those of ISO 26022 considering
equipment and instructions to ensure a valid application of this specification and its procedures.
© ISO 2012 – All rights reserved 3
---------------------- Page: 8 ----------------------
ISO/TS 14198:2012(E)
5.2 Types of calibration tasks
There are various possibilities to realize a calibration task. In the following subclauses, two example
alternatives are specified in detail. These alternatives include a system-paced secondary task (critical
tracking task) as well as a user-paced secondary task (surrogate reference task). Both alternatives represent
visual-manual tasks that can be used in a dual task setting and are recommended as calibration tasks.
5.3 Critical Tracking Task (CTT)
5.3.1 Description
The CTT is a visual-manual task, which requires continuous control activity by the participant.
The participant controls the position of a vertically moving target bar with respect to a fiducial line
(centreline) within a target area by manipulating up and down arrow keys. The arrow key control gives
discrete commands to the target bar which moves it up and down. The up key moves the target bar up,
and the down key moves it down.
The dynamics of the motion of the target bar are a first order instability. If the participant does nothing,
the target bar moves (divergently) towards the edge of the display. The participant then has to make
suitable corrective arrow key inputs to bring the target bar back towards the centre (the red dashed
line in Figure 2).
A control system block diagram of the CTT is shown in Figure 1.
5.3.2 Operation of the CTT
When the program is first started, the screen will look like Figure 2. Nothing will happen initially.
The centreline is displayed as a red dashed horizontal line in the centre of the display. The target bar
is displayed as a black line. The target bar will start to move away from the centreline showing an
increasing error. Two short blue reference lines are shown above and below the centre to subdivide the
screen for better orientation. The instruction to the participant is to control the position of the target
bar with the arrow keys (arrow up key and arrow down key) and keep it as close to the centreline as
possible so as to minimize the errors.
5.3.3 Setup for CTT
The setup consists of a 19 inch (483 mm) screen with SVGA resolution plus keyboard. The subtended
vision angle (width) of the display area relative to participant’s eyes shall be 13±1 degrees horizontally.
The width to height ratio of display area shall be 4:3. The centre of the secondary display shall be
positioned 28±2 degrees horizontally (right or left depending on the intended display position in the
vehicle, LHD or RHD) and 20±2 degrees vertically from participant’s straight ahead line of sight. For
further details see Annex A.
To control the target bar movement, the arrow key of a standard PC keyboard (or an equivalent
arrangement of keys, for example Figure 4) shall be used. Participants are allowed to place the keyboard
in a comfortable position on the same side of the steering wheel as the CTT display on a table or console
aside but not connected to their body.
5.3.4 Test conditions for CTT
The test in the dual task setting shall include one specified level of lambda, 0.5, the CTT easy condition.
The lambda level has to be set prior to each run and kept constant during the run. In each run the
primary task and the CTT task are to be performed for at least 2 min.
4 © ISO 2012 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/TS 14198:2012(E)
5.3.5 Participants for CTT
The CTT as a calibration task shall be performed in a dual task setting by at least 16 [n = 16] participants
who are licensed drivers. Participants should be familiar with the primary task as well as operation of
the CTT. The level of participant familiarity shall be documented in the protocol. In the case of calibration
using an LCT setting it is recommended to select the participants according to the ISO 26022 LCT
standard, and following the sample description regarding age, gender and familiarity with primary and
secondary task in Bengler, K., Mattes, S., Hamm, O., Hensel, M., 2010 [1]. Typically, this was an average
age of 32 to 45, gender balanced, and instructed and practiced in both the primary and secondary tasks.
5.3.6 Participant instruction of CTT
The following verbal/written instructions for CTT shall be given to the participant:
“A horizontal black target bar is displayed on the secondary task screen. There is also a red dashed
horizontal line in the centre of the screen. When the task is started, the black target bar will tend to
move away from the red centreline. The motion of the black target bar can be controlled by the arrow-
up- and arrow-down keys in order to bring the target bar back to the centre. These keys give discrete
commands to the target. (The down arrow key moves the target bar down; the up arrow key moves the
target bar up.) The goal is to control the motion of the black target bar to keep it as close as possible to
the red dashed centre line.”
Key
λ level of instability or rate of divergence (adjustable)
s Laplace transform variable
Figure 1 — Control system block diagram of the CTT
The lambda level is set to a fixed value at the beginning of a run and defines the secondary task difficulty.
© ISO 2012 – All rights reserved 5
---------------------- Page: 10 ----------------------
ISO/TS 14198:2012(E)
Key
1 target bar
2 centreline
Figure 2 — Typical screen of the CTT with target bar above the centreline
If the participant does nothing, the target bar moves towards the edge of the display. The further the
black target bar is away from the centre line, the faster it will move, so a larger (longer duration) input
is needed, to bring it back to the centreline.
The difficulty of controlling the black target bar can vary with the size of the error. The participant will
need to work quickly and correctly in order to keep the target bar near the centreline.
5.3.7 Practice trials
Participants shall practice the CTT secondary task alone at lambda levels of 0,5 and 1,0. Before proceeding
to the dual task conditions, participants shall be able to perform the CTT alone at a level of lambda = 1,0
for one minute successfully (i.e. without the target bar hitting the upper or lower limit of the display).
Only the lambda value of 0,5 is used in the dual task trials.
Primary task and dual task practice trials shall be conducted as prescribed in the corresponding
standard or other procedures.
5.3.8 Test metrics
The primary task measures shall be those prescribed for the primary task. If the Lane Change Test is
used, these are in ISO 26022.
Participants’ performance in the CTT is measured using:
— CTT root mean square deviation of the target bar from the centreline in pixels.
— CTT percentage of time with target bar at the upper or lower limit.
6 © ISO 2012 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/TS 14198:2012(E)
5.4 Surrogate Reference Task (SURT)
5.4.1 Description
The Surrogate Reference Task (SURT) is both visually and manually demanding. The level of demand
can be varied.
The SURT is a visual search paradigm. In a typical visual search paradigm, participants are asked to
report whether or not a pre-specified target is embedded in a multi-item display, usually comprised of
an array of alphanumeric symbols, forms, colours or words. The non-target items are called distractors.
To vary the time the participant will need to identify the target and to answer (reaction time), the
similarity between target and distractors can be manipulated. The more similar the distractors are
to the target the more time is typically needed to identify the target. Unless these distractors have
no unique feature distinguishing them from the target (feature with a visual “pop out” character), the
number of distractors can also increase the time participants will need for search (Treisman and Gelade,
1980). An example for the task is shown in Figure 3.
For the SURT calibration task the pre-specified target is always present and the participants have to
locate the target among more or less visually similar distractors (visual search is visually demanding).
The task for the participants is to locate the target and to manually confirm its location (manually
demanding). The stimuli for targets and distractors are circles. Only the diameter of the circles
distinguishes between target and distractors, while the diameter of the target remains constant. The
thickness of the circle shall be equal for both target and distractors.
For the participant to be able to indicate the position of the target, the display is divided into evenly
distributed vertically arranged rectangular areas. During search none of these areas is marked and no
highlight is visible. Once the participant visually locates the target, the participant should press the left
or right key (indicated by graphics on the keys) depending on the position of the target on the display,
and the highlight (as in Figure 3) will appear immediately to the left or right of the centreline of the
display. Consecutive presses on the keys moves the highlight to the left or right. The participant uses
the keys to move the highlight until s/he has positioned the highlight over the target. Finally s/he has to
confirm completion of the task by pressing a third key (confirmation key).
The response time is defined as the time from stimulus onset to the keypress of the confirmation key.
The task is participant paced, i.e. there is no time out. If the target is within the highlighted area when
the confirmation key is pressed, the answer is recorded as correct, otherwise as incorrect. Response
times are measured to estimate the participant’s performance with different difficulty levels of the
calibration task, and to confirm increasing calibration task difficulty.
Figure 3 — Visual display of SURT distractors and target (here the target is the right circle in
the upper area marked by the arrow). The visual display of SURT in operation does not include
the marking arrow
© ISO 2012 – All rights reserved 7
---------------------- Page: 12 ----------------------
ISO/TS 14198:2012(E)
5.4.2 Setup for SURT
Calibration task display set-up should be as follows, in order to make all values consistent between
tests or sites:
— Vision angle of the display area from participant’s eyes shall be 13 ±1 degrees horizontally.
— Width to height ratio of display area: 4:3.
— Centre of display shall be positioned 28 ±2 degrees horizontally (right or left depending expected
position of system to be tested) and 20 ±2 degrees vertically from participant’s straight ahead
line of sight.
— Line thickness of target and distractors: 0,07 degrees corresponding to approximately 1 mm when
viewed at a distance of 800 mm.
— Target size: 0,7 degrees corresponding to approx. 10 mm when viewed at a distance of 800 mm.
— Background colour: black (RGB-Values: 0, 0, 0).
— Target and distractor colours: light grey (RGB-Value: 192, 192, 192).
— Highlighting colour: medium grey (RGB-Value: 160, 160, 160).
— Minimum contrast (light/dark), target and distractors to background: 8:1 (ISO 15008).
An example setup is shown in Annex A.
Once the target circle is located, the manual answer is given using the arrow key of a standard PC
keyboard or an equivalent keypad. The arrangement of the keys shown in Figure 4 may be used. The
left/right keys move the shaded area over the target circle. Either up/down key can be used to enter the
answer. Participants can place the keyboard at their own comfort on the same side of the steering wheel
as the Calibration task display. Alternatively, the keyboard may be part of the Calibration task display
unit (lap-top computer).
Figure 4 — Example of key-pad to move the highlight and confirm target location result
The background colour of the Calibration task display is black (RGB-Values: 0, 0, 0). Each display consists
of one target and 50 distractors. Targets and distractors are randomly distributed over the whole
display. The items should not intersect with each other.
To control the visual demand of the Calibration task it is possible to manipulate the size of the distractors
(e.g. diameter). Distractor size closer to target size will result in higher visual demand. The more the
distractor size differs from the target size the less visual demanding the Calibration task will be.
8 © ISO 2012 – All rights reserved
---------------------- Page: 13 ----------------------
ISO/TS 14198:2012(E)
5.4.3 Test conditions for SURT
The test shall include one level of difficulty. The test duration should be approximately 2 min.
Figure 3 shows an example for the SURT hard condition to be used. Using a display distance of 78,5 cm,
– the target size is 61 arcmin (14 mm); the distractor 53 arcmin (12 mm).
5.4.4 Participants for SURT
The SURT as a calibration task shall be performed in a dual task setting by at least 16 participants [n =
16]. Participants shall be familiar with the primary task as well as the SURT. The level of participants’
familiarity with the primary task and the SURT shall be documented in the protocol. For example, state
that the participant performed this secondary task previously in a dual task condition. In the case of
calibration using the LCT procedure it is recommended to select the participants following the protocols
in ISO 26022 and the sample description in Bengler, K., Mattes, S., Hamm, O., Hensel, M., 2010. For LCT
being the primary driving like task the number of n = 16 participants is based on the experience of
Bengler et al. 2010 [10].
5.4.5 Participant instructions for SURT
All participants should be instructed the same way as follows:
An array of circles is displayed on the secondary task screen which is down and to the right of the
driving simulation scene. One of these circles is bigger than the other ones. The position of this circle is
randomly varying, but the positioning should not influence the difficulty to detect the difference. The
participant task is to f
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.