SIST EN ISO 10075-3:2004

Ergonomic principles related to mental workload - Part 3: Principles and requirements concerning methods for measuring and assessing mental workload (ISO 10075-3:2004)

This International Standard gives guidance on measurement and assessment of mental workload and specifies the requirements for measurement instruments to be met a different level of precision in measuring mental workload. This International Standard provides information to the parties involved for choosing appropriate methods and provides information to improve communication among the parties involved on aspects of assessing and measuring mental workload.

Ergonomische Grundlagen bezüglich psychischer Arbeitsbelastung - Teil 3: Grundsätze und Anforderungen an Verfahren zur Messung und Erfassung psychischer Arbeitsbelastung (ISO 10075-3:2004)

Dieser Teil der ISO 10075 legt Grundsätze und Anforderungen für die Messung und Erfassung der psychischen Arbeitsbelastung fest und spezifiziert die Anforderungen an Messverfahren. Dieser Teil der ISO 10075 stellt Informationen zur Auswahl geeigneter Verfahren bereit und enthält Informationen zu Aspekten der Erfassung und Messung der psychischen Arbeitsbelastung, um damit zur Verbesserung der Kommunikation zwischen den beteiligten Parteien beizutragen.
Dieser Teil der ISO 10075 ist hauptsächlich für die Anwendung durch Fachleute der Ergonomie vorgesehen, z. B. Psychologen, Arbeitsmediziner, Arbeitsphysiologen und andere medizinische Fachkräfte, mit einer angemessenen Ausbildung in den theoretischen Grundlagen und der Anwendung dieser Messverfahren sowie der Interpretation der Ergebnisse. Diese Fachleute werden die für die Entwicklung oder Bewertung von Verfahren zur Erfassung der psychischen Arbeitsbelastung benötigten Informationen finden.
Nichtfachleute, z. B. Arbeitgeber, Beschäftigte und deren Vertreter, Systemverantwortliche und -gestalter wie auch Behörden können nützliche Informationen zur Orientierung auf dem Gebiet der Erfassung und Messung der psychischen Arbeitsbelastung finden; z. B. darüber, welche Arten von Verfahren zur Verfügung stehen, welche Kriterien für die Bewertung von Messverfahren wesentlich sind und welche Art von Informationen sie für die Entscheidung, welches Instrument für ihren Zweck geeignet ist, fordern und beachten sollten.
ANMERKUNG   Ein Technischer Bericht mit weiteren Informationen über die Terminologie und die Anwendung dieses Teils der ISO 10075 wird für Nichtfachleute zur Verfügung gestellt werden.
Dieser Teil der ISO 10075 enthält Informationen, die eine begründete Auswahl eines der jeweiligen Untersuchungssituation angemessenen Verfahrens erlauben. Es existiert eine große Anzahl verschiedener Verfahren, die sich für unterschiedliche Zwecke, Situationen und Stufen der Messgenauigkeit eignen. Allgemein besteht ein Bedarf an

Principes ergonomiques relatifs a la charge de travail mental - Partie 3: Principes et exigences concernant les méthodes de mesurage et d'évaluation de la charge de travail mental (ISO 10075-3:2004)

L'ISO 10075-3:2004 établit des principes et spécifie des exigences pour le mesurage et l'évaluation de la charge de travail mental; elle spécifie également les exigences relatives aux instruments de mesure. La présente partie de l'ISO 10075 fournit des informations permettant de choisir les méthodes appropriées, ainsi que des renseignements sur les aspects liés à l'évaluation et au mesurage de la charge de travail mental, afin d'améliorer la communication entre les parties concernées.
L'ISO 10075-3:2004 est principalement destinée aux spécialistes en ergonomie, à savoir, par exemple, les psychologues, les spécialistes en médecine du travail et/ou les physiologistes ayant reçu une formation théorique adéquate, disposant d'une bonne maîtrise pratique de telles méthodes et capables d'interpréter les résultats. Ils y trouveront les informations requises lorsqu'ils seront appelés à élaborer ou analyser les méthodes d'évaluation de la charge de travail mental.

Ergonomska načela v zvezi s psihičnimi obremenitvami - 3. del: Načela in zahteve v zvezi z merjenjem in ocenjevanjem psihične obremenitve (ISO 10075-3:2004)

General Information

• Status: Published
• Publication Date: 31-Oct-2004
• ICS: 13.180 - Ergonomics
• Technical Committee: VSN - Safety of machinery
• Current Stage: 6060 - National Implementation/Publication (Adopted Project)
• Start Date: 01-Nov-2004
• Due Date: 01-Nov-2004
• Completion Date: 01-Nov-2004
• Ref Project: EN ISO 10075-3:2004 - Ergonomic principles related to mental workload - Part 3: Principles and requirements concerning methods for measuring and assessing mental workload (ISO 10075-3:2004)

Standards Content (Sample)

SLOVENSKI STANDARD
SIST EN ISO 10075-3:2004
01-november-2004
(UJRQRPVNDQDþHODY]YH]LVSVLKLþQLPLREUHPHQLWYDPLGHO1DþHODLQ]DKWHYH
Y]YH]L]PHUMHQMHPLQRFHQMHYDQMHPSVLKLþQHREUHPHQLWYH,62
Ergonomic principles related to mental workload - Part 3: Principles and requirements
concerning methods for measuring and assessing mental workload (ISO 10075-3:2004)
Ergonomische Grundlagen bezüglich psychischer Arbeitsbelastung - Teil 3: Grundsätze
und Anforderungen an Verfahren zur Messung und Erfassung psychischer
Arbeitsbelastung (ISO 10075-3:2004)
Principes ergonomiques relatifs a la charge de travail mental - Partie 3: Principes et
exigences concernant les méthodes de mesurage et d'évaluation de la charge de travail
mental (ISO 10075-3:2004)
Ta slovenski standard je istoveten z: EN ISO 10075-3:2004
ICS:
13.180 Ergonomija Ergonomics
SIST EN ISO 10075-3:2004 en
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

---------------------- Page: 1 ----------------------

SIST EN ISO 10075-3:2004

---------------------- Page: 2 ----------------------

SIST EN ISO 10075-3:2004
EUROPEAN STANDARD
EN ISO 10075-3
NORME EUROPÉENNE
EUROPÄISCHE NORM
August 2004
ICS 13.180

English version
Ergonomic principles related to mental workload - Part 3:
Principles and requirements concerning methods for measuring
and assessing mental workload (ISO 10075-3:2004)
Principes ergonomiques relatifs à la charge de travail Ergonomische Grundlagen bezüglich psychischer
mental - Partie 3: Principes et exigences concernant les Arbeitsbelastung - Teil 3: Grundsätze und Anforderungen
méthodes de mesurage et d'évaluation de la charge de an Verfahren zur Messung und Erfassung psychischer
travail mental (ISO 10075-3:2004) Arbeitsbelastung (ISO 10075-3:2004)
This European Standard was approved by CEN on 16 July 2004.
CEN members are bound to comply with the CEN/CENELEC Internal Regulations which stipulate the conditions for giving this European
Standard the status of a national standard without any alteration. Up-to-date lists and bibliographical references concerning such national
standards may be obtained on application to the Central Secretariat or to any CEN member.
This European Standard exists in three official versions (English, French, German). A version in any other language made by translation
under the responsibility of a CEN member into its own language and notified to the Central Secretariat has the same status as the official
versions.
CEN members are the national standards bodies of Austria, Belgium, Cyprus, Czech Republic, Denmark, Estonia, Finland, France,
Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Slovakia,
Slovenia, Spain, Sweden, Switzerland and United Kingdom.
EUROPEAN COMMITTEE FOR STANDARDIZATION
COMITÉ EUROPÉEN DE NORMALISATION
EUROPÄISCHES KOMITEE FÜR NORMUNG
Management Centre: rue de Stassart, 36  B-1050 Brussels
© 2004 CEN All rights of exploitation in any form and by any means reserved Ref. No. EN ISO 10075-3:2004: E
worldwide for CEN national Members.

---------------------- Page: 3 ----------------------

SIST EN ISO 10075-3:2004

EN ISO 10075-3:2004 (E)





Foreword


This document (EN ISO 10075-3:2004) has been prepared by Technical Committee ISO/TC 159
"Ergonomics" in collaboration with Technical Committee CEN/TC 122 "Ergonomics", the
secretariat of which is held by DIN.

This European Standard shall be given the status of a national standard, either by publication of
an identical text or by endorsement, at the latest by February 2005, and conflicting national
standards shall be withdrawn at the latest by February 2005.

According to the CEN/CENELEC Internal Regulations, the national standards organizations of
the following countries are bound to implement this European Standard: Austria, Belgium,
Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary,
Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland,
Portugal, Slovakia, Slovenia, Spain, Sweden, Switzerland and United Kingdom.


Endorsement notice

The text of ISO 10075-3:2004 has been approved by CEN as EN ISO 10075-3:2004 without any
modifications.

2

---------------------- Page: 4 ----------------------

SIST EN ISO 10075-3:2004


INTERNATIONAL ISO
STANDARD 10075-3
First edition
2004-08-15

Ergonomic principles related to mental
workload —
Part 3:
Principles and requirements concerning
methods for measuring and assessing
mental workload
Principes ergonomiques relatifs à la charge de travail mental —
Partie 3: Principes et exigences concernant les méthodes de mesurage
et d'évaluation de la charge de travail mental




Reference number
ISO 10075-3:2004(E)
©
ISO 2004

---------------------- Page: 5 ----------------------

SIST EN ISO 10075-3:2004
ISO 10075-3:2004(E)
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but
shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In
the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.


©  ISO 2004
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail [e-mail]
Web www.iso.org
Published in Switzerland

ii © ISO 2004 – All rights reserved

---------------------- Page: 6 ----------------------

SIST EN ISO 10075-3:2004
ISO 10075-3:2004(E)
Contents Page
Foreword. iv
Introduction . v
1 Scope. 1
2 Normative references . 1
3 Terms and definitions. 2
4 Measurement and assessment of mental workload. 3
4.1 General principles. 3
4.2 Procedural requirements. 5
4.3 Quantitative requirements for measuring instruments. 8
4.4 Documentation requirements . 10
Annex A (informative) Additional information concerning generalizability . 13
Annex B (informative) Checklist for choosing an instrument. 14
Bibliography . 15

© ISO 2004 – All rights reserved iii

---------------------- Page: 7 ----------------------

SIST EN ISO 10075-3:2004
ISO 10075-3:2004(E)
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards bodies
(ISO member bodies). The work of preparing International Standards is normally carried out through ISO
technical committees. Each member body interested in a subject for which a technical committee has been
established has the right to be represented on that committee. International organizations, governmental and
non-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely with the
International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of technical committees is to prepare International Standards. Draft International Standards
adopted by the technical committees are circulated to the member bodies for voting. Publication as an
International Standard requires approval by at least 75 % of the member bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO shall not be held responsible for identifying any or all such patent rights.
ISO 10075-3 was prepared by Technical Committee ISO/TC 159, Ergonomics, Subcommittee SC 1,
Ergonomic guiding principles.
ISO 10075 consists of the following parts, under the general title Ergonomic principles related to mental
workload:
 Part 1: General terms and definitions
 Part 2: Design principles
 Part 3: Principles and requirements concerning methods for measuring and assessing mental workload
A Technical Report will accompany these parts to explain to non-experts the basic concepts and how to use
these parts.
iv © ISO 2004 – All rights reserved

---------------------- Page: 8 ----------------------

SIST EN ISO 10075-3:2004
ISO 10075-3:2004(E)
Introduction
This part of ISO 10075 specifies technical information relevant in the context of constructing, evaluating and
choosing measurement instruments for assessing mental workload as defined and treated in ISO 10075 and
ISO 10075-2. Familiarity with the concepts discussed in these two documents is required to understand the
provisions of this part of ISO 10075.
Since mental workload is a part of the total workload, users of this part of ISO 10075 should also be familiar
with the concepts and provisions presented in ISO 6385.
This part of ISO 10075 aims at providing information for the development of measurement instruments, about
which specifications will be required to evaluate a given procedure with regard to its usability as a measuring
instrument for assessing mental workload.
This part of ISO 10075 addresses requirements for instruments measuring different aspects of mental
workload, but it does not specify which instruments should be used, e.g. psychological scaling or
psychophysiological methods. The choice of which instruments to use can be facilitated by the provision of
appropriate information.

© ISO 2004 – All rights reserved v

---------------------- Page: 9 ----------------------

SIST EN ISO 10075-3:2004

---------------------- Page: 10 ----------------------

SIST EN ISO 10075-3:2004
INTERNATIONAL STANDARD ISO 10075-3:2004(E)

Ergonomic principles related to mental workload —
Part 3:
Principles and requirements concerning methods for
measuring and assessing mental workload
1 Scope
This part of ISO 10075 establishes principles and requirements for the measurement and assessment of
mental workload and specifies the requirements for measurement instruments. This part of ISO 10075
provides information for choosing appropriate methods and provides information on aspects of assessing and
measuring mental workload to improve communication among the parties involved.
This part of ISO 10075 is intended for use mainly by ergonomic experts, for example, psychologists,
occupational health specialists, and/or physiologists, with appropriate training in the theoretical background
and usage of such methods, as well as in the interpretation of the results. They will find the information
needed when developing or evaluating methods of mental-workload assessment.
Non-experts, e.g. employers, employees and their representatives, system managers and designers, and
public authorities can find useful information for their orientation in the field of assessment and measurement
of mental workload, e.g. what kinds of methods are available, which criteria are relevant in the evaluation of
measurement instruments and what kind of information they should require and observe in deciding which
instrument will be suitable for their purpose and which can be used.
NOTE A Technical Report on the terminology and use of this part of ISO 10075 will be available for further
information for non-experts.
This part of ISO 10075 provides information on which to base a well-considered choice for an appropriate
method in different situations. There are a large number of different methods available which are suitable for
different purposes, situations and different levels of precision. There is a need for effective and efficient
methods of measurement. The information provided in this part of ISO 10075 will allow users to evaluate the
type of measurement approach most suitable for their specific purposes.
Conformance with the provisions of this part of ISO 10075 has to be provided by the documentation
requirements.
2 Normative references
The following referenced documents are indispensable for the application of this document. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced
document (including any amendments) applies.
ISO 6385:2004, Ergonomic principles in the design of work systems
© ISO 2004 – All rights reserved 1

---------------------- Page: 11 ----------------------

SIST EN ISO 10075-3:2004
ISO 10075-3:2004(E)
1)
ISO 10075:1991 , Ergonomic principles related to mental workload — General terms and definitions
ISO 10075-2:1996, Ergonomic principles related to mental workload — Part 2: Design principles
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO 6385:2004, ISO 10075:1991 and
ISO 10075-2:1996 and the following apply.
3.1
objectivity
degree to which results gained with an instrument are independent of the person administering the instrument,
analysing and interpreting the data
3.2
reliability
degree of precision to which a method or instrument is able to measure what it measures
NOTE Reliability can be assessed as homogeneity, consistency or stability of measurement, or in the case of two or
more raters, as inter-rater-reliability. Reliability is closely related to generalizability.
3.2.1
homogeneity
degree to which all parts or items of a measurement procedure measure the same characteristic
3.2.2
consistency
degree to which different parts or parallel forms of a measurement instrument lead to identical results, e.g. by
dividing a scale into two or more parts or applying two or more parallel forms of an instrument
3.2.3
stability
degree to which a (usually time delayed) replication of a measurement procedure leads to identical results
3.2.4
inter-rater-reliability
degree to which two or more raters produce the same results in measuring the same characteristics
3.3
validity
degree to which a method or instrument is able to measure what it is intended to measure
NOTE Validity can be assessed via concurrent validation (e.g. by agreement of results with the results of a
simultaneously applied procedure known to measure the intended aspect), criterion-related validation (e.g. by establishing
a relation with a relevant criterion), or factorial validity (e.g. by demonstrating that a measurement procedure assesses
specific facets of a construct).
3.4
sensitivity
degree to which a method or instrument is able to discriminate between different degrees of the object of
measurement, e.g. different degrees of mental strain or fatigue

1) If revised, this International Standard will become ISO 10075-1.
2 © ISO 2004 – All rights reserved

---------------------- Page: 12 ----------------------

SIST EN ISO 10075-3:2004
ISO 10075-3:2004(E)
3.5
diagnosticity
degree to which a method or instrument is able to discriminate between different kinds or sources of mental
workload e.g. perceptual demands etc., or its effects, e.g. discriminating between fatigue, monotony, satiation
or reduced vigilance
3.6
generalizability
degree to which an observed score (unit of measurement) can be generalized to a defined universe of
situations (stress conditions) and/or population of workers
3.6.1
relative generalizability
degree to which rank ordering of workload conditions/people, e.g. from low to high, is replicable (as in
decisions concerning relative positions)
3.6.2
absolute generalizability
degree to which an absolute level of stress/strain associated with specific work conditions/people without
regard to the stress/strain associated with other conditions/people can be replicated (as in decisions
concerning absolute values)
NOTE For more explanations concerning generalizability, see Annex A.
3.7
usability
extent to which a product can be used by specified users to achieve specified goals with effectiveness,
efficiency and satisfaction in a specified context of use
[ISO 9241-11:1998, Definition 3.1]
NOTE In the case of this part of ISO 10075, a product is an instrument.
3.8
critical values
reference standard for the evaluation of measured scores and/or pass-fail decisions
NOTE The establishment of critical values presupposes that the scores of the measurement instrument in question
allow absolute decisions with high precision. Furthermore, it has to be stated under which conditions the critical values are
valid (for example, post-test measurements etc.)
4 Measurement and assessment of mental workload
4.1 General principles
Mental workload is not a unitary and not a unidimensional concept, so the assessment and measurement of
mental workload cannot be a uniform procedure. There is no single best way to assess mental workload,
since the most appropriate way to assess or measure mental workload will depend on the purpose of the
assessment, which may require the assessment of different aspects of mental workload, the use of different
techniques of measurement, and different degrees of precision.
Thus, the model of workload assessment used in this part of ISO 10075 has a three-dimensional structure. It
takes into account
 different aspects of mental workload, e.g. mental stress, mental strain, mental fatigue, etc.,
 different techniques of measurement, e.g. task analysis, performance assessment, subjective ratings or
psychophysiological measurement, and
© ISO 2004 – All rights reserved 3

---------------------- Page: 13 ----------------------

SIST EN ISO 10075-3:2004
ISO 10075-3:2004(E)
 different degrees of precision, e.g. measurement at an orienting, screening or accurate level of
measurement.
According to ISO 10075:1991, different aspects have to be differentiated: mental stress, mental strain, and the
effects of mental strain within the individual. Assessment and measurement thus have to refer to these
different steps in the stress-strain-effects process, i.e:
 assessment of work conditions producing mental stress, as in the design and evaluation of work system
design;
 assessment and measurement of mental strain, produced by mental stress, e.g. in order to evaluate the
tolerability of the strain;
 measurement of the effects of strain in the employee, e.g. fatigue, monotony, satiation or reduced
vigilance;
which may affect her/his health and safety, well-being, performance and productivity.
Different purposes may require different approaches and/or degrees of precision, e.g. a risk analysis for
hazardous technologies as opposed to a survey where the intention is to give a general overview over
different work systems. This part of ISO 10075 does not prescribe which level of precision has to be used,
since this is a matter of the purpose and conditions of measurement, depending, for example, on legal
requirements or agreed regulations, or cost-benefits considerations.
Methods for the measurement and assessment of mental workload, independent of the technique to be used,
will be classified in this part of ISO 10075 according to three, i.e. low, intermediate and high levels of
precision:
 level 1: for accurate measurement purposes. The purpose of the mental-workload assessment at this
high precision level is to obtain reliable and valid information about the nature of the source of under-
overload in order to optimise work conditions. The methods at this level will most probably be applicable
only by specialists, e.g. psychologists, ergonomists, occupational health specialists, and/or physiologists,
with appropriate training in the theoretical background and usage of these methods, as well as in the
interpretation of the results.
 level 2: for screening purposes. Measurement at this level represents an intermediate level of precision,
and is most often used for screening purposes, which require more precision than orienting
measurements, e.g. where problems of mental workload can be anticipated or where causes of
inappropriate workload have to be identified. Procedures to be used at this level have higher reliabilities,
and demonstrated validities, and should be able to indicate if corrective measures should be taken.
 level 3: for orienting purposes. Methods at this level allow the user to gather information about mental
workload at a low level of precision. They provide general information about work conditions, subjective
and psychophysiological states of the worker with respect to mental workload, without being excessively
resource intensive. The information at this level should make it possible to prevent negative effects by
making management decisions at an operational level such as to change work tasks and/or methods and
conditions. Measurement at this level is usually restricted to orienting measurements, e.g. relating to
rough measures (with moderate levels of reliability, validity, etc.) of task analysis, subjective ratings of
acceptability of work conditions and subjective states with regard to all aspects of mental workload.
In specifying for which level of precision an instrument can be used, the object of measurement should be
indicated. It should be stated whether the object of measurement is a person or a situational characteristic. If
the intended object of measurement is the individual, psychometric properties have to be assessed and
specified for the assessment of people. If the intended object of measurement is a situational characteristic,
psychometric properties shall relate to assessing conditions. This will be demonstrated in more detail in the
context of testing generalizability (see 4.2.7).
In specifying for which level of precision an instrument can be used, a differentiation is also required
depending on whether the measurement is based on single scores/observations or mean scores/observations.
4 © ISO 2004 – All rights reserved

---------------------- Page: 14 ----------------------

SIST EN ISO 10075-3:2004
ISO 10075-3:2004(E)
All assessment procedures will inevitably carry a measurement error which can be reduced by averaging over
a number of scores/observations. An instrument which reaches the orienting level requirements for single
scores/observations may attain screening level requirements for average scores/observations if a sufficient
number of scores is used. The required number of scores/observations for attaining a higher level of precision
shall thus be specified.
An instrument claiming to assess mental workload shall have demonstrated validity in the assessment of
those aspects for which validity is claimed, and the domains of measurement, for which validity is claimed,
shall be clearly stated (e.g. mental fatigue or monotony). If validity is claimed for more than one aspect,
documentation of the validity shall include evidence for each field of measurement.
Different techniques may be used to assess mental workload, with some techniques being more suitable for
some domains of measurement than others. In particular, the following techniques can be applied:
 physiological measurements: these methods provide information about physiological states of employees
under given work conditions;
 subjective scaling: these methods provide information on how employees subjectively assess different
aspects of mental workload at their work stations, e.g. using psychometric scales, and how they feel
about their work conditions;
 performance assessment: these methods offer the possibility to evaluate human mental and psychomotor
performance under given work conditions, e.g. in order to assess decrements or variations in
performance due to the effects of mental workload;
 job and task analysis: these methods assess task elements, physical and psychosocial work conditions,
environmental conditions and the organization of the work process as sources of mental workload.
Different methods will be required to achieve different levels of reliability, e.g. a short questionnaire may be
sufficient for an orienting level measurement, whereas in order to verify that a system design does not lead to
monotony, scales with sufficient reliability for that purpose will be required. In order to safeguard safety-critical
or hazardous systems against any negative effects of mental workload, the highest reliabilities and validities
will be required. Where such instruments are not yet available, they should be developed. Until then, those
methods with the highest available psychometric criteria shall be used for such purposes. Precision is not
determined by the measurement technique itself, but by the development, the psychometric properties, and
the adequate application of the method or instrument.
If measurement methods have to be used which do not fulfil the requirements, special expertise in the field of
mental workload and its measurement will be required
 to assess the risks associated with using a less than perfect instrument, and
 to achieve a sound evaluation of the results.
However, the selection of an appropriate instrument will also be influenced by legal or agreed regulations, as
well as cost-benefit considerations.
4.2 Procedural requirements
4.2.1 General
Assessment of the objectivity, reliability, validity, sensitivity and diagnosticity of a measurement method can
be done by any suitable and scientifically acceptable method. This part of ISO 10075 does not specify one
single best way to be followed in this procedure. However, a clear account of the strategy used in assessing
the psychometric properties of a measurement method shall be provided in order to allow for an evaluation of
the adequacy of the procedure chosen and the results achieved. In fact, the validation procedure will depend
on the measurement model assumed, e.g. assuming a probabilistic measurement model will require
probabilistic psychometrics, whereas a generalizability approach will require the estimation of variance
© ISO 2004 – All rights reserved 5

---------------------- Page: 15 ----------------------

SIST EN ISO 10075-3:2004
ISO 10075-3:2004(E)
components and the calculation of generalizability coefficients (see 4.2.7). There is no single best way for
assessing the psychometric properties of an instrument, since this depends in part on the principles
underlying the construction of the instrument. However, a generalizability approach allows for effective and
efficient procedures to estimate objectivity, reliability, validity, sensitivity and diagnosticity parameters with
reasonable effort. In any case, the procedures shall be described in such a way as to enable a replication of
the assessment of the psychometric properties.
4.2.2 Objectivity
For documenting the objectivity of a measurement method, evidence shall be provided that the person
performing the assessment or measurement has no influence on the scores/observations.This can best be
achieved by having more than two intended users of the measurement method performing measurements
with this method, and analysing the resulting data by adequate analysis of variance (ANOVA) methods for
main or interaction effects of the users of the instrument. The results shall indicate that the corresponding
effects do not exist, respectively that the estimated variance components do not deviate significantly from 0. In
testing for the presence of such effects, it shall be ensured that any actual existing effect of the person
performing the assessment/measurement will not be missed. Therefore, the error probability β should be set
to 0,05 and the error probability α should be set at least to 0,20. If effects of the person performing the
assessment/measurement cannot be rejected, the measurement
...