SIST EN ISO 9241-304:2009
(Main)Ergonomics of human-system interaction - Part 304: User performance test methods for electronic visual displays (ISO 9241-304:2008)
Ergonomics of human-system interaction - Part 304: User performance test methods for electronic visual displays (ISO 9241-304:2008)
ISO 9241-304:2008 provides guidance for assessing the visual ergonomics of display technologies with user performance test methods (as opposed to the optical test methods given in ISO 9241-305). Its use will help to ensure that, for a given context of use, a display meets minimum visual ergonomics requirements. It covers only visual attributes and does not address the ergonomics or usability of the whole product that houses a visual display.
The main users of ISO 9241-304:2008 will be those who procure displays or who need to measure display performance during product development. Its application assumes a background in behavioural science.
Ergonomie der Mensch-System-Interaktion - Teil 304: Prüfverfahren zur Benutzerleistung (ISO 9241-304:2008)
Dieser Teil von ISO 9241 bietet einen Leitfaden zur Beurteilung der visuellen Ergonomie von Anzeigetechnologien
unter Anwendung von Verfahren zur Prüfung der Leistungsfähigkeit der Nutzer (im Gegensatz
zu den optischen Prüfverfahren in ISO 9241-305). Die Anwendung dieser Norm hilft in einem gegebenen
Anwendungskontext sicherzustellen, dass eine Anzeige die Mindestanforderungen an die visuelle Ergonomie
erfüllt. Die Norm deckt nur die visuellen Merkmale von Anzeigen ab. Sie bezieht sich nicht auf die Ergonomie
oder Gebrauchstauglichkeit des gesamten Produktes, in dem die visuelle Anzeige enthalten ist.
Die in diesem Teil von ISO 9241 festlegten allgemeinen Grundsätze der Norm gelten für jegliche farbige oder
einfarbige visuelle Anzeigen, die mit einem System verbunden sind, mit dem Menschen interagieren. Darin
eingeschlossen, aber nicht auf diese beschränkt, sind visuelle Anzeigen, die mit ortsfesten oder tragbaren
Computern verwendet werden, Anzeigen in mobilen Geräten wie beispielsweise Mobiltelefonen,
Digitalkameras und digitalen personenbezogenen Geräten sowie Statusanzeigen in elektronischen Geräten
für Verbraucher wie beispielsweise Drucker, Navigationssysteme in Fahrzeugen und Mikrowellengeräten.
Dadurch wird der Anwendungsbereich der in ISO 9241-3:1992/Amd. 1:2000 festgelegten Prüfung der
visuellen Leistung und Beeinträchtigungsfreiheit erweitert, um die Leistung der Endverbraucher selbst und
deren Beurteilung der Anzeige zur Bewertung der Qualität der betreffenden Anzeige zu verwenden, um einen
mannigfaltigeren Bereich von Technologien, Benutzern, Aufgaben und Umgebungen abzudecken.
Aufgrund dieser Mannigfaltigkeit ist es unmöglich, in diesem Teil von ISO 9241 ein einziges, generelles
Prüfverfahren festzulegen, das auf sämtliche Anzeigetechnologien anwendbar ist. Stattdessen wird in dieser
Norm das Grundprinzip zur Entwicklung eines Prüfverfahrens beschrieben.
Ergonomie de l'interaction homme-système - Partie 304: Méthodes d'essai de la performance de l'utilisateur pour écrans de visualisation électroniques (ISO 9241-304:2008)
L'ISO 9241-304:2008 fournit des lignes directrices relatives à l'évaluation de l'ergonomie visuelle des technologies d'affichage, au moyen de méthodes d'essai de la performance humaine (par opposition aux méthodes d'essai optique données dans l'ISO 9241-305). Son utilisation aidera à s'assurer que, pour un contexte d'utilisation donné, un affichage satisfait aux exigences ergonomiques visuelles minimales. Elle ne concerne que les attributs visuels, et ne couvre ni l'ergonomie ni l'utilisabilité du produit entier qui intègre l'écran de visualisation.
Les principaux utilisateurs de l'ISO 9241-304:2008 seront des personnes qui achètent des écrans de visualisation ou qui doivent mesurer la performance de l'affichage pendant l'élaboration d'un produit. Son application suppose une base de connaissance en science comportementale.
Ergonomija medsebojnega vpliva človek-sistem - 304. del: Preskusni postopki za določanje uporabnikove sposobnosti delovanja (ISO 9241-304:2008)
General Information
Relations
Standards Content (Sample)
SLOVENSKI STANDARD
01-februar-2009
(UJRQRPLMDPHGVHERMQHJDYSOLYDþORYHNVLVWHPGHO 3UHVNXVQLSRVWRSNL]D
GRORþDQMHXSRUDEQLNRYHVSRVREQRVWLGHORYDQMD ,62
Ergonomics of human-system interaction - Part 304: User performance test methods for
electronic visual displays (ISO 9241-304:2008)
Ergonomie der Mensch-System-Interaktion - Teil 304: Prüfverfahren zur
Benutzerleistung (ISO 9241-304:2008)
Ergonomie de l'interaction homme-système - Partie 304: Méthodes d'essai de la
performance de l'utilisateur pour écrans de visualisation électroniques (ISO 9241-
304:2008)
Ta slovenski standard je istoveten z: EN ISO 9241-304:2008
ICS:
13.180 Ergonomija Ergonomics
35.180 Terminalska in druga IT Terminal and other
periferna oprema IT peripheral equipment
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.
EUROPEAN STANDARD
EN ISO 9241-304
NORME EUROPÉENNE
EUROPÄISCHE NORM
November 2008
ICS 13.180; 35.180
English Version
Ergonomics of human-system interaction - Part 304: User
performance test methods for electronic visual displays (ISO
9241-304:2008)
Ergonomie de l'interaction homme-système - Partie 304: Ergonomie der Mensch-System-Interaktion - Teil 304:
Méthodes d'essai de la performance de l'utilisateur pour Prüfverfahren zur Benutzerleistung (ISO 9241-304:2008)
écrans de visualisation électroniques (ISO 9241-304:2008)
This European Standard was approved by CEN on 11 July 2008.
CEN members are bound to comply with the CEN/CENELEC Internal Regulations which stipulate the conditions for giving this European
Standard the status of a national standard without any alteration. Up-to-date lists and bibliographical references concerning such national
standards may be obtained on application to the CEN Management Centre or to any CEN member.
This European Standard exists in three official versions (English, French, German). A version in any other language made by translation
under the responsibility of a CEN member into its own language and notified to the CEN Management Centre has the same status as the
official versions.
CEN members are the national standards bodies of Austria, Belgium, Bulgaria, Cyprus, Czech Republic, Denmark, Estonia, Finland,
France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal,
Romania, Slovakia, Slovenia, Spain, Sweden, Switzerland and United Kingdom.
EUROPEAN COMMITTEE FOR STANDARDIZATION
COMITÉ EUROPÉEN DE NORMALISATION
EUROPÄISCHES KOMITEE FÜR NORMUNG
Management Centre: rue de Stassart, 36 B-1050 Brussels
© 2008 CEN All rights of exploitation in any form and by any means reserved Ref. No. EN ISO 9241-304:2008: E
worldwide for CEN national Members.
Contents Page
Foreword .3
Foreword
This document (EN ISO 9241-304:2008) has been prepared by Technical Committee ISO/TC 159
"Ergonomics" in collaboration with Technical Committee CEN/TC 122 "Ergonomics", the secretariat of which
is held by DIN.
This European Standard shall be given the status of a national standard, either by publication of an identical
text or by endorsement, at the latest by May 2009, and conflicting national standards shall be withdrawn at the
latest by May 2009.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. CEN [and/or CENELEC] shall not be held responsible for identifying any or all such patent rights.
According to the CEN/CENELEC Internal Regulations, the national standards organizations of the following
countries are bound to implement this European Standard: Austria, Belgium, Bulgaria, Cyprus, Czech
Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia,
Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Romania, Slovakia, Slovenia, Spain,
Sweden, Switzerland and the United Kingdom.
Endorsement notice
The text of ISO 9241-304:2008 has been approved by CEN as a EN ISO 9241-304:2008 without any
modification.
INTERNATIONAL ISO
STANDARD 9241-304
First edition
2008-11-15
Ergonomics of human-system
interaction —
Part 304:
User performance test methods for
electronic visual displays
Ergonomie de l'interaction homme-système
Partie 304: Méthodes d'essai de la performance de l'utilisateur pour
écrans de visualisation électroniques
Reference number
ISO 9241-304:2008(E)
©
ISO 2008
ISO 9241-304:2008(E)
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but
shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In
the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.
© ISO 2008
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO 2008 – All rights reserved
ISO 9241-304:2008(E)
Contents Page
Foreword .iv
Introduction.vi
1 Scope.1
2 Normative references.1
3 Terms and definitions .2
4 Guiding principles .2
5 Conformance .2
6 Specifying the visual ergonomics test objectives .3
6.1 General .3
6.2 Criterion description .3
6.3 Measuring method.3
6.4 Performance criteria.3
7 Defining the test procedure.4
7.1 General .4
7.2 Alphanumeric and non-alphanumeric text .4
8 Visual performance and comfort test — Carrying out the test and analysing the data.5
8.1 General .5
8.2 Purpose .5
8.3 Overview.5
8.4 Test participants.6
8.5 The displays.6
8.6 Test setup.6
8.7 Dependent measures .11
8.8 Statistical treatment of results.12
8.9 Critical values for Barnard's U test.14
Annex A (informative) Overview of the ISO 9241 series .16
Bibliography.20
ISO 9241-304:2008(E)
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards bodies
(ISO member bodies). The work of preparing International Standards is normally carried out through ISO
technical committees. Each member body interested in a subject for which a technical committee has been
established has the right to be represented on that committee. International organizations, governmental and
non-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely with the
International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of technical committees is to prepare International Standards. Draft International Standards
adopted by the technical committees are circulated to the member bodies for voting. Publication as an
International Standard requires approval by at least 75 % of the member bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO shall not be held responsible for identifying any or all such patent rights.
ISO 9241-304 was prepared by Technical Committee ISO/TC 159, Ergonomics, Subcommittee SC 4,
Ergonomics of human-system interaction.
This first edition of ISO 9241-304, together with ISO 9241-302:2008, ISO 9241-303:2008, ISO 9241-305:2008
and ISO 9241-307:2008, cancels and replaces ISO 9241-3:1992, of which it constitutes a technical revision. It
also incorporates the Amendment ISO 9241-3:1992/Amd.1:2000, replacing that Amendment's test method
with the one specified in Clause 8.
ISO 9241 consists of the following parts, under the general title Ergonomic requirements for office work with
visual display terminals (VDTs):
⎯ Part 1: General introduction
⎯ Part 2: Guidance on task requirements
⎯ Part 4: Keyboard requirements
⎯ Part 5: Workstation layout and postural requirements
⎯ Part 6: Guidance on the work environment
⎯ Part 9: Requirements for non-keyboard input devices
⎯ Part 11: Guidance on usability
⎯ Part 12: Presentation of information
⎯ Part 13: User guidance
⎯ Part 14: Menu dialogues
⎯ Part 15: Command dialogues
⎯ Part 16: Direct manipulation dialogues
⎯ Part 17: Form filling dialogues
iv © ISO 2008 – All rights reserved
ISO 9241-304:2008(E)
ISO 9241 also consists of the following parts, under the general title Ergonomics of human-system interaction:
⎯ Part 20: Accessibility guidelines for information/communication technology (ICT) equipment and services
⎯ Part 110: Dialogue principles
⎯ Part 151: Guidance on World Wide Web user interfaces
⎯ Part 171: Guidance on software accessibility
⎯ Part 300: Introduction to electronic visual display requirements
⎯ Part 302: Terminology for electronic visual displays
⎯ Part 303: Requirements for electronic visual displays
⎯ Part 304: User performance test methods for electronic visual displays
⎯ Part 305: Optical laboratory test methods for electronic visual displays
⎯ Part 306: Field assessment methods for electronic visual displays
⎯ Part 307: Analysis and compliance test methods for electronic visual displays
⎯ Part 308: Surface-conduction electron-emitter displays (SED) [Technical Report]
⎯ Part 309: Organic light-emitting diode (OLED) displays [Technical Report]
⎯ Part 400: Principles and requirements for physical input devices
⎯ Part 410: Design criteria for physical input devices
⎯ Part 920: Guidance on tactile and haptic interactions
For the other parts under preparation, see Annex A.
ISO 9241-304:2008(E)
Introduction
ISO 9241 was originally developed as a seventeen-part International Standard on the ergonomics
requirements for office work with visual display terminals. As part of the standards review process, a major
restructuring of ISO 9241 was agreed to broaden its scope, to incorporate other relevant standards and to
make it more usable. The general title of the revised ISO 9241, “Ergonomics of human-system interaction”,
reflects these changes and aligns the standard with the overall title and scope of Technical Committee
ISO/TC 159, Subcommittee SC 4. The revised multipart standard is structured as series of standards
numbered in the “hundreds”: the 100 series deals with software interfaces, the 200 series with human centred
design, the 300 series with visual displays, the 400 series with physical input devices, and so on.
See Annex A for an overview of the entire ISO 9241 series.
ISO 9241-3:1992, Annex C, offered users a provisional alternative method for testing the visual quality of a
display, intended for novel display technologies for which no optical test method was available. The
Amendment ISO 9241-3:1992/Amd.1:2000 replaced this test method and made the previously informative
Annex C normative. ISO 9241-7:1998, ISO 9241-8:1997 and ISO 13406-2:2001 (all three of which have since
been cancelled and replaced by other parts of the ISO 9241 “300” subseries) referred to that Amendment as
providing an alternative user performance test method.
This part of ISO 9421 not only incorporates the Amendment, but extends its basis to provide guidance on the
general process of assessing the visual ergonomics of displays in a specific context of use by means of a user
performance test method. The test method specified in this part of ISO 9241 is applicable only to user tasks
involving the handling and processing of text. However, it is expected that test procedures will also be
developed for using maps and for handling and interpreting photographs and moving images, with these then
being incorporated into a future edition.
The structure of this part of ISO 9241 is an exception in the ISO 9241 “300” subseries in that it establishes the
conformance of a visual display used for text rendition according to its own user performance test method,
instead of by means of a compliance route given in ISO 9241-307 (in which no compliance route relevant to
this part of ISO 9241 is provided).
vi © ISO 2008 – All rights reserved
INTERNATIONAL STANDARD ISO 9241-304:2008(E)
Ergonomics of human-system interaction —
Part 304:
User performance test methods for electronic visual displays
1 Scope
This part of ISO 9241 provides guidance for assessing the visual ergonomics of display technologies with user
performance test methods (as opposed to the optical test methods given in ISO 9241-305). Its use will help to
ensure that, for a given context of use, a display meets minimum visual ergonomics requirements. It covers
only visual attributes and does not address the ergonomics or usability of the whole product that houses a
visual display.
The general principles laid down by this part of ISO 9241 apply to any colour or monochrome visual display
attached to a system with which human beings interact. This includes, but is not limited to, visual displays
used with desktop and portable computers, those used on mobile devices such as mobile telephones, digital
cameras and personal digital assistants, and status displays used on consumer electronics equipment such
as printers, in-car navigation systems and microwave ovens. It extends the basic idea of the visual
performance and comfort test specified in ISO 9241-3:1992/Amd.1:2000 to the use of the performance and
judgment of the display end users themselves for evaluating the quality of a display, and includes a more
diverse range of technologies, users, tasks and environments.
Because of this diversity, it is not feasible for this part of ISO 9241 to stipulate a single, generic test method
that can be used with all display technologies. Instead, the basic principles for generating a test method are
given. This method will be valid for evaluating specific displays in specific contexts of use: the method
generated according to Clause 8 is applicable only to tasks involving the handling and processing of text. No
other examples are given. An essential property of the process is that it permits the verification of the usability
of a visual display for a representative task, performed by representative users, taking their performance and
judgment as measured quality values. It does not, however, permit the measurement of specific perceptual
attributes such as luminance contrast or display flicker in isolation.
The main users of this part of ISO 9241 will be those who procure displays or who need to measure display
performance during product development. Its application assumes a background in behavioural science.
2 Normative references
The following referenced documents are indispensable for the application of this document. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced
document (including any amendments) applies.
ISO 9241-5, Ergonomic requirements for office work with visual display terminals (VDTs) — Part 5:
Workstation layout and postural requirements
ISO 9241-6, Ergonomic requirements for office work with visual display terminals (VDTs) — Part 6: Guidance
on the work environment
ISO 9241-302, Ergonomics of human-system interaction — Part 302: Terminology for electronic visual
displays
ISO 9241-304:2008(E)
ISO 9241-303:2008, Ergonomics of human-system interaction — Part 303: Requirements for electronic visual
displays
ISO/IEC 8859 (all parts), Information technology — 8-bit single-byte coded graphic character sets
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO 9241-302 apply.
4 Guiding principles
The guiding principles of this part of ISO 9241 are that visual displays should help people carry out their tasks
effectively and efficiently, and that displays should be satisfying to use and not in any way be harmful to their
users' health.
Formal optical test methods as specified in ISO 9241-305 might not be available to support the procurement
of newer display technologies. In such cases, requiring manufacturers to demonstrate evidence of the
usability of their displays provides the most effective route for ensuring good ergonomics quality. This is the
approach taken by this part of ISO 9241.
It sets out four steps for generating test methods that can be used to measure the ergonomics visual quality of
visual displays:
a) specify the visual ergonomic test objectives (see Clause 6);
b) define the test procedure (see Clause 7);
c) carry out the test (see Clause 8);
d) analyse the data (see Clause 8).
5 Conformance
Whereas ISO 9241-303 and ISO 9241-305 refer to the compliance routes defined in ISO 9241-307 to
establish the conformance of a visual display, this part of ISO 9241 itself specifies a test method for
establishing such conformance.
If the test display is compared to a benchmark display and the test procedure is based on either alphanumeric
or non-alphanumeric text, conformance is achieved when both
⎯ the search speed for the test display is not statistically significantly lower than the search speed obtained
with the benchmark display, and
⎯ the perceived quality, as measured by its visual comfort rating, of the test display is not statistically
significantly lower than that of the benchmark display.
The procedure used for determining search speed and perceived quality shall be in accordance with Clause 8.
2 © ISO 2008 – All rights reserved
ISO 9241-304:2008(E)
6 Specifying the visual ergonomics test objectives
6.1 General
Visual ergonomics can be measured like any other engineering attribute. Although the data from user
performance tests are derived from objective and subjective measures of human performance, this does not
mean that the data are simply personal opinions. A good test design will generate data that are objective and
unbiased. Useful information on many practical aspects of test design in general is given in ISO 20282;
ISO 20282-1, in particular, provides valuable background information in this area.
Testing only makes sense if the test results are compared with criteria that define a display as acceptable or
unacceptable. The aim of this step is to define those criteria for the display to be tested.
6.2 Criterion description
The criterion description defines the context of the measurements and the performance characteristics that
will be measured. In most cases — when, for instance, a novel display technology is used in a visual display
that helps perform an existing task (such as word processing in an office) — the visual quality of the test
display is assessed against that of a benchmark display known to meet or exceed the requirements of
ISO 9241-303, using a measuring method according to ISO 9241-305 and a conformance method according
to ISO 9241-307.
EXAMPLE A test of a display that will be used for in-car navigation might use as the criterion: “Ease of reading
information from the display when it is used by experienced drivers in bright ambient lighting”.
6.3 Measuring method
The measuring method describes how the criterion will be measured, i.e. the scale that will be used for the
measurement and how the values will be derived.
As an example, in ISO 9241-11, three separate measures are taken:
⎯ effectiveness (the accuracy and completeness with which customers achieve specified goals);
⎯ efficiency (the accuracy and completeness of goals achieved in relation to resources);
⎯ satisfaction (freedom from discomfort, and positive attitudes towards the use of the visual display).
It needs be realized that these three measures from ISO 9241-11 are context-dependent; this means that, for
example, the effectiveness of a mobile phone display could be low — even very low compared to that of a
desktop display — yet entirely satisfactory in the context of mobile phone use.
EXAMPLE A test of a display used on a mobile phone measures the accuracy with which a participant can
distinguish different colours (effectiveness), the speed with which a participant can read text on the display (efficiency) as
well as the participant's overall attitude towards the display's image quality (satisfaction).
6.4 Performance criteria
If the test display is compared to a benchmark display, the performance criterion is normally that the test
display have at least the same visual quality as the benchmark one. But in other cases, making a decision on
the value that is acceptable can require some market analysis. Useful questions to ask at this stage include
the following.
⎯ Is there an earlier version of the tested display that is relevant in this context? If so, how is its visual
quality rated?
⎯ How do competitors displays perform?
ISO 9241-304:2008(E)
These values provide the engineer with a lower limit to place on the performance of the display. Human
factors specialists recommend that the response range be considered as a continuum ranging from
“Unacceptable”, through a “Minimum” range into a “Target” range, and finally into an “Exceeds” range, as
follows.
a) Unacceptable
If the display performs within this range, it cannot be released.
b) Minimum
If the display performs within this range, it is barely acceptable. Management must weigh the benefits of
releasing a barely adequate display now, versus waiting for the usability defects to be fixed.
c) Target
If the display performs within this range, it can be released. This is the performance range thought to be
necessary to succeed.
d) Exceeds
If the display performs within this range, it could be that the development team have put too much effort
into the design of the display and/or developed a product surpassing requirements.
This approach makes it unlikely that the development team will either under- or over-engineer the display.
7 Defining the test procedure
7.1 General
The test procedure shall be oriented towards a task carried out with the help of a visual display: a user
performance test method as defined in this part of ISO 9241 relies on a user behaving as when performing a
typical task of this kind.
7.2 Alphanumeric and non-alphanumeric text
The visual performance and comfort test specified in Clause 8 may be used to establish the conformance of a
visual display to a certain quality, according to user performance, applicable to tasks involving the handling
and processing of text. Its test procedure is suitable for such tasks, typical of an office environment. Test
methods for other types of tasks, with their appropriate devices, remain to be developed.
NOTE 1 So far, no test procedure for using maps has been developed.
NOTE 2 So far, no test procedure for handling and interpreting photographs has been developed.
NOTE 3 So far, no test procedure for handling and interpreting moving images has been developed.
4 © ISO 2008 – All rights reserved
ISO 9241-304:2008(E)
8 Visual performance and comfort test — Carrying out the test and analysing
the data
8.1 General
This clause specifies a method that can be used to test the visual quality of electronic visual displays where
the entire set of physical requirements (viz. ISO 9241-303) cannot be applied — for example, in the case of
novel display technologies such as new types of flat panel display (e.g. electroluminescent or field emission
displays). The test consists of a letter search task and an assessment of visual comfort.
The context of use is text processing, such as that which takes place in office work. The combination of test
results will be referred to as the visual quality of the display.
The various stages of the test, such as recruiting participants, running a pilot test and administering the final
test are described (see also ISO/TS 20282-2), as is the analysis of the test data, including application of
statistics.
8.2 Purpose
The method tests displays that cannot otherwise be tested for conformance with ISO 9241. It is not an
alternative test method in the sense that a display manufacturer has the option of choosing either the physical
requirements described in ISO 9241-303 or this method. Instead, the test method provides a testing route for
displays that cannot be tested according to the requirements specified in ISO 9241-303 because the display
under test uses a new technology that makes metrological measurements such as those specified in
ISO 9241-305 difficult or impossible.
8.3 Overview
8.3.1 General
This test procedure measures the effectiveness of the transfer of visual information in terms of participants'
search performance for targets embedded in alphanumerics on a candidate display versus those same
participants' performance for such a task on a benchmark display. Effectiveness in this context means that the
user is able to detect and recognize the visual targets accurately, quickly and without visual discomfort. If the
display passes this particular visual search and rating performance test, it can be assumed capable of other
presentations of information such as, but not limited to, non-alphanumeric languages.
NOTE For a general background on this type of test methodology, see ISO/IEC 25062.
The dependent variables of the test are the search speed achieved by the test participants in a visual letter
search task and subjective ratings of visual comfort using a category scale. Testing takes place in a simulated
office environment, with test participants representative of the anticipated user population.
The method, i.e. a letter search task applying pseudo-text in combination with scaling of experienced visual
comfort, was first developed and tested by researchers of the centre for research on user-system interaction
[12], [13]
(IPO) .
The visual quality of a display, referred to as a test display, is assessed against a benchmark display known to
meet or exceed the requirements of ISO 9241-303. Both the speed in the visual search task and the
subjective ratings must meet certain minimum requirements for the test display to pass. Sequential statistics,
or an equally robust equivalent statistical procedure, are used to determine if the participant's performance on
the test display exceeds, is equal to, or falls short of, performance on the benchmark display.
ISO 9241-304:2008(E)
8.3.2 Avoidance of bias
All tests are open to bias, and this is especially true in the area of psychological testing. The assessment shall
therefore be carried out under the supervision of those qualified to carry out such testing, having the
necessary education and at least one year of experience. Rules governing the ethical conduct of human
experimental testing shall be followed. Examples of such rules can be found in References [14] and [15].
The test administrator should ensure that all potential sources of error are minimized or controlled. The
following list describes some potential sources of bias and error; it is not intended to be exhaustive.
⎯ Selection of test participants: for example, avoid selecting particular age groups.
⎯ Configuration of displays: during the test, the benchmark display shall meet all requirements of
ISO 9241-303.
⎯ Environmental conditions: lighting and other conditions shall be equal for both displays, in order to avoid
detrimental conditions for one of them.
⎯ Instructions to the test participants: these should be impartial.
8.4 Test participants
Test participants should be a sample representing the anticipated user population. The following assumes this
population to have unimpaired vision; for people with visual impairments using visual displays, other criteria
for required image quality apply, and other tests would need to be designed. All test participants shall have
visual acuity that is normal, or corrected to normal, at the design viewing distance, and they shall be without
any obvious physical or physiological conditions that could influence either their search performance or the
quality of the images that they perceive.
8.5 The displays
The test display shall be a production or full-feature pre-production unit. It shall incorporate any treatments
that would be in the production unit, for example, surface treatments such as anti-glare and anti-reflection
filters and treatments. The benchmark display shall be supplied or nominated by the supplier of the test
display and shall meet or exceed all requirements of ISO 9241-303; it shall have approximately the same size
as the test display.
The displays may be labelled for identification purposes: for example, as “Display 1” and “Display 2”. Under
these conditions, the test participants should not be informed as to which is the test and which the benchmark
display, with half the number of test participants first having “Display 1”, the test display, and the other half first
having “Display 2”, the benchmark display.
8.6 Test setup
8.6.1 General
The test shall be conducted in an area that is free from distractions and external interference that could
influence the test results. The ambient conditions shall fall within the range defined in ISO 9241-6. These
conditions shall be comfortable and shall not be subject to significant variation during the test, either within a
test participant's session or between test participants.
8.6.2 Test environment
The thermal environment, the background noise level, the ambient lighting and the reflectance of work
surfaces shall meet the minimum requirements set out in ISO 9241-5 and ISO 9241-6. The ambient
illumination shall be designed so as to minimize glare and specular reflections. Constant lighting conditions
shall be maintained both within a participant's session and between test participants. The test participants
shall be light-adapted by being placed in the test room for 10 min prior to the test.
NOTE This period can be used by the experimenter to instruct each test participant about the test.
6 © ISO 2008 – All rights reserved
ISO 9241-304:2008(E)
8.6.3 Test workstation
The display and associated equipment (e.g. the keyboard) shall be supported by a work surface in
accordance with ISO 9241-5.
For both the benchmark and test displays, the viewing distance shall be set according to the design viewing
distance. This distance should be constrained by a head-and-chin rest, the height of which is adjustable. The
individual height adjustments for the test participants should be such that for both the test display and the
benchmark display, the position of the eyes with respect to the display is equal for all test participants. The
position of the test participant's eyes shall comply with the line-of-sight angle requirements given in
ISO 9241-303:2008, 5.1.4.
The brightness and contrast settings of the benchmark display shall be specified by the manufacturer who
nominates the display: at these settings it shall meet or exceed all respective requirements given in
ISO 9241-303.
NOTE A measurement procedure for specifying brightness and contrast is given in ISO 9241-305.
At the option of the manufacturer, the brightness and contrast of the test display should be either
a) fixed at settings specified by the manufacturer, or
b) adjustable by test participants to their personal optimum settings.
Both displays shall be allowed to warm-up for at least 20 min prior to testing.
The test participant shall be seated in accordance with ISO 9241-5.
8.6.4 Test material
The test material shall be pseudo-text generated from a character set associated with an 8-bit single-byte
coded graphic character set as specified ISO/IEC 8859, which describes a collection of character sets for
various languages. If a system cannot display text in an alphabet familiar to the users, text should be
displayed by double-byte coded characters (e.g. Asian characters). In this case, the language used shall be
specified in the compliance statement. Each test will use a specified character subset (e.g. “A” to “Z”, “a” to
“z”, and “0” to “9”). The same subset shall be used for both displays.
Pseudo-text shall be generated from the character set according to the following constraints.
⎯ Pseudo-text shall consist of blocks of random strings of characters separated by spaces.
⎯ The texts, on both test and benchmark displays, shall consist of a constant number of lines and a
constant number of characters per line (including space characters).
⎯ The number of characters per line shall be chosen so that the line length (in centimetres) is less than
25 times the line-to-line distance (i.e. the height of the display area divided by the maximum number of
lines). However, a line should contain at least 30 characters (including embedded spaces). The total
number of characters in a pseudo-text shall be between 400 and 600, embedded spaces included. The
pseudo-text blocks shall be sized such that, if five blocks could be displayed at once (one in each corner
and one in the middle), they would have minimum overlap while maximizing coverage of the display area.
⎯ Each test participant shall be instructed to count the occurrences of a single target character over the
entire test (e.g. test participant X is instructed to search for “A”s during the entire test, test participant Y is
instructed to search for “R”s, etc.).
⎯ The number of targets shall be 2 % to 3 % of the total number of characters in the text, including
embedded spaces.
ISO 9241-304:2008(E)
⎯ The position of the targets shall be randomly chosen with the restriction that a line shall not start or end
with the target character.
⎯ The text shall contain a constant number of spaces. The space fraction shall be 15 % (i.e. the number of
spaces relative to the total number of characters, including embedded spaces).
NOTE Although the average word length does vary over different languages, pseudo-texts with 15 % space
fraction do resemble normal texts with respect to their string length distributions.
The position of the spaces shall be randomly chosen with the following restriction:
a) a line shall neither start nor end with a space character (all spaces are embedded);
b) a space character shall not be adjacent to another space character (strings are separated by single
spaces);
c) the minimum string length shall be two characters.
8.6.5 Test procedure
Display pseudo-text as a block of characters in one of five screen locations. The test participant's task is to
scan the text and identify each instance of the target character.
Place the blocks of pseudo-text in the upper left, upper right, lower left, lower right and centre of the screen.
Locate the centre block so that the middle character of the block is approximately in the centre of the active
area of the screen. Place text in each of the four corners so that it abuts the extreme corners of the screen.
Inform the test participants that the objective of the test is to evaluate the quality of the image on the display.
If, for the purposes of the experiment, the manufacturer of the test display has decided that the brightness and
contrast may be adjusted by test participants, give the test participants the opportunity to adjust the test
display to their preferred settings. Set the brightness and contrast settings of the benchmark display in
accordance with the manufacturer's instructions. This shall not be adjusted by the test participant.
Manufacturers should be aware that, if the user is allowed to adjust the display, this can give the user an
indication of the display under test and therefore could affect the results of the test. This may be prevented by
asking the user to adjust the settings before the test and then performing the test with the controls hidden from
view.
Present the five test blocks at the five locations in random order. Instruct each test participant to scan the
pseudo-text from the top to the bottom line and indicate each occurrence of the target character. In order to
overcome the problem of initial learning effects, train the test participants before the main experiment by
performing the task for at least 10 pseudo-texts (i.e. 10 trials). Residual learning shall be controlled by
counter-balancing the stimulus order within the main experiment. These practice trials shall use pseudo-text
placed in any of the five possible screen locations. Practice trials shall be presented on both test and
benchmark displays.
Continue practice trials until the test participant's performance on any one block of pseudo-text is error-free.
Do not use data collected from the practice trials to evaluate the quality of the display.
For the experimental trials, measure the time taken for the test participant to identify each instance of the
target character in each block of pseudo-text and the number of errors made by the test participant (see 8.8).
Allow the test participant a rest break of up to 1 min between trials, with a minimum break of 10 s.
Instruct test participants to respond by pressing predefined keys or buttons to: initiate a trial; count spotted
targets; and stop a trial.
8 © ISO 2008 – All rights reserved
ISO 9241-304:2008(E)
A keyboard or any other appropriate input device may be used for this purpose. If the keyboard is used, the
“ENTER” key should be defined to initiate/stop a trial, and the space bar should be defined to register spotted
targets.
Register the interval between initiation and stopping of a trial as the search time for this trial.
Instruct test participants to minimize errors as far as possible and yet work as quickly as possible. They shall
be instructed to minimize their error rate, regardless of the visual quality of the display under test; for example,
if the display has deteriorated in comparison to a previous one. It is very important to give the proper
instruction to the test participants in this respect, who then generally are well able to keep their error rate
constant and low.
Half of the test participants shall use the benchmark display first, and the other half shall use the test display
first.
On completion of the visual search task with a display, ask the test participants to rate the visual quality of that
display on a nine-point numerical scale (shown below), with 1 being “Poor” and 9 being “Excellent”. After
completion of the trials with the test display or the benchmark display, ask the test participants to assess the
perceptual quality of that display with respect to its visual comfort.
1 2 3 4 5 6 7 8 9
Poor Fair Excellent
The following written instructions shall be gi
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.
Loading comments...