ASTM E1499-16(2023)
(Guide)Standard Guide for Selection, Evaluation, and Training of Observers
Standard Guide for Selection, Evaluation, and Training of Observers
SIGNIFICANCE AND USE
5.1 The term appearance (see 3.2.1) implies the essential presence of human visual observations. The results of visual observation involve not only the step of observing, accomplished by the eye, but also the inseparable step of interpretation in the brain. Instrumental test methods currently cannot duplicate this second step, and therefore can now only approximate, but not fully measure, appearance. Such instrumental measures of appearance properties are useful only to the extent that they can be correlated to the results of visual observations by observers of the appearance phenomena being evaluated.
5.2 Almost invariably, too little attention has been paid to ensuring that the essential visual observations have been properly obtained to provide the basis for correlating visual and instrumental test results.
5.3 This guide provides the means for assessing observers, by outlining the requirements and tests for their selection, evaluation, and training. This guide should be useful to all experimenters designing or using visual test methods to provide either direct results in terms of the observation of appearance properties, or the experiments correlating such results with instrumental measures approximating the same appearance properties. The user is cautioned to avoid the substitution of validated vision tests with replicas of any kind, either printed, photographed or digitally displayed.
SCOPE
1.1 This guide describes criteria and tests for selecting, evaluating, and training human visual-sensory observers for tasks involving the perception and scaling of properties and phenomena relating to appearance.
1.2 Examples of tests requiring the use of trained observers include but are not limited to those described in the following ASTM standards: on color, Practice D1535 and Practice E1360; on color difference, Practice D1729 and Test Method D2616; on gloss, Test Method D4449; on metamerism, Practice D4086; and on setting tolerances, Practice D3134.
1.3 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility of the user of this standard to establish appropriate safety, health, and environmental practices and determine the applicability of regulatory limitations prior to use.
1.4 This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
General Information
Relations
Standards Content (Sample)
This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation: E1499 − 16 (Reapproved 2023)
Standard Guide for
Selection, Evaluation, and Training of Observers
This standard is issued under the fixed designation E1499; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope ences Between Surfaces of Similar Appearance
E284 Terminology of Appearance
1.1 This guide describes criteria and tests for selecting,
E1360 Practice for Specifying Color by Using the Optical
evaluating, and training human visual-sensory observers for
Society of America Uniform Color Scales System
tasks involving the perception and scaling of properties and
phenomena relating to appearance.
3. Terminology
1.2 Examples of tests requiring the use of trained observers
3.1 Definitions—Definitions of appearance terms in Termi-
include but are not limited to those described in the following
nology E284 are applicable to this guide.
ASTM standards: on color, Practice D1535 and Practice
3.2 Definitions of Terms Specific to This Standard:
E1360; on color difference, Practice D1729 and Test Method
3.2.1 appearance, n—in psychophysical studies, perception
D2616; on gloss, Test Method D4449; on metamerism, Prac-
in which the spectral and geometric aspects of a visual stimulus
tice D4086; and on setting tolerances, Practice D3134.
are integrated with its illuminating and viewing environment.
1.3 This standard does not purport to address all of the
3.2.2 observer, n—one who judges visually, qualitatively or
safety concerns, if any, associated with its use. It is the
quantitatively, the content of one or more appearance attributes
responsibility of the user of this standard to establish appro-
in each member of a set of objects or stimuli.
priate safety, health, and environmental practices and deter-
3.2.3 scale, v—to assess the content of one or more appear-
mine the applicability of regulatory limitations prior to use.
ance attributes in the members of a set of stimuli.
1.4 This international standard was developed in accor-
3.2.3.1 Discussion—Alternatively, scales may be deter-
dance with internationally recognized principles on standard-
mined by assessing the difference in content of an attribute
ization established in the Decision on Principles for the
with respect to the differences in that attribute among the
Development of International Standards, Guides and Recom-
members of the set.
mendations issued by the World Trade Organization Technical
Barriers to Trade (TBT) Committee.
4. Summary of Guide
4.1 This guide provides descriptions of techniques and tests
2. Referenced Documents
for the selection of candidates for observers for use in visual
2.1 ASTM Standards:
testing, for the evaluation of their capabilities in this field, and
D1535 Practice for Specifying Color by the Munsell System
for their training to enhance these capabilities.
D1729 Practice for Visual Appraisal of Colors and Color
4.2 Discussion is provided of precautions required for the
Differences of Diffusely-Illuminated Opaque Materials
efficient use of observers in visual tests, including avoidance of
D2616 Test Method for Evaluation of Visual Color Differ-
overtaxing the observers and the control of test variables.
ence With a Gray Scale
D3134 Practice for Establishing Color and Gloss Tolerances
4.3 Other considerations of test design, including the num-
D4086 Practice for Visual Evaluation of Metamerism
bers of observers and observations required and the precision
D4449 Test Method for Visual Evaluation of Gloss Differ-
of the visual results, are to be covered elsewhere.
5. Significance and Use
This guide is under the jurisdiction of ASTM Committee E12 on Color and
5.1 The term appearance (see 3.2.1) implies the essential
Appearance and is the direct responsibility of Subcommittee E12.11 on Visual
presence of human visual observations. The results of visual
Methods.
observation involve not only the step of observing, accom-
Current edition approved June 1, 2023. Published July 2023. Originally approved
plished by the eye, but also the inseparable step of interpreta-
in 1992. Last previous edition approved in 2016 as E1499 – 16. DOI: 10.1520/
E1499-23.
tion in the brain. Instrumental test methods currently cannot
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
duplicate this second step, and therefore can now only
contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
approximate, but not fully measure, appearance. Such instru-
Standards volume information, refer to the standard’s Document Summary page on
the ASTM website. mental measures of appearance properties are useful only to the
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E1499 − 16 (2023)
extent that they can be correlated to the results of visual value and chroma, and the observer’s failure is usually by wide
observations by observers of the appearance phenomena being margins in one or more limited regions of the hue circle. The
evaluated. presence of such abnormal results of the test should be grounds
for dismissing the candidate observer.
5.2 Almost invariably, too little attention has been paid to
ensuring that the essential visual observations have been
6.3 Visual Acuity and Discrimination Tests—Having deter-
properly obtained to provide the basis for correlating visual and
mined that the candidate observers have normal color vision, it
instrumental test results.
is next necessary to test their level of discrimination of small
differences in color or another appearance attribute of interest.
5.3 This guide provides the means for assessing observers,
by outlining the requirements and tests for their selection,
6.3.1 Farnsworth-Munsell 100 Hue Test—Use of the
evaluation, and training. This guide should be useful to all Farnsworth-Munsell 100 Hue Test as a color-discrimination
experimenters designing or using visual test methods to pro-
test does not require readministration of the test, but merely
vide either direct results in terms of the observation of
reexamination of the test results. For the purposes of assessing
appearance properties, or the experiments correlating such
color (more precisely, hue) discrimination, the test results are
results with instrumental measures approximating the same
examined for the presence of an approximately constant but
appearance properties. The user is cautioned to avoid the
significant error level in the arrangement of the test chips
substitution of validated vision tests with replicas of any kind,
throughout the hue circle. This may be interpreted as an
either printed, photographed or digitally displayed.
inability to discriminate the small color differences between
neighboring chips. While a weakness of this type might, for
6. Selection and Evaluation of Observers
example, interfere with an observer’s ability to participate in
threshold scaling experiments, the observer might still be
6.1 The process used for selecting observers depends a great
competent to perform magnitude scaling of larger differences
deal upon the type of experiment being carried out, but should
among specimens.
essentially evaluate the potential capability of the observer to
execute a series of visual evaluation tasks (1, 2). When these
6.3.2 Triangle Test—This test is part of a series known as
tasks involve appearance attributes, color or related spectral the Japanese Color Aptitude Test. The candidate observers are
phenomena are often among the task subjects, and if instead
shown, one at a time, a series of 20 sets of three colored chips
geometric phenomena such as gloss are involved, many of the
each. In each set, two of the chips are identical and the third is
same considerations apply. Accordingly, the emphasis in this
slightly different in color. The observer is asked to identify
guide is upon selecting observers for color-related measure-
which one is different, the differences being so small that there
ments. Thus, observers must be screened to rule out those with
is considerable uncertainty in the judgment. A lower than
any color- or task-oriented deficiencies.
average score in this test indicates that the observer does not
differentiate small differences well.
6.2 Color Vision Tests (3):
6.2.1 Pseudoisochromatic Plates—As a preliminary color
6.4 Magnitude Scaling Tests:
vision test, a pseudoisochromatic-plate test should be admin-
6.4.1 Length Estimation—A simple magnitude-scaling test
istered to the candidate observers. The instructions and scoring
may be devised to familiarize the observer with scaling
techniques supplied by the manufacturer should be followed.
procedures and the experimental task of matching a given
In particular, the illumination level should be kept well within
anchor scale with a perceived difference in stimuli. In an
the photopic range (1000 1x is recommended as a minimum
example (1), the observer was asked to judge the apparent
value) and the spectral quality of the illuminating source
length of a line in comparison to the length of a standard line.
should be near that of north-sky daylight. Failure to identify
The lines were drawn with a heavy black marker on 100 mm
correctly the required number of the plates in the test used
by 150 mm index cards, one line to each of 21 cards. The
should be considered grounds for dismissing the candidate
standard or anchor line, 125 mm long, was assigned a value of
observer.
10 units of length. The other 20 cards had lines of various
6.2.2 Farnsworth-Munsell 100 Hue Test—The Farnsworth-
lengths, both longer and shorter than the anchor line. The
Munsell 100 Hue Test (4) should next be administered to the
anchor and one test card were displayed side by side at a
candidate. While the pseudoisochromatic-plate tests isolate
distance of 0.6 m. Of course, no rulers or other aids were
certain factors of color deficiency, the Farnsworth-Munsel
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.