ASTM E3000-17
(Guide)Standard Guide for Measuring and Tracking Performance of Assessors on a Descriptive Sensory Panel
Standard Guide for Measuring and Tracking Performance of Assessors on a Descriptive Sensory Panel
SIGNIFICANCE AND USE
5.1 This guide is meant to be used with and applied to individual trained descriptive assessors.
5.2 The procedures recommended in this guide can be used by the panel leader to periodically appraise the performance of individual descriptive assessors.
5.3 Tracking assessor performance will provide information as to the quality of the data being generated. Performance information may be used to decide whether to use the data to interpret product profiles.
5.4 Monitoring assessor performance will enable the panel leader to identify retraining needs or to identify assessors who are not performing well enough to continue participating on a panel.
SCOPE
1.1 This guide provides guidelines for measuring and tracking the performance of individual assessors on a descriptive sensory panel.
1.2 This guide provides guidelines to assist sensory professionals in measuring performance for given assessors. Measuring performance will form the basis for (1) determining the reliability of the results, and (2) establishing remedial actions for an individual assessor.
1.3 This guide examines various aspects of trained assessor performance; such as repeatability, discrimination, and agreement and demonstrates some ways to measure them. The procedures will help the sensory professional determine areas of good performance as well as those that require improvement.
1.4 Individual assessor performance is tracked using established statistical procedures. These procedures depend on whether replicates are collected and if they are collected over multiple sessions or within a single session.
1.5 This guide provides suggested procedures, including statistical procedures that can be done using standard statistical software, for evaluating performance and is not meant to exclude other methods that may be effectively used for a similar purpose.
1.6 Methods for training and screening assessors are not within the scope of this guide. This guide does not address how to communicate performance feedback information to individual assessors. This monitoring of panel reproducibility, a measure of the panel’s ability to reproduce the results of other panels, is also not within the scope of this guide.
1.7 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility of the user of this standard to establish appropriate safety, health, and environmental practices and determine the applicability of regulatory limitations prior to use.
1.8 This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
General Information
Relations
Standards Content (Sample)
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Contact ASTM International (www.astm.org) for the latest information
Designation: E3000 − 17
Standard Guide for
Measuring and Tracking Performance of Assessors on a
1
Descriptive Sensory Panel
This standard is issued under the fixed designation E3000; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope 1.8 This international standard was developed in accor-
dance with internationally recognized principles on standard-
1.1 This guide provides guidelines for measuring and track-
ization established in the Decision on Principles for the
ing the performance of individual assessors on a descriptive
Development of International Standards, Guides and Recom-
sensory panel.
mendations issued by the World Trade Organization Technical
1.2 This guide provides guidelines to assist sensory profes-
Barriers to Trade (TBT) Committee.
sionals in measuring performance for given assessors. Measur-
ing performance will form the basis for (1) determining the
2. Referenced Documents
reliability of the results, and (2) establishing remedial actions
2
2.1 ASTM Standards:
for an individual assessor.
E253 Terminology Relating to Sensory Evaluation of Mate-
1.3 This guide examines various aspects of trained assessor
rials and Products
performance; such as repeatability, discrimination, and agree-
E456 Terminology Relating to Quality and Statistics
ment and demonstrates some ways to measure them. The 2
2.2 Other Documents:
procedures will help the sensory professional determine areas
ASTMSTP758 GuidelinesfortheSelectionandTrainingof
of good performance as well as those that require improve-
Sensory Panel Members
ment.
ASTM MNL13 Manual on DescriptiveAnalysis for Sensory
1.4 Individual assessor performance is tracked using estab- Evaluation
3
lished statistical procedures. These procedures depend on
2.3 ISO Standards:
whether replicates are collected and if they are collected over
ISO 11132:2012 Sensory Analysis – Methodology—Guide-
multiple sessions or within a single session.
lines for Monitoring the Performance of a Quantitative
Sensory Panel
1.5 This guide provides suggested procedures, including
statisticalproceduresthatcanbedoneusingstandardstatistical
3. Terminology
software, for evaluating performance and is not meant to
exclude other methods that may be effectively used for a
3.1 Please refer to Terminologies E253 and E456, ASTM
similar purpose. STP 758, ASTM MNL13 and ISO 11132:2012 for any terms
related to assessor performance that are not listed below.
1.6 Methods for training and screening assessors are not
withinthescopeofthisguide.Thisguidedoesnotaddresshow 3.2 Definitions:
to communicate performance feedback information to indi-
3.2.1 agreement—ability of an assessor to give similar
vidual assessors. This monitoring of panel reproducibility, a scores (rate) or to order the intensity of stimuli similarly to the
measure of the panel’s ability to reproduce the results of other
rest of the panel (rank) on a given attribute.
panels, is also not within the scope of this guide.
3.2.2 performance—ability of an assessor to make repeat-
1.7 This standard does not purport to address all of the able assessments that are in agreement with other assessors on
safety concerns, if any, associated with its use. It is the the panel and discriminate perceptible differences between
responsibility of the user of this standard to establish appro- attributes when they are present.
priate safety, health, and environmental practices and deter-
mine the applicability of regulatory limitations prior to use.
2
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
1
This guide is under the jurisdiction of ASTM Committee E18 on Sensory Standards volume information, refer to the standard’s Document Summary page on
Evaluation and is the direct responsibility of Subcommittee E18.03 on Sensory the ASTM website.
3
Theory and Statistics. Available from International Organization for Standardization (ISO), ISO
Current edition approved Nov. 1, 2017. Published December 2017. DOI: Central Secretariat, BIBC II, Chemin de Blandonnet 8, CP 401, 1214 Vernier,
10.1520/E3000-17. Geneva, Switzerland, http://www.iso.org.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
1
---------------------- Page: 1 ----------------------
E3000 − 17
3.2.3 scale usage—the extent to which the assessor(s) uses preventing overall discrimination between samples. In some
the scale with respect to the intensities of the attributes being cases, a poorly performing assessor can cause p
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.