ASTM E3000-18
(Guide)Standard Guide for Measuring and Tracking Performance of Assessors on a Descriptive Sensory Panel
Standard Guide for Measuring and Tracking Performance of Assessors on a Descriptive Sensory Panel
SIGNIFICANCE AND USE
5.1 This guide is meant to be used with and applied to individual trained descriptive assessors.
5.2 The procedures recommended in this guide can be used by the panel leader to periodically appraise the performance of individual descriptive assessors.
5.3 Tracking assessor performance will provide information as to the quality of the data being generated. Performance information may be used to decide whether to use the data to interpret product profiles.
5.4 Monitoring assessor performance will enable the panel leader to identify retraining needs or to identify assessors who are not performing well enough to continue participating on a panel.
SCOPE
1.1 This guide provides guidelines for measuring and tracking the performance of individual assessors on a descriptive sensory panel.
1.2 This guide provides guidelines to assist sensory professionals in measuring performance for given assessors. Measuring performance will form the basis for (1) determining the reliability of the results, and (2) establishing remedial actions for an individual assessor.
1.3 This guide examines various aspects of trained assessor performance; such as repeatability, discrimination, and agreement and demonstrates some ways to measure them. The procedures will help the sensory professional determine areas of good performance as well as those that require improvement.
1.4 Individual assessor performance is tracked using established statistical procedures. These procedures depend on whether replicates are collected and if they are collected over multiple sessions or within a single session.
1.5 This guide provides suggested procedures, including statistical procedures that can be done using standard statistical software, for evaluating performance and is not meant to exclude other methods that may be effectively used for a similar purpose.
1.6 Methods for training and screening assessors are not within the scope of this guide. This guide does not address how to communicate performance feedback information to individual assessors. This monitoring of panel reproducibility, a measure of the panel’s ability to reproduce the results of other panels, is also not within the scope of this guide.
1.7 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility of the user of this standard to establish appropriate safety, health, and environmental practices and determine the applicability of regulatory limitations prior to use.
1.8 This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
General Information
Relations
Buy Standard
Standards Content (Sample)
This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation: E3000 − 18
Standard Guide for
Measuring and Tracking Performance of Assessors on a
1
Descriptive Sensory Panel
This standard is issued under the fixed designation E3000; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope priate safety, health, and environmental practices and deter-
mine the applicability of regulatory limitations prior to use.
1.1 This guide provides guidelines for measuring and track-
1.8 This international standard was developed in accor-
ing the performance of individual assessors on a descriptive
dance with internationally recognized principles on standard-
sensory panel.
ization established in the Decision on Principles for the
1.2 This guide provides guidelines to assist sensory profes-
Development of International Standards, Guides and Recom-
sionals in measuring performance for given assessors. Measur-
mendations issued by the World Trade Organization Technical
ing performance will form the basis for (1) determining the
Barriers to Trade (TBT) Committee.
reliability of the results, and (2) establishing remedial actions
for an individual assessor.
2. Referenced Documents
2
1.3 This guide examines various aspects of trained assessor 2.1 ASTM Standards:
performance; such as repeatability, discrimination, and agree-
E253 Terminology Relating to Sensory Evaluation of Mate-
ment and demonstrates some ways to measure them. The rials and Products
procedures will help the sensory professional determine areas
E456 Terminology Relating to Quality and Statistics
2
of good performance as well as those that require improve-
2.2 Other Documents:
ment.
ASTMSTP758 GuidelinesfortheSelectionandTrainingof
Sensory Panel Members
1.4 Individual assessor performance is tracked using estab-
ASTM MNL13 Manual on DescriptiveAnalysis for Sensory
lished statistical procedures. These procedures depend on
Evaluation
whether replicates are collected and if they are collected over
3
multiple sessions or within a single session. 2.3 ISO Standards:
ISO 11132:2012 Sensory Analysis – Methodology—Guide-
1.5 This guide provides suggested procedures, including
lines for Monitoring the Performance of a Quantitative
statisticalproceduresthatcanbedoneusingstandardstatistical
Sensory Panel
software, for evaluating performance and is not meant to
exclude other methods that may be effectively used for a
3. Terminology
similar purpose.
3.1 Please refer to Terminologies E253 and E456, ASTM
1.6 Methods for training and screening assessors are not
STP 758, ASTM MNL13 and ISO 11132:2012 for any terms
withinthescopeofthisguide.Thisguidedoesnotaddresshow
related to assessor performance that are not listed below.
to communicate performance feedback information to indi-
3.2 Definitions:
vidual assessors. This monitoring of panel reproducibility, a
3.2.1 agreement—ability of an assessor to give similar
measure of the panel’s ability to reproduce the results of other
scores (rate) or to order the intensity of stimuli similarly to the
panels, is also not within the scope of this guide.
rest of the panel (rank) on a given attribute.
1.7 This standard does not purport to address all of the
3.2.2 performance—ability of an assessor to make repeat-
safety concerns, if any, associated with its use. It is the
able assessments that are in agreement with other assessors on
responsibility of the user of this standard to establish appro-
2
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
1
This guide is under the jurisdiction of ASTM Committee E18 on Sensory contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
Evaluation and is the direct responsibility of Subcommittee E18.03 on Sensory Standards volume information, refer to the standard’s Document Summary page on
Theory and Statistics. the ASTM website.
3
Current edition approved April 1, 2018. Published April 2018. Originally Available from International Organization for Standardization (ISO), ISO
approved in 2017. Last previous edition approved in 2017 as E3000 – 17. DOI: Central Secretariat, BIBC II, Chemin de Blandonnet 8, CP 401, 1214 Vernier,
10.1520/E3000-18. Geneva, Switzerland, http://www.iso.org.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
1
---------------------- Page: 1 ----------------------
E3000 − 18
the panel and discriminate perceptible differences between criteriafortheindividualass
...
This document is not an ASTM standard and is intended only to provide the user of an ASTM standard an indication of what changes have been made to the previous version. Because
it may not be technically possible to adequately depict all changes accurately, ASTM recommends that users consult prior editions as appropriate. In all cases only the current version
of the standard as published by ASTM is to be considered the official document.
Designation: E3000 − 17 E3000 − 18
Standard Guide for
Measuring and Tracking Performance of Assessors on a
1
Descriptive Sensory Panel
This standard is issued under the fixed designation E3000; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope
1.1 This guide provides guidelines for measuring and tracking the performance of individual assessors on a descriptive sensory
panel.
1.2 This guide provides guidelines to assist sensory professionals in measuring performance for given assessors. Measuring
performance will form the basis for (1) determining the reliability of the results, and (2) establishing remedial actions for an
individual assessor.
1.3 This guide examines various aspects of trained assessor performance; such as repeatability, discrimination, and agreement
and demonstrates some ways to measure them. The procedures will help the sensory professional determine areas of good
performance as well as those that require improvement.
1.4 Individual assessor performance is tracked using established statistical procedures. These procedures depend on whether
replicates are collected and if they are collected over multiple sessions or within a single session.
1.5 This guide provides suggested procedures, including statistical procedures that can be done using standard statistical
software, for evaluating performance and is not meant to exclude other methods that may be effectively used for a similar purpose.
1.6 Methods for training and screening assessors are not within the scope of this guide. This guide does not address how to
communicate performance feedback information to individual assessors. This monitoring of panel reproducibility, a measure of the
panel’s ability to reproduce the results of other panels, is also not within the scope of this guide.
1.7 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility
of the user of this standard to establish appropriate safety, health, and environmental practices and determine the applicability of
regulatory limitations prior to use.
1.8 This international standard was developed in accordance with internationally recognized principles on standardization
established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued
by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
2. Referenced Documents
2
2.1 ASTM Standards:
E253 Terminology Relating to Sensory Evaluation of Materials and Products
E456 Terminology Relating to Quality and Statistics
2
2.2 Other Documents:
ASTM STP 758 Guidelines for the Selection and Training of Sensory Panel Members
ASTM MNL13 Manual on Descriptive Analysis for Sensory Evaluation
3
2.3 ISO Standards:
ISO 11132:2012 Sensory Analysis – Methodology—Guidelines for Monitoring the Performance of a Quantitative Sensory Panel
1
This guide is under the jurisdiction of ASTM Committee E18 on Sensory Evaluation and is the direct responsibility of Subcommittee E18.03 on Sensory Theory and
Statistics.
Current edition approved Nov. 1, 2017April 1, 2018. Published December 2017April 2018. Originally approved in 2017. Last previous edition approved in 2017 as
E3000 – 17. DOI: 10.1520/E3000-17.10.1520/E3000-18.
2
For referenced ASTM standards, visit the ASTM website, www.astm.org, or contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM Standards
volume information, refer to the standard’s Document Summary page on the ASTM website.
3
Available from International Organization for Standardization (ISO), ISO Central Secretariat, BIBC II, Chemin de Blandonnet 8, CP 401, 1214 Vernier, Geneva,
Switzerland, http://www.iso.org.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
1
---------------------- Page: 1 ----------------------
E3000 − 18
3. Terminology
3.1 Please refer to Terminologies E253 and E456, ASTM STP 758,ASTM MNL13 and ISO 11132:2012 for any terms related to
assessor performance that are not listed below.
3.2 Definitions:
3.2.1 agreement—ability of an assessor to give similar scores (rate) or to order the intensity of stimuli similarly to the rest of
the panel (rank) on a given attribute.
3.2.2 performance—ability of an
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.