ASTM E2610-18

Standard Test Method for Sensory Analysis—Duo-Trio Test

SIGNIFICANCE AND USE
5.1 The test method is effective for the following test objectives:  
5.1.1 To determine whether a perceivable difference results or a perceivable difference does not result, for example, when a change is made in ingredients, processing, packaging, handling or storage; or  
5.1.2 To select, train and monitor assessors.  
5.2 The test method itself does not change whether the purpose of the duo-trio test is to determine that two products are perceivably different versus that the products are not perceivably different. Only the selected values of pd, α, and β change. If the objective of the test is to determine if there is a perceivable difference between two products, then the value selected for α is typically smaller than the value selected for β. If the objective is to determine if the two products are sufficiently similar to be used interchangeably, then the value selected for β is typically smaller than the value selected for α and the value of pd is selected to define “sufficiently similar.”  
5.3 The test method may change based on the test objective or the assessors’ familiarity with the product. The balanced-reference technique (see 9.1.1) typically is used when neither product is more familiar than the other. The constant-reference technique (see 9.1.2) frequently is used when one product is a control/current product or is familiar to the assessors.
SCOPE
1.1 This test method covers a procedure for determining whether a perceptible sensory difference exists between samples of two products.  
1.2 This test method applies whether a difference may exist in a single sensory attribute or in several.  
1.3 This test method is applicable when the nature of the difference between the samples is unknown. It does not determine the size or the direction of the difference. The attribute(s) responsible for the difference are not identified.  
1.4 Compared to the triangle test, the duo-trio test is statistically less efficient, but easier to perform by the assessors. For details on how the duo-trio test compares to other three-sample tests, see Refs (1-4).2  
1.5 This test method is applicable only if the products are homogeneous. If two samples of the same product can often be distinguished, then another method, for example, descriptive analysis, may be more appropriate.  
1.6 This test method is applicable only when the products do not cause excessive sensory fatigue, carryover or adaptation.  
1.7 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility of the user of this standard to establish appropriate safety, health, and environmental practices and determine the applicability of regulatory limitations prior to use.  
1.8 This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.

General Information

• Status: Published
• Publication Date: 31-Jul-2018
• ICS: 67.240 - Sensory analysis
• Technical Committee: E18 - Sensory Evaluation
• Drafting Committee: E18.04 - Test Methods

Relations

• Replaces: ASTM E2610-08(2011) - Standard Test Method for Sensory Analysis—Duo-Trio Test
• Effective Date: 01-Aug-2018
• Refers: ASTM E456-13a(2022)e1 - Standard Terminology Relating to Quality and Statistics
• Effective Date: 01-Apr-2022
• Refers: ASTM E253-19 - Standard Terminology Relating to Sensory Evaluation of Materials and Products
• Effective Date: 15-Oct-2019
• Refers: ASTM E253-18a - Standard Terminology Relating to Sensory Evaluation of Materials and Products
• Effective Date: 01-Oct-2018
• Refers: ASTM E1885-18 - Standard Test Method for Sensory Analysis—Triangle Test
• Effective Date: 15-Aug-2018
• Refers: ASTM E253-18 - Standard Terminology Relating to Sensory Evaluation of Materials and Products
• Effective Date: 15-Jun-2018
• Refers: ASTM E456-13A(2017)e3 - Standard Terminology Relating to Quality and Statistics
• Effective Date: 01-Oct-2017
• Refers: ASTM E456-13A(2017)e1 - Standard Terminology Relating to Quality and Statistics
• Effective Date: 01-Oct-2017
• Refers: ASTM E1871-17 - Standard Guide for Serving Protocol for Sensory Evaluation of Foods and Beverages
• Effective Date: 01-Sep-2017
• Refers: ASTM E253-17 - Standard Terminology Relating to Sensory Evaluation of Materials and Products
• Effective Date: 01-May-2017
• Refers: ASTM E253-16 - Standard Terminology Relating to Sensory Evaluation of Materials and Products
• Effective Date: 01-Jun-2016
• Refers: ASTM E253-15b - Standard Terminology Relating to Sensory Evaluation of Materials and Products
• Effective Date: 01-Dec-2015
• Refers: ASTM E253-15a - Standard Terminology Relating to Sensory Evaluation of Materials and Products
• Effective Date: 01-Jun-2015
• Refers: ASTM E253-15 - Standard Terminology Relating to Sensory Evaluation of Materials and Products
• Effective Date: 15-Jan-2015
• Refers: ASTM E456-13ae3 - Standard Terminology Relating to Quality and Statistics
• Effective Date: 15-Nov-2013

Overview

ASTM E2610-18: Standard Test Method for Sensory Analysis-Duo-Trio Test provides a standardized procedure for determining whether a perceptible sensory difference exists between samples of two products. This method is widely used in food, beverage, personal care, and consumer goods industries to assess changes in ingredients, processing, packaging, handling, or storage. The duo-trio test is designed to identify whether a difference can be detected, not to measure the size or direction of any difference or to pinpoint the responsible attribute.

The method focuses on providing reliable, reproducible results to support quality control, product development, and consistency in sensory analysis. It prescribes procedures for sample presentation, assessor selection, testing environment, and result analysis, maintaining internationally recognized principles of standardization.

Key Topics

• Purpose: To detect if a perceptible sensory difference exists between two product samples, whether in a single attribute or multiple attributes.
• Panelist Selection: Guidance on using trained or untrained assessors depending on the test objective.
• Test Design: Utilizes a triad of samples-one reference and two coded samples (one matching the reference)-to determine similarity or difference.
• Objective-Driven Sensitivity: Test settings (number of assessors, α-risk, β-risk, proportion of distinguishers) are adjusted to fit specific objectives, such as confirming differences or demonstrating product similarity.
• Testing Techniques:
  • Balanced-reference technique: Used when neither sample is more familiar to assessors.
  • Constant-reference technique: Preferred when one product is a known control or familiar to assessors.
• Limitations: Applicable only to homogeneous products and where sensory fatigue or adaptation does not impair the test.
• Statistical Analysis: Interpretation is based on the number of correct responses compared against significance thresholds for α and β errors.
• Reporting: Documentation should include test objectives, sample details, methodology, assessor information, environment, and statistical results to ensure transparency and repeatability.

Applications

The ASTM E2610-18 duo-trio test delivers practical value through various sensory evaluation applications:

• Quality Control: Detects unintended changes in product formulation, ingredient sourcing, packaging, or processing that may impact perceptible qualities.
• Product Development: Assists R&D teams in validating new processes or ingredient substitutions by verifying that product differences are or are not detectable by sensory means.
• Shelf Life and Packaging Studies: Evaluates the sensory effects of different packaging materials or storage durations on product acceptability and consistency.
• Panel Training and Monitoring: Functions as a tool to select, train, and periodically monitor the performance of sensory panelists, ensuring ongoing accuracy and reliability.
• Consumer Studies: While primarily for analytical testing, results can inform broader consumer research related to sensory perception and preference.

Related Standards

The duo-trio test method is part of a broader set of sensory analysis protocols. Key related standards include:

• ASTM E253: Terminology Relating to Sensory Evaluation of Materials and Products
• ASTM E456: Terminology Relating to Quality and Statistics
• ASTM E1871: Guide for Serving Protocol for Sensory Evaluation of Foods and Beverages
• ASTM E1885: Standard Test Method for Sensory Analysis-Triangle Test
• ASTM E2262: Practice for Estimating Thurstonian Discriminal Distances
• ISO 4120: Sensory Analysis-Methodology-Triangle Test
• ISO 10399: Sensory Analysis-Methodology-Duo-Trio Test

Conclusion

ASTM E2610-18 is a foundational sensory analysis standard for determining perceptible differences between products, with wide application in manufacturing, product development, and quality assurance. By standardizing the duo-trio test procedure, the standard helps organizations ensure product consistency, support claims of similarity or difference, and maintain robust sensory evaluation programs, all while aligning with internationally recognized best practices.

Standards Content (Sample)

This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation: E2610 − 18
Standard Test Method for
Sensory Analysis—Duo-Trio Test
This standard is issued under the fixed designation E2610; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope 2. Referenced Documents
1.1 This test method covers a procedure for determining 2.1 ASTM Standards:
whether a perceptible sensory difference exists between E253 Terminology Relating to Sensory Evaluation of Mate-
samples of two products. rials and Products
E456 Terminology Relating to Quality and Statistics
1.2 This test method applies whether a difference may exist
E1871 Guide for Serving Protocol for Sensory Evaluation of
in a single sensory attribute or in several.
Foods and Beverages
1.3 This test method is applicable when the nature of the E1885 Test Method for Sensory Analysis—Triangle Test
difference between the samples is unknown. It does not
E2262 Practice for Estimating Thurstonian Discriminal Dis-
determine the size or the direction of the difference. The tances
attribute(s) responsible for the difference are not identified. 4
2.2 ISO Standards:
ISO 4120 Sensory Analysis—Methodology—Triangle Test
1.4 Compared to the triangle test, the duo-trio test is
ISO 10399 Sensory Analysis—Methodology—Duo-Trio
statistically less efficient, but easier to perform by the asses-
Test
sors. For details on how the duo-trio test compares to other
three-sample tests, see Refs (1-4).
3. Terminology
1.5 This test method is applicable only if the products are
homogeneous. If two samples of the same product can often be 3.1 Definitions—For definition of terms relating to sensory
analysis, see Terminology E253, and for terms relating to
distinguished, then another method, for example, descriptive
analysis, may be more appropriate. statistics, see Terminology E456.
3.2 Definitions of Terms Specific to This Standard:
1.6 This test method is applicable only when the products
3.2.1 α (alpha) risk—probability of concluding that a per-
do not cause excessive sensory fatigue, carryover or adapta-
tion. ceptible difference exists when, in reality, one does not. (Also
known as Type I Error or significance level.)
1.7 This standard does not purport to address all of the
safety concerns, if any, associated with its use. It is the 3.2.2 β (beta) risk—probability of concluding that no per-
responsibility of the user of this standard to establish appro- ceptible difference exists when, in reality, one does. (Also
priate safety, health, and environmental practices and deter-
known as Type II Error.)
mine the applicability of regulatory limitations prior to use.
3.2.3 p —probability of a correct response.
c
1.8 This international standard was developed in accor-
3.2.4 p (proportion of discriminators)—proportion of the
d
dance with internationally recognized principles on standard-
population represented by the assessors that can distinguish
ization established in the Decision on Principles for the
between the two products.
Development of International Standards, Guides and Recom-
mendations issued by the World Trade Organization Technical
3.2.5 product—material to be evaluated.
Barriers to Trade (TBT) Committee.
3.2.6 sample—unit of product prepared, presented, and
evaluated in the test.
This test method is under the jurisdiction ofASTM Committee E18 on Sensory
Evaluation and is the direct responsibility of Subcommittee E18.04 on Fundamen-
tals of Sensory. For referenced ASTM standards, visit the ASTM website, www.astm.org, or
Current edition approved Aug. 1, 2018. Published August 2018. Originally contact ASTM Customer Service at [e-mail]. For Annual Book of ASTM
approved in 2008. Last previous edition approved in 2011 as E2610 – 08 (2011). Standards volume information, refer to the standard’s Document Summary page on
DOI: 10.1520/E2610-18. the ASTM website.
2 4
The boldface numbers in parentheses refer to the list of references at the end of Available fromAmerican National Standards Institute (ANSI), 25 W. 43rd St.,
this standard. 4th Floor, New York, NY 10036, http://www.ansi.org.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E2610 − 18
3.2.7 sensitivity—general term used to summarize the per- selected for α is typically smaller than the value selected for β.
formance characteristics of the test. The sensitivity of the test If the objective is to determine if the two products are
is rigorously defined, in statistical terms, by the values selected sufficiently similar to be used interchangeably, then the value
for α, β, and p . selected for β is typically smaller than the value selected for α
d
and the value of p is selected to define “sufficiently similar.”
d
3.2.8 δ—Thurstonian measure of sensory difference (effect
size) relative to perceptual noise (standard deviation) (see
5.3 The test method may change based on the test objective
Practice E2262).
or the assessors’ familiarity with the product. The balanced-
reference technique (see 9.1.1) typically is used when neither
3.2.9 triad—three samples given to an assessor in the
product is more familiar than the other. The constant-reference
duo-trio test; one sample is labeled as a reference the other two
technique (see 9.1.2) frequently is used when one product is a
samples are labeled with different codes. One of the coded
control/current product or is familiar to the assessors.
samples is the same product as the reference. The other coded
sample is different.
6. Apparatus
4. Summary of Test Method
6.1 Carry out the test under conditions that prevent contact
4.1 Clearly define the test objective in writing.
between assessors until the evaluations have been completed,
for example, using booths that comply with Ref (5).
4.2 Choose the number of assessors based on the level of
sensitivity desired for the test. The sensitivity of the test is, in
6.2 Sample preparation and serving sizes should comply
part, a function of two competing risks: the risk of declaring
with Guide E1871. See Refs (6) or (7).
the samples different when they are not (that is, α-risk) and the
risk of not declaring the samples different when they are (that 7. Assessors
is, β-risk).Acceptable values of α and β vary depending on the
7.1 All assessors must be familiar with the mechanics of the
test objective and should be determined before the test (see for
duo-trio test (the format, the task, and the procedure of
example Appendix X1 and Appendix X2).
evaluation). Experience and familiarity with the product and
4.3 Each assessor receives a triad where one sample is testmethodmayincreasethesensitivityofanassessorandmay
labeled as the reference and the other two samples are labeled
therefore increase the likelihood of finding a significant differ-
with different codes.The assessors are informed that one of the ence. Monitoring the performance of assessors over time may
coded samples is the same as the reference and that one is
be useful.
different.Theassessorsreportwhichofthecodedsamplesthey
7.2 Choose assessors in accordance with test objectives. For
believe to be the same as (or different from) the reference.
example, to project results to a general consumer population,
4.4 Results are tallied and significance determined by ref- assessors with unknown sensitivity might be selected. To
erence to a statistical table.
increase protection of product quality, assessors with demon-
strated acuity should be selected.
5. Significance and Use
7.3 Thedecisiontousetrainedoruntrainedassessorsshould
5.1 The test method is effective for the following test
be addressed prior to testing. Training may include a prelimi-
objectives:
nary presentation on the nature of the samples and the problem
5.1.1 To determine whether a perceivable difference results
concerned. If the test concerns the detection of a particular
or a perceivable difference does not result, for example, when
taint, consider the inclusion of samples during training that
a change is made in ingredients, processing, packaging, han-
demonstrate its presence and absence. Such demonstration will
dling or storage; or
increase the panel’s acuity for the taint but may detract from
5.1.2 To select, train and monitor assessors.
other differences. See Ref (8) for details.Allow adequate time
5.2 The test method itself does not change whether the between the exposure to the training samples and the actual
purpose of the duo-trio test is to determine that two products duo-trio test to avoid carryover.
are perceivably different versus that the products are not
7.4 During the test sessions, avoid giving information about
perceivably different. Only the selected values of p , α, and β
d
product identity, expected treatment effects or individual per-
change. If the objective of the test is to determine if there is a
formance until all testing is complete.
perceivable difference between two products, then the value
7.5 Avoid replicate evaluations by the same assessor when-
ever possible. However, if replications are needed to produce a
Organizations differ in the instructions they give their assessors. Some
sufficient number of total evaluations, every effort should be
organizations instruct their assessors to select the sample that is most similar to the
made to have each assessor perform the same number of
reference. Others instruct their assessors to select the sample that is most different
replicate evaluations.
from the reference. Either approach is acceptable.
E2610 − 18
8. Number of Assessors to the assessors (for example, a control sample from the
production line), use the constant reference technique (9.1.2).
8.1 Choose the number of assessors to yield the level of
9.1.1 Balanced-Reference Technique—Prepare worksheets
sensitivity called for by the test objectives. The sensitivity of
and scoresheets (see Appendix X1) in advance of the test so as
the test is a function of three values: the α-risk, and the β-risk,
to utilize an equal number of the four possible sequences of
and the maximum allowable proportion of distinguishers, p .
d
two products, A and B:
8.2 Priortoconductingthetest,selectvaluesforα,βand p .
d
A AB A BA
R R
The following can be considered as general guidelines.
B AB B BA
R R
8.2.1 For α-risk—A statistically significant result at:
9.1.1.1 Distribute these at random among the assessors so
10 to 5 % (0.10 to 0.05) indicates “slight” evidence
that serving order is balanced.
that a difference was apparent
5 to 1 % (0.05 to 0.01) indicates “moderate” evidence 9.1.2 Constant-Reference Technique—Prepare worksheets
that a difference was apparent
and scoresheets (see Appendix X2) in advance of the test so as
1 to 0.1 % (0.01 to 0.001) indicates “strong” evidence
toutilizeanequalnumberofthetwopossiblesequencesoftwo
that a difference was apparent
Below 0.1 % (<0.001) indicates “very strong” evidence products, A and B:
that a difference was apparent
A AB A BA
R R
8.2.2 For β-risk—The strength of the evidence that a differ-
9.1.2.1 Distribute these at random among the assessors so
ence was not apparent is assessed using the same criteria as
that serving order is balanced.
above (substituting “was not apparent” for “was apparent”).
9.2 Present each triad simultaneously if possible, following
8.2.3 For p —The maximum allowable proportion of
d
the same spatial arrangement for each assessor (on a line to be
distinguishers, p , falls into three ranges:
d
sampled always from left to right, in a triangular array, etc.).
p < 25 % represent small values
d
25%< p < 35 % represent medium sized values Within the triad, assessors are typically allowed to make
d
p > 35 % represent large values
d
repeated evaluations of each sample as desired. If the condi-
8.3 Having defined the required level of sensitivity for the tions of the test require the prevention of repeat evaluations for
test using 8.2, use Table A1.1 to determine the number of
example, if samples are bulky, leave an aftertaste, or show
assessors necessary. Enter Table A1.1 in the section corre- slight differences in appearance that cannot be masked, present
sponding to the selected value of p and the column corre-
thesamplessequentiallyanddonotallowrepeatedevaluations.
d
sponding to the selected value of β. The minimum required In addition, if the samples change over time, for example,
number of assessors is found in the row corresponding to the
chewing gum or cereal with milk, samples should be tested
selected value of α. Alternatively, Table A1.1 can be used to
sequentially.
develop a set of values for p , α and β that provide acceptable
d
9.3 Instruct the assessors to evaluate the reference sample
sensitivity while maintaining the number of assessors within
first and then evaluate the two coded samples in the order in
practical limits. The approach is presented in detail in Ref (9).
which they were presented. The assessor should then indicate
8.4 If one wishes to use Thurstonian δ as a measure of
which of the two coded samples is the same as the reference.
sensory effect size, use Tables A2.1 and A2.2 to convert
9.4 Each scoresheet should provide for a single triad of
between p and δ. See Ref (10) for further discussion on the
d
samples. If a different set of products is to be evaluated by an
relationship between Thurstonian δ and p .
d
assessor in a single session, the completed scoresheet and any
8.5 Often in practice, the number of assessors is determined
remaining product should be returned to the test administrator
by material conditions (for example, duration of the
prior to receiving the subsequent triad. The assessor cannot go
experiment, number of available assessors, quantity of prod-
back to any of the previous samples or change the verdict on
uct). However, increasing the number of assessors increases
any previous test.
the likelihood of detecting small proportions of distinguishers.
9.5 Do not ask questions about preference, acceptance, or
Thus, one should expect to use larger numbers of assessors
degreeofdifferenceaftertheinitialselectionofthesamplethat
when trying to demonstrate that products are similar compared
matches the reference.The selection the assessor has just made
towhenoneistryingtoprovetheyaredifferent.Often20to36
may bias the reply to any additional questions. Responses to
assessors are used when testing for a difference. For compa-
such questions may be obtained through separate tests for
rable sensitivity when testing for similarity, 40 to 78 assessors
preference, acceptance, degree of difference, etc. (see Ref
are needed.
(11)).Acommentsectionaskingwhythechoicewasmademay
9. Procedure
be included for the assessor’s remarks.
9.1 Ifneitherproductismorefamiliarthantheother,usethe
9.6 The duo-trio test is a forced-choice procedure; assessors
balanced reference technique (9.1.1). If the product is familiar
are not allowed the option of reporting “no difference.” An
assessor who detects no difference between the samples and
In this test method, the probability of a correct response, p , is modeled as requests to report “no difference” should be instructed to
c
p 51·p 1 1/2 · 12p , where p is the proportion of the entire population
~ ! ~ !
c d d d randomly select one of the coded samples as being the same as
of assessors who can distinguish between the two products. It is a strictly statistical
the reference. In such situations the assessor can indicate that
“guessing model” of the assessor’s behavior. It is not a psychometric model of the
the selection was only a guess in the comments section of the
assessor’s decision process, such as the Thurstone-Ura model that could also be
applied in discrimination testing. scoresheet.
E2610 − 18
10. Analysis and Interpretation of Results storage handling and preparation was done in such a way as to
yield samples that differ only due to the variable of interest, if
10.1 Use Table A1.2 to analyze the data obtained from a
at all);
duo-trio test. The actual number of assessors can be greater
11.1.3 The number of assessors, the number of correct
than the minimum value given in TableA1.1. If the number of
selections, and the result of the statistical evaluation;
correct responses is greater than or equal to the number given
11.1.4 Assessors—Age, gender, experience in sensory
in Table A1.2, conclude that a perceptible difference exists
testing, experience with the product category, experience with
between the samples. If the number of correct answers is less
the samples in the test;
thanthenumbergiveninTableA1.2,concludethatthesamples
11.1.5 Any information and any specific instructions given
are sufficiently similar.Again, the conclusions are based on the
the assessor in connection with the test;
risksacceptedwhenthelevelofsensitivity(thatis, p ,αandβ)
d 11.1.6 The test environment: use of booths, simultaneous or
was selected in determining the number of assessors (Table
sequential presentation, light conditions, whether the identity
A1.1).
of the samples was disclosed after the test and the manner in
which it was done; and
10.2 If desired, calculate a confidence interval on the
11.1.7 The location and date of the test and the name of the
proportion of the population that can distinguish the samples.
panel leader.
This method is described in Appendix X3.
12. Precision and Bias
11. Report
...

This document is not an ASTM standard and is intended only to provide the user of an ASTM standard an indication of what changes have been made to the previous version. Because
it may not be technically possible to adequately depict all changes accurately, ASTM recommends that users consult prior editions as appropriate. In all cases only the current version
of the standard as published by ASTM is to be considered the official document.
Designation: E2610 − 08 (Reapproved 2011) E2610 − 18
Standard Test Method for
Sensory Analysis—Duo-Trio Test
This standard is issued under the fixed designation E2610; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope
1.1 This test method covers a procedure for determining whether a perceptible sensory difference exists between samples of two
products.
1.2 This test method applies whether a difference may exist in a single sensory attribute or in several.
1.3 This test method is applicable when the nature of the difference between the samples is unknown. It does not determine the
size or the direction of the difference. The attribute(s) responsible for the difference are not identified.
1.4 Compared to the triangle test, the duo-trio test is statistically less efficient, but easier to perform by the assessors. For details
on how the duo-trio test compares to other three-sample tests, see Refs (1-4).
1.5 This test method is applicable only if the products are homogeneous. If two samples of the same product can often be
distinguished, then another method, for example, descriptive analysis, may be more appropriate.
1.6 This test method is applicable only when the products do not cause excessive sensory fatigue, carryover or adaptation.
1.7 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility
of the user of this standard to establish appropriate safety safety, health, and healthenvironmental practices and determine the
applicability of regulatory limitations prior to use.
1.8 This international standard was developed in accordance with internationally recognized principles on standardization
established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued
by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
2. Referenced Documents
2.1 ASTM Standards:
E253 Terminology Relating to Sensory Evaluation of Materials and Products
E456 Terminology Relating to Quality and Statistics
E1871 Guide for Serving Protocol for Sensory Evaluation of Foods and Beverages
E1885 Test Method for Sensory Analysis—Triangle Test
E2262 Practice for Estimating Thurstonian Discriminal Distances
2.2 ISO Standards:
ISO 4120 Sensory Analysis—Methodology—Triangle Test
ISO 10399 Sensory Analysis—Methodology—Duo-Trio Test
3. Terminology
3.1 Definitions—For definition of terms relating to sensory analysis, see Terminology E253, and for terms relating to statistics,
see Terminology E456.
3.2 Definitions of Terms Specific to This Standard:
This test method is under the jurisdiction of ASTM Committee E18 on Sensory Evaluation and is the direct responsibility of Subcommittee E18.04 on Fundamentals
of Sensory.
Current edition approved Nov. 15, 2011Aug. 1, 2018. Published March 2012August 2018. Originally approved in 2008. Last previous edition approved in 20082011 as
E2610 – 08.E2610 – 08 (2011). DOI: 10.1520/E2610-08R11.10.1520/E2610-18.
The boldface numbers in parentheses refer to the list of references at the end of this standard.
For referenced ASTM standards, visit the ASTM website, www.astm.org, or contact ASTM Customer Service at [e-mail]. For Annual Book of ASTM Standards
volume information, refer to the standard’s Document Summary page on the ASTM website.
Available from American National Standards Institute (ANSI), 25 W. 43rd St., 4th Floor, New York, NY 10036, http://www.ansi.org.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E2610 − 18
3.2.1 α (alpha) risk—probability of concluding that a perceptible difference exists when, in reality, one does not. (Also known
as Type I Error or significance level.)
3.2.2 β (beta) risk—probability of concluding that no perceptible difference exists when, in reality, one does. (Also known as
Type II Error.)
3.2.3 p —probability of a correct response.
c
3.2.4 p (proportion of discriminators)—proportion of the population represented by the assessors that can distinguish between
d
the two products.
3.2.5 product—material to be evaluated.
3.2.6 sample—unit of product prepared, presented, and evaluated in the test.
3.2.7 sensitivity—general term used to summarize the performance characteristics of the test. The sensitivity of the test is
rigorously defined, in statistical terms, by the values selected for α, β, and p .
d
3.2.8 δ—Thurstonian measure of sensory difference (effect size) relative to perceptual noise (standard deviation) (see Practice
E2262).
3.2.9 triad—three samples given to an assessor in the duo-trio test; one sample is labeled as a reference the other two samples
are labeled with different codes. One of the coded samples is the same product as the reference. The other coded sample is different.
4. Summary of Test Method
4.1 Clearly define the test objective in writing.
4.2 Choose the number of assessors based on the level of sensitivity desired for the test. The sensitivity of the test is, in part,
a function of two competing risks: the risk of declaring the samples different when they are not (that is, α-risk) and the risk of not
declaring the samples different when they are (that is, β-risk). Acceptable values of α and β vary depending on the test objective
and should be determined before the test (see for example Appendix X1 and Appendix X2).
4.3 Each assessor receives a triad where one sample is labeled as the reference and the other two samples are labeled with
different codes. The assessors are informed that one of the coded samples is the same as the reference and that one is different.
The assessors report which of the coded samples they believe to be the same as (or different from) the reference.
4.4 Results are tallied and significance determined by reference to a statistical table.
5. Significance and Use
5.1 The test method is effective for the following test objectives:
5.1.1 To determine whether a perceivable difference results or a perceivable difference does not result, for example, when a
change is made in ingredients, processing, packaging, handling or storage; or
5.1.2 To select, train and monitor assessors.
5.2 The test method itself does not change whether the purpose of the duo-trio test is to determine that two products are
perceivably different versus that the products are not perceivably different. Only the selected values of p , α, and β change. If the
d
objective of the test is to determine if there is a perceivable difference between two products, then the value selected for α is
typically smaller than the value selected for β. If the objective is to determine if the two products are sufficiently similar to be used
interchangeably, then the value selected for β is typically smaller than the value selected for α and the value of p is selected to
d
define “sufficiently similar.”
5.3 The test method may change based on the test objective or the assessors’ familiarity with the product. The balanced-
reference technique (see 9.1.1) typically is used when neither product is more familiar than the other. The constant-reference
technique (see 9.1.2) frequently is used when one product is a control/current product or is familiar to the assessors.
6. Apparatus
6.1 Carry out the test under conditions that prevent contact between assessors until the evaluations have been completed, for
example, using booths that comply with Ref (5).
6.2 Sample preparation and serving sizes should comply with Guide E1871. See Refs (6) or (7).
7. Assessors
7.1 All assessors must be familiar with the mechanics of the duo-trio test (the format, the task, and the procedure of evaluation).
Experience and familiarity with the product and test method may increase the sensitivity of an assessor and may therefore increase
the likelihood of finding a significant difference. Monitoring the performance of assessors over time may be useful.
Organizations differ in the instructions they give their assessors. Some organizations instruct their assessors to select the sample that is most similar to the reference.
Others instruct their assessors to select the sample that is most different from the reference. Either approach is acceptable.
E2610 − 18
7.2 Choose assessors in accordance with test objectives. For example, to project results to a general consumer population,
assessors with unknown sensitivity might be selected. To increase protection of product quality, assessors with demonstrated acuity
should be selected.
7.3 The decision to use trained or untrained assessors should be addressed prior to testing. Training may include a preliminary
presentation on the nature of the samples and the problem concerned. If the test concerns the detection of a particular taint, consider
the inclusion of samples during training that demonstrate its presence and absence. Such demonstration will increase the panel’s
acuity for the taint but may detract from other differences. See Ref (8) for details. Allow adequate time between the exposure to
the training samples and the actual duo-trio test to avoid carryover.
7.4 During the test sessions, avoid giving information about product identity, expected treatment effects or individual
performance until all testing is complete.
7.5 Avoid replicate evaluations by the same assessor whenever possible. However, if replications are needed to produce a
sufficient number of total evaluations, every effort should be made to have each assessor perform the same number of replicate
evaluations.
8. Number of Assessors
8.1 Choose the number of assessors to yield the level of sensitivity called for by the test objectives. The sensitivity of the test
is a function of three values: the α-risk, and the β-risk, and the maximum allowable proportion of distinguishers, p .
d
8.2 Prior to conducting the test, select values for α, β and p . The following can be considered as general guidelines.
d
8.2.1 For α-risk—A statistically significant result at:
10 to 5 % (0.10 to 0.05) indicates “slight” evidence
that a difference was apparent
5 to 1 % (0.05 to 0.01) indicates “moderate” evidence
that a difference was apparent
1 to 0.1 % (0.01 to 0.001) indicates “strong” evidence
that a difference was apparent
Below 0.1 % (<0.001) indicates “very strong” evidence
that a difference was apparent
8.2.2 For β-risk—The strength of the evidence that a difference was not apparent is assessed using the same criteria as above
(substituting “was not apparent” for “was apparent”).
8.2.3 For p —The maximum allowable proportion of distinguishers, p , falls into three ranges:
d d
p < 25 % represent small values
d
25 % < p < 35 % represent medium sized values
d
p > 35 % represent large values
d
8.3 Having defined the required level of sensitivity for the test using 8.2, use Table A1.1 to determine the number of assessors
necessary. Enter Table A1.1 in the section corresponding to the selected value of p and the column corresponding to the selected
d
value of β. The minimum required number of assessors is found in the row corresponding to the selected value of α. Alternatively,
Table A1.1 can be used to develop a set of values for p , α and β that provide acceptable sensitivity while maintaining the number
d
of assessors within practical limits. The approach is presented in detail in Ref (9).
8.4 If one wishes to use Thurstonian δ as a measure of sensory effect size, use Tables A2.1 and A2.2 to convert between p and
d
δ. See Ref (10) for further discussion on the relationship between Thurstonian δ and p .
d
8.5 Often in practice, the number of assessors is determined by material conditions (for example, duration of the experiment,
number of available assessors, quantity of product). However, increasing the number of assessors increases the likelihood of
detecting small proportions of distinguishers. Thus, one should expect to use larger numbers of assessors when trying to
demonstrate that products are similar compared to when one is trying to prove they are different. Often 20 to 36 assessors are used
when testing for a difference. For comparable sensitivity when testing for similarity, 40 to 78 assessors are needed.
9. Procedure
9.1 If neither product is more familiar than the other, use the balanced reference technique (9.1.1). If the product is familiar to
the assessors (for example, a control sample from the production line), use the constant reference technique (9.1.2).
9.1.1 Balanced-Reference Technique—Prepare worksheets and scoresheets (see Appendix X1) in advance of the test so as to
utilize an equal number of the four possible sequences of two products, A and B:
A AB A BA
R R
B AB B BA
R R
9.1.1.1 Distribute these at random among the assessors so that serving order is balanced.
In this test method, the probability of a correct response, p , is modeled as p 51·p 1 1/2 · 12p , where p is the proportion of the entire population of assessors
~ ! ~ !
c c d d d
who can distinguish between the two products. It is a strictly statistical “guessing model” of the assessor’s behavior. It is not a psychometric model of the assessor’s decision
process, such as the Thurstone-Ura model that could also be applied in discrimination testing.
E2610 − 18
9.1.2 Constant-Reference Technique—Prepare worksheets and scoresheets (see Appendix X2) in advance of the test so as to
utilize an equal number of the two possible sequences of two products, A and B:
A AB A BA
R R
9.1.2.1 Distribute these at random among the assessors so that serving order is balanced.
9.2 Present each triad simultaneously if possible, following the same spatial arrangement for each assessor (on a line to be
sampled always from left to right, in a triangular array, etc.). Within the triad, assessors are typically allowed to make repeated
evaluations of each sample as desired. If the conditions of the test require the prevention of repeat evaluations for example, if
samples are bulky, leave an aftertaste, or show slight differences in appearance that cannot be masked, present the samples
sequentially and do not allow repeated evaluations. In addition, if the samples change over time, for example, chewing gum or
cereal with milk, samples should be tested sequentially.
9.3 Instruct the assessors to evaluate the reference sample first and then evaluate the two coded samples in the order in which
they were presented. The assessor should then indicate which of the two coded samples is the same as the reference.
9.4 Each scoresheet should provide for a single triad of samples. If a different set of products is to be evaluated by an assessor
in a single session, the completed scoresheet and any remaining product should be returned to the test administrator prior to
receiving the subsequent triad. The assessor cannot go back to any of the previous samples or change the verdict on any previous
test.
9.5 Do not ask questions about preference, acceptance, or degree of difference after the initial selection of the sample that
matches the reference. The selection the assessor has just made may bias the reply to any additional questions. Responses to such
questions may be obtained through separate tests for preference, acceptance, degree of difference, etc. (see Ref (1011)). A comment
section asking why the choice was made may be included for the assessor’s remarks.
9.6 The duo-trio test is a forced-choice procedure; assessors are not allowed the option of reporting “no difference.” An assessor
who detects no difference between the samples and requests to report “no difference” should be instructed to randomly select one
of the coded samples as being the same as the reference. In such situations the assessor can indicate that the selection was only
a guess in the comments section of the scoresheet.
10. Analysis and Interpretation of Results
10.1 Use Table A1.2 to analyze the data obtained from a duo-trio test. The actual number of assessors can be greater than the
minimum value given in Table A1.1. If the number of correct responses is greater than or equal to the number given in Table A1.2,
conclude that a perceptible difference exists between the samples. If the number of correct answers is less than the number given
in Table A1.2, conclude that the samples are sufficiently similar. Again, the conclusions are based on the risks accepted when the
level of sensitivity (that is, p , α and β) was selected in determining the number of assessors (Table A1.1).
d
10.2 If desired, calculate a confidence interval on the proportion of the population that can distinguish the samples. This method
is described in Appendix X3.
11. Report
11.1 Report the test objective, the results, and the conclusions. The following additional information is recommended:
11.1.1 The purpose of the test and the nature of the treatment studied;
11.1.2 Full Identification of the Samples—Origin, age, lot number, packaging, where obtained, method of preparation, quantity,
shape, storage prior to testing, serving size, and temperature (sample information should communicate that all storage handling and
preparation was done in such a way as to yield samples that differ only due to the variable of interest, if at all);
11.1.3 The number of assessors, the number of correct selections, and the result of the statistical evaluation;
11.1.4 Assessors—Age, gender, experience in sensory testing, experience with the product category, experience with the
samples in the test;
11.1.5 Any information and any specific instructions given the assessor in connection with the test;
11.1.6 The test environment: use of booths, simultaneous or sequential presentation, light conditions, whether the identity of the
samples was disclosed after the test and the manner in which it was done; and
11.1.7 The location and date of the test and the name of the panel leader.
12. Precision and Bias
12.1 Because results of sensory difference tests are functions of individual
...