ASTM E2262-03(2020)
(Practice)Standard Practice for Estimating Thurstonian Discriminal Distances
Standard Practice for Estimating Thurstonian Discriminal Distances
SIGNIFICANCE AND USE
5.1 Under the assumptions of the model, the Thurstonian model approach to measuring the perceived difference between two samples (whether overall or for a specific attribute) is independent of the sensory method used to collect the data. Converting results obtained from different test methods to d' values permits the assessment of relative differences among samples without requiring that the samples be compared to each other directly or that the same test methods be used for all pairs of samples.
5.2 Thurstonian scaling has been applied to:
5.2.1 Creating a historical database to track differences between production and reference samples over periods in which different test methods were used to measure the difference,
5.2.2 Comparing the relative sensitivities of different user groups and consumer segments,
5.2.3 Comparing trained panels that use different measuring techniques,
5.2.4 Comparing the relative sensitivities of consumers versus trained panels,
5.2.5 Comparing different methods of consumer testing (for example, CLT versus HUT, preference versus hedonic scales, etc.), and
5.2.6 Comparing different discrimination test methods.
SCOPE
1.1 This practice describes procedures to estimate Thurstonian discriminal distances (that is, d' values) from data obtained on two samples. Procedures are presented for four forced-choice methods (that is, the triangle, the Duo-Trio, the 3-alternative-forced-choice (or 3-AFC) and the 2-AFC (also called the directional difference test)), the A/Not-A method, the Same-Different method, and for data obtained from ordered category scales. Procedures for estimating the variance of d' are also presented. Thus, confidence intervals and statistical tests can be calculated for d'.
1.2 The procedures in this practice pertain only to the unidimensional, equal-variance model. Other, more complicated Thurstonian models, involving multiple dimensions and unequal variances exist but are not addressed in this practice. The procedure for forced-choice methods is limited to dichotomous responses. The procedure for the A/Not-A method assumes equal sample sizes for the two samples. The procedure for the Same-Different method assumes equal sample sizes for the matched and unmatched pairs of samples. For all methods, only unreplicated tests are considered. (Tests in which each assessor performs multiple (that is, replicated) evaluations require different analyses.)
1.3 Thurstonian scaling is a method for measuring the perceptual difference between two samples based on a probabilistic model for categorical choice decision making. The magnitude of the perceived difference, δ, can be estimated from the assessors' categorical choices using the methods described in this practice. (See Appendix X3 for a more detailed description of Thurstonian scaling.)
1.4 In theory, the Thurstonian δ does not depend on the method used to measure the difference between two samples. As such, δ provides a common scale of measure for comparing samples measured under a variety of test conditions. For example, Thurstonian scaling can be used to compare products measured under different test conditions, to compare panels (trained, consumer or both) that have evaluated the same samples (using the same or different test methods) and to compare test methods on their ability to discriminate samples that exhibit a fixed sensory difference.
1.5 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility of the user of this standard to establish appropriate safety, health, and environmental practices and determine the applicability of regulatory limitations prior to use.
1.6 This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the Development of International Standards, Guides and Recommendations is...
General Information
Relations
Standards Content (Sample)
This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation: E2262 − 03 (Reapproved 2020)
Standard Practice for
Estimating Thurstonian Discriminal Distances
This standard is issued under the fixed designation E2262; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope (trained, consumer or both) that have evaluated the same
samples (using the same or different test methods) and to
1.1 This practice describes procedures to estimate Thursto-
compare test methods on their ability to discriminate samples
nian discriminal distances (that is, d’ values) from data
that exhibit a fixed sensory difference.
obtained on two samples. Procedures are presented for four
1.5 This standard does not purport to address all of the
forced-choice methods (that is, the triangle, the Duo-Trio, the
safety concerns, if any, associated with its use. It is the
3-alternative-forced-choice (or 3-AFC) and the 2-AFC (also
responsibility of the user of this standard to establish appro-
calledthedirectionaldifferencetest)),theA/Not-Amethod,the
priate safety, health, and environmental practices and deter-
Same-Different method, and for data obtained from ordered
mine the applicability of regulatory limitations prior to use.
category scales. Procedures for estimating the variance of d’
1.6 This international standard was developed in accor-
are also presented. Thus, confidence intervals and statistical
dance with internationally recognized principles on standard-
tests can be calculated for d’.
ization established in the Decision on Principles for the
1.2 The procedures in this practice pertain only to the
Development of International Standards, Guides and Recom-
unidimensional, equal-variance model. Other, more compli-
mendations issued by the World Trade Organization Technical
cated Thurstonian models, involving multiple dimensions and
Barriers to Trade (TBT) Committee.
unequal variances exist but are not addressed in this practice.
Theprocedureforforced-choicemethodsislimitedtodichoto-
2. Referenced Documents
mous responses. The procedure for the A/Not-A method
2.1 ASTM Standards:
assumesequalsamplesizesforthetwosamples.Theprocedure
E253Terminology Relating to Sensory Evaluation of Mate-
for the Same-Different method assumes equal sample sizes for
rials and Products
the matched and unmatched pairs of samples. For all methods,
E456Terminology Relating to Quality and Statistics
only unreplicated tests are considered. (Tests in which each
assessor performs multiple (that is, replicated) evaluations
2.2 ASTM Manual:
require different analyses.)
Manual 26Sensory Testing Methods, 2nd Edition
1.3 Thurstonian scaling is a method for measuring the 2.3 ISO Standard:
perceptual difference between two samples based on a proba- ISO 5495Sensory Analysis—Methodology—Paired Com-
bilistic model for categorical choice decision making. The parison
magnitude of the perceived difference, δ, can be estimated
3. Terminology
from the assessors’ categorical choices using the methods
described in this practice. (See Appendix X3 for a more
3.1 Definitions:
detailed description of Thurstonian scaling.)
3.1.1 For definitions of terms relating to sensory analysis,
see Terminology E253. For terms relating to statistics, see
1.4 In theory, the Thurstonian δ does not depend on the
Terminology E456.
method used to measure the difference between two samples.
3.2 Definitions of Terms Specific to This Standard:
Assuch, δprovidesacommonscaleofmeasureforcomparing
3.2.1 δ—theThurstoniandiscriminaldistanceisthedistance
samples measured under a variety of test conditions. For
between the means of the distributions of sensory magnitudes
example,Thurstonian scaling can be used to compare products
of the two samples in the test (see Appendix X3).
measured under different test conditions, to compare panels
1 2
This practice is under the jurisdiction of ASTM Committee E18 on Sensory For referenced ASTM standards, visit the ASTM website, www.astm.org, or
Evaluation and is the direct responsibility of Subcommittee E18.03 on Sensory contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
Theory and Statistics. Standards volume information, refer to the standard’s Document Summary page on
Current edition approved Aug. 1, 2020. Published September 2020. Originally the ASTM website.
approved in 2003. Last previous edition approved in 2014 as E2262–03 (2014). Available fromAmerican National Standards Institute (ANSI), 25 W. 43rd St.,
DOI: 10.1520/E2262-03R20. 4th Floor, New York, NY 10036, http://www.ansi.org.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E2262 − 03 (2020)
3.2.2 d’—the statistic used to estimate δ based on the data “A” sample and the other sample is selected to be the “Not-A”
obtained from the test. sample. Choice proportions are tallied for each sample and the
values of d’ and its variance, S (d’), are obtained from Tables
3.2.3 choice proportion (P )—the expected proportion of
c
X1.9 and X1.10, respectively, by the same techniques used in
responses from a forced-choice method. (For example, if there
the A/Not-A method.
is no perceptible difference between the samples in a triangle
test, P = 1/3. If there is a perceptible difference, P > 1/3.)
c c
5. Significance and Use
3.2.4 observed choice proportion (p )—the statistic used to
c
5.1 Under the assumptions of the model, the Thurstonian
estimate choice proportion, P , where p = x/n, where x is the
c c
modelapproachtomeasuringtheperceiveddifferencebetween
observednumberofcorrectresponsesand nisthesamplesize.
two samples (whether overall or for a specific attribute) is
independent of the sensory method used to collect the data.
4. Summary of Practice
Converting results obtained from different test methods to d’
4.1 Determinethetypeofdatacollectedonthetwosamples:
values permits the assessment of relative differences among
data from a forced-choice test, an A/Not-A test, a Same-
samples without requiring that the samples be compared to
Different test or an ordered category scale.
eachotherdirectlyorthatthesametestmethodsbeusedforall
pairs of samples.
4.2 For forced-choice tests, reference the table that corre-
spondstothetestmethod(thatis,triangletest(TablesX1.1and
5.2 Thurstonian scaling has been applied to:
X1.2), Duo-Trio test (Tables X1.3 and X1.4), 3-AFC test
5.2.1 Creating a historical database to track differences
(Tables X1.5 and X1.6), or 2-AFC test (Tables X1.7 and
between production and reference samples over periods in
X1.8)). Identify the entry in the table closest to the observed
which different test methods were used to measure the
choice proportion (p ) from the test. Read the estimated value
c
difference,
of δ (that is, d’) from the corresponding row and column
5.2.2 Comparing the relative sensitivities of different user
headings of the table. Estimate the variance of d’ by referenc-
groups and consumer segments,
ing the appropriate table for the test method. Find the value of
5.2.3 Comparingtrainedpanelsthatusedifferentmeasuring
B that corresponds to the value of d’ obtained in the first step
techniques,
(see Note 1). The estimated variance of d’is S (d’) = B/n,
5.2.4 Comparing the relative sensitivities of consumers
where n is the sample size. Use the estimates d’ and S (d’) to
versus trained panels,
constructconfidenceintervalsandtestsofhypothesesrelatedto
5.2.5 Comparing different methods of consumer testing (for
the objectives of the research.
example, CLT versus HUT, preference versus hedonic scales,
etc.), and
NOTE 1—The variance of d’ is a complicated function of the true value
5.2.6 Comparing different discrimination test methods.
ofδandthedecisionrulewhenassociatedwiththetestmethodbeingused
(see Appendix X3). However, regardless of the test method, the variance
of d’ can always be expressed as S (d’)= B/n, where the parameter B
6. Procedure
captures all of the information concerning the test method, and n is the
6.1 Forced-choice Methods—The relationship between δ
sample size. The values of B have been tabulated to make the calculation
of the variance of d’ a simple task.
andtheexpectedchoiceproportion, P ,isdifferentfordifferent
c
forced-choice methods because the decision rule used by the
4.3 For the A/Not-A method, tally the observed choice
assessors varies from one method to another (see Appendix
proportions of “A” responses for the A sample and the “A”
X3). As a result, different tables are required to estimate δ
responses for the Not-A sample. Read the value of d’ from
depending on the method used. Tables for the four most
Table X1.9 in the column that corresponds to the observed
commonlyusedmethodsarepresented.Theestimatedvalueof
choice proportion of the “A” responses for the Not-A sample
δ (that is, d’) is obtained as follows:
(p ) and the row that corresponds to the observed choice
na
6.1.1 Compute the observed choice proportion as p = x/n,
proportion of the “A” responses for the A sample (p ). The c
a
where x is the observed number of correct responses and n is
same method is used to estimate the variance of d’, S (d’),
the sample size.
using Table X1.10.
6.1.2 Obtain d’ by entering the table in Appendix X1 that
4.4 For the Same-Different method, tally the proportion of
correspondstothetestmethodused:triangletest(TableX1.1),
“same” responses for the matched pairs of samples (that is,
Duo-Trio (Table X1.3), 3-AFC (Table X1.5), or 2-AFC (Table
A/A or B/B) and the proportion of “same” responses for the
X1.7).Findtheentryinthetablethatisclosesttotheobserved
unmatched pairs of samples (that is, A/B or B/A). Read the
value of p . The value of d’, accurate to one decimal place, is
c
valueof d’fromTableX1.11inthecolumnthatcorrespondsto
the row-label of the table corresponding to the selected entry.
the observed proportion of “same” responses for the un-
Theseconddecimalplaceof d’isthecolumn-labelofthetable
matched pairs (p / ) and the row that corresponds to the
s u
corresponding to the selected entry.
observed proportion of the “same” responses for the matched
6.1.3 Obtain the estimated variance of d’ as follows. Enter
pairs (p / ). The same method is used to estimate the variance
s m
the appropriate table in Appendix X1: triangle test (Table
of d’, S (d’), using Table X1.12.
X1.2), Duo-Trio (Table X1.4), 3-AFC (Table X1.6), or 2-AFC
4.5 For ordered category scales, a rapid, table-look-up (Table X1.8). Find the value of B in the row and column that
approach is used. For each sample, the category scale data are correspond to the value of d’ obtained in 6.1.2. Compute the
collapsed into two categories. One sample is selected to be the estimated variance of d’as S (d’) = B/n, where n is the sample
E2262 − 03 (2020)
size. Use the estimates d’ and S (d’) to construct confidence each sample separately and record the totals in the two-by-two
intervalsandtestsofhypothesesrelatedtotheobjectivesofthe table under “High” (that is, the x and x tallies, below).
na a
research.
Sample Low High
Not-A y x
na na
6.2 A/Not-A Method—Compute the choice proportions of
A y x
a a
the two samples, p = x /n and p = x /n, where x is the
a a na na a
6.4.3 Compute the choice proportions of the two samples,
number of times the “A” sample is chosen as being “A,” x is
na
p = x /n and p = x /n, where x and x are obtained from
a a na na a na
the number of times the “Not-A” sample is chosen as being
the table above and n is the sample size, common to both
“A” and n is the sample size.
samples.
NOTE 2—This practice only considers the case where the number of
6.4.4 ApplythesametechniqueusedintheA/Not-Amethod
“A” samples equals the number of “Not-A” samples, n = n = n .
a na
(see 6.2). Read the value of d’ from Table X1.9 in Appendix
X1 in the column that corresponds to the observed choice
6.2.1 Readthevalueof d’fromTableX1.9inAppendixX1
proportion of the Not-A sample (p ) and the row that
in the column that corresponds to the observed choice propor-
na
corresponds to the observed choice proportion of theAsample
tion of the “Not-A” sample (p ) and the row that corresponds
na
(p ).
to the observed choice proportion of the “A” sample (p ).
a
a
6.2.2 To obtain an estimate of the variance of d’, read the 6.4.5 To obtain an estimate of the variance of d’, read the
value of B from Table X1.10 in Appendix X1 using the same
value of B from Table X1.10 in Appendix X1 using the same
2 2
technique as in 6.2.1. The variance estimate is S (d’) = B/n, technique as in 6.4.4. The variance estimate is S (d’) = B/n,
where n is the sample size.
where n is the sample size.
6.3 Same-Different Method—Compute the choice propor- 6.5 Statistical Tests and Confidence Intervals—Often the
tions for the matched (m) and unmatched (u) pairs of samples, objective of a sensory discrimination test is to determine if the
p = x /n and p = x /n, where x is the number of samples in the test are perceptibly different. In other instances
s/m s/m s/u s/u s/m
“same” responses for the matched pairs (A/A or B/B)
it is of interest to obtain an estimate of the size of the
evaluated, x is the number of “same” responses for the perceptible difference (and to measure the precision of the
s/u
unmatched pair and n is the number of matched or unmatched
estimated difference). Because testing for a difference and
pairs evaluated. estimating the size of a difference address different goals, it is
not surprising that different statistical methods apply to each.
NOTE 3—This practice only considers the case where the number of
For the purpose of testing if a perceptible difference exists, the
matched pairs equals the number of unmatched pairs, n = n = n .
m u
binomial and chi-square tests traditionally associated with the
6.3.1 Read the value of d’ from Table X1.11 in Appendix
test methods discussed in this practice are appropriate. For the
X1 in the column that corresponds to the observed proportion
purposes of estimating the size of the difference and assessing
of “same” responses for unmatched pair (p ) and the row that
s/u
the precision of that estimate, confidence intervals are appro-
corresponds to the observed proportion of “same” responses
priate. Because δ is the difference between the means of two
for the matched pair (p ).
s/m
normal distributions and d’ is an estimate of δ, it can be
6.3.2 To obtain an estimate of the variance of d’, read the
assumed t
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.