ASTM E2263-12(2018)
(Test Method)Standard Test Method for Paired Preference Test
Standard Test Method for Paired Preference Test
SIGNIFICANCE AND USE
5.1 The paired preference test determines whether or not there is a preference for one product over another product among a specific target population. Knowledge of consumer segments, brand loyalties, the range of product offerings in the marketplace, and the decision risk must be understood when planning a paired preference test.
5.2 The paired preference method is commonly used in tests with one or more of the following objectives: (1) to establish superiority in preference versus the competition for advertising claims support; (2) to establish the preference of a new product for launch versus a competitor's product; (3) to establish the preference of a reformulated product in a product improvement or product modification project (for example, process change or ingredient change); and (4) to establish the preference of a cost improved product versus the current formulation in a cost savings project. Selected values of Pmax, α, and β will change with all four types of test objectives. These should be selected prior to determination of N.
5.2.1 Preference versus Competition or Launching a New Product versus Competition—Select a Pmax to represent what you expect a reasonable preference split to be. The main risk to avoid is to wrongly claim your product is preferred over the competitors. Thus, low values of α are selected, for example, 0.05, 0.01, or 0.001. The desired outcome of this test is to reject the null hypothesis. The alternative hypothesis is one sided: A new or improved product (A) is preferred over the competitor’s product (B). The test is one-sided. The value of β will be determined by the sample size chosen and the size of the preference in the consumer segment selected for the test. Selection of the appropriate number of respondents is determined by Pmax, α, and β, as well as the market segment that must be included in the test (for issues specific to conducting a paired preference test for an advertising claim, refer to Guide E1958).
5.2.2 C...
SCOPE
1.1 This document covers a procedure for determining preference between two products using either a two-alternative forced-choice task, or with the option of choosing no preference. Preference testing is a type of hedonic testing.
1.2 A paired preference test determines whether there is a statistically significant preference between two products for a given population of respondents. The target population must be carefully considered.
1.3 This method establishes preference in a single evaluation context. Replicated tests will not be covered within the scope of this document.
1.4 Paired preference testing can address overall preference or preference for a specified sensory attribute.
1.5 The method does not directly determine the magnitude of preference.
1.6 This method does not address whether or not two samples are perceived as different. Refer to Test Method E2164 for directional difference test.
1.7 A paired preference test is a simple task for respondents, and can be used with populations that have minimal reading or comprehension skills, or both.
1.8 Preference is not an intrinsic attribute of the product, such as hue is, but is a subjective measure relating to respondents' affective or hedonic response. It differs from paired comparison testing which measures objective characteristics of the product. Preference results are always dependent on the population sampled.
1.9 This standard does not purport to address all of the safety problems associated with its use, when testing includes hazardous materials, operations, or equipment. It is the responsibility of the user of this standard to establish appropriate safety, health, and environmental practices and to determine the applicability of regulatory limitations prior to use.
1.10 This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles fo...
General Information
Relations
Standards Content (Sample)
This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation: E2263 − 12 (Reapproved 2018)
Standard Test Method for
Paired Preference Test
This standard is issued under the fixed designation E2263; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope 1.10 This international standard was developed in accor-
dance with internationally recognized principles on standard-
1.1 This document covers a procedure for determining
ization established in the Decision on Principles for the
preference between two products using either a two-alternative
Development of International Standards, Guides and Recom-
forced-choice task, or with the option of choosing no prefer-
mendations issued by the World Trade Organization Technical
ence. Preference testing is a type of hedonic testing.
Barriers to Trade (TBT) Committee.
1.2 A paired preference test determines whether there is a
2. Referenced Documents
statistically significant preference between two products for a
givenpopulationofrespondents.Thetargetpopulationmustbe 2
2.1 ASTM Standards:
carefully considered.
E253 Terminology Relating to Sensory Evaluation of Mate-
1.3 This method establishes preference in a single evalua- rials and Products
E456 Terminology Relating to Quality and Statistics
tion context. Replicated tests will not be covered within the
scope of this document. E1871 Guide for Serving Protocol for Sensory Evaluation of
Foods and Beverages
1.4 Paired preference testing can address overall preference
E1958 Guide for Sensory Claim Substantiation
or preference for a specified sensory attribute.
E2164 Test Method for Directional Difference Test
1.5 The method does not directly determine the magnitude
2.2 ISO Standard:
of preference.
ISO 5495 Sensory Analysis—Methodology—Paired Com-
parison
1.6 This method does not address whether or not two
samplesareperceivedasdifferent.RefertoTestMethodE2164
3. Terminology
for directional difference test.
3.1 For definition of terms relating to sensory analysis, see
1.7 Apairedpreferencetestisasimpletaskforrespondents,
Terminology E253, and for terms relating to statistics, see
and can be used with populations that have minimal reading or
Terminology E456.
comprehension skills, or both.
3.2 Definitions of Terms Specific to This Standard:
1.8 Preference is not an intrinsic attribute of the product,
3.2.1 α (alpha) risk—the probability of concluding that a
such as hue is, but is a subjective measure relating to
preference exists when, in reality, one does not. (Also known
respondents’ affective or hedonic response. It differs from
as Type I Error or significance level.)
paired comparison testing which measures objective character-
3.2.2 β (beta) risk—the probability of concluding that no
istics of the product. Preference results are always dependent
preference exists when, in reality, one does. (Also known as
on the population sampled.
Type II Error.)
1.9 This standard does not purport to address all of the
3.2.3 common responses—for a one-sided test, the number
safety problems associated with its use, when testing includes
of respondents selecting the product that is expected to be
hazardous materials, operations, or equipment. It is the re-
preferred. For a two-sided test, the largest number of respon-
sponsibility of the user of this standard to establish appropriate
dents selecting either product.
safety, health, and environmental practices and to determine
the applicability of regulatory limitations prior to use.
3.2.4 one-sided test—a test in which the researcher has an a
priori assumption concerning the direction of the preference.
1 2
This test method is under the jurisdiction ofASTM Committee E18 on Sensory For referenced ASTM standards, visit the ASTM website, www.astm.org, or
Evaluation and is the direct responsibility of Subcommittee E18.04 on Fundamen- contactASTM Customer Service at service@astm.org. ForAnnual Book ofASTM
tals of Sensory. Standards volume information, refer to the standard’s Document Summary page on
Current edition approved Aug. 1, 2018. Published August 2018. Originally theASTM website.
approved in 2004. Last previous edition approved in 2012 as E2263 – 12. DOI: Available fromAmerican National Standards Institute (ANSI), 25 W. 43rd St.,
10.1520/E2263-12R18. 4th Floor, New York, NY 10036.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E2263 − 12 (2018)
In this case, the alternative hypothesis will express that a 4.4 Results are tallied and significance determined by ref-
specific product is preferred over another product (that is only, erence to a statistical table (or calculation).
A > B or A < B), depending on the a priori belief.
4.5 Testingisgenerallyconductedforonepairofsamplesto
3.2.5 two-sided test—a test in which the researcher does not
avoid bias from one set of samples to another.
have any a priori assumption concerning direction of the
preference. In this case, the alternative hypothesis is that the
5. Significance and Use
two products are not equally preferred (that is, A ≠ B).
5.1 The paired preference test determines whether or not
3.2.6 P — a test sensitivity parameter established prior to
max there is a preference for one product over another product
testing and used along with the selected values of α and β to
among a specific target population. Knowledge of consumer
determinethenumberofrespondentsneededinastudy. P is
max segments, brand loyalties, the range of product offerings in the
the proportion of common responses that the researcher wants
marketplace, and the decision risk must be understood when
the test to be able to detect with a probability of 1-β. For
planning a paired preference test.
example, if a researcher wants to have a 90 % confidence level
5.2 Thepairedpreferencemethodiscommonlyusedintests
of detecting a 60:40 split in preference, then P = 60 % and
max
with one or more of the following objectives: (1) to establish
β = 0.10.
superiority in preference versus the competition for advertising
3.2.7 sensitivity—a general term used to summarize the
claims support; (2) to establish the preference of a new product
performance characteristics of the paired preference test. The
for launch versus a competitor’s product; (3) to establish the
sensitivity of the test is defined, in statistical terms, by the
preferenceofareformulatedproductinaproductimprovement
values selected for α, β, and P . Smaller values of α, β, and
max
or product modification project (for example, process change
P indicate a more sensitive test.
max
or ingredient change); and (4) to establish the preference of a
3.2.8 p —the proportion of common responses which is cost improved product versus the current formulation in a cost
c
savings project. Selected values of P , α, and β will change
calculated from the test data.
max
with all four types of test objectives. These should be selected
3.2.9 product—the material from which samples are se-
prior to determination of N.
lected.
5.2.1 Preference versus Competition or Launching a New
3.2.10 sample—the unit of product prepared, presented, and
Product versus Competition—Select a P to represent what
max
evaluated in the test.
youexpectareasonablepreferencesplittobe.Themainriskto
3.2.11 respondent—also known as assessor; a general term avoid is to wrongly claim your product is preferred over the
for any individual responding to stimuli in a sensory test. competitors. Thus, low values of α are selected, for example,
Trained panelists or experienced discrimination panelists do 0.05,0.01,or0.001.Thedesiredoutcomeofthistestistoreject
not serve as respondents in a paired preference test. the null hypothesis. The alternative hypothesis is one sided: A
neworimprovedproduct(A)ispreferredoverthecompetitor’s
product (B). The test is one-sided. The value of β will be
4. Summary of Test Method
determined by the sample size chosen and the size of the
4.1 Clearly define the test objective in writing, specifying
preference in the consumer segment selected for the test.
the type of audience or population you wish to recruit as
Selection of the appropriate number of respondents is deter-
respondents. (If objective involves substantiating an advertis-
mined by P , α, and β, as well as the market segment that
max
ing claim, refer to Guide E1958.)
must be included in the test (for issues specific to conducting
4.2 Choose the number of respondents (N) to be recruited
a paired preference test for an advertising claim, refer to Guide
based on the sensitivity level desired for the test (P , α, and
E1958).
max
β). The sensitivity of the test is, in part, a function of two
5.2.2 Cost Reduction or Reformulation of an Existing
competing risks—the risk of declaring a preference when there
Product—When parity preference is the desired test outcome,
is none (that is, α-risk) and the risk of not declaring that a
values of α are increased and values of β are decreased. For
preference exists when there is a preference (that is, β-risk).
example, if a product is developed which represents a signifi-
Acceptable values of α and β vary depending on the test
cant cost savings over the current formulation and there is
objective. The values should be agreed upon by all parties
concern over alienation of current users,α might be selected at
affected by the results of the test before the test is conducted.
0.20 and β might be selected at 0.01. Parity testing can be
either one-or two-sided depending on the action standards of
4.3 In paired preference testing, an assessor receives a pair
the test. The test is one-sided if the action standard is that the
of coded samples that are identified with appropriate non-
product must be parity or better. The test is two-sided if the
biasing codes. The assessor is asked to choose the sample that
action standard is parity only. The number of respondents
is preferred.
chosen must reflect the risk of replacing the current product
4.3.1 When using a forced choice procedure, a sample must
with the cost-reduced product.
be chosen even if the selection is based only on a random
selection by the assessor.
5.3 Atest result of superiority or parity does not ensure that
4.3.2 If a choice is not forced, a “no preference” option the test conclusion is correct. An incorrect test result can be
should be included, and the data must be handled in a different obtained when the sample of respondents is selected in a way
way. that does not reflect the true preference in the population of
E2263 − 12 (2018)
interest, or when the number of respondents is too small to 8.1.2 Whentheresearcherwantstotakeonlyasmallchance
correctly reflect the preference status of the two products of concluding that a preference exists when it does not (for
among the target consumer group. Careful selection of P ,α, example, when testing to support a claim of superiority), the
max
and β and an appropriate selection of respondents is needed to most commonly used values for α-risk and β-risk are α = 0.05
minimize the risk of drawing an incorrect conclusion in and β = 0.20. These values can be adjusted on a case-by-case
forced-choice paired preference testing. basis to reflect the sensitivity desired versus the number of
respondents available. When testing for a preference with a
6. Apparatus
limited number of respondents, hold the α-risk at a relatively
small value and allow the β-risk to increase in order to control
6.1 Carry out the test under conditions that prevent contact
the risk of falsely concluding that a preference is present.
between respondents until the evaluations have been com-
8.1.3 Whentheresearcherwantstotakeonlyasmallchance
pleted.
of missing a preference that exists (for example, when testing
6.2 Sample preparation and serving sizes should comply
to support a claim of parity preference), the most commonly
with Practice E1871, or see Herz and Cupchik or Todrank et
used values for α-risk and β-risk are α = 0.20 and β = 0.05.
al.
These values can be adjusted on a case-by-case basis to reflect
the sensitivity desired versus the number of respondents
7. Respondents
available. When testing for parity with a limited number of
7.1 Choose the appropriate set of respondents on the basis
respondents, hold the β-risk at a relatively small value and
of the test objective. Selecting the appropriate set of assessors
allow the α-risk to increase in order to control the risk of
for a preference test is critical since preference responses vary
missing a preference that truly exists.
depending on the consumer group targeted. The most appro-
8.1.4 For P , the proportion of common responses falls
max
priate respondents to determine product preference are the
into three ranges: (1) P < 55 % represents “small” values;
max
current or potential consumers of the product category.
(2)55%≤ P ≤65 %represents“mediumsized”values;and
max
7.2 Respondents must be selected based upon the objective (3) P > 65 % represents “large” values.
max
ofthestudyandaredependentonthebusinessimplication.For
8.2 Having defined the required sensitivity for the test using
a new product, the respondents should represent target con-
8.1, use Table X1.1 to determine the number of respondents
sumers.Foranexistingproduct,respondentsmayincludeusers
necessary for a one-sided test, or Table X2.1 to determine the
of the product. If your business objective is to ensure that
number of respondents necessary for two-sided test. Select the
market share is not lost when making formula changes,
section of the table corresponding to the selected P value
max
respondents should include heavy category or product users.
and the column corresponding to the selected β value. The
minimum required number of respondents is found in the row
8. Number of Respondents
corresponding to the selected value of α. Alternatively, Table
8.1 Once the target population has been clearly defined,
X1.1 can be used to develop a set of values for P , α, and β
max
choose the number of respondents required for the test as
that provide acceptable sensitivity while maintaining the num-
follows:(1)firstdetermineifthetestisone-sidedortwo-sided,
ber of respondents within practical limits.
and (2) establish the sensitivity required by the test objectives
8.2.1 Using the parameters: α = 0.05, β = 0.20, and P =
max
by selecting values for the three test-sensitivity parameters: the
60 %, the researcher would use t
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.