SIST EN ISO 11132:2021
(Main)Sensory analysis - Methodology - Guidelines for the measurement of the performance of a quantitative descriptive sensory panel (ISO 11132:2021)
Sensory analysis - Methodology - Guidelines for the measurement of the performance of a quantitative descriptive sensory panel (ISO 11132:2021)
This document gives guidelines for assessing the overall performance of a quantitative descriptive
panel and the performance of each panel member.
This document is applicable to the validation of the training of individual assessors or panels, as well as
to the performance monitoring of established panels.
This document does not apply to the panel performance for descriptive methods where the individual
scores of each assessor are not recorded, where there is no single list of attributes that is common to all
the assessors, or where dominance rather than intensity is measured. Consequently, the performance
of descriptive panels using methods such as consensus profile, free-choice profile, flash profile and
temporal dominance of sensations (TDS) are out of scope.
The methods specified in this document are for monitoring and assessing the ability of a panel and its
assessors to discriminate between products, the agreement between assessors of the same panel and
the repeatability of these assessors in their intensity scoring.
Reproducibility, including both the comparison between panels and the comparison within the same
panel of several evaluations conducted under different conditions (i.e. separated in time), is out of scope
of this document.
The methods specified in this document can be used, in full or a selection only, by the panel leader to
appraise continuously the performance of panels or individual assessors. The methods listed are not
exhaustive and other appropriate methods can also be used.
Sensorische Analyse - Methodologie - Leitlinien zur Überprüfung der Leistungsfähigkeit eines quantitativen sensorischen Panels (ISO 11132:2021)
Dieses Dokument enthält Leitlinien zur Bewertung der allgemeinen Leistungsfähigkeit eines quantitativ beschreibenden Panels und der Leistungsfähigkeit jedes Panelmitglieds.
Dieses Dokument ist anwendbar auf die Validierung der Schulung einzelner Prüfpersonen oder Panels sowie auf die Überprüfung der Leistungsfähigkeit bestehender Panels.
Dieses Dokument gilt nicht für die Leistungsfähigkeit eines Panels für beschreibende Verfahren, bei denen die einzelnen Punktzahlen jeder Prüfperson nicht aufgezeichnet werden, oder für Fälle, bei denen es keine einzelne Liste mit Merkmalen gibt, die allen Prüfpersonen gemein ist, oder für den Fall, dass eher die Dominanz als die Intensität gemessen wird. Folglich ist die Leistungsfähigkeit von beschreibenden Panels, die Verfahren wie Konsensprofil, freies Auswahlprofil, Flash Profil und zeitliche Dominanz von Empfindungen (TDS, en: temporal dominance of sensations) anwenden, nicht im Anwendungsbereich erfasst.
Die in diesem Dokument festgelegten Verfahren dienen der Überprüfung und Bewertung der Fähigkeit eines Panels und dessen Prüfpersonen, zwischen Produkten zu unterscheiden, der Übereinstimmung zwischen den Prüfpersonen desselben Panels und der Wiederholbarkeit dieser Prüfpersonen in ihren Intensitätspunktzahlen.
Die Vergleichbarkeit, einschließlich des Vergleiches zwischen Panels und des Vergleiches innerhalb desselben Panels bei verschiedenen Bewertungen, die unter verschiedenen Bedingungen durchgeführt werden (d. h. zeitlich gesondert), liegt nicht im Anwendungsbereich dieses Dokumentes.
Die in diesem Dokument festgelegten Verfahren können im Ganzen oder nur in einer Auswahl durch den Leiter des Panels angewendet werden, um die Leistungsfähigkeit des Panels oder einzelner Prüfpersonen fortlaufend einzuschätzen. Bei den aufgeführten Verfahren handelt es sich nicht um eine vollständige Aufführung und andere geeignete Verfahren können ebenfalls angewendet werden.
Analyse sensorielle - Méthodologie - Lignes directrices pour le mesurage de la performance d'un jury descriptif quantitatif (ISO 11132:2021)
Le présent document donne des lignes directrices pour évaluer la performance globale d’un jury descriptif quantitatif et la performance de chaque membre du jury.
Le présent document est applicable à la validation de l’entraînement des sujets ou des jurys, ainsi qu’au contrôle de la performance des jurys établis.
Le présent document ne s’applique pas à la performance du jury pour les méthodes descriptives dans lesquelles les notes individuelles de chaque sujet ne sont pas enregistrées, dans lesquelles il n’existe pas une liste unique d’attributs communs à tous les sujets, ou dans lesquelles la dominance plutôt que l’intensité est mesurée. Par conséquent, la performance des jurys descriptifs utilisant des méthodes telles que le profil par consensus, le profil libre choix, le profil flash et la dominance temporelle des sensations (DTS) ne fait pas partie du domaine d’application.
Les méthodes spécifiées dans le présent document servent à contrôler et évaluer la capacité d’un jury et de ses sujets à discriminer les produits, l’accord entre les sujets d’un même jury et la répétabilité de ces sujets lorsqu’ils attribuent des notes d’intensité.
La reproductibilité, y compris la comparaison entre jurys et la comparaison au sein d’un même jury, de plusieurs évaluations effectuées sous différentes conditions (c’est-à-dire, séparées dans le temps), ne fait pas partie du domaine d’application du présent document.
Les méthodes spécifiées dans le présent document peuvent être utilisées, en totalité ou en partie, par le responsable du jury pour évaluer de façon continue la performance du jury ou de chaque sujet. Les méthodes énumérées ne sont pas exhaustives et d’autres méthodes appropriées peuvent également être utilisées.
Senzorična analiza - Metodologija - Smernice za merjenje izvajanja kvantitativnega opisnega senzoričnega panela (ISO 11132:2021)
General Information
Relations
Standards Content (Sample)
SLOVENSKI STANDARD
SIST EN ISO 11132:2021
01-december-2021
Nadomešča:
SIST EN ISO 11132:2017
Senzorična analiza - Metodologija - Smernice za merjenje izvajanja kvantitativnega
opisnega senzoričnega panela (ISO 11132:2021)
Sensory analysis - Methodology - Guidelines for the measurement of the performance of
a quantitative descriptive sensory panel (ISO 11132:2021)
Sensorische Analyse - Methodologie - Leitlinien zur Überprüfung der Leistungsfähigkeit
eines quantitativen sensorischen Panels (ISO 11132:2021)
Analyse sensorielle - Méthodologie - Lignes directrices pour le mesurage de la
performance d'un jury descriptif quantitatif (ISO 11132:2021)
Ta slovenski standard je istoveten z: EN ISO 11132:2021
ICS:
67.240 Senzorična analiza Sensory analysis
SIST EN ISO 11132:2021 en
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.
---------------------- Page: 1 ----------------------
SIST EN ISO 11132:2021
---------------------- Page: 2 ----------------------
SIST EN ISO 11132:2021
EN ISO 11132
EUROPEAN STANDARD
NORME EUROPÉENNE
September 2021
EUROPÄISCHE NORM
ICS 67.240 Supersedes EN ISO 11132:2017
English Version
Sensory analysis - Methodology - Guidelines for the
measurement of the performance of a quantitative
descriptive sensory panel (ISO 11132:2021)
Analyse sensorielle - Méthodologie - Lignes directrices Sensorische Analyse - Methodologie - Leitlinien zur
pour le mesurage de la performance d'un jury Überprüfung der Leistungsfähigkeit eines
descriptif quantitatif (ISO 11132:2021) quantitativen sensorischen Panels (ISO 11132:2021)
This European Standard was approved by CEN on 29 August 2021.
CEN members are bound to comply with the CEN/CENELEC Internal Regulations which stipulate the conditions for giving this
European Standard the status of a national standard without any alteration. Up-to-date lists and bibliographical references
concerning such national standards may be obtained on application to the CEN-CENELEC Management Centre or to any CEN
member.
This European Standard exists in three official versions (English, French, German). A version in any other language made by
translation under the responsibility of a CEN member into its own language and notified to the CEN-CENELEC Management
Centre has the same status as the official versions.
CEN members are the national standards bodies of Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia,
Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway,
Poland, Portugal, Republic of North Macedonia, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and
United Kingdom.
EUROPEAN COMMITTEE FOR STANDARDIZATION
COMITÉ EUROPÉEN DE NORMALISATION
EUROPÄISCHES KOMITEE FÜR NORMUNG
CEN-CENELEC Management Centre: Rue de la Science 23, B-1040 Brussels
© 2021 CEN All rights of exploitation in any form and by any means reserved Ref. No. EN ISO 11132:2021 E
worldwide for CEN national Members.
---------------------- Page: 3 ----------------------
SIST EN ISO 11132:2021
EN ISO 11132:2021 (E)
Contents Page
European foreword . 3
2
---------------------- Page: 4 ----------------------
SIST EN ISO 11132:2021
EN ISO 11132:2021 (E)
European foreword
This document (EN ISO 11132:2021) has been prepared by Technical Committee ISO/TC 34 "Food
products" in collaboration with Technical Committee CEN/SS C01 “Food Products” the secretariat of
which is held by CCMC.
This European Standard shall be given the status of a national standard, either by publication of an
identical text or by endorsement, at the latest by March 2022, and conflicting national standards shall
be withdrawn at the latest by March 2022.
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. CEN shall not be held responsible for identifying any or all such patent rights.
This document supersedes EN ISO 11132:2017.
Any feedback and questions on this document should be directed to the users’ national standards
body/national committee. A complete listing of these bodies can be found on the CEN websites.
According to the CEN-CENELEC Internal Regulations, the national standards organizations of the
following countries are bound to implement this European Standard: Austria, Belgium, Bulgaria,
Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland,
Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Republic of
North Macedonia, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and the
United Kingdom.
Endorsement notice
The text of ISO 11132:2021 has been approved by CEN as EN ISO 11132:2021 without any modification.
3
---------------------- Page: 5 ----------------------
SIST EN ISO 11132:2021
---------------------- Page: 6 ----------------------
SIST EN ISO 11132:2021
INTERNATIONAL ISO
STANDARD 11132
Second edition
2021-09
Sensory analysis — Methodology —
Guidelines for the measurement of
the performance of a quantitative
descriptive sensory panel
Analyse sensorielle — Méthodologie — Lignes directrices pour le
mesurage de la performance d'un jury descriptif quantitatif
Reference number
ISO 11132:2021(E)
©
ISO 2021
---------------------- Page: 7 ----------------------
SIST EN ISO 11132:2021
ISO 11132:2021(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO 2021
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO 2021 – All rights reserved
---------------------- Page: 8 ----------------------
SIST EN ISO 11132:2021
ISO 11132:2021(E)
Contents Page
Foreword .iv
Introduction .vi
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Principle . 3
4.1 Two possible approaches . 3
4.1.1 General. 3
4.1.2 Performance measurement via a dedicated procedure . 3
4.1.3 Ongoing monitoring via routine product profiling . 4
4.2 Indicators of panel or individual assessor performance . 4
4.3 Statistical analyses . 5
5 Prerequisites . 5
5.1 Experimental conditions . 5
5.2 Qualification of assessors . 5
6 Performance measurement via a dedicated procedure . 5
6.1 Sample and attribute selection . 5
6.2 Experimental designs . 5
6.2.1 General. 5
6.2.2 Randomized block design . 6
6.2.3 Balanced and random designs . 6
6.2.4 Same order design . 6
6.3 Statistical analyses . 7
6.4 Performance of the overall panel — Interpretation of statistical output . 9
6.4.1 Key attribute discrimination . 9
6.4.2 Agreement at panel level . 9
6.4.3 Repeatability of the panel .10
6.5 Performance of individual assessors — Interpretation of statistical output .10
6.5.1 Discrimination ability of an assessor .10
6.5.2 Repeatability of an assessor .10
6.5.3 Consistency of an assessor .10
6.5.4 Agreement among assessors .11
6.5.5 Bias — Different use of scale .11
6.6 Performance issues .12
6.6.1 General.12
6.6.2 Panel .12
6.6.3 Individual assessor .12
6.7 Experimental design for following up the performance over time .12
7 Procedure for ongoing monitoring via routine product profiling .12
7.1 Attribute selection .12
7.2 Statistical analyses .12
7.3 Following up the performance over time .12
7.4 Statistical analysis of data over time .12
7.5 Statistical analysis of complete profiles .13
Annex A (informative) Example of a practical application .14
Bibliography .22
© ISO 2021 – All rights reserved iii
---------------------- Page: 9 ----------------------
SIST EN ISO 11132:2021
ISO 11132:2021(E)
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www .iso .org/ patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO’s adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www .iso .org/
iso/ foreword .html.
This document was prepared by Technical Committee ISO/TC 34, Food products, Subcommittee SC 12,
Sensory analysis, in collaboration with the European Committee for Standardization (CEN) Technical
Committee CEN/SS C01, Food Products, in accordance with the Agreement on technical cooperation
between ISO and CEN (Vienna Agreement).
This second edition cancels and replaces the first edition (ISO 11132:2012), which has been technically
revised. The main changes compared with the previous edition are as follows:
— the title has been changed to specify that the document is applicable to descriptive sensory panels;
— the Scope has been revised:
— in order to provide a distinction of application for validation and monitoring, with improved
wording to clarify;
— it has been reduced to measure repeatability only, and reproducibility has been stated to be out
of scope;
— the type of quantitative descriptive sensory panels for which the document is applicable to has
been specified;
— the definitions have been revised and new terminological entries have been added;
— the process for the dedicated procedure has been improved;
— experimental designs have been reviewed and augmented;
— statistical analyses related to analysis of variance have been reviewed and augmented to include
more models, especially regarding sessions and panellists (fixed or random) effects and interactions;
— the subclauses (specifically the original 6.4.4 and 7.4) and Annexes B and C related to reproducibility
have been removed to align with the changes to the Scope.
iv © ISO 2021 – All rights reserved
---------------------- Page: 10 ----------------------
SIST EN ISO 11132:2021
ISO 11132:2021(E)
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/ members .html.
© ISO 2021 – All rights reserved v
---------------------- Page: 11 ----------------------
SIST EN ISO 11132:2021
ISO 11132:2021(E)
Introduction
A panel of assessors can be used as an instrument to identify products’ sensory attributes and to assess
the magnitude of sensory attributes.
Performance is the measure of the ability of a panel or an assessor to make reliable and valid attribute
assessments across the products being evaluated. It can be assessed at a given time point, typically after
a training period (validation) or tracked over time (monitoring). Performance comprises the ability of
a panel to detect, identify and measure an attribute, use attributes in a similar way to other panels or
between assessors within a panel, discriminate between stimuli, use a scale properly, repeat their own
results, and reproduce results in comparison to other panels or assessors.
Measuring performance enables the panel leader to improve panel and assessor output, to identify
issues and retraining needs or to identify assessors who are not performing well enough to continue
participating.
vi © ISO 2021 – All rights reserved
---------------------- Page: 12 ----------------------
SIST EN ISO 11132:2021
INTERNATIONAL STANDARD ISO 11132:2021(E)
Sensory analysis — Methodology — Guidelines for the
measurement of the performance of a quantitative
descriptive sensory panel
1 Scope
This document gives guidelines for assessing the overall performance of a quantitative descriptive
panel and the performance of each panel member.
This document is applicable to the validation of the training of individual assessors or panels, as well as
to the performance monitoring of established panels.
This document does not apply to the panel performance for descriptive methods where the individual
scores of each assessor are not recorded, where there is no single list of attributes that is common to all
the assessors, or where dominance rather than intensity is measured. Consequently, the performance
of descriptive panels using methods such as consensus profile, free-choice profile, flash profile and
temporal dominance of sensations (TDS) are out of scope.
The methods specified in this document are for monitoring and assessing the ability of a panel and its
assessors to discriminate between products, the agreement between assessors of the same panel and
the repeatability of these assessors in their intensity scoring.
Reproducibility, including both the comparison between panels and the comparison within the same
panel of several evaluations conducted under different conditions (i.e. separated in time), is out of scope
of this document.
The methods specified in this document can be used, in full or a selection only, by the panel leader to
appraise continuously the performance of panels or individual assessors. The methods listed are not
exhaustive and other appropriate methods can also be used.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO 5492, Sensory analysis — Vocabulary
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO 5492 and the following apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
3.1
agreement
ability of different panels or assessors to exhibit the same product differences when assigning scores on
a given attribute to the same set of products
© ISO 2021 – All rights reserved 1
---------------------- Page: 13 ----------------------
SIST EN ISO 11132:2021
ISO 11132:2021(E)
3.2
panel drift
phenomenon where a panel, over time, changes in sensitivity or becomes susceptible to biases and as
a consequence changes the location on the scale where an attribute is rated for a constant, reference
product
3.3
performance
ability of a panel or an assessor to make reliable and valid assessments of stimuli and stimulus attributes
3.4
validation
process of establishing that a panel or assessor is able to meet specified performance (3.3) criteria
3.5
session
period of time in which products are assessed
Note 1 to entry: In a single session either one or several products may be assessed by one or several assessors.
For an assessor, whether alone or as part of a panel, sessions are separated in time.
[SOURCE: ISO 5492:2008/Amd.1:2016, 4.63]
3.6
replicate
occurrence of a particular condition in an experimental design
Note 1 to entry: The term usually implies that the occurrence is one of several of the same kind, but it can refer to
a single occurrence. When the condition is performed twice, the wording is “two repetitions”, etc.
Note 2 to entry: To specify more than one occurrence of a condition, the terms “replication” or “replicate session”
are more explicit.
Note 3 to entry: A “replicate session” is a session (3.5) in which the assessors, products, test conditions and task
are the same.
3.7
assessor bias
tendency of an assessor to give scores which are consistently above or below the true score when that is
known or the panel mean when it is not
[SOURCE: ISO 5492:2008/Amd.1:2016, 1.40]
3.8
order bias
arising from a product’s spatial or temporal position relative to a group of products being assessed
Note 1 to entry: The term includes both “position bias” and “sequential bias”.
[SOURCE: ISO 5492:2008/Amd.1:2016, 1.42]
3.9
repeatability
agreement (3.1) in assessments of the same products under the same test conditions by the same
assessor or panel
Note 1 to entry: Repeatability can be measured within one session (3.5) or over several distinctly separate
sessions, provided that the replicate (3.6) evaluations are conducted under the test conditions that can be
considered to be the same. If replicate evaluations are conducted in distinctly separate sessions/sittings, the
sessions are generally separated by several days only. In this case, the distinction between repeatability and
reproducibility (3.10) in the short term is minor and relates to the test conditions being considered the same or
not.
2 © ISO 2021 – All rights reserved
---------------------- Page: 14 ----------------------
SIST EN ISO 11132:2021
ISO 11132:2021(E)
[SOURCE: ISO 5492:2008/Amd.1:2016, 1.45, modified — Note 1 to entry has been added.]
3.10
reproducibility
agreement (3.1) in assessments of the same products under different test conditions or by different
assessors or panels
Note 1 to entry: Reproducibility may be measured as any of the following:
— the reproducibility of a panel (or an assessor) in the short term, measured between two or more sessions (3.5)
separated by several days;
— the reproducibility of a panel (or an assessor) in the medium or long term, measured among sessions separated
by several months;
— the reproducibility between different panels, in the same laboratory or in different laboratories.
[SOURCE: ISO 5492:2008/Amd.1:2016, 1.46]
4 Principle
4.1 Two possible approaches
4.1.1 General
This document is concerned with sensory panels used to assess the magnitude of one or more sensory
attributes in order to make quantitative descriptions or profiles of products (see ISO 13299). Different
methods are appropriate to the measurement of the performance of panels used for difference testing.
The performance of a quantitative sensory panel may be evaluated from panel sessions conducted
specifically for the purpose of obtaining performance (called “dedicated procedure”) by using
assessments already available (called “ongoing monitoring”).
4.1.2 Performance measurement via a dedicated procedure
A dedicated procedure is the method of choice for the certification of individual assessors and for other
validation purposes. For the certification renewal, this dedicated procedure should be repeated at
periodic intervals, as needed. Figure 1 is a flow chart for this procedure.
This approach can typically be used at the end of the training phase of a panel to ensure that the panel
and the individual assessors have achieved the desired level of performance and can be considered as
trained sensory assessors or expert sensory assessors (depending on the performance criteria).
© ISO 2021 – All rights reserved 3
---------------------- Page: 15 ----------------------
SIST EN ISO 11132:2021
ISO 11132:2021(E)
Figure 1 — Process steps for the performance measurement via a dedicated procedure
4.1.3 Ongoing monitoring via routine product profiling
Another approach consists of monitoring profile data that was already collected. To review ongoing
profile data generated by a panel, it can be appropriate to use data that originated from quite different
profiling experiments using different product types, product numbers, etc. The procedure is the
same as that shown in Figure 1. However, as there are no predefined differences, it is recommended
that attributes for which the products are significantly discriminated by the panel as a whole for a
given profile be used as the key measures to check the performance of individual sensory assessors.
Attributes that result in no significant difference cannot be reliably used to check consistency since the
lack of agreement within and between sensory assessors probably means that the products are very
similar for those characteristics.
In this case, over a given period, it will be necessary to check on a set of products more different that
the panel is indeed capable of highlighting difference in these characteristics.
4.2 Indicators of panel or individual assessor performance
For one assessment, the following indicators can be determined:
— discrimination of the panel, measured as the ability of the panel to exhibit significant differences
among products;
— discrimination of an assessor, measured as the ability of the assessor to exhibit significant differences
among products;
— agreement of an assessor, measured as the degree of alignment between the assessor’s average
product scores and the ones of the panel;
— agreement of the panel, measured as the degree of alignment between the assessors’ average
product scores.
For replicate assessments:
— repeatability of an assessor, measured as the degree of homogeneity between replicated assessments
of the same product;
4 © ISO 2021 – All rights reserved
---------------------- Page: 16 ----------------------
SIST EN ISO 11132:2021
ISO 11132:2021(E)
— repeatability of a panel, measured as the average degree of homogeneity between replicated
assessments of the same product for each assessor.
4.3 Statistical analyses
A single, consistent approach to statistical analysis of the results is described in this document. However,
some indicators of panel performance can be assessed by more than one measure. For instance, error
mean square and error standard deviation (SD) (its square root) both express variability in the
evaluation of a product
...
SLOVENSKI STANDARD
oSIST prEN ISO 11132:2020
01-marec-2020
Senzorična analiza - Metodologija - Smernice za nadzorovanje izvajanja
kvantitativnega senzoričnega panelnega testa (ISO/DIS 11132:2020)
Sensory analysis - Methodology - Guidelines for monitoring the performance of a
quantitative sensory panel (ISO/DIS 11132:2020)
Sensorische Analyse - Methodologie - Leitlinien zur Überprüfung der Leistungsfähigkeit
eines quantitativen sensorischen Panels (ISO/DIS 11132:2020)
Analyse sensorielle - Méthodologie - Lignes directrices pour le contrôle de la
performance d'un jury sensoriel quantitatif (ISO/DIS 11132:2020)
Ta slovenski standard je istoveten z: prEN ISO 11132
ICS:
67.240 Senzorična analiza Sensory analysis
oSIST prEN ISO 11132:2020 en
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.
---------------------- Page: 1 ----------------------
oSIST prEN ISO 11132:2020
---------------------- Page: 2 ----------------------
oSIST prEN ISO 11132:2020
DRAFT INTERNATIONAL STANDARD
ISO/DIS 11132
ISO/TC 34/SC 12 Secretariat: IRAM
Voting begins on: Voting terminates on:
2020-01-30 2020-04-23
Sensory analysis — Methodology — Guidelines for
monitoring the performance of a quantitative sensory panel
Analyse sensorielle — Méthodologie — Lignes directrices pour le contrôle de la performance d'un jury
sensoriel quantitatif
ICS: 67.240
THIS DOCUMENT IS A DRAFT CIRCULATED
This document is circulated as received from the committee secretariat.
FOR COMMENT AND APPROVAL. IT IS
THEREFORE SUBJECT TO CHANGE AND MAY
NOT BE REFERRED TO AS AN INTERNATIONAL
STANDARD UNTIL PUBLISHED AS SUCH.
IN ADDITION TO THEIR EVALUATION AS
ISO/CEN PARALLEL PROCESSING
BEING ACCEPTABLE FOR INDUSTRIAL,
TECHNOLOGICAL, COMMERCIAL AND
USER PURPOSES, DRAFT INTERNATIONAL
STANDARDS MAY ON OCCASION HAVE TO
BE CONSIDERED IN THE LIGHT OF THEIR
POTENTIAL TO BECOME STANDARDS TO
WHICH REFERENCE MAY BE MADE IN
Reference number
NATIONAL REGULATIONS.
ISO/DIS 11132:2020(E)
RECIPIENTS OF THIS DRAFT ARE INVITED
TO SUBMIT, WITH THEIR COMMENTS,
NOTIFICATION OF ANY RELEVANT PATENT
RIGHTS OF WHICH THEY ARE AWARE AND TO
©
PROVIDE SUPPORTING DOCUMENTATION. ISO 2020
---------------------- Page: 3 ----------------------
oSIST prEN ISO 11132:2020
ISO/DIS 11132:2020(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO 2020
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO 2020 – All rights reserved
---------------------- Page: 4 ----------------------
oSIST prEN ISO 11132:2020
ISO/DIS 11132:2020(E)
Contents Page
Foreword .iv
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Principle . 2
4.1 Two possible approaches . 2
4.1.1 Panel validation via a dedicated procedure. 2
4.1.2 On-going monitoring via routine product profiling. 3
4.2 Indicators of panel or individual assessor performance . 3
4.3 Statistical analyses . 4
5 Prerequisites . 4
5.1 Experimental conditions . 4
5.2 Qualification of assessors . 4
6 Procedure for performance validation . 4
6.1 Sample and attribute selection . 4
6.2 Experimental designs . 4
6.2.1 Randomized block design . 4
6.2.2 Balanced and random designs . 5
6.2.3 Same order design . 5
6.3 Statistical analysis of data from performance validation (a single session) . 6
6.4 Overall panel performance from performance validation . 7
6.4.1 Key attribute discrimination . 7
6.4.2 Agreement at panel level . 7
6.4.3 Repeatability of the panel . 8
6.4.4 Reproducibility of the panel. 8
6.5 Individual assessor performance from performance validation . 9
6.5.1 Discrimination ability of an assessor . 9
6.5.2 Repeatability of an assessor . 9
6.5.3 Consistency of an assessor . 9
6.5.4 Agreement among assessors . 9
6.5.5 Different use of scale/bias .10
6.6 Performance issues .10
6.6.1 General.10
6.6.2 Panel .10
6.6.3 Individual assessor .10
7 Procedure for on-going monitoring via routine product profiling .10
7.1 Attribute selection .10
7.2 Experimental design for study of performance over time .10
7.3 Statistical analysis of data over time .11
7.4 Reproducibility between panels .11
7.5 Statistical analysis of complete profiles .11
Annex A (informative) Example of practical application .12
Annex B (informative) Example of use of cusum analysis .19
Annex C (informative) Example of use of Shewhart chart .22
Bibliography .24
© ISO 2020 – All rights reserved iii
---------------------- Page: 5 ----------------------
oSIST prEN ISO 11132:2020
ISO/DIS 11132:2020(E)
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www .iso .org/ patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see the following
URL: www .iso .org/ iso/ foreword .html.
This document was prepared by Technical Committee ISO/TC 34, Food products, Subcommittee SC 12,
Sensory analysis.
This second edition cancels and replaces the first edition (ISO 11132:2012), which has been technically
revised.
iv © ISO 2020 – All rights reserved
---------------------- Page: 6 ----------------------
oSIST prEN ISO 11132:2020
DRAFT INTERNATIONAL STANDARD ISO/DIS 11132:2020(E)
Sensory analysis — Methodology — Guidelines for
monitoring the performance of a quantitative sensory panel
1 Scope
This International Standard gives guidelines for monitoring and assessing the overall performance of a
quantitative descriptive panel and the performance of each member.
A panel of assessors can be used as an instrument to identify products’ sensory attributes and to assess
the magnitude of sensory attributes.
Performance is the measure of the ability of a panel or an assessor to make reliable and valid attribute
assessments across the products being evaluated. It can be monitored at a given time point or tracked
over time. Performance comprises the ability of a panel to detect, identify, and measure an attribute, use
attributes in a similar way to other panels or between assessors within a panel, discriminate between
stimuli, use a scale properly, repeat their own results, and reproduce results in comparison to other
panels or assessors.
The methods specified allow to monitor and assess the consistency, repeatability, freedom from bias
and the ability of discrimination of panels and assessors. Monitoring and assessment can be carried out
in one session or over time.
Monitoring performance data enables the panel leader to improve panel and assessor performance, to
identify issues and retraining needs or to identify assessors who are not performing well enough to
continue participating.
The methods specified in this International Standard can be used, in full or a selection only, by the
panel leader to appraise continuously the performance of panels or individual assessors. The methods
listed are not exhaustive and other appropriate methods may also be used.
This International Standard applies to individuals or panels in training as well as for established panels.
2 Normative references
The following referenced documents are indispensable for the application of this document. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced
document (including any amendments) applies.
ISO 5492, Sensory analysis — Vocabulary
ISO 8586, Sensory analysis — General guidelines for the selection, training and monitoring of selected
assessors and expert sensory assessors
ISO 8589, Sensory analysis — General guidance for the design of test rooms
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO 5492 and the following apply.
3.1 agreement
ability of different panels or assessors to exhibit the same product differences when assigning scores on
a given attribute to the same set of products.
© ISO 2020 – All rights reserved 1
---------------------- Page: 7 ----------------------
oSIST prEN ISO 11132:2020
ISO/DIS 11132:2020(E)
3.2 panel drift
phenomenon where a panel, over time, changes in sensitivity or becomes susceptible to biases and as a
consequence changes the location on the scale where an attribute is rated for a constant, reference product
3.3 performance
ability of a panel or an assessor to make reliable and valid assessments of stimuli and stimulus attributes
3.4 validation
process of establishing that sensory data correlate with other data on samples of the same product (e.g.
laboratory measurements, consumer perception, results from other panels, consumer complaints) or
that a panel or assessor is able to meet specified performance criteria
3.5 replicate sessions
sessions in which the assessors, the products, the test conditions, and the task are the same
4 Principle
4.1 Two possible approaches
This International Standard is concerned with sensory panels used to assess the magnitude of one or
more sensory attributes in order to make quantitative descriptions or profiles of products. Different
methods are appropriate to the assessment and monitoring of the performance of panels used for
difference testing.
The performance of a quantitative sensory panel may be evaluated by using assessments already
available or from panel sessions conducted specifically for the purpose of obtaining performance data.
4.1.1 Panel validation via a dedicated procedure
A dedicated monitoring procedure at periodic intervals is appropriate for accreditation and other
purposes. Figure 1 is a flow chart for this procedure.
2 © ISO 2020 – All rights reserved
---------------------- Page: 8 ----------------------
oSIST prEN ISO 11132:2020
ISO/DIS 11132:2020(E)
Figure 1 — Process steps for the performance validation
4.1.2 On-going monitoring via routine product profiling
To review ongoing profile data generated by a panel, it can be appropriate to use data that originated
from quite different profiling experiments using different product types, product numbers, etc. The
procedure is the same as that shown in Figure 1. However, as there are no predefined differences,
it is recommended that attributes that are significantly discriminated by the panel as a whole for a
given profile be used as the key measures to check the performance of individual panelists. Attributes
that result in no significant difference cannot be reliably used to check consistency since the lack of
agreement within and between panelists probably means that the products are very similar for those
characteristics.
4.2 Indicators of panel or individual assessor performance
In a single session, the following indicators can be determined:
— Discrimination of an assessor, measured as the ability of the assessor to exhibit significant
differences among products.
— Discrimination of the panel, measured as the ability of the panel to exhibit significant differences
among products.
— Agreement of an assessor, measured as the degree of alignment between the assessor’s average
product scores and the ones of the panel
— Agreement of the panel, measured as the degree of alignment between the assessors’ average
product scores
— Repeatability of an assessor, measured as the degree of homogeneity between replicated assessments
of the same product.
Across several sessions, the following indicator can be determined:
— Reproducibility of the panel, Repeatability of the panel, measured as the agreement in assessments
of the same set of products under similar test conditions by the same or different assessors at
different time points.
© ISO 2020 – All rights reserved 3
---------------------- Page: 9 ----------------------
oSIST prEN ISO 11132:2020
ISO/DIS 11132:2020(E)
4.3 Statistical analyses
A single, consistent approach to statistical analysis of the results is described in this document. However,
some indicators of panel performance can be assessed by more than one measure. For instance, error
mean square and error SD (its square root) both express variability in the evaluation of a product. The
measures used should be those that are usual in the field of application.
Other relevant measures of agreement between assessors in the use of the scale for an attribute are the
interaction of assessor and product and the coefficient of correlation between an assessor’s scores and
the panel means. An assessor may have no bias, but may use the scale in a different way. A correlation
close to 1, a regression slope close to 1, and a regression intercept close to 0 indicate good agreement
between an assessor and the rest of the panel.
With a small number of assessments (fewer than six) the correlation coefficient should be interpreted
with caution, as it can be high (up to 0,7), by chance alone.
5 Prerequisites
5.1 Experimental conditions
The test facilities shall be in accordance with ISO 8589.
5.2 Qualification of assessors
The panel shall have the level of qualification and experience of selected assessors (ISO 8586) or better.
6 Procedure for performance validation
6.1 Sample and attribute selection
At each validation session, the panel of assessors should be presented with a set of samples similar
to those the panel are to assess when evaluating products and for which statistically significant
differences between at least one pair of the samples are expected for the list of relevant attributes.
As much as possible, it is recommended to use at least eight attributes to encourage panel leaders or
sensory managers to identify and select validation samples aligned with the level of difference among
products the panel is expected to detect.
These key attributes are used as key measures against which to assess panel performance. The sample
set should include replicates. There shall be the same number of replicates of each sample. The number
of assessors, samples, and replicates depends on the products, the sensory attributes assessed and
the purpose of the procedure. For example, 2 or 3 replicates of three or four samples might be used.
Care should be taken to limit the number of assessments required so as to avoid sensory fatigue.
The attributes of the samples should be similar to the range of values that the panel assesses when
evaluating products.
6.2 Experimental designs
Several types of experimental designs can be used in the performance validation procedure, depending
on the most important objective to answer.
6.2.1 Randomized block design
A randomized block experimental design can be used, in which the assessors are the "blocks". This
design is appropriate when no carry-over effect from one sample to the next is expected, otherwise a
balanced design should be used instead (see 6.2.2.).
4 © ISO 2020 – All rights reserved
---------------------- Page: 10 ----------------------
oSIST prEN ISO 11132:2020
ISO/DIS 11132:2020(E)
6.2.2 Balanced and random designs
If a carry-over effect is expected from one sample to the next, a suitable experimental design is the
Williams Latin square, see table 1. The basic design uses four assessors and four samples.
Table 1 — Williams Latin square
Order
Assessor Session
1 2 3 4
1 1 A B C D
2 1 B D A C
3 1 C A D B
4 1 D C B A
1 2 B D A C
2 2 C A D B
3 2 D C B A
4 2 A B C D
In this design, each assessor samples the four products in a different order in a given session and any
particular product is followed by a different one for each assessor, for example in session 1, A is followed
by B for assessor 1, C for assessor 2, D for assessor 3 and none for assessor 4.
For each replicate of the products’ evaluation, it is recommended to use a different product order for
each assessor, in order to reduce the order effect and the carry-over effect.
If multiples of four assessors are available, the same design can be repeated for each set of four.
It is also possible to choose a random product order design, i.e. to randomly affect each product to each
position in each session.
The advantage of these approaches is to minimize the carry-over effect at panel level and therefore
get better estimates of the product means at panel level for performance evaluation. However, if the
product order does have an impact, the agreement between assessors will be impacted because each
assessor will not experience the same product order. In order to compare the assessors on the exact
same task, one can use the same product order for all panellists (see 6.2.3.).
6.2.3 Same order design
In order to focus on individual assessor performance and in an effort to compare assessors under the
most similar conditions, an alternative design is proposed, whereby all assessors evaluate the products
in the same order, see Table 2.
© ISO 2020 – All rights reserved 5
---------------------- Page: 11 ----------------------
oSIST prEN ISO 11132:2020
ISO/DIS 11132:2020(E)
Table 2 — Same order design
Order
Assessor Session
1 2 3 4
1 1 A B C D
2 1 A B C D
3 1 A B C D
4 1 A B C D
1 2 A B C D
2 2 A B C D
3 2 A B C D
4 2 A B C D
It is worth mentioning that in this case the assessors are not evaluating the products per se but the
products at a given position (product and position effects are confounded). This will conduct to a biased
estimate of the product effect (biased by the order effect), but to an unbiased estimate of the assessor
effect and the product*assessor interaction.
6.3 Statistical analysis of data from performance validation (a single session)
Table 3 illustrates one way to tabulate and summarize the results. Some computer software may require
a different organization of the data, for instance with the samples in columns and the assessors in rows.
Table 3 — Results of the assessors
Assessor
Sample 1 … j … n Mean
q
Scores Mean Scores Mean Scores Mean
Y Y
111 1j1
Y Y
112 1j2
Y
1 Y Y
1.j
11. 1.
… …
Y Y
11n 1jn
r r
…
Y Y
i11 ij1
Y Y
i12 ij2
Y
i Y Y
ij.
i1. i .
… …
Y Y
in1 ijn
r r
…
Y .
n
p
Error!
n
p
Not a
valid
link.
Y .
Y
Y . Y
Mean
n
..j
1 .
q
In this table it is assumed that:
n ≡ number of samples (i = 1,2 … n );
p p
n ≡ number of assessors ( j = 1,2 … n );
q q
n ≡ number of replicates per sample (k = 1,2 … n ).
r r
6 © ISO 2020 – All rights reserved
---------------------- Page: 12 ----------------------
oSIST prEN ISO 11132:2020
ISO/DIS 11132:2020(E)
Measures of the performance of the panel as a whole and individual assessors, other than bias, require
the data to be analysed by ANOVA.
The details of the basic calculations are not shown in this International Standard, since the analyses are
normally carried out by a computer package.
Each assessor’s data are analysed by one-way ANOVA (Table 4).
Table 4 — ANOVA for an individual assessor for one attribute
Source of variation Degrees of freedom Sum of squares Mean square F-ratio
Between samples v = n − 1 S MS = s /v
1 p 1 1 1 1
F = MS /MS
1 2
Error v = n (n − 1) S MS = s /v
2 p r 2 2 2 2
Total v = n n − 1 S
3 p r 3
n ≡ number of samples
p
n ≡ number of replicates per sample
r
The data for the complete session are analysed by randomized block ANOVA (Table 5).
Table 5 — ANOVA for a complete data set (with repetitions) for one attribute
Source of variation Degrees of freedom Sum of squares Mean square F-ratio
Between samples v = n − 1 S MS = s /v
4 p 4 4 4 4
a
F = MS /MS
4 7
Between assessors v = n − 1 S MS = s /v
5 q 5 5 5 5
a
F = MS /MS
5 7
Interaction v = (n − 1)(n − 1) S MS = s /v
6 p q 6 6 6 6
F = MS /MS
6 7
Error v = n n (n − 1) S MS = s /v
7 p q r 7 7 7 7
Total v = n n n − 1 S
8 p q r 8
n ≡ number of samples
p
n ≡ number of assessors
q
n ≡ number of replicates per sample
r
a
Formulas are given considering the assessor effect as fixed. Considering a random assessor effect, the denominator of
the F ratio for the between samples effect becomes MS instead of MS .
6 7
6.4 Overall panel performance from performance validation
6.4.1 Key attribute discrimination
The proportion of key attributes that have been significantly discriminated as expected should be
determined. For each attribute, this is indicated by significant variation between samples at a level of
0,05 in the ANOVA table for a data set (Table 5). The higher the proportion of key attributes significantly
discriminated, the better the panel is performing. The panel should receive further training on key
attributes that are not significantly discriminated as expected.
6.4.2 Agreement at panel level
A panel is not in agreement when any assessor is in disagreement with the rest of the panel (see 7.4.4.
agreement among assessors).
A panel is not in agreement if the interaction of sample and assessor in the ANOVA is significant at a
level of 0,05.
© ISO 2020 – All rights reserved 7
---------------------- Page: 13 ----------------------
oSIST prEN ISO 11132:2020
ISO/DIS 11132:2020(E)
The degree of agreement of the panel is inversely related to the interaction SD, s .
i
MS −MS
67
s =
i
n
r
See Table 5.
The number of key attributes giving significant interaction of sample and assessor is determined based
on the ANOVA (table 5) for each attribute. The higher the number of key attributes giving significant
interaction, the less consistently the panel is performing. At least, the panel should receive further
training on key attributes that are giving significant interaction and not significant product effect.
6.4.3 Repeatability of the panel
The repeatability of the panel can be estimated from the repeatability of the individual assessors. This
is inversely related to the error SD, s :
e
sM= S
e 7
See Table 5.
6.4.4 Reproducibility of the panel
To check for reproducibility of the panel, make evaluations of ot
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.