SIST EN ISO 16140-2:2016/A1:2024
(Amendment)Microbiology of the food chain - Method validation - Part 2: Protocol for the validation of alternative (proprietary) methods against a reference method - Amendment 1 (ISO 16140-2:2016/Amd 1:2024)
Microbiology of the food chain - Method validation - Part 2: Protocol for the validation of alternative (proprietary) methods against a reference method - Amendment 1 (ISO 16140-2:2016/Amd 1:2024)
Mikrobiologie der Lebensmittelkette - Verfahrensvalidierung - Teil 2: Arbeitsvorschrift für die Validierung von alternativen (urheberrechtlich geschützten) Verfahren anhand eines Referenzverfahrens (ISO 16140-2:2016/Amd 1:2024)
Microbiologie de la chaîne alimentaire - Validation des méthodes - Partie 2: Protocole pour la validation de méthodes alternatives (commerciales) par rapport à une méthode de référence - Amendement 1 (ISO 16140-2:2016/Amd 1:2024)
Mikrobiologija v prehranski verigi - Validacija metode - 2. del: Protokol za validacijo alternativnih (lastniških) metod glede na referenčno metodo - Dopolnilo A1 (ISO 16140-2:2016/Amd 1:2024)
General Information
Relations
Standards Content (Sample)
SLOVENSKI STANDARD
01-november-2024
Mikrobiologija v prehranski verigi - Validacija metode - 2. del: Protokol za
validacijo alternativnih (lastniških) metod glede na referenčno metodo - Dopolnilo
A1 (ISO 16140-2:2016/Amd 1:2024)
Microbiology of the food chain - Method validation - Part 2: Protocol for the validation of
alternative (proprietary) methods against a reference method - Amendment 1 (ISO
16140-2:2016/Amd 1:2024)
Mikrobiologie der Lebensmittelkette - Verfahrensvalidierung - Teil 2: Arbeitsvorschrift für
die Validierung von alternativen (urheberrechtlich geschützten) Verfahren anhand eines
Referenzverfahrens (ISO 16140-2:2016/Amd 1:2024)
Microbiologie de la chaîne alimentaire - Validation des méthodes - Partie 2: Protocole
pour la validation de méthodes alternatives (commerciales) par rapport à une méthode
de référence - Amendement 1 (ISO 16140-2:2016/Amd 1:2024)
Ta slovenski standard je istoveten z: EN ISO 16140-2:2016/A1:2024
ICS:
07.100.30 Mikrobiologija živil Food microbiology
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.
EN ISO 16140-2:2016/A1
EUROPEAN STANDARD
NORME EUROPÉENNE
September 2024
EUROPÄISCHE NORM
ICS 07.100.30
English Version
Microbiology of the food chain - Method validation - Part 2:
Protocol for the validation of alternative (proprietary)
methods against a reference method - Amendment 1 (ISO
16140-2:2016/Amd 1:2024)
Microbiologie de la chaîne alimentaire - Validation des Mikrobiologie der Lebensmittelkette -
méthodes - Partie 2: Protocole pour la validation de Verfahrensvalidierung - Teil 2: Arbeitsvorschrift für die
méthodes alternatives (commerciales) par rapport à Validierung von alternativen (urheberrechtlich
une méthode de référence - Amendement 1(ISO 16140 geschützten) Verfahren anhand eines
2:2016/Amd 1:2024) Referenzverfahrens - ÄNDERUNG 1(ISO 16140
2:2016/Amd 1:2024)
This amendment A1 modifies the European Standard EN ISO 16140-2:2016; it was approved by CEN on 25 July 2024.
CEN members are bound to comply with the CEN/CENELEC Internal Regulations which stipulate the conditions for inclusion of
this amendment into the relevant national standard without any alteration. Up-to-date lists and bibliographical references
concerning such national standards may be obtained on application to the CEN-CENELEC Management Centre or to any CEN
member.
This amendment exists in three official versions (English, French, German). A version in any other language made by translation
under the responsibility of a CEN member into its own language and notified to the CEN-CENELEC Management Centre has the
same status as the official versions.
CEN members are the national standards bodies of Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia,
Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway,
Poland, Portugal, Republic of North Macedonia, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Türkiye and
United Kingdom.
EUROPEAN COMMITTEE FOR STANDARDIZATION
COMITÉ EUROPÉEN DE NORMALISATIO N
EUROPÄISCHES KOMITEE FÜR N O RMU N G
CEN-CENELEC Management Centre: Rue de la Science 23, B-1040 Brussels
© 2024 CEN All rights of exploitation in any form and by any means reserved Ref. No. EN ISO 16140-2:2016/A1:2024 E
worldwide for CEN national Members.
Contents Page
European foreword . 3
European foreword
This document (EN ISO 16140-2:2016/A1:2024) has been prepared by Technical Committee ISO/TC 34
"Food products" in collaboration with Technical Committee CEN/TC 463 “Microbiology of the food
chain” the secretariat of which is held by AFNOR.
This Amendment to the European Standard EN ISO 16140-2:2016 shall be given the status of a national
standard, either by publication of an identical text or by endorsement, at the latest by March 2025, and
conflicting national standards shall be withdrawn at the latest by March 2025.
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. CEN shall not be held responsible for identifying any or all such patent rights.
Any feedback and questions on this document should be directed to the users’ national standards
body/national committee. A complete listing of these bodies can be found on the CEN website.
According to the CEN-CENELEC Internal Regulations, the national standards organizations of the
following countries are bound to implement this European Standard: Austria, Belgium, Bulgaria,
Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland,
Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Republic of
North Macedonia, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Türkiye and the
United Kingdom.
Endorsement notice
The text of ISO 16140-2:2016/Amd 1:2024 has been approved by CEN as EN ISO 16140-
2:2016/A1:2024 without any modification.
International
Standard
ISO 16140-2
First edition
Microbiology of the food chain —
2016-06-15
Method validation —
AMENDMENT 1
Part 2:
2024-09
Protocol for the validation of
alternative (proprietary) methods
against a reference method
AMENDMENT 1: Revision of
qualitative method comparison
study data evaluation, relative level
of detection calculations in the
interlaboratory study, calculation and
interpretation of the relative trueness
study, and inclusion of a commercial
sterility testing protocol for specific
products
Microbiologie de la chaîne alimentaire — Validation des
méthodes —
Partie 2: Protocole pour la validation de méthodes alternatives
(commerciales) par rapport à une méthode de référence
AMENDEMENT 1: Révision de l’évaluation des données des études
de comparaison de méthodes qualitatives, des calculs du niveau
de détection de l'étude interlaboratoires et de l’interprétation
de l’étude de justesse relative, et ajout d’un protocole pour la
détermination de la stérilité commerciale pour des produits
spécifiques
Reference number
ISO 16140-2:2016/Amd.1:2024(en) © ISO 2024
ISO 16140-2:2016/Amd.1:2024(en)
© ISO 2024
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on
the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below
or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii
ISO 16140-2:2016/Amd.1:2024(en)
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out through
ISO technical committees. Each member body interested in a subject for which a technical committee
has been established has the right to be represented on that committee. International organizations,
governmental and non-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely
with the International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are described
in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the different types
of ISO document should be noted. This document was drafted in accordance with the editorial rules of the
ISO/IEC Directives, Part 2 (see www.iso.org/directives).
ISO draws attention to the possibility that the implementation of this document may involve the use of (a)
patent(s). ISO takes no position concerning the evidence, validity or applicability of any claimed patent
rights in respect thereof. As of the date of publication of this document, ISO had not received notice of (a)
patent(s) which may be required to implement this document. However, implementers are cautioned that
this may not represent the latest information, which may be obtained from the patent database available at
www.iso.org/patents. ISO shall not be held responsible for identifying any or all such patent rights.
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and expressions
related to conformity assessment, as well as information about ISO’s adherence to the World Trade
Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www.iso.org/iso/foreword.html.
This document was prepared by Technical Committee ISO/TC 34, Food products, Subcommittee SC 9,
Microbiology, in collaboration with the European Committee for Standardization (CEN) Technical Committee
CEN/TC 463, Microbiology of the food chain, in accordance with the Agreement on technical cooperation
between ISO and CEN (Vienna Agreement).
A list of all parts in the ISO 16140 series can be found on the ISO website.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www.iso.org/members.html.
iii
ISO 16140-2:2016/Amd.1:2024(en)
Microbiology of the food chain — Method validation —
Part 2:
Protocol for the validation of alternative (proprietary)
methods against a reference method
AMENDMENT 1: Revision of qualitative method comparison study
data evaluation, relative level of detection calculations in the
interlaboratory study, calculation and interpretation of the relative
trueness study, and inclusion of a commercial sterility testing
protocol for specific products
Introduction
Replace the text with the following:
Introduction
0.1 The ISO 16140 series
The ISO 16140 series has been expanded in response to the need for various ways to validate or verify test
methods. It is the successor to ISO 16140:2003. The ISO 16140 series consists of six parts with the general
title, Microbiology of the food chain — Method validation:
— Part 1: Vocabulary;
— Part 2: Protocol for the validation of alternative (proprietary) methods against a reference method;
— Part 3: Protocol for the verification of reference methods and validated alternative methods in a single
laboratory;
— Part 4: Protocol for method validation in a single laboratory;
— Part 5: Protocol for factorial interlaboratory validation for non-proprietary methods;
— Part 6: Protocol for the validation of alternative (proprietary) methods for microbiological confirmation and
typing procedures.
ISO 17468 is a closely linked International Standard, which establishes technical rules for the development
and validation of standardized methods.
In general, two stages are needed before a method can be used in a laboratory:
— The first stage is the validation of the method. Validation is conducted using a study in a single laboratory
followed by an interlaboratory study (see this document, ISO 16140-5 and ISO 16140-6). In the case when
a method is validated within one laboratory (see ISO 16140-4), no interlaboratory study is conducted.
— The second stage is method verification, where a laboratory demonstrates that it can satisfactorily
perform a validated method. This is described in ISO 16140-3. Verification is only applicable to methods
that have been validated using an interlaboratory study.
ISO 16140-2:2016/Amd.1:2024(en)
In general, two types of methods are distinguished: reference methods and alternative methods.
A reference method is defined in ISO 16140-1:2016, 2.59, as an “internationally recognized and widely
accepted method”. The note to entry clarifies that “these are ISO standards and standards jointly published
by ISO and CEN or other regional/national standards of equivalent standing”.
In the ISO 16140 series, reference methods include standardized reference (ISO and CEN) methods as
defined in ISO 17468:2023, 3.7, as a “reference method described in a standard”.
An alternative method (method submitted for validation) is defined in ISO 16140-1:2016, 2.4, as a “method
of analysis that detects or quantifies, for a given category of products, the same analyte as is detected or
quantified using the corresponding reference method”. The note to entry clarifies that: “The method can be
proprietary. The term ‘alternative’ is used to refer to the entire ‘test procedure and reaction system’. This
term includes all ingredients, whether material or otherwise, required for implementing the method.”
ISO 16140-4 addresses validation within a single laboratory. The results are therefore only valid for the
laboratory that conducted the study. In this case, verification (as described in ISO 16140-3) is not applicable.
ISO 16140-5 describes protocols for non-proprietary methods where a more rapid validation is required or
when the method to be validated is highly specialized and the number of participating laboratories required
by this document cannot be reached. ISO 16140-4 and ISO 16140-5 can be used for validation against a
reference method. ISO 16140-4 (regarding qualitative and quantitative methods) and ISO 16140-5 (regarding
quantitative methods only) can also be used for validation without a reference method.
The flow chart in Figure 0.1 gives an overview of the links between the different parts mentioned above. It
also guides the user in selecting the right part of the ISO 16140 series, taking into account the purpose of the
study and the remarks given above.
Figure 0.1 — Flow chart for application of the ISO 16140 series
ISO 16140-2:2016/Amd.1:2024(en)
NOTE In this document, the words “category”, “type” and/or “item” are sometimes combined with “(food)” to
improve readability. However, the word “(food)” is interchangeable with “(feed)” and other areas of the food chain as
mentioned in Clause 1.
ISO 16140-6 is somewhat different from the other parts in the ISO 16140 series in that it relates to a very
specific situation where only the confirmation procedure of a method is to be validated [e.g. the biochemical
confirmation of Enterobacteriaceae (see ISO 21528-2)]. The confirmation procedure advances a suspected
(presumptive) result to a confirmed positive result. The validation of alternative typing techniques (e.g.
serotyping of Salmonella) is also covered by ISO 16140-6. The validation study in ISO 16140-6 clearly defines
the selective agar(s) from which strains can be confirmed using the alternative confirmation method. If
successfully validated, the alternative confirmation method can only be used if strains are recovered on an
agar that was used and shown to be acceptable within the validation study. Figure 0.2 shows the possibilities
where an alternative confirmation method validated in accordance with ISO 16140-6 can be applied (see
text in the boxes).
Figure 0.2 — Use of validated alternative confirmation methods (see ISO 16140-6)
EXAMPLE An example application of a validated alternative confirmation method is as follows.
An alternative confirmation method based on ELISA has been validated (in accordance with ISO 16140-6) to replace
the biochemical confirmation for Salmonella as described in ISO 6579-1. In the validation study, XLD (mandatory agar
in accordance with ISO 6579-1) plus BGA and a specified chromogenic agar (two optional agars for second plating in
accordance with ISO 6579-1) were used as the agars to start the confirmation. The validated confirmation method can
be used to replace the biochemical confirmation under the following conditions:
— by laboratories using ISO 6579-1; or
— by laboratories using an ISO 16140-2 validated alternative method that refers to ISO 6579-1 for confirmation; or
— by laboratories using an ISO 16140-2 validated alternative method that starts the confirmation from XLD and/or
BGA agar and/or the specified chromogenic agar.
The validated confirmation method cannot be used under the following conditions:
— by laboratories using an ISO 16140-2 validated alternative method that refers only to agars other than those
included in the validation to start the confirmation (e.g. Hektoen agar and SS agar only); or
— by laboratories using an ISO 16140-2 validated alternative method that refers only to a confirmation procedure
that does not require isolation on agar.
ISO 16140-2:2016/Amd.1:2024(en)
0.2 Validation protocols in the ISO 16140 series
This document describes the general approach to method validation in the field of microbiology of the food
chain and serves as a fundamental basis to the other parts of the ISO 16140 series, which cross-reference to
it. An understanding of the performance characteristics, the (food) categories, the technical protocol and
data analysis as outlined in this document provides support in the application of the ISO 16140 series in
general.
Clause 4
Add the following text at the end of the clause:
For the validation of an alternative qualitative method, a corresponding qualitative reference method
is selected for carrying out the validation study. This is commonly done using test portions of 10 gram,
25 gram or higher. In some cases, it can be of interest to validate a qualitative alternative method against a
quantitative reference method, using smaller test portion sizes.
EXAMPLE 1 Enterobacteriaceae criterion for pasteurized milk and other liquid pasteurized products in Regulation
(EC) No 2073/2005 is < 10 cfu/ml and refers to the quantitative method ISO 21528-2.
In such situations, it is of interest to validate the performance of qualitative alternative methods against the
specified (quantitative) reference method. To that end, the technical protocol for the validation of qualitative
methods (see Clause 5) is to be used. For such a validation study, the quantitative results of the reference
method have to be converted into qualitative results prior to interpretation according to Clause 5.
−2
EXAMPLE 2 When one or more colonies are observed on a plate using 1 ml of a 10 dilution, this result corresponds
to a positive detection in 0,01 gram.
NOTE Annex J provides the special case of validation of a method for commercial sterility testing for specific
products [sterilized or ultra high temperature (UHT) dairy and plant-based liquid products].
If a technical change in a validated alternative (proprietary) method is evaluated as being major, a re-
validation of this alternative method in accordance with this document is needed.
When the re-validation of the alternative method is conducted, the impact on the performance
characteristics shall be evaluated to determine if the changes are to be regarded as major (performance
characteristics have substantially changed) or minor (no or minor impact on performance characteristics
observed). In certain cases, a major technical change in the method can be considered to be minor, if the re-
validation study shows that it has no significant impact on the performance characteristics or test results.
A major (technical) change that, after re-validation, has a major impact on the performance characteristics
of the alternative method, requires re-verification of the method by the user laboratory in accordance with
ISO 16140-3.
5.1.1
Add the following text at the end of the subclause:
The organizing laboratory shall be competent to perform both the reference method as well as the
alternative method.
NOTE Competence can be demonstrated in different ways (e.g. for the reference method, a documented proof of
meeting the requirements of ISO/IEC 17025, and for the alternative method, a documented training).
ISO 16140-2:2016/Amd.1:2024(en)
5.1.2
Add the following text at the end of the subclause:
When the reference and alternative methods are based on two different principles and are performed
with the same test portion, but do not share a common enrichment procedure, an unpaired data study is
performed. For example, when a qualitative alternative method is validated against a quantitative reference
method at a limit of 100 cfu/g. In this case, a suspension of the (food) item can be used to inoculate both
culture media for the reference method and the alternative method before any enrichment/multiplication of
the microorganism.
5.1.3.3
Add the following text at the end of the subclause:
The alternative method shall be evaluated for a defined test portion size (e.g. 25 g, 200 g, 375 g) during the
validation study. The method is considered to be validated for any test portion size up to the validated test
portion size if the testing protocol (dilution ratio, incubation time and incubation temperature) is the same
as that used during the validation study.
EXAMPLE A reference method used in a validation study of this document was validated for a “broad range of
foods” using a 25 g test portion and a 1:10 dilution ratio. The alternative method was validated for a “broad range of
foods” using a 375 g test portion and a 1:5 dilution ratio at a determined incubation time. In practice, a user laboratory
can use the alternative method for all food items (broad range of foods) using a test portion size up to 375 g test
portion and a 1:5 dilution ratio at the validated incubation time (unless stated differently by the organization involved
in the method validation).
5.1.3.4
Add the following text before the last sentence of the second paragraph:
The interpretation of the results (positive agreement, negative agreement, etc.) is based on a comparison of
the reference method result (column 1 in Tables 1 and 2) and the alternative method result, including any
confirmations as described in the alternative method protocol (column 2 in Tables 1 and 2). When positive or
negative deviations are obtained, a footnote should be included at the end of each table to provide additional
explanations for the interpretation of the deviations. The footnotes indicate if the result is due to a false
positive or false negative result of the alternative method. The footnote is a comparison of the results of the
alternative method (including any confirmations as described in the alternative-method protocol) (column 2
in Tables 1 and 2) and the confirmed alternative method (by any means) (column 3 in Tables 1 and 2).
ISO 16140-2:2016/Amd.1:2024(en)
5.1.3.4, after the second paragraph
Replace the text with the following:
Table 1 — Comparison and interpretation of sample results between the reference and alternative
methods for a paired study
Result of the (reference or alternative) method per sample
Reference Alternative method Confirmed Interpretation
method (including any confirmations alternative (based on the confirmed
as described in the method alternative-method result)
a
alternative-method protocol) (by any means)
b
+ + Not needed Positive Agreement (PA)
b
- - Not needed Negative Agreement (NA)
Negative Deviation due to false
b
+ - Not needed negative alternative-method result
(ND )
FN(alt)
- + + Positive Deviation (PD)
Positive Deviation due to false
- + - positive alternative-method result
c
(PD )
FP(alt)
a
Confirmation of the alternative-method result is done according to 5.1.3.3.
b
No need for additional confirmation test(s). Confirmed alternative-method result is the same as the alternative-method
result.
c
This false positive result (FP) shall also be used to calculate the false positive ratio.
Table 2 — Comparison and interpretation of sample results between the reference and alternative
methods for an unpaired study
Result of the (reference or alternative) method per sample
Reference Alternative method Confirmed Interpretation
method (including any confirmations alternative method (based on the confirmed
a,b
as described in the (by any means) alternative-method result)
alternative-method protocol)
+ + + Positive Agreement (PA)
Positive Agreement due to false
+ + - positive alternative-method result
c
(PA )
FP(alt)
- - - Negative Agreement (NA)
Negative Agreement due to false
- - + negative alternative-method result
(NA )
FN(alt)
+ - - Negative Deviation (ND)
Negative Deviation due to false
+ - + negative alternative-method result
(ND )
FN(alt)
- + + Positive Deviation (PD)
Positive Deviation due to false
- + - positive alternative-method result
c
(PD )
FP(alt)
a
Confirmation of the alternative-method result is done according to 5.1.3.3
b
Confirmation by any means is only required when the result of the alternative method does not produce viable organisms.
This is used as the confirmed alternative method result in comparison to the reference method result.
c
These false positive results (FP) shall also be used to calculate the false positive ratio.
ISO 16140-2:2016/Amd.1:2024(en)
Determine the Total Negative Deviation (TND) and Total Negative Agreement (TNA) for the validation study.
Paired evaluation: Total Negative Deviation: TND = ND
FN(alt)
Total Negative Agreement: TNA = NA + PD
FP(alt)
Unpaired evaluation: Total Negative Deviation: TND = ND + ND + PA
FN(alt) FP(alt)
Total Negative Agreement: TNA = NA + NA + PD
FN(alt) FP(alt)
Table 3 — Summary of results obtained with the reference and alternative methods (after
confirmation) of all samples for each category
Reference-method positive Reference-method negative
(R+) (R-)
+/+ -/+
Alternative-method positive (A+)
Positive Agreement (PA) Positive Deviation (PD)
+/– −/−
Alternative-method negative (A-)
Total Negative Deviation (TND) Total Negative Agreement (TNA)
Based on data summarized in Table 3 for the combined categories per category and per type, calculate
the values for sensitivity of the alternative method (see Formula (1)) and of the reference method (see
Formula (2)), as well as the relative trueness (see Formula (3)) and false positive ratio and false negative
ratio for the alternative method after the additional confirmation of the results (see Formula (4)).
Sensitivity for the alternative method:
()PA+PD
SE = ×100% (1)
alt
PA++TNDPD
()
Sensitivity for the reference method:
()PA+TND
SE = ×100% (2)
ref
()PA+TND+PD
Relative trueness:
PA+TNA
()
RT= ×100% (3)
N
False positive ratio (FPR) and false negative ratio (FNR) for the alternative method:
PD
FP()alt
Paired evaluation: FPR=×100 %
TNA
(4)
PA + PD
FP()altFPa()lt
Unpaired evaluationn: FPR= ×100 %
TNA
NA +ND
FN()altFNa()lt
Falsen egativeratio: FNR =
PA +TND ++ PD
where
N is the total number of samples (PA + PD + TND + TNA);
FP is the false positive results;
FN is the false negative results.
For explanation of the abbreviated terms used, see Tables 1 to 3.
ISO 16140-2:2016/Amd.1:2024(en)
The confirmed alternative-method results shall be used to determine whether the alternative method
produces comparable results to the reference method.
Calculate the difference between (TND – PD) for both paired and unpaired studies and the sum of (TND +
PD) for paired studies. Check whether the difference and/or sum of PD and TND conform to the Acceptability
Limit (AL) stated in Table 4 with respect to the type of study (paired or unpaired) and the number of
categories used in the evaluation.
NOTE 1 Acceptability Limits (AL) are based on data and consensus expert opinion. The AL are not based on
statistical analysis of the data.
The interpretation of results shall be done per category and for all categories used in the validation study.
An interpretation of results shall also be done per enrichment protocol in case different protocols are used
for different types of samples. A sensitivity study can also exist of a partly paired and unpaired study. In that
case the results for (TND + PD) shall be evaluated based on the number of positive samples obtained for the
categories tested using the paired study design. The results for (TND – PD) shall be evaluated based on the
number of positive samples obtained for the full study (so all categories belonging to both the paired and
unpaired study design).
The AL is not met when the observed value is higher than the AL. When the AL is not met, if the number
of positive samples is higher than expected according to the number of categories (e.g. having 60 or more
positive samples for one single category), it is possible to use the second column of Table 4 and switch to
higher AL. When the AL is not met, investigations should be made (e.g. root cause analysis) in order to provide
an explanation of the observed results. Based on the AL and the additional information, it is decided whether
the alternative method is regarded as not fit for purpose for the category or categories involved. The reasons
for acceptance of the alternative method in case the AL is not met shall be stated in the study report.
Table 4 — Acceptability limit parameters and values for a paired and unpaired study design in
relation to the number of positive samples obtained
c
Paired study Unpaired study Mixed study
Number of Number of positive
categories samples (N+) a b
(TND - PD ) (TND + PD) (TND - PD) (TND - PD) (TND + PD)
1 30 to 59 3 6 3 3 6
2 60 to 89 4 8 4 4 8
3 90 to 119 5 10 5 5 10
4 120 to 149 5 12 5 5 12
5 150 to 179 5 14 5 5 14
6 180 to 209 6 16 6 6 16
7 210 to 239 6 18 7 7 18
8 240 to 269 6 20 7 7 20
9 270 to 299 7 22 8 8 22
10 300 to 329 7 24 8 8 24
11 330 to 359 7 26 9 9 26
12 360 to 389 8 28 9 9 28
13 390 to 419 8 30 10 10 30
14 420 to 449 8 32 10 10 32
15 450 to 479 9 34 11 11 34
16 480 to 509 9 36 11 11 36
17 510 to 539 9 38 12 12 38
18 540 to 569 10 40 12 12 40
a
TND = total number of samples with Negative Deviation results.
b
PD = number of samples with Positive Deviation results.
c
Mixed study includes both paired and unpaired study design.
ISO 16140-2:2016/Amd.1:2024(en)
TTabablele 4 4 ((ccoonnttiinnueuedd))
c
Paired study Unpaired study Mixed study
Number of Number of positive
categories samples (N+) a b
(TND - PD ) (TND + PD) (TND - PD) (TND - PD) (TND + PD)
19 570 to 599 10 42 13 13 42
20 600 to 629 10 44 13 13 44
21 630 to 659 11 46 14 14 46
22 660 to 689 11 48 14 14 48
23 690 to 719 11 50 15 15 50
24 720 to 749 12 52 15 15 52
25 750 to 779 12 54 16 16 54
a
TND = total number of samples with Negative Deviation results.
b
PD = number of samples with Positive Deviation results.
c
Mixed study includes both paired and unpaired study design.
NOTE 2 A negative value for (TND – PD) is acceptable as this indicates a better performance of the alternative
method compared to the reference method.
NOTE 3 Information on differences observed between results of the alternative method before and after
confirmation of the results (step 1 and step 2) according to the alternative-method protocol is commonly presented
in the validation report as additional information but is not used in the overall assessment of the alternative-method
performance.
5.1.4, last sentence
Replace the text with the following:
The level of contamination shall be determined. This allows calculation of the LOD of the alternative
method, which is required in order to verify the performance of the alternative method upon implementation
of the validated method in a laboratory in accordance with ISO 16140-3. The level of contamination is
determined by performing a most probable number (MPN) analysis on the (stabilized) inoculated samples
(preferably) and/or through the enumeration of the inoculum at the time of inoculation, see 5.1.4.3.
5.1.4.1, second paragraph
Replace the text with the following:
A minimum of three levels per type shall be prepared consisting of at least a negative control level, a low
level and a higher level. Ideally, the low level shall be the theoretical detection level (i.e. 0,7 cfu per test
portion) and the higher level just above the theoretical detection level (e.g. 1 cfu to 1,5 cfu per test portion).
For fastidious bacteria, the low level can significantly exceed the theoretical detection level; thus, the low
and high level should be adjusted appropriately. A fixed ratio (e.g. 1:2) between the low- and high-level
contamination should be used to aid in determining the final contamination levels. At least the low level
should provide fractional recovery by either the reference method or the alternative method (fractional
recovery at the low level should be between 25 % and 75 % of the number of samples tested). An evaluation
shall be performed to ensure the relationship and consistency of the number of positives of the intermediate
and high level. In cases where the alternative method produces fractional recovery at the low level and the
reference method produces all positive results, the results of the RLOD study are not valid and a root cause
analysis shall be performed.
ISO 16140-2:2016/Amd.1:2024(en)
The level of contamination of the sample used (except for the negative control) shall be determined as in
5.1.4. At the negative control level, at least five replicate samples should be tested by both methods. For
the second (low) level (theoretical detection level), at least 20, and for the third (higher) level, at least five
replicate samples should be tested by both methods. The negative control level shall not produce positive
(by isolation of the target organism) results. When positive results are obtained, the experiments have to be
repeated for all levels.
5.1.4
Add the following text at the end of the subclause:
5.1.4.3 Calculation of the LOD
LOD shall be calculated for each category for the alternative method and optionally for the reference
method. The LOD is used in method verification (see ISO 16140-3).
For each category evaluated, determine the contamination of the low level by performing a 3-level MPN for
the particular (food) item tested using the reference method (preferably) at the time of the RLOD experiment
and/or by the enumeration of the inoculum at the time of inoculation using a non-selective medium. When
enumeration is performed, the inoculum for the low and high contamination levels should be determined.
The MPN and 95 % confidence interval shall be determined for the fractional level only. For the low (or
fractional) contamination level, analyse 20 test portions plus 5 test portions at approximately 2 times the
test portion size and 5 test portions at approximately half of the test portion size evaluated in the validation
study (e.g. if the reference method test portions were 25 g, evaluate 5 test portions at 50 g and 5 test portions
at 10 g). To each test portion, add a proportionate amount of enrichment broth as described in the reference
method to maintain the enrichment volume to mass ratio (e.g. a reference method with a 1:10 enrichment
ratio, add 450 ml to the 50 g test portions and 90 ml to the 10 g test portions). Analyse the test portions
following the reference method from enrichment to confirmation. Use the number of positive results per
1)
test portion size to calculate the MPN value. An Excel®-based program for calculating MPN values is freely
available for download at https:// standards .iso .org/ iso/ 7218 (download the file “MPN calculation Excel
program”).
Enumerate the inoculum at the time of inoculation by plating onto non-selective agar (see ISO 7218 for
guidance on plating). Agar plates should be incubated under conditions to allow for optimal growth of the
target microorganism.
Use the number of positive results per test portion size and the MPN value or the results of the enumeration of
the inoculum to calculate the LOD and 95 % confidence interval. An Excel®-based program for calculating
LOD values is freely available for download at https:// standards .iso .org/ iso/ 16140/ -2/ ed -1/ en/ amd/ 1/
(download the file “PODLOD_ver12”). The LOD value is calculated per category tested in the RLOD study
and shall be expressed as cfu/test portion.
NOTE The 20 test portions from the low level are the same as the 20 test portions used in the RLOD study.
Therefore, only 5 test portions at 2 times and 5 at approximately half of the test portion size are analysed in addition
to the RLOD study.
1) Excel® is an example of a suitable product available commercially. This information is given for the convenience of
users of this document and does not constitute an endorsement by ISO of this product.
ISO 16140-2:2016/Amd.1:2024(en)
5.2.1
Add the following text after the second sentence of the first paragraph:
The interlaboratory study shall be conducted with collaborators belonging to more than one country.
The collaborators shall be competent to perform both the reference method as well as the alternative method.
NOTE Competence can be demonstrated in different ways (e.g. for the reference method, a documented proof of
meeting the requirements of ISO/IEC 17025, and for the alternative method, a documented training).
5.2.2, third bullet
Replace the text with the following:
— at least three different levels of contamination shall be used: a negative control (L ) and two levels (L
0 1
and L ). At least one of these shall produce fractional positive results. The level of contamination needed
to obtain fractional recovery shall be based on the RLOD study data of the reference method in the
method comparison study. Theoretically, an average level of contamination of 1 cfu/sample is adequate to
obtain fractional recovery. The level of contamination shall be determined. This allows calculation of the
LOD of the alternative method, which is required in order to verify the performance of the alternative
method upon implementation of the validated method in a laboratory in accordance with ISO 16140-3.
The level of contamination is determined by performing an MPN analysis at the time of the start of the
interlaboratory study, see 5.2.4.4.
5.2.3, title
Replace the text with the following:
Summary of data and trueness calculations
5.2.3, after the third paragraph
Replace the text with the following:
Table 9 — Summarized results for all collaborators for a paired study
Result of the (reference or alternative) method per sample
Reference Alternative Confirmed alternative Interpretation
a b
method method method (based on the confirmed alternative-method result)
c
+ + Not needed Positive Agreement (PA)
c
- - Not needed Negative Agreement (NA)
Negative Deviation due to false negative
c
+ - Not needed
alternative-method result (ND )
FN(alt)
- + + Positive Deviation (PD)
Positive Deviation due to false positive
- + -
d
alternative-method result (PD )
FP(alt)
a
The alternative-method results includes any confirmations as described in the alternative-method protocol.
b
The confirmed alternative-method result is the result after additional confirmation as described in the protocol for the
validation study.
c
No need for additional confirmation test(s). Confirmed alternative-method result is the same as the alternative-method
result.
d
This false positive result (FP) shall also be used to calculate the false positive ratio.
ISO 16140-2:2016/Amd.1:2024(en)
Table 10 — Summarized results for all collaborators for an unpaired study
Result of the (reference or alternative) method per sample
Reference Alternative Confirmed alternative Interpretation
a b
method method method (based on the confirmed alternative-method result)
+ + + Positive Agreement (PA)
+ + - Positive Agreement due to false positive
c
alternative-method result (PA )
FP(alt)
- - - Negative Agreement (NA)
- - + Negative Agreement due to false negative
alternative-method result (NA )
FN(alt)
+ - - Negative Deviation (ND)
+ - + Negative Deviation due to false negative
alternative-method result (ND )
FN(alt)
- + + Positive Deviation (PD)
- + - Positive Deviation due to false positive
alternative-method result (PD )
FP(alt)
a
The alternative-method result includes any confirmations as described in the alternative-method protocol.
b
The confirmed alternative-method result is the result after additional confirmation as described in the protocol for the
validation study.
Determine the Total Negative Deviation (TND) and Total Negative Agreement (TNA) for the validation study.
Paired evaluation: Total Negative Deviation: TND = ND
FN(alt)
Total Negative Agreement: TNA = NA + PD
FP(alt)
Unpaired evaluation: Total Negative Deviation: TND = ND + ND + PA
FN(alt) FP(alt)
Total Negative Agreement: TNA = NA + NA + PD
FN(alt) FP(alt)
Table 11 — Summary of results for all collaborators obtained with the reference and alternative
methods (after confirmation) for level L or L
1 2
Reference-method positive Reference-method negative
(R+) (R-)
+/+ -/+
Alternative-method positive (A+)
Positive Agreement (PA) Positive Deviation (PD)
+/− −/−
Alternative-method negative (A-)
Total Negative Deviation (TND) Total Negative Agreement (TNA)
Based on data summarized in Table 11, calculate the values for sensitivity of the alternative method (see
Formula (8)) and of the reference method (see Formula (9)), as well as the relative trueness (see Formula (10))
and false positive ratio and false negative ratio for the alternative method after the additional confirmation
of the results (see Formula (11)).
ISO 16140-2:2016/Amd.1:2024(en)
Sensitivity for the alternative method:
()PA+PD
SE = ×100% (8)
alt
PA++TNDPD
()
Sensitivity for the reference method:
()PA+TND
SE = ×100% (9)
ref
PA++TNDPD
()
Relative trueness:
()PA+TNA
RT= ×100% (10)
N
False positive ratio (FPR) and false negative ratio (FNR) for the alternative method:
PD
FP()alt
Paired evaluation: FPR=×100 %
TNA
(11)
PA + PD
FP()altFPa()lt
Unpaired evaluationn: FPR= ×100 %
TNA
NA +ND
FN()altFNa()lt
Falsen egativeratio: FNR =
PA +TND ++ PD
where
N is the total number of samples (TNA + PA + PD + TND);
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.
Loading comments...