SIST-V CEN Guide 13:2009
(Main)Validation of environmental test methods
Validation of environmental test methods
This document provides guidance on the validation tasks in the standardisation process of environmental test methods.
It deals with the two main steps of such validation tasks, the robustness testing and the interlaboratory testing (determining the repeatability and reproducibility), both interacting with the elaboration of the draft standards as shown in the flow chart given in clause 2.1. It applies to the different inter-related phases of the environmental test methods, typically sampling and production of a laboratory sample, storage and transportation of the laboratory sample, extraction, analysis or quantification of a test portion and finally reporting. Consequently this document focuses on the ‘why’ and ‘what’ of validation tasks in direct relation to the different steps of the standardisation process. This document is focussed on the validation tasks in the standardisation process of reference methods either for the whole measurement process or for part thereof.
Given the guidance aim of this document, it does not contain detailed procedures for performing the validation tasks (such as number of laboratories, number of samples, etc.).
Validacija okoljskih preskusnih metod
General Information
Standards Content (Sample)
SLOVENSKI STANDARD
SIST-V CEN Guide 13:2009
01-junij-2009
Validacija okoljskih preskusnih metod
Validation of environmental test methods
Ta slovenski standard je istoveten z: CEN Guide 13:2008
ICS:
19.040 Preskušanje v zvezi z Environmental testing
okoljem
SIST-V CEN Guide 13:2009 en
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.
---------------------- Page: 1 ----------------------
SIST-V CEN Guide 13:2009
---------------------- Page: 2 ----------------------
SIST-V CEN Guide 13:2009
CEN
GUIDE
13
Validation of environmental
test methods
Version dated 2008-10-29
---------------------- Page: 3 ----------------------
SIST-V CEN Guide 13:2009
CEN Guide 13 Validation of environmental test methods
Contents
Contents .2
Foreword.3
Introduction.4
1 Scope.7
2 Validation of reference methods.8
2.1 General.8
2.2 Validation of the whole measurement process .10
2.3 Validation of the sampling steps .11
2.4 First step of validation (robustness testing).12
2.5 Second step of validation (interlaboratory testing – repeatability – reproducibility).14
2.6 Final draft standard.14
3 Validation of alternative methods.16
4 Validation of guidelines .17
5 Validation of non-experimental methods.18
Annex: Explanations of terms used in this document .19
References – Bibliography .21
2
Edition 1 / October 2008
---------------------- Page: 4 ----------------------
SIST-V CEN Guide 13:2009
CEN Guide 13 Validation of environmental test methods
validation tasks interacts with the elaboration of the
draft standards, and so there is also a general need
Foreword
to document the performed validation tasks and their
results in the standard.
This document has been produced by the CEN-
This document focuses on the validation tasks in the
SABE Environmental TCs Cooperation Team (ENV
standardisation process of reference methods, being
TCs) as a policy document on validation tasks in
either the whole measurement process or one of its
the standardisation process of environmental
constituent parts.
test methods.
The environmental TCs recognise that these
Decision of TC
validation tasks are complex. They consist of two
to start a New
Work Item
main steps, the robustness testing and the inter-
Experts
laboratory testing (determining repeatability and meetings in
CEN/TC/WG
reproducibility), both interacting with the elaboration
draft standard
of the draft standards as shown in the flow chart.
based on experts
Furthermore, they apply to the different inter-related
opinions
Robustness
phases of encountered in environmental testing,
testing by
expert
typically sampling and production of laboratory
laboratory
revised draft
sample, storage and transportation, extraction,
standard based on
results robustness
analysis and reporting. Consequently, this document
testing
focuses on the ‘why’ and ‘what’ of validation tasks in
Interlaboratory
direct relation to the different steps of the standardi-
testing
prEN standard
sation process. Given the policy aim of this
based on
document, it does not contain detailed procedures repeatability and
reproducibility
for performing the validation (such as number of
testing
laboratories, number of samples, etc.).
Publication as
The environmental TCs recognise that the
EN standard ?
environmental test methods published as standards
Decision of TC
are very often used as reference methods in
to launch the
regulations and/or in contracts between several CEN enquiry
parties. Therefore, a known quality is considered
as vital prior to publishing an environmental test Flow chart of the validation tasks in the standardisa-
tion process
method as a standard. Hence a general need for
3
Edition 1 / October 2008
---------------------- Page: 5 ----------------------
SIST-V CEN Guide 13:2009
CEN Guide 13 Validation of environmental test methods
Introduction
View of the Environmental TCs on validation
This paper is intended by the CEN-SABE Environmental TCs Cooperation Team to be a policy document on
validation. It defines the view of the Environmental TCs on the role of validation in the process of the
standardisation of environmental test methods.
Consequently, this document focuses on the ‘why’ and ‘what’ of validation in direct relation to the different steps
of the standardisation process. Given the policy aim of the document, this document does not contain detailed
procedures for performing the validation (such as number of laboratories, number of samples, etc.).
Uncertainty
The tests results in the environmental fields are often applied for the enforcement of regulation or for contract
1
execution. In such legal situations it is vital that the associated uncertainties in the tests' results are known.
The relation between the test result (TR) and the uncertainty (U) is generally presented as TR ± U. When the
regulatory or contractual limit value is above TR + U or below TR – U, the conclusion is clear, respectively
fulfilling or exceeding the limit value. If the limit value lies between these two boundaries, it is not possible to
come to a clear conclusion.
This is an even bigger problem when the associated uncertainty is unknown, as, despite the test result itself, it is
impossible to ascertain that the test result is really above or below the limit value.
Validation of an environmental test method is aimed at providing sound information on the uncertainty of the
tests' results, and by that means, providing the possibility to come to sound conclusions based on the
standardised measurements (see in bibliography the IPPC-REF document on monitoring).
Request from CEN/SABE
Resolution 26/2004 of SABE taken on 19 October 2004 invites all environmental TCs to establish their own policy
regarding the publication of validated or non-validated standards. The environmental TCs agreed that a
commonly developed policy on validation would be preferable, therefore giving the lead for the development of
this policy statement to the Environmental TC Cooperation Team.
In relation to its request, SABE wished to highlight two issues:
• there may be a financial liability if action is taken on the basis of a CEN document, and as such the
uncertainty of the test result should be known;
• the subject of uncertainty influences the credibility of CEN.
Indeed, these two points are fully recognised by the environmental TCs.
Views within CEN
1
Uncertainty: 'A parameter, associated with the result of a measurement that characterises the dispersion of
the values that could reasonably be attributed to the measurand' (VIM and GUM).
4
Edition 1 / October 2008
---------------------- Page: 6 ----------------------
SIST-V CEN Guide 13:2009
CEN Guide 13 Validation of environmental test methods
When developing a policy view on validation, benefit should be taken of already established policies on validation
within other CEN-sectors.
In general, the common view is that a test method can only be published as an EN when fully validated (first and
second validation steps have been performed). It may happen that the results are considered by the WG expert
as very poor and that they recommend to the TC to publish a TS instead. When no or only partial (e.g. first step)
validation results are available at the time of completion of the CEN enquiry, the test method is to be published as
a Technical Specification (TS).
When (partial) validation has been performed, the resulting performance characteristics are to be included in a
separate section of the test method (specific clause or annex performance characteristics).
Consideration should be given to ENV 13005 Guide on Uncertainty in Measurement (GUM). It is to be noted in
EN-ISO 17025 that 'General requirements for the competence of testing and calibration laboratories' requires that
laboratories provide results with the associated uncertainties.
Major impact
There are different interpretations of the term ‘validation’, even within the environmental fields. However, there is
a consensus that ‘validation’ is a key step in the standardisation process of environmental tests' methods.
Consequently, the definition of the term ‘validation’ has a major impact on the quality of the standards that
describe a test method.
Defining a common policy on validation within the environmental sector of CEN also has, in turn, an impact on
the work of the environmental TCs.
General principle within the environmental sector
The previously mentioned common view within CEN is embraced by the environmental TCs. This implies that
only validated test methods can be published as EN standards. Test methods that are not or only partly validated
are to be published as TSs. At the same time, the environmental TCs recognise the fact that not all standards are
indeed test methods and, therefore, there might be a necessity to differentiate this general principle to some
extent.
This policy document aims to clarify in which situations validation is to be considered as essential, and in which
cases it is of less or of no importance. In addition, the validation activities during the different steps of the
standardisation process are clarified.
Whenever there is a deviation from the general principle to publish validated standards, this should be a
conscious decision of the involved TC.
2
Validation of the whole measurement process
For the user / the customer of a standard or a series of standards, the reliability of the final overall result of a test
is of major importance. That is the reliability that is obtained through all steps of the measurement process.
Consequently, validation should not be just aimed at a single step of that procedure (like the analysis), but indeed
should be aimed at quantifying the uncertainty that is associated with the full test procedure.
Depending on the matrix and the components that are to be assessed, this whole measurement process can be
relatively simple or very complex. At least for part of the measurements in the environmental field, the whole
2
i.e. sampling plan, taking of sample, sample pre-treatment in the field, packaging, storage and
transportation, storage and conservation, sample pre-treatment, extraction, destruction, leaching, clean-up,
analysis / quantification, data management, report
5
Edition 1 / October 2008
---------------------- Page: 7 ----------------------
SIST-V CEN Guide 13:2009
CEN Guide 13 Validation of environmental test methods
measurement process involving the appliance of a series of standards and full validation of the whole
measurement process is not that simple. Therefore, this document starts with the validation of the individual
steps of the whole measurement process, like the analytical determination of the content, and only after that will
look at the validation of the whole measurement process.
6
Edition 1 / October 2008
---------------------- Page: 8 ----------------------
SIST-V CEN Guide 13:2009
CEN Guide 13 Validation of environmental test methods
1 Scope
This document provides guidance on the validation tasks in the standardisation process of environmental
test methods.
It deals with the two main steps of such validation tasks, the robustness testing and the interlaboratory testing
(determining the repeatability and reproducibility), both interacting with the elaboration of the draft standards as
shown in the flow chart given in clause 2.1. It applies to the different inter-related phases of the environmental
test methods, typically sampling and the production of a laboratory sample, storage and transportation of the
laboratory sample, extraction, analysis or quantification of a test portion and finally reporting. Consequently, this
document focuses on the ‘why’ and ‘what’ of validation tasks in direct relation to the different steps of the
standardisation process. This document is focussed on the validation tasks in the standardisation process of
reference methods either for the whole measurement process or for one of its constituent parts.
Given the guidance aim of this document, it does not contain detailed procedures for performing the validation
tasks (such as number of laboratories, number of samples, etc.).
7
Edition 1 / October 2008
---------------------- Page: 9 ----------------------
SIST-V CEN Guide 13:2009
CEN Guide 13 Validation of environmental test methods
2 Validation of reference methods
2.1 General
In this document reference methods are test methods that have been validated and of which the quality of the
test method is, given a specific field of application, accepted by experts and users. It might be the experts that
state that the method is a reference method, but in general, the claim that it is a reference method is not made
within the standardisation process. In order to have knowledge on the quality of the method and accepting that,
information on the quality is essential in order to be defined as a reference method. Validation is therefore an
essential step in the standardisation process from which this method originates.
Reference methods can be used as a legal reference in legislation/regulation and/or in contracts between two or
more parties. They need, therefore, to be self supportive. Reference methods are not necessarily of the highest
metrological quality, however, experts define a reference method as ‘reliable’ and a good basis for decisions. In
general, reference methods are ‘fit for purpose’.
Standardised reference methods are developed for ‘common and repeated use’. They are not of the same nature
as the utmost metrological quality that is required for Certified Reference Material developed in National
Reference Laboratories (BAM, LNE, NPL….etc).
Validation of standardised reference methods is generally performed in two steps including performance
characteristics relevant for the considered method:
• robustness testing;
• interlaboratory testing (repeatability and reproducibility)
As the first step is based on a first draft of the standard and each of the validation steps will result in a revised
draft standard, the implementation of validation in the standardisation process normally relates to three different
draft standards, the last one of which will be published as an EN-standard. These steps are depicted in Figure 1.
It is to be noted that the actual state of the art may not be sufficient for the efficient further development of the
envisaged standard. In such a case, a so-called pre-normative research may be needed prior to any
standardisation with validation.
8
Edition 1 / October 2008
---------------------- Page: 10 ----------------------
SIST-V CEN Guide 13:2009
CEN Guide 13 Validation of environmental test methods
Decision of TC
to start a New
Work Item
Experts
meetings in
CEN/TC/WG
draft standard
based on experts
opinions
Robustness
testing by
expert
laboratory
revised draft
standard based on
results robustness
testing
Interlaboratory
testing
prEN standard
based on
repeatability and
reproducibility
testing
Publication as
EN standard ?
Decision of TC
to launch the
CEN enquiry
Figure 1: Flow chart of the validation tasks in the standardisation process
The robustness testing is generally performed in one highly competent laboratory which ideally already has
experience with the new test method. The repeatability and reproducibility are determined through inter-
laboratory experiments. Both steps are needed and contribute to the evaluation of the uncertainty of the test
results.
In order to secure comparable data, the associated uncertainty of the test methods will often be based on
3
traceability to SI units . However, this is not, per definition, possible in all cases. Alternatively, the quantification of
the uncertainty can also be based on the analysis of certified reference materials. Indeed, in the environmental
3
e.g. weighing traceable to kg SI unit
9
Edition 1 / October 2008
---------------------- Page: 11 ----------------------
SIST-V CEN Guide 13:2009
CEN Guide 13 Validation of environmental test methods
field certified reference materials are, in most cases, more logical (and applicable) than SI units. Unfortunately,
there is also a clear disadvantage with certified reference materials as these are only available for a limited
number of components and matrices and are, in general, so expensive that it is not financially possible to use
certified reference materials for validation or routine checks to determine if the analyses are still within the
predefined bandwidth. A third option, therefore, is also important in the environmental field using (informal)
reference materials that are not certified.
Certainly in the environmental field, more often the uncertainty is to be quantified by means of a relative
comparison to (informal) reference materials. Therefore, the standardised method shall specify the minimum
requirements to be fulfilled in the analysis by:
• a quantification using a set of measurement traceable to SI units; or
• a quantification using certified reference materials; or
• a quantification using reference material specified in the standard
In specific cases the introduction of a new reference method will result in the withdrawal of a previous reference
method. For example, due to the general application of new analytical instruments, as a consequence of which
the old reference method is in practice no longer applicable. Validation of the new reference method and cross
comparison of the results of both the old and the new reference methods allows the continuous use of data
collected in the past (the 'historical' data).
When a CEN TC starts a work item, this is generally given to a dedicated WG. The aim of the work is to
harmonise as far as possible the European practices on a specific topic. Through a series of steps, consensus
among the experts is achieved resulting in a first draft standard that, to the opinion of the experts, reflects the
state of the art (in terms of standardisation) and is assumed to be fit for purpose.
This first draft standard will be used as a starting point for the validation work, or, if funds or means for validation
are not sufficiently available, may be adopted by the CEN-members and published as a TS. When published as a
TS, the document should state that it has not been validated.
4
2.2 Validation of the whole measurement process
The following clauses deal with the validation of the whole measurement process or of individual steps of the
whole measurement process. The standardisation work should ultimately apply to the whole measurement
process that consists of different steps and might be described as:
4
The whole measurement process includes not only the quantification / analysis step but also all the other
steps needed to obtain a measurement result
10
Edition 1 / October 2008
---------------------- Page: 12 ----------------------
SIST-V CEN Guide 13:2009
CEN Guide 13 Validation of environmental test methods
Definition of sampling plan → taking of sample → sample pre-treatment in the field → packaging, storage and
transportation → storage and conservation → sample pre-treatment → extraction, destruction, leaching → clean-
5
up → analysis → data management → report
One standard or a coherent set of standards might comprise all steps of the measurement process and,
consequently, validation of that standard or of that coherent set of standards would imply the validation of all
activities that are part of that measurement process.
Technical characteristics of the measurement process may impose such validation cover in the same experiment
most of the measurement steps e.g. measurement at stack, physical properties of sludge, etc.
As mentioned in relation to the validation of sampling steps in clause 2.3, it is questionable if full validation of a
measurement process is possible. This will, of course, depend on the number of steps in the measurement
process, the complexity of these steps and the variety of situations in which the method can be applied. But this
will mainly depend on the extent to which it is possible to validate the sampling steps as discussed in clause 2.3.
Consequently, the publication of a method describing the whole measurement process as a standard might be
discussed, specifically if no or only partial validation is achievable. At the same time, it might be desirable to
publish such a method as a standard, despite the lack of validation results.
The TC concerned should make a conscious decision on the publication of such a method as a standard.
It should be noted that in the application of a coherent set of steps, all such steps of a measurement process
have a part in the overall uncertainty of the measurand. Therefore, validation of only part of the whole
measurement process is likely to be found insufficient later on when applying the whole measurement process.
It might be noted that validation might be possible when a method is applied for a certain period, and the results
are fit for validation as well as for the assessment for which the results were produced originally. In order to be
applicable for validation purposes, it is essential that the test method provides duplicate results.
2.3 Validation of the sampling steps
However, relevant for the full measurement process (see also clause 2.2), sampling standards describe only a
portion of the full measurement process and they do not deliver a result on their own. Validation of the sampling
standard is, therefore, impossible without mutual validation of the test methods (or the application of already
validated test methods). Such overall testing is per definition expensive due to the overall testing and the
necessity of assessing the variability of the materials to be sampled. Even partial validation, applying a specific
sampling protocol on a specific material under predefined conditions, might still ask for major resources or might
result in such a limited ‘degree of validation’ that this cannot be seen as a full validation.
Consequently, it might be argued that due to a lack of validation results the sampling method cannot be
published as a standard. However, this might result in a largely negative image of the whole measurement
process and is, therefore, undesirable. So sampling methods too may, when there is agreement amongst
experts, be published as standards. In this specific situation, the standard should contain a paragraph on
validation, obviously in this case mentioning and motivating the achieved degree of validation. Depending on the
5
Depending on the measurement process one should keep in mind that there might be differences in the
mentioned steps. Steps might be combined in a standard or are irrelevant for a specific measurement
process. However, in all situations the measurement process consists of different activities that are to be
performed, indifferent of the question if these activities are incorporated into one standard, or that a series
of standards needs to be applied.
11
Edition 1 / October 2008
---------------------- Page: 13 ----------------------
SIST-V CEN Guide 13:2009
CEN Guide 13 Validation of environmental test methods
characteristics of the sampling scenario being considered, including the properties of the material to be sampled,
paired and repeated measurements by several laboratories may provide a significant degree of validation.
In addition, sampling might be carried out on the basis of a predefined procedure that was accepted by all parties
involved, without the need for prior validation (acceptance might be based on the experts' opinion and might, for
example, be formalised in a contract or in legislation). When the associated uncertainty is to be quantified, it will
in such a situation be based on the variability between laboratory samples obtained with the same procedure
from the same quantity of material. It should be kept in mind that the quantified uncertainty might originate both
from the heterogeneity of the sampled material, the sampling activity itself and the subsequent analytical steps.
When all these sources of variability are to be quantified, these different sources of variability are to be taken into
account when designing the validation scheme.
2.4 First step of validation (robustness testing)
Robustness testing is performed by means of comparison tests. These tests are designed to assure that there is
an appropriate control of the test method, in doing so, establishing the ‘robustness’ of the test method.
Depending on the test method that is assessed, the robustness testing might aim at an individual step within the
test method, or aim at the totality of the test method (see clause 6 on the validation of the whole measurement
process). Which approach is chosen will depend on the complexity of the test method.
When a standard incorporates different procedures, possibly as part of the whole measurement method, it is vital
to establish that the different procedures produce comparable results. This is to be done at the latest during the
robustness testing.
The robustness testing of a reference method requires tests with parallel measurements, measuring the same
6
measurand in the same medium .
In addition, when relevant for the considered steps of the measurement process, the robustness testing in the
first validation stage should also determine the correctness (based on reference materials applied in the
analytical / quantification step itself as specified in the standard) of the test method, as well as the limitations of
the test method (lowest and highest concentrations that can be measured, interferences, detection limit, etc.).
The results of those tests are taken on board in a second draft of the test method which is the basis for the
second step of the validation. Again, when no further validation is performed, this second draft may be adopted
and published as a TS. If so, the TS shall contain the results of the robustness testing.
Test methods may be defined for two or more specific matrices, while, in other situations, an overall definition of
a matrix (like ‘soil’ or ‘waste’) might actually incorporate a large number of different ‘sub’-matrices (for soil this
6
An alternative route is to use Certified Reference Material (CRM). But one should keep in mind that in the
environmental fields a CRM is seldom available. Furthermore, when available these CRM are expensive
and not easily applicable for the whole measurement process or for a significant part of it.
12
Edition 1 / October 2008
---------------------- Page: 14 ----------------------
SIST-V CEN Guide 13:2009
CEN Guide 13 Validation of environmental test methods
might be sand, clay, peat, etc.). Prior to robustness testing, (and the subsequent second validation phase) it is
essential to define for which (‘sub’-)matrices the test method will be validated. Obviously, full validation of a
method that incorporates a large variety of (‘sub’-)matrices is practically impossible. However, it might well be
unnecessary to perform such a wide validation when a limited number of (‘sub’-)matrices can be, to the experts'
opinion, representative for the whol
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.