SIST CR 14377:2002
(Main)Air quality - Approach to uncertainty estimation for ambient air reference measurement methods
Air quality - Approach to uncertainty estimation for ambient air reference measurement methods
The Report is to provide guidance on uncertainty evaluation to CEN/TC 264 Working Group(s), who are involved in the preparation of Reference Methods to measure ambient air quality, as required by the EU Daughter Directives of the European Framework Directive on Ambient Air-quality Assessment and Management (96/62/EC)
Kakovost zraka - Pristop k ocenjevanju merilne negotovosti referenčnih merilnih metod za zunanji zrak
General Information
Standards Content (Sample)
SLOVENSKI STANDARD
SIST CR 14377:2002
01-maj-2002
.DNRYRVW]UDND3ULVWRSNRFHQMHYDQMXPHULOQHQHJRWRYRVWLUHIHUHQþQLKPHULOQLK
PHWRG]D]XQDQML]UDN
Air quality - Approach to uncertainty estimation for ambient air reference measurement
methods
Ta slovenski standard je istoveten z: CR 14377:2002
ICS:
13.040.20 Kakovost okoljskega zraka Ambient atmospheres
SIST CR 14377:2002 en
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.
---------------------- Page: 1 ----------------------
SIST CR 14377:2002
---------------------- Page: 2 ----------------------
SIST CR 14377:2002
CEN REPORT
CR 14377
RAPPORT CEN
CEN BERICHT
January 2002
ICS
English version
Air quality - Approach to uncertainty estimation for ambient air
reference measurement methods
This CEN Report was approved by CEN on 10 November 2001. It has been drawn up by the Technical Committee CEN/TC 264.
CEN members are the national standards bodies of Austria, Belgium, Czech Republic, Denmark, Finland, France, Germany, Greece,
Iceland, Ireland, Italy, Luxembourg, Malta, Netherlands, Norway, Portugal, Spain, Sweden, Switzerland and United Kingdom.
EUROPEAN COMMITTEE FOR STANDARDIZATION
COMITÉ EUROPÉEN DE NORMALISATION
EUROPÄISCHES KOMITEE FÜR NORMUNG
Management Centre: rue de Stassart, 36 B-1050 Brussels
© 2002 CEN All rights of exploitation in any form and by any means reserved Ref. No. CR 14377:2002 E
worldwide for CEN national Members.
---------------------- Page: 3 ----------------------
SIST CR 14377:2002
CR 14377:2002 (E)
Contents
Foreword.4
1 Introduction .5
2 Assumptions and interpretations in this Report of the EC Air Quality Framework and Daughter
Directives.5
2.1 Annexes to the Daughter Directives .5
2.2 Definitions and interpretation of measurement uncertainty, level of confidence, and
confidence interval, within the Ambient Air Quality Daughter Directives .6
2.2.1 General.6
2.2.2 Uncertainty of measurement .6
2.2.3 Level of confidence .6
2.2.4 Relationship between confidence interval and measurement uncertainty .7
2.3 EC Terms of Reference .8
3 Approaches to uncertainty estimation .8
3.1 General.8
3.2 Guide to the Expression of Uncertainty in Measurement.9
3.3 International Standards ISO 5725 Parts 1 to 6.9
3.4 International Vocabulary of Basic and General Terms in Metrology.10
3.5 Other International Standards .10
4 Recommendations for the assessment of uncertainty of ambient air measurement methods .11
4.1 Introduction .11
4.1.1 General.11
4.1.2 Automated measurement methods.11
4.1.3 Non-automated measurement methods .12
4.1.4 Utilisation of traceable calibration standards and certified reference materials.13
4.1.5 Specification of the regime of field calibrations and QA/QC procedures.13
4.2 Consistent approach to the assessment of uncertainty in the Ambient Air Quality Daughter
Directives.13
4.2.1 General.13
4.2.2 General types of test procedures to be employed .14
4.2.3 Examples of the approach to uncertainty estimation given in this Report .18
4.2.4 Applicability of the general types of tests.18
5 Requirements for on-going quality assurance and quality control of field measurements
covered by the EC Directives .18
5.1 Overview of requirements.18
5.2 Specific requirements of the Framework Directive.18
5.3 Types of organisations involved.19
5.4 Requirement for traceability to national or international accepted standards .19
5.5 Demonstration of the competence of organisations involved with QA/QC activities .20
5.6 The role of quality assurance and quality control for the on-going field measurements.20
5.7 Procedures for the regular auditing of the performance of laboratories involved in QA/QC
activities.21
5.8 Conclusions.22
6 General philosophy for the development of future ambient air measurement methods .22
Annex A Principles of the Reference Methods prescribed in the EC Air Quality Daughter Directives .23
Annex B Data quality objectives contained in the Ambient Air Quality Daughter Directive covering
benzene and carbon monoxide .24
Annex C Reference measurement methods for the implementation of EU Air Quality Directives .25
Annex D Examples illustrating the approach specified in this Report for uncertainty estimation for
ambient air measurement methods .27
Annex E Definitions of measurement uncertainty terms used in this Report.35
2
---------------------- Page: 4 ----------------------
SIST CR 14377:2002
CR 14377:2002 (E)
Annex F Outline description of a methodology for a proficiency testing scheme which could be used
to underpin European harmonisation of ambient air quality measurements.36
Annex G Members of the Ad-hoc Group on an approach to uncertainty estimation for ambient air
measurement methods .38
Annex H Acknowledgements .39
Bibliography .40
3
---------------------- Page: 5 ----------------------
SIST CR 14377:2002
CR 14377:2002 (E)
Foreword
This Technical Report has been prepared by Technical Committee CEN/TC 264, "Air quality", the secretariat of
which is held by DIN.
This document is a working document.
This CEN Report has been prepared by an Ad-Hoc-Group of CEN/TC 264 "Air quality" in co-operation with the
European Commission’s Joint Research Centre, Ispra, Italy.
This CEN Report is an informative document.
4
---------------------- Page: 6 ----------------------
SIST CR 14377:2002
CR 14377:2002 (E)
1 Introduction
The European Framework Directive on Ambient Air-quality Assessment and Management, and its related Daughter
Directives, require measurements to be made using specified Reference Methods (given in outline in Annex A of
this Report), and the results to be reported, on specific air pollutants, with specified data quality objectives, by all
Member States within the EU. The main purpose of this CEN Report is to provide guidance to those CEN/TC 264
working groups which are involved in the preparation of Reference Methods to measure ambient air quality as
required by the EC Daughter Directives, on the uncertainty evaluation to be carried out in order to conform to these
data quality objectives. One further purpose of this Report is to emphasize the essential requirements for
appropriate quality assurance (QA) and quality control (QC) procedures in order to ensure that the ongoing field
measurements are valid – so as to ensure the EC data quality objectives continue to be met.
It is important to understand that these Daughter Directives specify the principles of the Reference Method to be
used, but they rely on the European standardisation body (CEN) to produce detailed Reference Method(s) for each
pollutant as European Standard(s), and to arrange for their publication so that they can be made available to
Member States. As noted above, the principles of these Reference Methods, which are to be used in the
implementation of the Directives, are prescribed in the relevant EC Directives. In some cases, these Methods are
specified as continuous or semi-continuous automated instruments, which comprise fully integrated systems,
usually including a sampling line and the analytical equipment. In other cases, the Reference Methods are
specified as manual or discontinuous methods, which comprise a field sampler and a separate laboratory analytical
component. In addition, it should be noted that in certain scenarios the Directives allow indicative methods to be
used, which are allowed to have poorer measurement uncertainties.
It is also a requirement of some of the Daughter Directives that Member States meet air quality limit values for a
variety of different averaging periods (covering e. g. hourly, daily, and annual averages). For some of the pollutants
specified, however, the requirement is only for annual averages.
The European Framework Directive on Ambient Air-quality, and the associated Daughter Directives, also state that
the approaches given in the Guide to the Expression of Uncertainty in Measurement published by ISO, and given in
the International Standards ISO 5725, are to be used for the estimation of the measurement uncertainties of these
Reference Methods. Both approaches are mentioned in these Directives as alternatives for the estimation of
measurement uncertainty. This Report describes, therefore, in broad terms, the methodology which should be
employed by the relevant CEN working groups for the integration of the approaches described in these documents
into European Standards, whilst still maintaining a valid overarching statistically-based methodology for determining
these measurement uncertainties.
The issues outlined above are covered in this Report. The Report was produced following consultations with an Ad-
hoc Group of experts which were convened to consider the issues. These experts are experienced in both ambient
air monitoring and in measurement uncertainty statistics. The members of this group and its Chairman are listed in
Annex G. It should be noted that all the existing working groups of CEN/TC 264, which are currently involved with
the standardisation of ambient air quality measurements, were represented on this Ad-hoc Group.
It should also be noted that Member States may use other methods, as alternatives to these Reference Methods –
provided they can demonstrate that these other methods produce results which have been shown to be equivalent
to those obtained by the relevant Reference Method. It is not, however, the remit of this Report to discuss the issue
of equivalence, or to define how it may be demonstrated.
2 Assumptions and interpretations in this Report of the EC Air Quality Framework and
Daughter Directives
2.1 Annexes to the Daughter Directives
Each of the Daughter Directives contains an Annex which specifies the data quality objectives required for the air
quality assessment for the particular pollutant species they address. These Annexes specify the ‘accuracy’ or
‘uncertainty’ that must be met by the Member States when reporting results, in order to satisfy the requirements of
the Directives. For clarity, it is assumed in this Report that the two terms accuracy and uncertainty are
synonymous, and therefore in the text of this Report reference will only be made to uncertainty, or to the
uncertainty of measurement, as defined in the Guide to the Expression of Uncertainty of Measurement (1993)
published by ISO – known hereafter in this Report as the GUM. An overview of the GUM is presented in clause 3.2
of this Report.
5
---------------------- Page: 7 ----------------------
SIST CR 14377:2002
CR 14377:2002 (E)
Included in each of the Annexes of the Daughter Directives, noted above, is an explanatory paragraph on their
accuracy (uncertainty) requirements, in which it is explained that “the Guide to the Expression of Uncertainty of
Measurements 1993, or ISO 5725-1, Accuracy (Trueness and Precision) of Measurement Methods and Results,
shall be used”. As an example of this, the relevant Annex of one of the (draft) Daughter Directives is reproduced in
Annex B of this Report, where it can be seen that the uncertainty values required of the Reference Methods for
benzene and carbon monoxide are listed.
It is also assumed in this Report that the GUM and ISO 5725 are not mutually exclusive, and that there is no
intention in the Daughter Directives to restrict the use by CEN and others to only one or the other. Instead, it is
proposed here that the principles of GUM is employed to identify and combine all uncertainties of the appropriate
Reference Method, and that ISO 5725 (including some of its six parts – not just the one part mentioned in the
Directives) may be used, where appropriate, as an aid to this evaluation (see clause 3 of this Report). It should also
be noted in this context that the GUM is a (voluntary) European Standard (ENV 13005).
It has also been necessary for the Ambient-air working groups in CEN/TC 264 to assume that the data quality
objectives of the Daughter Directives will be met, provided that the uncertainty of the measurement results
obtained with the Reference Methods are within the uncertainties prescribed by these data quality objectives. This
is also assumed in this Report, as it is judged that this is a valid assumption for the relevant European Standards.
There are also other statistical ISO standards that may be useful in this evaluation. Some of these are noted below
in this Report. It is assumed that the statements in the Daughter Directives do not preclude the use of any of these,
or of other relevant standards, where they are applicable, as long as the over-riding principles described in the
GUM are followed.
2.2 Definitions and interpretation of measurement uncertainty, level of confidence, and
confidence interval, within the Ambient Air Quality Daughter Directives
2.2.1 General
As discussed above, each of the EC Ambient Air Quality Daughter Directives specify data quality objectives
(quantified in their Annexes) that EU Member States must conform to, when reporting the results obtained to the
European Commission (EC). The definitions associated with measurement uncertainty that also relate to, or are
referred to, in the Annexes of these Daughter Directives are discussed below for, clarity, and also to serve as a
background to the remainder of this Report.
2.2.2 Uncertainty of measurement
The Air Quality Directives require an evaluation of the ‘uncertainty of the assessment (or measurement) methods’
according to GUM and therefore it is clear that the definition of uncertainty used in GUM shall be used. The GUM
definition of the term ‘uncertainty of measurement’ is therefore reproduced in this Report for completeness as:
Uncertainty of measurement: Parameter, associated with the result of a measurement, that characterizes the
dispersion that would reasonably be attributed to the measurand.
It is also essential to express clearly the conditions under which this uncertainty of the measurement results is to be
evaluated. This issue is also covered in this Report.
2.2.3 Level of confidence
The above term ‘uncertainty of measurement’ is used in the Annexes of the Daughter Directives. The term
‘confidence interval’ is, in addition, used in these Annexes of the Daughter Directives, and this is also defined in the
GUM. The relationship between the terms ‘confidence interval’ and ‘uncertainty of measurement’ is discussed
below in clause 2.2.4. However, it is first useful, for clarification purposes, to discuss the meaning of the term ‘level
of confidence’ which is used throughout these discussions.
As described in the GUM, the usual method of determining the level of uncertainty of a given measurement
parameter is to determine the combined standard uncertainty u of that measured parameter by combining all the
c
individual standard uncertainties, which may be Type A or Type B. It is then necessary to provide a measure of this
uncertainty that defines an interval about the measurement result that is expected to encompass a large fraction of
the distribution of values that could reasonably be attributed to the measurement. This is illustrated
diagrammatically in Figure 1. This measure of uncertainty is termed the expanded uncertainty, denoted by U, which
is obtained by multiplying the standard uncertainty by a coverage factor k:
6
---------------------- Page: 8 ----------------------
SIST CR 14377:2002
CR 14377:2002 (E)
Uk u
c
As is well known, the result of the measurement is then traditionally expressed as:
YyU
This is interpreted to mean that the best estimate of the value attributed to Y is y and also that y–U to y+U is an
Y
interval that may be expected to encompass a large fraction of the distribution of the value.
More specifically, U may be interpreted as defining the interval about the measurement result that encompasses a
large fraction p of the probability characterised by that result and its associated uncertainty. This fraction p is then
the ‘level of confidence’ of that interval. Generally, and in the specific case of the requirements of the EC Air-quality
Daughter Directives this level of confidence p is defined as 95 % – i.e. 95 % of all the individual measurement
parameters lie within ±U of the best estimated value y and only 5 % lie outside of this interval.
It is clear, however, from this discussion that whatever level of confidence is actually used, it shall be stated
unambiguously, together with the result and its expanded level of uncertainty. It should also be noted that the
coverage factor k which is used as a multiplicand with u to obtain an appropriate level of confidence will, of course,
c
depend on the number of independent (uncorrelated) determinations of that measurand (i.e. the number of degrees
of freedom), which make up the probability distribution (as described in the GUM). Therefore k = 2 shall be used
only when there are a sufficiently large number of such degrees of freedom.
2.2.4 Relationship between confidence interval and measurement uncertainty
The relationship between uncertainty of measurement and the confidence interval I of a statistical distribution of
measurement results is illustrated, in an exemplar manner, in Figure 1, where
T
1 = – 1,96 0 is the lower confidence limit for a large number of degrees of freedom - expressed in this
example at a level of confidence of 95%
T
2 = + 1,96 0 is the upper confidence limit for a large number of degrees of freedom - expressed in
this example at a level of confidence of 95%
0 is the normally defined standard deviation (uncertainty) associated with the statistical
population distribution
The length of the confidence interval I defined in the GUM, for a large number of degrees of freedom expressed at
a level of confidence of 95 %, is then:
ITT21 ,96
21
I
Thus the standard deviation (uncertainty) associated with this confidence interval is:
I
21,96
This then enables the expanded uncertainty U associated with the confidence interval I to be expressed (for a large
number of degrees of freedom, expressed at a level of confidence of 95%) as:
UI19,,6 0 5
This interpretation is then fully consistent with the definition and use of the term ‘expanded uncertainty’ within the
GUM, and it is also consistent with the intentions within the relevant Annexes of the Air Quality Daughter Directives.
As noted above, the key terms which are used to describe the statistical procedures associated with measurement
uncertainty are defined in Annex E of this Report.
7
---------------------- Page: 9 ----------------------
SIST CR 14377:2002
CR 14377:2002 (E)
I = 2 1,96
1 -
n
T - + T
1 2
Figure 1 — Graphical illustration of the uncertainty distribution
of a measurement quantity derived from repeated observations
2.3 EC Terms of Reference
The EC has produced certain Terms of Reference (ToR) which specify requirements for the contents of European
Standards covering ambient air quality, and also to assist in their preparation. The latest draft of these is
reproduced in Annex C of this Report. It should be noted that, although these ToR are still draft, and thus are liable
to revision, they will retain the same under-lying principles given in the current draft. It should also be noted that
because different types of instruments or equipment are available to measure air quality, the EC ToR allow for the
type-approval of instruments, and therefore any instruments which operate on the same principle as those stated
as meeting all the requirements of the Reference Method prescribed in the relevant European Standard are also
acceptable – as long as the latter have been validated by demonstrating that they conform to all the relevant
performance criteria applicable to the Reference Method, and also to the overall requirements of the data quality
objectives in the appropriate EC Daughter Directive.
The Daughter Directives also contain air-quality concentration limit values set by the European Commission. These
are expressed as limits to be met over stated temporal averaging periods, and some of these may be as long as a
year. For discontinuous methods this poses potential difficulties for the development and validation of the relevant
European Standards, not only with respect to the sampling method, but also in evaluation of the uncertainty of the
results of the measurements. In these cases, it should be assumed that the uncertainty of the Reference Method,
which is derived for the shorter averaging period used during the laboratory and field validation trials, applies to the
longer averaging times specified in the Directives. This, although not completely rigorous, is a pragmatic solution.
However, in these circumstances the longest averaging times which it is practical to employ during the laboratory
and field trials should be used.
3 Approaches to uncertainty estimation
3.1 General
As noted above, the Air Quality Daughter Directives place specific data quality objectives on the uncertainty of the
measurements carried out. This uncertainty of the measurement result is expressed in terms of a maximum
allowable percentage of the air quality limit value(s), and it should be calculated with respect to the stated
averaging period(s) referred to in the Directives. The ToR also state that these data quality requirements
(‘objectives’ in Daughter Directives) are criteria against which the Reference Method is assessed using a
combination of laboratory and field tests. The primary concern is therefore to ensure that the assessment of
uncertainty of measurement that is used within the CEN/TC 264 standards to derive uncertainties from these
assessment tests, allows valid comparisons to be made with the Directives’ objectives, both in terms of the
8
---------------------- Page: 10 ----------------------
SIST CR 14377:2002
CR 14377:2002 (E)
concentration levels actually present, the prescribed limit values, and the associated averaging periods. This has
implications for the subsequent use of the Reference Methods and the on-going quality assurance and quality
control (QA/QC) regime which is adopted, as knowledge of these will be necessary to derive, for example, an
uncertainty on a yearly average when the assessment tests are conducted over a matter of weeks or months.
The Daughter Directives allow the use of the GUM or the International Standards ISO 5725 (or equivalents) to
assess the measurement uncertainty of the Reference Methods. It could be argued that these two approaches are
not compatible. However, this is not necessarily correct, and the two procedures may be used in such a way that
they are complimentary. This approach is explained further in clause 4 of this Report. However, it is useful first to
review briefly the published documents covering uncertainty of measurements that are most relevant to this Report.
3.2 Guide to the Expression of Uncertainty in Measurement
The GUM provides the general concept for the harmonised estimation of measurement uncertainties, agreed on
and adopted by important organisations in the field of general physical metrology and analytical chemistry (e.g.
Bureau International des Poids et Measures (BIPM), International Union of Pure and Applied Chemistry (IUPAC),
International Union of Pure and Applied Physics (IUPAP), International Electrotechnical Commission (IEC), and
International Organisation of Legal Metrology (OIML)), and published by ISO. In overview, the GUM provides a
procedure for assessing the combined uncertainty of a measurement method by assessing and combining all
potential sources of uncertainty. In this procedure it is necessary to:
Establish a model equation (or measurement equation) and a related uncertainty budget which should list fully
all potential sources of uncertainty. It is also important at this stage to consider carefully the identified
uncertainty contributions of the model equation, so as to ensure that there is no double counting and that any
correlations between uncertainty sources are understood.
Quantify the individual sources of uncertainty, as standard uncertainties (standard deviations), either by
experiment and statistical analysis of repeated measurement (Type A uncertainties) or by other evaluation
(generally an assumption about the likely distribution describing the uncertainty of the source).
NOTE The above methodology can, in principle, be implemented in all circumstances. However, in some cases,
particularly those involving a complex measurement equation, it may not be practical to establish and evaluate all the
uncertainty components of the measurement equation, and in these cases some components of the uncertainty budget
might potentially be ignored. In these circumstances, the uncertainty evaluations described in clause 3.3 of this Report,
which involve International Standards ISO 5725, may be used, where appropriate, to check and evaluate whether there
are any neglected components of the uncertainty that are of significant magnitude. Any additional components identified
by these means should be incorporated into the uncertainty of the measurement results, in an appropriate manner
according to GUM. Where this is not practical, the field evaluations prescribed by the International Standard
prEN ISO 14956 should, where possible, be used to establish appropriate (additional) components in the uncertainty
budget (see clause 4.1 of this Report).
Combine the individual uncertainties, taking into account the sensitivity coefficients of each component
determined from the partial differential of
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.