Air quality - Approach to uncertainty estimation for ambient air reference measurement methods

The Report is to provide guidance on uncertainty evaluation to CEN/TC 264 Working Group(s), who are involved in the preparation of Reference Methods to measure ambient air quality, as required by the EU Daughter Directives of the European Framework Directive on Ambient Air-quality Assessment and Management (96/62/EC)

Kakovost zraka - Pristop k ocenjevanju merilne negotovosti referenčnih merilnih metod za zunanji zrak

General Information

Status
Published
Publication Date
30-Apr-2002
Technical Committee
Current Stage
6060 - National Implementation/Publication (Adopted Project)
Start Date
01-May-2002
Due Date
01-May-2002
Completion Date
01-May-2002

Buy Standard

Standardization document
SIST CR 14377:2002
English language
40 pages
sale 10% off
Preview
sale 10% off
Preview
e-Library read for
1 day

Standards Content (sample)

SLOVENSKI STANDARD
SIST CR 14377:2002
01-maj-2002
.DNRYRVW]UDND3ULVWRSNRFHQMHYDQMXPHULOQHQHJRWRYRVWLUHIHUHQþQLKPHULOQLK
PHWRG]D]XQDQML]UDN

Air quality - Approach to uncertainty estimation for ambient air reference measurement

methods
Ta slovenski standard je istoveten z: CR 14377:2002
ICS:
13.040.20 Kakovost okoljskega zraka Ambient atmospheres
SIST CR 14377:2002 en

2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

---------------------- Page: 1 ----------------------
SIST CR 14377:2002
---------------------- Page: 2 ----------------------
SIST CR 14377:2002
CEN REPORT
CR 14377
RAPPORT CEN
CEN BERICHT
January 2002
ICS
English version
Air quality - Approach to uncertainty estimation for ambient air
reference measurement methods

This CEN Report was approved by CEN on 10 November 2001. It has been drawn up by the Technical Committee CEN/TC 264.

CEN members are the national standards bodies of Austria, Belgium, Czech Republic, Denmark, Finland, France, Germany, Greece,

Iceland, Ireland, Italy, Luxembourg, Malta, Netherlands, Norway, Portugal, Spain, Sweden, Switzerland and United Kingdom.

EUROPEAN COMMITTEE FOR STANDARDIZATION
COMITÉ EUROPÉEN DE NORMALISATION
EUROPÄISCHES KOMITEE FÜR NORMUNG
Management Centre: rue de Stassart, 36 B-1050 Brussels

© 2002 CEN All rights of exploitation in any form and by any means reserved Ref. No. CR 14377:2002 E

worldwide for CEN national Members.
---------------------- Page: 3 ----------------------
SIST CR 14377:2002
CR 14377:2002 (E)
Contents

Foreword......................................................................................................................................................................4

1 Introduction ....................................................................................................................................................5

2 Assumptions and interpretations in this Report of the EC Air Quality Framework and Daughter

Directives........................................................................................................................................................5

2.1 Annexes to the Daughter Directives ............................................................................................................5

2.2 Definitions and interpretation of measurement uncertainty, level of confidence, and

confidence interval, within the Ambient Air Quality Daughter Directives ...............................................6

2.2.1 General............................................................................................................................................................6

2.2.2 Uncertainty of measurement ........................................................................................................................6

2.2.3 Level of confidence .......................................................................................................................................6

2.2.4 Relationship between confidence interval and measurement uncertainty .............................................7

2.3 EC Terms of Reference .................................................................................................................................8

3 Approaches to uncertainty estimation ........................................................................................................8

3.1 General............................................................................................................................................................8

3.2 Guide to the Expression of Uncertainty in Measurement..........................................................................9

3.3 International Standards ISO 5725 Parts 1 to 6............................................................................................9

3.4 International Vocabulary of Basic and General Terms in Metrology......................................................10

3.5 Other International Standards ....................................................................................................................10

4 Recommendations for the assessment of uncertainty of ambient air measurement methods ..........11

4.1 Introduction ..................................................................................................................................................11

4.1.1 General..........................................................................................................................................................11

4.1.2 Automated measurement methods............................................................................................................11

4.1.3 Non-automated measurement methods ....................................................................................................12

4.1.4 Utilisation of traceable calibration standards and certified reference materials..................................13

4.1.5 Specification of the regime of field calibrations and QA/QC procedures..............................................13

4.2 Consistent approach to the assessment of uncertainty in the Ambient Air Quality Daughter

Directives......................................................................................................................................................13

4.2.1 General..........................................................................................................................................................13

4.2.2 General types of test procedures to be employed ...................................................................................14

4.2.3 Examples of the approach to uncertainty estimation given in this Report ...........................................18

4.2.4 Applicability of the general types of tests.................................................................................................18

5 Requirements for on-going quality assurance and quality control of field measurements

covered by the EC Directives .....................................................................................................................18

5.1 Overview of requirements...........................................................................................................................18

5.2 Specific requirements of the Framework Directive..................................................................................18

5.3 Types of organisations involved................................................................................................................19

5.4 Requirement for traceability to national or international accepted standards .....................................19

5.5 Demonstration of the competence of organisations involved with QA/QC activities ..........................20

5.6 The role of quality assurance and quality control for the on-going field measurements....................20

5.7 Procedures for the regular auditing of the performance of laboratories involved in QA/QC

activities........................................................................................................................................................21

5.8 Conclusions..................................................................................................................................................22

6 General philosophy for the development of future ambient air measurement methods .....................22

Annex A Principles of the Reference Methods prescribed in the EC Air Quality Daughter Directives ..........23

Annex B Data quality objectives contained in the Ambient Air Quality Daughter Directive covering

benzene and carbon monoxide ..................................................................................................................24

Annex C Reference measurement methods for the implementation of EU Air Quality Directives .................25

Annex D Examples illustrating the approach specified in this Report for uncertainty estimation for

ambient air measurement methods ...........................................................................................................27

Annex E Definitions of measurement uncertainty terms used in this Report....................................................35

---------------------- Page: 4 ----------------------
SIST CR 14377:2002
CR 14377:2002 (E)

Annex F Outline description of a methodology for a proficiency testing scheme which could be used

to underpin European harmonisation of ambient air quality measurements........................................36

Annex G Members of the Ad-hoc Group on an approach to uncertainty estimation for ambient air

measurement methods ...............................................................................................................................38

Annex H Acknowledgements ..................................................................................................................................39

Bibliography ..............................................................................................................................................................40

---------------------- Page: 5 ----------------------
SIST CR 14377:2002
CR 14377:2002 (E)
Foreword

This Technical Report has been prepared by Technical Committee CEN/TC 264, "Air quality", the secretariat of

which is held by DIN.
This document is a working document.

This CEN Report has been prepared by an Ad-Hoc-Group of CEN/TC 264 "Air quality" in co-operation with the

European Commission’s Joint Research Centre, Ispra, Italy.
This CEN Report is an informative document.
---------------------- Page: 6 ----------------------
SIST CR 14377:2002
CR 14377:2002 (E)
1 Introduction

The European Framework Directive on Ambient Air-quality Assessment and Management, and its related Daughter

Directives, require measurements to be made using specified Reference Methods (given in outline in Annex A of

this Report), and the results to be reported, on specific air pollutants, with specified data quality objectives, by all

Member States within the EU. The main purpose of this CEN Report is to provide guidance to those CEN/TC 264

working groups which are involved in the preparation of Reference Methods to measure ambient air quality as

required by the EC Daughter Directives, on the uncertainty evaluation to be carried out in order to conform to these

data quality objectives. One further purpose of this Report is to emphasize the essential requirements for

appropriate quality assurance (QA) and quality control (QC) procedures in order to ensure that the ongoing field

measurements are valid – so as to ensure the EC data quality objectives continue to be met.

It is important to understand that these Daughter Directives specify the principles of the Reference Method to be

used, but they rely on the European standardisation body (CEN) to produce detailed Reference Method(s) for each

pollutant as European Standard(s), and to arrange for their publication so that they can be made available to

Member States. As noted above, the principles of these Reference Methods, which are to be used in the

implementation of the Directives, are prescribed in the relevant EC Directives. In some cases, these Methods are

specified as continuous or semi-continuous automated instruments, which comprise fully integrated systems,

usually including a sampling line and the analytical equipment. In other cases, the Reference Methods are

specified as manual or discontinuous methods, which comprise a field sampler and a separate laboratory analytical

component. In addition, it should be noted that in certain scenarios the Directives allow indicative methods to be

used, which are allowed to have poorer measurement uncertainties.

It is also a requirement of some of the Daughter Directives that Member States meet air quality limit values for a

variety of different averaging periods (covering e. g. hourly, daily, and annual averages). For some of the pollutants

specified, however, the requirement is only for annual averages.

The European Framework Directive on Ambient Air-quality, and the associated Daughter Directives, also state that

the approaches given in the Guide to the Expression of Uncertainty in Measurement published by ISO, and given in

the International Standards ISO 5725, are to be used for the estimation of the measurement uncertainties of these

Reference Methods. Both approaches are mentioned in these Directives as alternatives for the estimation of

measurement uncertainty. This Report describes, therefore, in broad terms, the methodology which should be

employed by the relevant CEN working groups for the integration of the approaches described in these documents

into European Standards, whilst still maintaining a valid overarching statistically-based methodology for determining

these measurement uncertainties.

The issues outlined above are covered in this Report. The Report was produced following consultations with an Ad-

hoc Group of experts which were convened to consider the issues. These experts are experienced in both ambient

air monitoring and in measurement uncertainty statistics. The members of this group and its Chairman are listed in

Annex G. It should be noted that all the existing working groups of CEN/TC 264, which are currently involved with

the standardisation of ambient air quality measurements, were represented on this Ad-hoc Group.

It should also be noted that Member States may use other methods, as alternatives to these Reference Methods –

provided they can demonstrate that these other methods produce results which have been shown to be equivalent

to those obtained by the relevant Reference Method. It is not, however, the remit of this Report to discuss the issue

of equivalence, or to define how it may be demonstrated.

2 Assumptions and interpretations in this Report of the EC Air Quality Framework and

Daughter Directives
2.1 Annexes to the Daughter Directives

Each of the Daughter Directives contains an Annex which specifies the data quality objectives required for the air

quality assessment for the particular pollutant species they address. These Annexes specify the ‘accuracy’ or

‘uncertainty’ that must be met by the Member States when reporting results, in order to satisfy the requirements of

the Directives. For clarity, it is assumed in this Report that the two terms accuracy and uncertainty are

synonymous, and therefore in the text of this Report reference will only be made to uncertainty, or to the

uncertainty of measurement, as defined in the Guide to the Expression of Uncertainty of Measurement (1993)

published by ISO – known hereafter in this Report as the GUM. An overview of the GUM is presented in clause 3.2

of this Report.
---------------------- Page: 7 ----------------------
SIST CR 14377:2002
CR 14377:2002 (E)

Included in each of the Annexes of the Daughter Directives, noted above, is an explanatory paragraph on their

accuracy (uncertainty) requirements, in which it is explained that “the Guide to the Expression of Uncertainty of

Measurements 1993, or ISO 5725-1, Accuracy (Trueness and Precision) of Measurement Methods and Results,

shall be used”. As an example of this, the relevant Annex of one of the (draft) Daughter Directives is reproduced in

Annex B of this Report, where it can be seen that the uncertainty values required of the Reference Methods for

benzene and carbon monoxide are listed.

It is also assumed in this Report that the GUM and ISO 5725 are not mutually exclusive, and that there is no

intention in the Daughter Directives to restrict the use by CEN and others to only one or the other. Instead, it is

proposed here that the principles of GUM is employed to identify and combine all uncertainties of the appropriate

Reference Method, and that ISO 5725 (including some of its six parts – not just the one part mentioned in the

Directives) may be used, where appropriate, as an aid to this evaluation (see clause 3 of this Report). It should also

be noted in this context that the GUM is a (voluntary) European Standard (ENV 13005).

It has also been necessary for the Ambient-air working groups in CEN/TC 264 to assume that the data quality

objectives of the Daughter Directives will be met, provided that the uncertainty of the measurement results

obtained with the Reference Methods are within the uncertainties prescribed by these data quality objectives. This

is also assumed in this Report, as it is judged that this is a valid assumption for the relevant European Standards.

There are also other statistical ISO standards that may be useful in this evaluation. Some of these are noted below

in this Report. It is assumed that the statements in the Daughter Directives do not preclude the use of any of these,

or of other relevant standards, where they are applicable, as long as the over-riding principles described in the

GUM are followed.

2.2 Definitions and interpretation of measurement uncertainty, level of confidence, and

confidence interval, within the Ambient Air Quality Daughter Directives
2.2.1 General

As discussed above, each of the EC Ambient Air Quality Daughter Directives specify data quality objectives

(quantified in their Annexes) that EU Member States must conform to, when reporting the results obtained to the

European Commission (EC). The definitions associated with measurement uncertainty that also relate to, or are

referred to, in the Annexes of these Daughter Directives are discussed below for, clarity, and also to serve as a

background to the remainder of this Report.
2.2.2 Uncertainty of measurement

The Air Quality Directives require an evaluation of the ‘uncertainty of the assessment (or measurement) methods’

according to GUM and therefore it is clear that the definition of uncertainty used in GUM shall be used. The GUM

definition of the term ‘uncertainty of measurement’ is therefore reproduced in this Report for completeness as:

Uncertainty of measurement: Parameter, associated with the result of a measurement, that characterizes the

dispersion that would reasonably be attributed to the measurand.

It is also essential to express clearly the conditions under which this uncertainty of the measurement results is to be

evaluated. This issue is also covered in this Report.
2.2.3 Level of confidence

The above term ‘uncertainty of measurement’ is used in the Annexes of the Daughter Directives. The term

‘confidence interval’ is, in addition, used in these Annexes of the Daughter Directives, and this is also defined in the

GUM. The relationship between the terms ‘confidence interval’ and ‘uncertainty of measurement’ is discussed

below in clause 2.2.4. However, it is first useful, for clarification purposes, to discuss the meaning of the term ‘level

of confidence’ which is used throughout these discussions.

As described in the GUM, the usual method of determining the level of uncertainty of a given measurement

parameter is to determine the combined standard uncertainty u of that measured parameter by combining all the

individual standard uncertainties, which may be Type A or Type B. It is then necessary to provide a measure of this

uncertainty that defines an interval about the measurement result that is expected to encompass a large fraction of

the distribution of values that could reasonably be attributed to the measurement. This is illustrated

diagrammatically in Figure 1. This measure of uncertainty is termed the expanded uncertainty, denoted by U, which

is obtained by multiplying the standard uncertainty by a coverage factor k:
---------------------- Page: 8 ----------------------
SIST CR 14377:2002
CR 14377:2002 (E)
Uk u

As is well known, the result of the measurement is then traditionally expressed as:

YyU

This is interpreted to mean that the best estimate of the value attributed to Y is y and also that y–U to y+U is an

interval that may be expected to encompass a large fraction of the distribution of the value.

More specifically, U may be interpreted as defining the interval about the measurement result that encompasses a

large fraction p of the probability characterised by that result and its associated uncertainty. This fraction p is then

the ‘level of confidence’ of that interval. Generally, and in the specific case of the requirements of the EC Air-quality

Daughter Directives this level of confidence p is defined as 95 % – i.e. 95 % of all the individual measurement

parameters lie within ±U of the best estimated value y and only 5 % lie outside of this interval.

It is clear, however, from this discussion that whatever level of confidence is actually used, it shall be stated

unambiguously, together with the result and its expanded level of uncertainty. It should also be noted that the

coverage factor k which is used as a multiplicand with u to obtain an appropriate level of confidence will, of course,

depend on the number of independent (uncorrelated) determinations of that measurand (i.e. the number of degrees

of freedom), which make up the probability distribution (as described in the GUM). Therefore k = 2 shall be used

only when there are a sufficiently large number of such degrees of freedom.
2.2.4 Relationship between confidence interval and measurement uncertainty

The relationship between uncertainty of measurement and the confidence interval I of a statistical distribution of

measurement results is illustrated, in an exemplar manner, in Figure 1, where
T  

1 = – 1,96 0 is the lower confidence limit for a large number of degrees of freedom - expressed in this

example at a level of confidence of 95%
T  

2 = + 1,96 0 is the upper confidence limit for a large number of degrees of freedom - expressed in

this example at a level of confidence of 95%

0 is the normally defined standard deviation (uncertainty) associated with the statistical

population distribution

The length of the confidence interval I defined in the GUM, for a large number of degrees of freedom expressed at

a level of confidence of 95 %, is then:
ITT21 ,96

Thus the standard deviation (uncertainty) associated with this confidence interval is:

21,96

This then enables the expanded uncertainty U associated with the confidence interval I to be expressed (for a large

number of degrees of freedom, expressed at a level of confidence of 95%) as:
UI19,,6  0 5

This interpretation is then fully consistent with the definition and use of the term ‘expanded uncertainty’ within the

GUM, and it is also consistent with the intentions within the relevant Annexes of the Air Quality Daughter Directives.

As noted above, the key terms which are used to describe the statistical procedures associated with measurement

uncertainty are defined in Annex E of this Report.
---------------------- Page: 9 ----------------------
SIST CR 14377:2002
CR 14377:2002 (E)
I = 2  1,96  
1 -
 
n 
T  -  + T
 
1 2
Figure 1 — Graphical illustration of the uncertainty distribution
of a measurement quantity derived from repeated observations
2.3 EC Terms of Reference

The EC has produced certain Terms of Reference (ToR) which specify requirements for the contents of European

Standards covering ambient air quality, and also to assist in their preparation. The latest draft of these is

reproduced in Annex C of this Report. It should be noted that, although these ToR are still draft, and thus are liable

to revision, they will retain the same under-lying principles given in the current draft. It should also be noted that

because different types of instruments or equipment are available to measure air quality, the EC ToR allow for the

type-approval of instruments, and therefore any instruments which operate on the same principle as those stated

as meeting all the requirements of the Reference Method prescribed in the relevant European Standard are also

acceptable – as long as the latter have been validated by demonstrating that they conform to all the relevant

performance criteria applicable to the Reference Method, and also to the overall requirements of the data quality

objectives in the appropriate EC Daughter Directive.

The Daughter Directives also contain air-quality concentration limit values set by the European Commission. These

are expressed as limits to be met over stated temporal averaging periods, and some of these may be as long as a

year. For discontinuous methods this poses potential difficulties for the development and validation of the relevant

European Standards, not only with respect to the sampling method, but also in evaluation of the uncertainty of the

results of the measurements. In these cases, it should be assumed that the uncertainty of the Reference Method,

which is derived for the shorter averaging period used during the laboratory and field validation trials, applies to the

longer averaging times specified in the Directives. This, although not completely rigorous, is a pragmatic solution.

However, in these circumstances the longest averaging times which it is practical to employ during the laboratory

and field trials should be used.
3 Approaches to uncertainty estimation
3.1 General

As noted above, the Air Quality Daughter Directives place specific data quality objectives on the uncertainty of the

measurements carried out. This uncertainty of the measurement result is expressed in terms of a maximum

allowable percentage of the air quality limit value(s), and it should be calculated with respect to the stated

averaging period(s) referred to in the Directives. The ToR also state that these data quality requirements

(‘objectives’ in Daughter Directives) are criteria against which the Reference Method is assessed using a

combination of laboratory and field tests. The primary concern is therefore to ensure that the assessment of

uncertainty of measurement that is used within the CEN/TC 264 standards to derive uncertainties from these

assessment tests, allows valid comparisons to be made with the Directives’ objectives, both in terms of the

---------------------- Page: 10 ----------------------
SIST CR 14377:2002
CR 14377:2002 (E)

concentration levels actually present, the prescribed limit values, and the associated averaging periods. This has

implications for the subsequent use of the Reference Methods and the on-going quality assurance and quality

control (QA/QC) regime which is adopted, as knowledge of these will be necessary to derive, for example, an

uncertainty on a yearly average when the assessment tests are conducted over a matter of weeks or months.

The Daughter Directives allow the use of the GUM or the International Standards ISO 5725 (or equivalents) to

assess the measurement uncertainty of the Reference Methods. It could be argued that these two approaches are

not compatible. However, this is not necessarily correct, and the two procedures may be used in such a way that

they are complimentary. This approach is explained further in clause 4 of this Report. However, it is useful first to

review briefly the published documents covering uncertainty of measurements that are most relevant to this Report.

3.2 Guide to the Expression of Uncertainty in Measurement

The GUM provides the general concept for the harmonised estimation of measurement uncertainties, agreed on

and adopted by important organisations in the field of general physical metrology and analytical chemistry (e.g.

Bureau International des Poids et Measures (BIPM), International Union of Pure and Applied Chemistry (IUPAC),

International Union of Pure and Applied Physics (IUPAP), International Electrotechnical Commission (IEC), and

International Organisation of Legal Metrology (OIML)), and published by ISO. In overview, the GUM provides a

procedure for assessing the combined uncertainty of a measurement method by assessing and combining all

potential sources of uncertainty. In this procedure it is necessary to:

 Establish a model equation (or measurement equation) and a related uncertainty budget which should list fully

all potential sources of uncertainty. It is also important at this stage to consider carefully the identified

uncertainty contributions of the model equation, so as to ensure that there is no double counting and that any

correlations between uncertainty sources are understood.

 Quantify the individual sources of uncertainty, as standard uncertainties (standard deviations), either by

experiment and statistical analysis of repeated measurement (Type A uncertainties) or by other evaluation

(generally an assumption about the likely distribution describing the uncertainty of the source).

NOTE The above methodology can, in principle, be implemented in all circumstances. However, in some cases,

particularly those involving a complex measurement equation, it may not be practical to establish and evaluate all the

uncertainty components of the measurement equation, and in these cases some components of the uncertainty budget

might potentially be ignored. In these circumstances, the uncertainty evaluations described in clause 3.3 of this Report,

which involve International Standards ISO 5725, may be used, where appropriate, to check and evaluate whether there

are any neglected components of the uncertainty that are of significant magnitude. Any additional components identified

by these means should be incorporated into the uncertainty of the measurement results, in an appropriate manner

according to GUM. Where this is not practical, the field evaluations prescribed by the International Standard

prEN ISO 14956 should, where possible, be used to establish appropriate (additional) components in the uncertainty

budget (see clause 4.1 of this Report).

 Combine the individual uncertainties, taking into account the sensitivity coefficients of each component

determined from the partial differential of
...

Questions, Comments and Discussion

Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.