ASTM D7440-08
(Practice)Standard Practice for Characterizing Uncertainty in Air Quality Measurements
Standard Practice for Characterizing Uncertainty in Air Quality Measurements
SIGNIFICANCE AND USE
A primary use intended for this practice is for qualifying ASTM International Standards as Standard Test Methods. In the past, a “Precision and Bias” report has been required. However, recently a statement of uncertainty has become an acceptable alternative to D 3670 – 91: Guide for Determination of Precision and Bias of Methods of Committee D-22. Inclusion of such a statement with a method description simplifies comparison of ASTM Test Methods to analogous ISO and CEN standards, now required to have uncertainty statements.
Standardizing the characterization of sampling/analytical method performance is expected to be useful in other applications as well. For example, performance details are a necessity for justifying compliance decisions based on experimental air quality assessments (6). Documented uncertainty can form a basis for specific criteria defining acceptable sampling/analytical method performance.
Furthermore, high quality atmospheric measurements are vital for making decisions as to how hazardous substances are to be controlled. Valid data are required for drawing reasonable epidemiological conclusions, for making sound decisions as to acceptable limits, as well as for determining the efficacy of a hazard control system.
Finally, because of developing world-wide acceptance of ISO GUM for detailing measurements when statistics are simple, the practice should be useful in comparing ASTM International Test Methods to others’ published methods. The codification of statistical procedures may in fact minimize the difficulty in interpreting a plethora of individual, albeit possibly valid, approaches.
SCOPE
1.1 This practice is for assisting developers and users of air quality methods for sampling concentrations of both airborne and settled materials in characterizing measurements as to uncertainty. Where possible, analysis into uncertainty components as recommended in the ISO Guide to the Expression of Uncertainty in Measurement (1, ISO GUM) is suggested. Aspects of uncertainty estimation particular to air quality measurement are emphasized. For example, air quality assessment is often complicated by: the difficulty of taking replicate measurements owing to the large spatio-temporal variation in concentration values to be measured; systematic error or bias, both corrected and uncorrected; and the (rare) non-normal distribution of errors. This practice operates mainly through example. Background and mathematical development are relegated to appendices for optional reading.
1.2 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility of the user of this standard to establish appropriate safety and health practices and determine the applicability of regulatory limitations prior to use.
General Information
Relations
Standards Content (Sample)
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Contact ASTM International (www.astm.org) for the latest information
Designation: D7440 − 08
StandardPractice for
Characterizing Uncertainty in Air Quality Measurements
This standard is issued under the fixed designation D7440; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope E691Practice for Conducting an Interlaboratory Study to
Determine the Precision of a Test Method
1.1 This practice is for assisting developers and users of air
2.2 Other International Standards:
quality methods for sampling concentrations of both airborne
ISO GUMGuide to the Expression of Uncertainty in
and settled materials in characterizing measurements as to
Measurement, ISO Guide 98, 1995 (See Ref (1), giving
uncertainty. Where possible, analysis into uncertainty compo-
initial publication.)
nents as recommended in the ISO Guide to the Expression of
2 ISO7708AirQuality—ParticleSizeFractionDefinitionsfor
Uncertainty in Measurement (1, ISO GUM) is suggested.
Health-Related Sampling
Aspects of uncertainty estimation particular to air quality
ISO 15767WorkplaceAtmospheres—Controlling and Char-
measurement are emphasized. For example, air quality assess-
acterizing Errors in Weighing Collected Aerosol
ment is often complicated by: the difficulty of taking replicate
ISO 16107Workplace Atmospheres—Protocol for Evaluat-
measurements owing to the large spatio-temporal variation in
ing the Performance of Diffusive Samplers, 2007
concentration values to be measured; systematic error or bias,
EN 482Workplace Atmospheres—General Requirements
both corrected and uncorrected; and the (rare) non-normal
forthePerformanceofProceduresfortheMeasurementof
distribution of errors. This practice operates mainly through
Chemical Agents
example. Background and mathematical development are rel-
egated to appendices for optional reading.
3. Terminology
1.2 This standard does not purport to address all of the
3.1 Definitions—For definitions of terms used in this
safety concerns, if any, associated with its use. It is the
practice, see Terminology D1356.
responsibility of the user of this standard to establish appro-
3.2 Other terms defined as follows are taken from ISO GUM
priate safety and health practices and determine the applica-
unless otherwise noted:
bility of regulatory limitations prior to use.
3.2.1 accuracy—closeness of agreement between the result
2. Referenced Documents
of a measurement and a true value of the measurand.
2.1 ASTM Standards:
3.2.2 combined standard uncertainty, u —standard uncer-
c
D1356Terminology Relating to Sampling and Analysis of tainty of the result of a measurement when that result is
Atmospheres
obtainedfromthevaluesofanumberofotherquantities,equal
D3670Guide for Determination of Precision and Bias of to the positive square root of a sum of terms, the terms being
Methods of Committee D22
the variances or covariances of these other quantities weighted
D6061Practice for Evaluating the Performance of Respi- according to how the measurement result varies with changes
rable Aerosol Samplers in these quantities.
D6246Practice for Evaluating the Performance of Diffusive 3.2.2.1 Discussion—As within ISO GUM, the “other quan-
Samplers tities” are designated uncertainty components u from source j.
j
D6552Practice for Controlling and Characterizing Errors in The component u is taken as the standard deviation estimate
j
Weighing Collected Aerosols from source j in the case of a source of random variation.
3.2.3 coverage factor, k—numerical factor used as a multi-
plier of the combined standard uncertainty (u ) in order to
ThispracticeisunderthejurisdictionofASTMCommitteeD22onAirQuality c
and is the direct responsibility of Subcommittee D22.01 on Quality Control. obtain an expanded uncertainty (U).
Current edition approved April 1, 2008. Published May 2008. DOI: 10.1520/
3.2.3.1 Discussion—The factor k depends on the specific
D7440-08.
meaning attributed to the expanded uncertainty U. However,
Theboldfacenumbersinparenthesesrefertothelistofreferencesattheendof
for simplicity this practice adopts the now nearly traditional
this standard.
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
Standards volume information, refer to the standard’s Document Summary page on Available fromAmerican National Standards Institute (ANSI), 25 W. 43rd St.,
the ASTM website. 4th Floor, New York, NY 10036, http://www.ansi.org.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
D7440 − 08
coverage factor as the value 2, determining the specific 3.2.15 TypeAevaluation (of uncertainty)—methodofevalu-
meaning of the expanded uncertainty U in different circum- ation of uncertainty by the statistical analysis of series of
stances. Other coverage factors if needed are then easily
observations.
implemented simply by multiplication of the traditional ex-
3.2.16 TypeBevaluation(ofuncertainty)—methodofevalu-
panded uncertainty U (see 7.1 – 7.4).
ation of uncertainty by means other than the statistical analysis
3.2.3.2 Discussion—The use of a single coverage factor,
of series of observations.
often through approximation, avoids the overly conservative
use of individual component confidence limits rather than root
4. Background Information
variance estimates as uncertainty components.
4.1 Uncertainty in a measurement result can be taken as the
3.2.4 error (of measurement)—result of a measurement
range about an estimate, corrected for bias if known, contain-
minus a true value of the measurand.
ing the true, or mean reference value—in the language of ISO
3.2.5 expanded uncertainty, U—quantity defining an inter-
GUM, the measurand value at given confidence. Uncertainty
val about the result of a measurement that may be expected to
accounts not only for variation in a method’s results at
encompass a large fraction of the distribution of values that
application, but also for incomplete characterization of the
could reasonably be attributed to the measurand.
method when evaluated. Per ISO GUM, uncertainty may often
3.2.5.1 Discussion—This definition has the breadth to en-
usefully be analyzed into individual components.
compass a wide variety of conceptions.
3.2.5.2 Discussion—The expanded uncertainty U in some
4.2 Thereareseveralaspectsofuncertaintycharacterization
cases is expressed in absolute terms, but sometimes as relative
specific to air quality measurements. One of these aspects
to the measurement result. What is meant is generally clear
concerns known, that is, correctible, systematic error or mean
from the context.
bias of a measurement relative to a true measurand value.
Several measurement methods exist with such bias left uncor-
3.2.6 influence quantity—quantity that is not the measurand
rectedbecauseofpolicy,tradition,orotherreason. Uncertainty
but that affects the result of the measurement.
deals only with what is unknown about a measurement, and as
3.2.7 measurand—particular quantity subject to measure-
suchdoesnotincludecorrectible(known)bias.Themagnitude
ment.
of the difference between estimate and measurand value is
3.2.8 measurand value—(adapted from ISO GUM), un-
covered by accuracy as defined qualitatively in ISO GUM,
known quantity whose measurement is sought, often called the
rather than uncertainty, particularly when the bias is known,
true value. Examples are the concentration (mg/m)ofa
but uncorrected. Such methods require specification of both
substance in the air at a particular time and place, the
uncertaintyandasmuchasisknownoftheuncorrectedbias,or
time-weighted average of a concentration at a particular
alternatively the adoption of an accuracy measure.
position, or the expected mean concentration estimate as
obtained by a reference method at a specific time and position.
4.3 Oftenbiasisknowntoexist,butwithunknownvalue.In
the case where only limits may be placed on the magnitude of
3.2.9 (population) variance (of a random variable)—the
the bias, ISO GUM generally recommends treating the bias as
expectation of the square of the centered random variable.
uniformly distributed within the known limits. Such a distri-
3.2.10 random error—result of a measurement minus the
bution refers to independent situations, for example,
mean that would result from an infinite number of measure-
calibrations, where bias may arise (see 7.4 and Appendix X2),
ments of the same measurand carried out under the same
rather than variation at the point of method application. Even
(repeatability) conditions of measurement.
though such an equal-likelihood bias distribution may be
3.2.10.1 Discussion—Random error is equal to error minus
unrealistic, nevertheless a standard deviation estimate may be
systematic error.
madethatrevealsthelimitsonthebias.Iftheeven-distribution
3.2.11 (sample) variance—the sum of the squared devia-
approximation is clearly invalid for a relevant set of
tions of observations from their average divided by one less
measurements, the procedure may be adjusted slightly by
than the number of observations.
adopting an accuracy measure tailored to the assumed limits.
3.2.11.1 Discussion—The sample variance is an unbiased
4.4 Another issue concerns the distribution of measure-
estimator of the population variance.
ments. ISO GUM deals only with normally distributed first-
3.2.12 standard deviation—positive square root of the vari-
order (that is, “small”) variations relative to measurand values.
ance.
An example to the contrary is afforded by normally distributed
3.2.13 symmetric accuracy range A—the range symmetric
data confounded by a small number of apparent outliers (3),
about (true) measurand values containing 95% of measure-
which may not detract from the method performance (see
ment estimates. A is a specific quantification of accuracy. (2)
Appendix X4 for details). Another example is the determina-
ISO 16107
tion of an aerosol concentration at one location (perhaps at a
worker’slapel)asanestimateoftheconcentrationataseparate
3.2.14 systematic error (bias)—meanthatwouldresultfrom
an infinite number of measurements of the same measurand point (such as a breathing zone). In this case the variations can
beoftheorderoftheestimateitselfandmayhavethecharacter
carriedoutunderrepeatabilityconditionsminusatruevalueof
the measurand. of a log-normal distribution.
D7440 − 08
4.5 The spatial inhomogeneity alluded to in 4.4 relates to uncertaintyevaluationmaysometimesseeminexpensive,there
another point regarding the focus of this practice. The spatio- isadifficultyincoveringessentialcontingenciesofthemethod
temporal variations in air quality characteristics are generally
application.
so large (4) as to preclude evaluation of a method during
6.2 Uncertainty component analysis further has several
application through the use of replicate measurements. In this
specific advantages over global analysis. The results may be
case, often an initial single method evaluation is undertaken
applicable to a variety of situations. For example, an aerosol
with the purpose of determining uncertainty present in subse-
sampler might be (globally) evaluated as to particle-size-
quent applications of the method. Confidence in such an
dependent error by side-by-side comparison to a reference
evaluation can be specified and relates to the concept of
sampler in several coal mines. The knowledge obtained may
prediction-intervals(5) (see 7.2).
not be as easily applied for sampler use in iron mines, for
4.6 A related subject is measurement system control. The
example, as more detailed information on how the sampler
measurement system must remain in a state of statistical
performs over given dust size distributions may be needed.
control if an introductory evaluation is to characterize later
Furthermore, specific problem areas of a given method may be
practical applications of the method. Measurement system
pinpointed. The detailed itemization of uncertainty sources
control is evaluated using an ongoing quality control program,
leads to a transparency in covering the essential problems of a
testing critical performance aspects for detecting problems
measurement method. Examples of potentially significant un-
which may develop in the method.
certainty components are listed in Table 1.
5. Significance and Use
6.3 Type A and B Uncertainty Components:
5.1 Aprimaryuseintendedforthispracticeisforqualifying 6.3.1 Components that have been statistically evaluated
ASTM International Standards as Standard Test Methods. In
during method application may be classified as Type A. (See
the past, a “Precision and Bias” report has been required.
Section 7 for specific examples.)
However, recently a statement of uncertainty has become an
6.3.2 Some components are often statistically evaluated
acceptable alternative to D3670–91: Guide for Determination
during an initial method evaluation, rather than at application.
ofPrecisionandBiasofMethodsofCommitteeD22.Inclusion
Also acknowledged is a common situation that components
of such a statement with a method description simplifies
may not have been characterized in a statistically valid manner
comparison of ASTM Test Methods to analogous ISO and
andthereforemayrequireprofessionaljudgmentforitemizing.
CEN standards, now required to have uncertainty statements.
Such components are termed Type B uncertainties. Type B
5.2 Standardizing the characterization of sampling/ uncertainties are often associated with unknown systematic
analyticalmethodperformanceisexpectedtobeusefulinother error or bias; however, random variation may also fall into this
applications as well. For example, performance details are a
category. For example, a common assumption (see, for
necessity for justifying compliance decisions based on experi-
mental air quality assessments (6). Documented uncertainty
can form a basis for specific criteria defining acceptable
sampling/analytical method performance.
TABLE 1 Common Potential Uncertainty Components
Sampling
5.3 Furthermore, high quality atmospheric measurements
personal sampling pump flow rate: setting the pump and subsequent drift
are vital for making decisions as to how hazardous substances
sampling rate of diffusive sampler
are to be controlled. Valid data are required for drawing
sampler dimension (aerosol and diffusive sampling)
collection efficiency of a sampler or sampling medium
reasonable epidemiological conclusions, for making sound
(also, see (7))
decisionsastoacceptablelimits,aswellasfordeterminingthe
Analytical
efficacy o
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.