ASTM D3670-91(2001)
(Guide)Standard Guide for Determination of Precision and Bias of Methods of Committee D22
Standard Guide for Determination of Precision and Bias of Methods of Committee D22
SCOPE
1.1 This standard provides guidance to task groups of Committee D22 on Sampling and Analysis of Atmospheres in planning and conducting collaborative testing of candidate methods.
1.2 It is intended for use with other ASTM practices for the determination of precision and bias.
1.3 It is applicable to most manual and automated methods and to most components of monitoring systems. It is recognized that the evaluation of monitoring systems may provide special problems. Practice D 3249 should be considered for general guidance in this respect.
1.4 It is directly applicable to chemical methods and in principle to most physical methods, sampling methods, and calibration procedures.
1.5 The processes described are for the general validation of methods of test. A user has the obligation and responsibility to validate any method it uses for a specific application and to demonstrate its own competence in the use of validated methods.
General Information
Relations
Standards Content (Sample)
Designation:D3670–91(Reapproved 2001)
Standard Guide for
Determination of Precision and Bias of Methods of
Committee D22
This standard is issued under the fixed designation D 3670; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (e) indicates an editorial change since the last revision or reapproval.
1. Scope Determine the Precision of a Test Method
1.1 This standard provides guidance to task groups of
3. Terminology
Committee D22 on Sampling and Analysis of Atmospheres in
3.1 The terms used in this practice are consistent with those
planning and conducting collaborative testing of candidate
defined in Practices D 2777, E 177, E 180, and E 691.
methods.
3.2 Definitions:
1.2 It is intended for use with otherASTM practices for the
3.2.1 accuracy—the degree of conformity of a value gen-
determination of precision and bias.
erated by a specific procedure to the assumed or accepted true
1.3 It is applicable to most manual and automated methods
value. It includes both precision and bias.
and to most components of monitoring systems. It is recog-
3.2.2 bias—a systematic (nonrandom) deviation of the
nized that the evaluation of monitoring systems may provide
method average value or the measured value from an accepted
special problems. Practice D 3249 should be considered for
reference value.
general guidance in this respect.
3.2.3 candidate method—an analytical method or measure-
1.4 It is directly applicable to chemical methods and in
mentprocessbeingconsideredforstandardization.Amethodis
principle to most physical methods, sampling methods, and
a “candidate” until completion of all phases of the consensus
calibration procedures.
process specified by ASTM regulations for a proposal, an
1.5 Theprocessesdescribedareforthegeneralvalidationof
emergency standard, or a standard.
methods of test.Auser has the obligation and responsibility to
3.2.4 collaborative test—an interlaboratory study of a test
validate any method it uses for a specific application and to
method wherein the participants analyze or make measure-
demonstrate its own competence in the use of validated
ments on sub-samples of the same test material. If the test
methods.
method includes the sampling of atmospheres, the participants
2. Referenced Documents should sample the same test atmosphere, as possible.
3.2.5 laboratory bias—systematic differences between the
2.1 ASTM Standards:
true value and a value reported by a laboratory due to errors of
D 2777 Practice for Determination of Precision and Bias of
application such as losses, contamination, miscalibration, and
Applicable Test Methods of Committee D19 on Water
faulty manipulations, for example.
D 3249 Practice for General Ambient Air Analyzer Proce-
3.2.6 method bias—systematic departures of the limiting
dures
meanfromthetruevalueoftheparametermeasured,causedby
E 177 Practice for Use of the Terms Precision and Bias in
physical or chemical phenomena inherent in the methodology.
ASTM Test Methods
3.2.7 over-all precision—a value including components of
E 180 Practice for Determining the Precision of ASTM
within-laboratory and between-user variability.
Methods for Analysis and Testing of Industrial and Spe-
3.2.8 precision—the degree of mutual agreement between
cialty Chemicals
individual measurements using an analytical method or mea-
E 691 Practice for Conducting an Interlaboratory Study to
surement process. In practice, the standard deviation of an
entire array of reviewed and acceptable data is calculated to
provide the value to be stated as the precision of the method.
This guide is under the jurisdiction of ASTM Committee D22 on Air Quality
and is the direct responsibility of Subcommittee D22.01 on Quality Control.
Current edition approved March 15, 1991. Published May 1991. Originally
published as D 3670 – 78. Last previous edition D 3670 – 81.
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
Standards volume information, refer to the standard’s Document Summary page on
the ASTM website.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.
D3670–91 (2001)
3.2.9 ruggedness test—a factorial test designed to explore test and achieve satisfactory results, using the method as
the sensitivity of the method to variations in the procedure (see written. It also provides guidance to the user as to what levels
Youden and Steiner, 1975). of precision and accuracy may be expected in such usage.
3.2.10 single-operator precision—a measure of the replica- 5.2 The write-up of the method describes the media for
tionofrepeatedmeasurementsobtainedbyasingleoperatoron
which the test method is believed to be appropriate. The
a given sample. collaborative test corroborates the write-up within the limita-
3.2.10.1 Discussion—Other classifications of precision
tions of the test design. A collaborative test can only use
which are useful in evaluating a method, a measurement, or representative media so that universal applicability cannot be
performance within a single laboratory are: multioperator
implied from the results.
precision, single or multi-apparatus precision, and single or
5.3 The fundamental assumption of the collaborative test is
multi-day precision.
that the media tested, the concentrations used, and the partici-
3.2.10.2 Discussion—The terms “repeatability” and “repro-
pating laboratories are representative and provide a fair evalu-
ducibility” are not standardized, but have generally come to
ation of the scope and applicability of the test method as
mean “single-laboratory-operator-material precision” and
written.
“multi-laboratory-multi-operator-single material precision,”
respectively. Such usage is maintained in the text of this
6. General Policy
practice.
6.1 This section describes the general policy to be followed
3.2.10.3 Discussion—Further classifications of bias which
by Committee D22, its subcommittees, and task groups in the
are useful in evaluating performance are: operator bias, appa-
development of ASTM standard methods and practices. The
ratus bias, and day bias.
objective of Committee D22 is to develop fully evaluated
standard methods and practices as far as possible. In cases
4. Summary of Guide
where this is not expedient, proposed methods, as defined in
4.1 Data supporting a statement of single-operator repeat-
6.2, may be developed. In each case, an appropriate task group
ability is the entrance requirement for any candidate method to
shallhavetheresponsibilitytocriticallyexaminethemethodor
be considered for standardization by Committee D22. The task
practice, conduct evaluation tests by round robins or other
group to which a candidate method is assigned will review it
techniques including ruggedness tests, and to recommend it, if
for adequacy in this respect, and conduct further tests as
meritorious,forsubcommitteeballoting.Nomethodorpractice
necessary to evaluate its precision and bias, as technically
shall be released and recommended for balloting unless the
feasible. A method may be accepted as a proposed method,
precision or accuracy requirements, or both, as set forth in the
providedtherepeatabilityisknownorhasbeenascertainedand
following, have been satisfied.
provided all other criteria for acceptance have been met.
6.1.1 Collaborative testing by D22 is the preferred method
Independenttestsbyatleastthreelaboratoriesshallberequired
of validation. Data obtained by collaborative testing by others
tosubstantiatetherepeatabilityofamethodbeforeitattainsthe
may be used in lieu of D22 testing, provided that such testing
status of a standard method. Collaborative testing by at least
was equivalent to ASTM approved procedures. In either case,
five laboratories to estimate the interlaboratory bias and, if
a copy of the test procedures and data must be filed in a
applicable to evaluate the method’s inherent bias with respect
research file maintained at ASTM for such purposes.
to the “true” value is needed for all standard methods and must
6.2 Proposed Method—A proposed method is one that has
be accomplished within 5 years of its initial issuance as a
foundfavorableusageinaspecificlaboratory,orhasbeenused
standard, if such testing has not already been done. Failure to
by several laboratories, but has not yet been standardized. In
subject such methods to appropriate collaborative testing,
each case, the test method is submitted by its proponents to
constitutes valid grounds for disallowing its reapproval as a
Committee D22 for standardization.
standard.
6.2.1 The minimum requirement for balloting of a proposed
4.2 Procedures that may be used in collecting the required
method shall be the inclusion in it of a single laboratory’s
data are given with particular emphasis upon the applicability
statement of single-operator precision, together with support-
to analysis of atmospheres. Documentation requirements are
ing experimental data. Test methods meeting this requirement
established. Terms that are useful in expressing statements of
will be referred to a Task Group, following procedures estab-
precision and bias are presented.
lished by Committee D22.
6.2.2 The experimental data needed to support a proposal
5. Significance and Use
must reflect a test of the method as a whole, that is, sampling,
5.1 The objective of this standard is to provide guidelines to
apparatus, reagents and, calibration, and must use a procedure
Committee D22 for the evaluation of the precision and bias, or
that is essentially identical to that described in the proposal.
both, of ASTM standard methods and practices at the time of
Any significant deviations between the procedure used to
their development. Such an evaluation is necessary to assure
gather the data and the proposed procedure shall be clearly
that a cross section of interested laboratories could perform the
identified.
6.2.3 If such data are missing or inadequate, but the method
itself is considered by consensus of Committee D22 to be
Youden, W. J. and Steiner, G. H., Statistical Manual of the Association of
worthy of further study, a task group may be assigned to
Offıcial Analytical Chemists, AOAC International, 481 North Frederick Ave., Suite
500, Gaithersburg, MD 20877-2417, 1975. conduct experimental studies or enlist the services of at least
D3670–91 (2001)
onecompetentlaboratorytoobtainthedatauponwhichtobase during the tests. Because actual atmospheric samples cannot be
a statement of single-operator precision. collected and stabilized for long periods of time, two proce-
6.3 Standard Method–Initial Acceptance—A method that dures are acceptable. Reproducibility and repeatability may be
has found favorable acceptance and for which the within- evaluated by simultaneous measurement by participating labo-
laboratory repeatability has been verified by a multilaboratory ratories sampling the same atmosphere at substantially the
test program, shall be examined by the task group for compli- same time. Alternatively, comparison of a candidate method
ance with the following requirements. with a standard method of known precision and bias will
6.3.1 An initial minimum requirement for establishing a constitute an acceptable technique for evaluation of precision
standard method is a statement of within-laboratory precision andaccuracy.Suchmeasurementsmadebyseverallaboratories
based on data from three laboratories similar to that described may be statistically treated to evaluate the reproducibility of
in 6.2.1-6.2.3. the candidate method. In this latter case, the measurements
6.3.2 If the method purports to measure the concentration of need not be made at the same place and time by the collabo-
a substance, an investigation of the bias of the method by rating laboratories.
comparison with a standard must be made by at least one 7.3 A test sample or series of test samples that are stable
laboratory and the results included in an accuracy statement. during the period required to perform a limited series of
6.3.3 Astandard can only be carried under the provisions of measurements are adequate for evaluation of single-operator
6.3 for five years. Conditions for reapproval are specified in precision to satisfy the requirements for consideration as a
6.4. proposed method. Three levels of concentration are recom-
6.4 Standard Method–Reapproval—Astandardmethodmay mended, with such levels sufficiently well established to
be retained if it has found extensive use and between- determine whether, and to what extent, the repeatability is
laboratory precision data have been obtained. Before doing a dependent or independent of concentration level.
collaborative study, a ruggedness test should be performed by 7.4 A series of test samples of at least three concentration
at least one laboratory. levels,andavailableinsufficientnumber,isrequiredforuseby
6.4.1 The minimum requirement for retaining a standard collaborating laboratories to evaluate the repeatability and
method shall be a statement of the between-laboratory preci- reproducibility of a candidate method. The samples should be
sion of the method as established in a collaborative test stable during the entire test period, which should include a
including at least five participants. reasonable time following the collaborative test to permit
6.4.2 If a bias statement is appropriate for the method, the resolution of any discrepancies encountered during the evalu-
data supporting the statement should be obtained by at least ation procedures. The compositions of the test samples do not
two laboratories. At least one such test shall include the need to be known accurately, but the samples furnished to each
introduction of potential interferences. collaboratormustbesufficientlysimilartopermitevaluationof
6.5 In all testing, the minimum number of participants measurement errors in excess of compositional inhomogeneity.
shouldbeexceededtotheextentpossible.Thestatisticalpower The test samples for repeatability and reproducibility should
of collaborative testing is greatly enhanced as such numbers closelysimulateactualsourceoratmosphericaircompositions,
are increased. The possibility of invalidation of a test by including the presence of any known interferents. The statisti-
outliers or missing data is also minimized. calstatementsmustreflectthetypeoftestsampleforwhichthe
precision or bias, or both, are specified. The statement should
7. Sample Requirements
include the concentration levels studied and the number of
7.1 The precision and bias of test methods are typically
laboratories participating.
evaluated by the data obtained in the measurement of test
7.5 Accuracy tests to determine the inherent b
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.