ASTM D3670-91(2007)
(Guide)Standard Guide for Determination of Precision and Bias of Methods of Committee D22
Standard Guide for Determination of Precision and Bias of Methods of Committee D22
SIGNIFICANCE AND USE
The objective of this standard is to provide guidelines to Committee D22 for the evaluation of the precision and bias, or both, of ASTM standard methods and practices at the time of their development. Such an evaluation is necessary to assure that a cross section of interested laboratories could perform the test and achieve satisfactory results, using the method as written. It also provides guidance to the user as to what levels of precision and accuracy may be expected in such usage.
The write-up of the method describes the media for which the test method is believed to be appropriate. The collaborative test corroborates the write-up within the limitations of the test design. A collaborative test can only use representative media so that universal applicability cannot be implied from the results.
The fundamental assumption of the collaborative test is that the media tested, the concentrations used, and the participating laboratories are representative and provide a fair evaluation of the scope and applicability of the test method as written.
SCOPE
1.1 This standard provides guidance to task groups of Committee D22 on Sampling and Analysis of Atmospheres in planning and conducting collaborative testing of candidate methods.
1.2 It is intended for use with other ASTM practices for the determination of precision and bias.
1.3 It is applicable to most manual and automated methods and to most components of monitoring systems. It is recognized that the evaluation of monitoring systems may provide special problems. Practice D 3249 should be considered for general guidance in this respect.
1.4 It is directly applicable to chemical methods and in principle to most physical methods, sampling methods, and calibration procedures.
1.5 The processes described are for the general validation of methods of test. A user has the obligation and responsibility to validate any method it uses for a specific application and to demonstrate its own competence in the use of validated methods.
General Information
Relations
Standards Content (Sample)
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Contact ASTM International (www.astm.org) for the latest information
Designation: D3670 − 91 (Reapproved2007)
Standard Guide for
Determination of Precision and Bias of Methods of
Committee D22
This standard is issued under the fixed designation D3670; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope cialty Chemicals (Withdrawn 2009)
E691 Practice for Conducting an Interlaboratory Study to
1.1 This standard provides guidance to task groups of
Determine the Precision of a Test Method
Committee D22 on Sampling and Analysis of Atmospheres in
E1169 Practice for Conducting Ruggedness Tests
planning and conducting collaborative testing of candidate
methods.
3. Terminology
1.2 It is intended for use with otherASTM practices for the
3.1 The terms used in this practice are consistent with those
determination of precision and bias.
defined in Practices D2777, E177, E180, and E691.
1.3 It is applicable to most manual and automated methods
3.2 Definitions:
and to most components of monitoring systems. It is recog-
3.2.1 accuracy—the degree of conformity of a value gener-
nized that the evaluation of monitoring systems may provide
ated by a specific procedure to the assumed or accepted true
special problems. Practice D3249 should be considered for
value. It includes both precision and bias.
general guidance in this respect.
3.2.2 bias—a systematic (nonrandom) deviation of the
1.4 It is directly applicable to chemical methods and in
method average value or the measured value from an accepted
principle to most physical methods, sampling methods, and
reference value.
calibration procedures.
3.2.3 candidate method—an analytical method or measure-
1.5 The processes described are for the general validation of
mentprocessbeingconsideredforstandardization.Amethodis
methods of test.Auser has the obligation and responsibility to
a “candidate” until completion of all phases of the consensus
validate any method it uses for a specific application and to
process specified by ASTM regulations for a proposal, an
demonstrate its own competence in the use of validated
emergency standard, or a standard.
methods.
3.2.4 collaborative test—an interlaboratory study of a test
method wherein the participants analyze or make measure-
2. Referenced Documents
ments on sub-samples of the same test material. If the test
2.1 ASTM Standards: method includes the sampling of atmospheres, the participants
D2777 Practice for Determination of Precision and Bias of should sample the same test atmosphere, as possible.
Applicable Test Methods of Committee D19 on Water
3.2.5 laboratory bias—systematic differences between the
D3249 Practice for General Ambient Air Analyzer Proce-
true value and a value reported by a laboratory due to errors of
dures
application such as losses, contamination, miscalibration, and
E177 Practice for Use of the Terms Precision and Bias in
faulty manipulations, for example.
ASTM Test Methods
3.2.6 method bias—systematic departures of the limiting
E180 Practice for Determining the Precision of ASTM
meanfromthetruevalueoftheparametermeasured,causedby
Methods for Analysis and Testing of Industrial and Spe-
physical or chemical phenomena inherent in the methodology.
3.2.7 over-all precision—a value including components of
within-laboratory and between-user variability.
This guide is under the jurisdiction of ASTM Committee D22 on Air
3.2.8 precision—the degree of mutual agreement between
Qualityand is the direct responsibility of Subcommittee D22.01 on Quality Control.
Current edition approved March 1, 2007. Published April 2007. Originally
individual measurements using an analytical method or mea-
approved in 1978. Last previous edition approved in 2001 as D3670 - 91(2001).
surement process. In practice, the standard deviation of an
DOI: 10.1520/D3670-91R07.
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
Standards volume information, refer to the standard’s Document Summary page on The last approved version of this historical standard is referenced on
the ASTM website. www.astm.org.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
D3670 − 91 (2007)
entire array of reviewed and acceptable data is calculated to both, of ASTM standard methods and practices at the time of
provide the value to be stated as the precision of the method. their development. Such an evaluation is necessary to assure
that a cross section of interested laboratories could perform the
3.2.9 ruggedness test—a factorial test designed to explore
test and achieve satisfactory results, using the method as
the sensitivity of the method to variations in the procedure (see
written. It also provides guidance to the user as to what levels
Youden and Steiner, 1987).
of precision and accuracy may be expected in such usage.
3.2.10 single-operator precision—a measure of the replica-
5.2 The write-up of the method describes the media for
tionofrepeatedmeasurementsobtainedbyasingleoperatoron
which the test method is believed to be appropriate. The
a given sample.
collaborative test corroborates the write-up within the limita-
3.2.10.1 Discussion—Other classifications of precision
tions of the test design. A collaborative test can only use
which are useful in evaluating a method, a measurement, or
representative media so that universal applicability cannot be
performance within a single laboratory are: multioperator
implied from the results.
precision, single or multi-apparatus precision, and single or
multi-day precision.
5.3 The fundamental assumption of the collaborative test is
3.2.10.2 Discussion—The terms “repeatability” and “repro-
that the media tested, the concentrations used, and the partici-
ducibility” are not standardized, but have generally come to
pating laboratories are representative and provide a fair evalu-
mean “single-laboratory-operator-material precision” and
ation of the scope and applicability of the test method as
“multi-laboratory-multi-operator-single material precision,”
written.
respectively. Such usage is maintained in the text of this
practice.
6. General Policy
3.2.10.3 Discussion—Further classifications of bias which
6.1 This section describes the general policy to be followed
are useful in evaluating performance are: operator bias, appa-
by Committee D22, its subcommittees, and task groups in the
ratus bias, and day bias.
development of ASTM standard methods and practices. The
objective of Committee D22 is to develop fully evaluated
4. Summary of Guide
standard methods and practices as far as possible. In cases
4.1 Data supporting a statement of single-operator repeat-
where this is not expedient, proposed methods, as defined in
ability is the entrance requirement for any candidate method to
6.2, may be developed. In each case, an appropriate task group
be considered for standardization by Committee D22. The task
shallhavetheresponsibilitytocriticallyexaminethemethodor
group to which a candidate method is assigned will review it
practice, conduct evaluation tests by round robins or other
for adequacy in this respect, and conduct further tests as
techniques including ruggedness tests, and to recommend it, if
necessary to evaluate its precision and bias, as technically
meritorious,forsubcommitteeballoting.Nomethodorpractice
feasible. A method may be accepted as a proposed method,
shall be released and recommended for balloting unless the
providedtherepeatabilityisknownorhasbeenascertainedand
precision or accuracy requirements, or both, as set forth in the
provided all other criteria for acceptance have been met.
following, have been satisfied.
Independenttestsbyatleastthreelaboratoriesshallberequired
6.1.1 Collaborative testing by D22 is the preferred method
tosubstantiatetherepeatabilityofamethodbeforeitattainsthe
of validation. Data obtained by collaborative testing by others
status of a standard method. Collaborative testing by at least
may be used in lieu of D22 testing, provided that such testing
five laboratories to estimate the interlaboratory bias and, if
was equivalent to ASTM approved procedures. In either case,
applicable to evaluate the method’s inherent bias with respect
a copy of the test procedures and data must be filed in a
to the “true” value is needed for all standard methods and must
research file maintained at ASTM for such purposes.
be accomplished within 5 years of its initial issuance as a
standard, if such testing has not already been done. Failure to 6.2 Proposed Method—A proposed method is one that has
subject such methods to appropriate collaborative testing, foundfavorableusageinaspecificlaboratory,orhasbeenused
constitutes valid grounds for disallowing its reapproval as a by several laboratories, but has not yet been standardized. In
standard. each case, the test method is submitted by its proponents to
Committee D22 for standardization.
4.2 Procedures that may be used in collecting the required
6.2.1 The minimum requirement for balloting of a proposed
data are given with particular emphasis upon the applicability
method shall be the inclusion in it of a single laboratory’s
to analysis of atmospheres. Documentation requirements are
statement of single-operator precision, together with support-
established. Terms that are useful in expressing statements of
ing experimental data. Test methods meeting this requirement
precision and bias are presented.
will be referred to a Task Group, following procedures estab-
lished by Committee D22.
5. Significance and Use
6.2.2 The experimental data needed to support a proposal
5.1 The objective of this standard is to provide guidelines to
must reflect a test of the method as a whole, that is, sampling,
Committee D22 for the evaluation of the precision and bias, or
apparatus, reagents and, calibration, and must use a procedure
that is essentially identical to that described in the proposal.
Any significant deviations between the procedure used to
Youden, W. J. and Steiner, G. H., Statistical Manual of the Association of
gather the data and the proposed procedure shall be clearly
Offıcial Analytical Chemists, AOAC International, 481 North Frederick Ave., Suite
500, Gaithersburg, MD 20877-2417, 1987. identified.
D3670 − 91 (2007)
6.2.3 If such data are missing or inadequate, but the method course, depend on the simulation achieved, and the statements
itself is considered by consensus of Committee D22 to be of precision and accuracy must define the test conditions.
worthy of further study, a task group may be assigned to
7.2 The ideal test sample is the actual atmosphere for which
conduct experimental studies or enlist the services of at least
the method is intended. However the use of such offers
onecompetentlaboratorytoobtainthedatauponwhichtobase
complications because the composition may not be known at
a statement of single-operator precision.
the moment of test and furthermore may undergo change
6.3 Standard Method–Initial Acceptance—A method that
during the tests. Because actual atmospheric samples cannot be
has found favorable acceptance and for which the within-
collected and stabilized for long periods of time, two proce-
laboratory repeatability has been verified by a multilaboratory
dures are acceptable. Reproducibility and repeatability may be
test program, shall be examined by the task group for compli-
evaluated by simultaneous measurement by participating labo-
ance with the following requirements.
ratories sampling the same atmosphere at substantially the
6.3.1 An initial minimum requirement for establishing a
same time. Alternatively, comparison of a candidate method
standard method is a statement of within-laboratory precision
with a standard method of known precision and bias will
based on data from three laboratories similar to that described
constitute an acceptable technique for evaluation of precision
in 6.2.1-6.2.3.
andaccuracy.Suchmeasurementsmadebyseverallaboratories
6.3.2 If the method purports to measure the concentration of
may be statistically treated to evaluate the reproducibility of
a substance, an investigation of the bias of the method by
the candidate method. In this latter case, the measurements
comparison with a standard must be made by at least one
need not be made at the same place and time by the collabo-
laboratory and the results included in an accuracy statement.
rating laboratories.
6.3.3 Astandard can only be carried under the provisions of
6.3 for five years. Conditions for reapproval are specified in 7.3 A test sample or series of test samples that are stable
6.4.
during the period required to perform a limited series of
measurements are adequate for evaluation of single-operator
6.4 Standard Method–Reapproval—Astandard method may
precision to satisfy the requirements for consideration as a
be retained if it has found extensive use and between-
proposed method. Three levels of concentration are
laboratory precision data have been obtained. Before doing a
recommended, with such levels sufficiently well established to
collaborative study, a ruggedness test should be performed by
determine whether, and to what extent, the repeatability is
at least one laboratory. (see Guide E1169)
dependent or independent of concentration level.
6.4.1 The minimum requirement for retaining a standard
method shall be a statement of the between-laboratory preci-
7.4 A series of test samples of at least three concentration
sion of the method as established in a collaborative test
levels, and available in sufficient number, is required for use by
including at least five participants.
collaborating laboratories to evaluate the repeatability and
6.4.2 If a bias statement is appropriate for the method, the
reproducibility of a candidate method. The samples should be
data supporting the statement should be obtained by at least
stable during the entire test period, which should include a
two laboratories. At least one such test shall include the
reasonable time following the collaborative test to permit
introduction of potential interferences.
resolution of any discrepancies encountered during the evalu-
6.5 In all testing, the minimum number of participants
ation procedures. The compositions of the test samples do not
shouldbeexceededtotheextentpossible.Thestatisticalpower need to be known accurately, but the samples furnished to each
of collaborative testing is greatly enhanced as such numbers
collaboratormustbesufficientlysimilartopermitevaluationof
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.