ASTM D3670-91(1996)e1
(Guide)Standard Guide for Determination of Precision and Bias of Methods of Committee D22
Standard Guide for Determination of Precision and Bias of Methods of Committee D22
SCOPE
1.1 This standard provides guidance to task groups of Committee D22 on Sampling and Analysis of Atmospheres in planning and conducting collaborative testing of candidate methods.
1.2 It is intended for use with other ASTM practices for the determination of precision and bias.
1.3 It is applicable to most manual and automated methods and to most components of monitoring systems. It is recognized that the evaluation of monitoring systems may provide special problems. Practice D 3249 should be considered for general guidance in this respect.
1.4 It is directly applicable to chemical methods and in principle to most physical methods, sampling methods, and calibration procedures.
1.5 The processes described are for the general validation of methods of test. A user has the obligation and responsibility to validate any method it uses for a specific application and to demonstrate its own competence in the use of validated methods.
General Information
Relations
Standards Content (Sample)
NOTICE: This standard has either been superseded and replaced by a new version or discontinued.
Contact ASTM International (www.astm.org) for the latest information.
e1
Designation: D 3670 – 91 (Reapproved 1996) An American National Standard
Standard Guide for
Determination of Precision and Bias of Methods of
Committee D-22
This standard is issued under the fixed designation D 3670; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (e) indicates an editorial change since the last revision or reapproval.
e NOTE—Section 13 was added editorially in November 1996.
1. Scope Determine the Precision of a Test Method
1.1 This standard provides guidance to task groups of
3. Terminology
Committee D-22 on Sampling and Analysis of Atmospheres in
3.1 The terms used in this practice are consistent with those
planning and conducting collaborative testing of candidate
defined in Practices D 2777, E 177, E 180, and E 691.
methods.
3.2 Definitions:
1.2 It is intended for use with other ASTM practices for the
3.2.1 accuracy—the degree of conformity of a value gen-
determination of precision and bias.
erated by a specific procedure to the assumed or accepted true
1.3 It is applicable to most manual and automated methods
value. It includes both precision and bias.
and to most components of monitoring systems. It is recog-
3.2.2 bias—a systematic (nonrandom) deviation of the
nized that the evaluation of monitoring systems may provide
method average value or the measured value from an accepted
special problems. Practice D 3249 should be considered for
reference value.
general guidance in this respect.
3.2.3 candidate method—an analytical method or measure-
1.4 It is directly applicable to chemical methods and in
ment process being considered for standardization. A method is
principle to most physical methods, sampling methods, and
a “candidate” until completion of all phases of the consensus
calibration procedures.
process specified by ASTM regulations for a proposal, an
1.5 The processes described are for the general validation of
emergency standard, or a standard.
methods of test. A user has the obligation and responsibility to
3.2.4 collaborative test—an interlaboratory study of a test
validate any method it uses for a specific application and to
method wherein the participants analyze or make measure-
demonstrate its own competence in the use of validated
ments on sub-samples of the same test material. If the test
methods.
method includes the sampling of atmospheres, the participants
2. Referenced Documents should sample the same test atmosphere, as possible.
3.2.5 laboratory bias—systematic differences between the
2.1 ASTM Standards:
true value and a value reported by a laboratory due to errors of
D 2777 Practice for Determination of Precision and Bias of
application such as losses, contamination, mis-calibration, and
Applicable Methods of Committee D-19 on Water
faulty manipulations, for example.
D 3249 Practice for General Ambient Air Analyzer Proce-
3.2.6 method bias—systematic departures of the limiting
dures
mean from the true value of the parameter measured, caused by
E 177 Practice for Use of the Terms Precision and Bias in
physical or chemical phenomena inherent in the methodology.
ASTM Test Methods
3.2.7 over-all precision—a value including components of
E 180 Practice for Determining the Precision of ASTM
within-laboratory and between-user variability.
Methods for Analysis and Testing of Industrial Chemicals
3.2.8 precision—the degree of mutual agreement between
E 691 Practice for Conducting an Interlaboratory Study to
individual measurements using an analytical method or mea-
surement process. In practice, the standard deviation of an
This guide is under the jurisdiction of ASTM Committee D-22 on Sampling and
entire array of reviewed and acceptable data is calculated to
Analysis of Atmospheres, and is the direct responsibility of Subcomittee D 22.01 on
provide the value to be stated as the precision of the method.
Quality Control.
3.2.9 ruggedness test—a factorial test designed to explore
Current edition approved March 15, 1991. Published May 1991. Originally
published as D 3670 – 78. Last previous edition D 3670 – 81.
the sensitivity of the method to variations in the procedure (see
Annual Book of ASTM Standards, Vol 11.01.
Annual Book of ASTM Standards, Vol 11.03.
Annual Book of ASTM Standards, Vol 14.02.
Annual Book of ASTM Standards, Vol 15.05.
Copyright © ASTM, 100 Barr Harbor Drive, West Conshohocken, PA 19428-2959, United States.
D 3670
Youden and Steiner, 1975). 5.2 The write-up of the method describes the media for
3.2.10 single-operator precision—a measure of the replica- which the test method is believed to be appropriate. The
tion of repeated measurements obtained by a single operator on collaborative test corroborates the write-up within the limita-
a given sample. tions of the test design. A collaborative test can only use
3.2.10.1 Discussion—Other classifications of precision representative media so that universal applicability cannot be
which are useful in evaluating a method, a measurement, or implied from the results.
performance within a single laboratory are: multioperator
5.3 The fundamental assumption of the collaborative test is
precision, single or multi-apparatus precision, and single or
that the media tested, the concentrations used, and the partici-
multi-day precision.
pating laboratories are representative and provide a fair evalu-
3.2.10.2 Discussion—The terms “repeatability” and “repro-
ation of the scope and applicability of the test method as
ducibility” are not standardized, but have generally come to
written.
mean “single-laboratory-operator-material precision” and
“multi-laboratory-multi-operator-single material precision,”
6. General Policy
respectively. Such usage is maintained in the text of this
6.1 This section describes the general policy to be followed
practice.
by Committee D-22, its subcommittees, and task groups in the
3.2.10.3 Discussion—Further classifications of bias which
development of ASTM standard methods and practices. The
are useful in evaluating performance are: operator bias, appa-
objective of Committee D-22 is to develop fully evaluated
ratus bias, and day bias.
standard methods and practices as far as possible. In cases
where this is not expedient, proposed methods, as defined in
4. Summary of Guide
6.2, may be developed. In each case, an appropriate task group
4.1 Data supporting a statement of single-operator repeat-
shall have the responsibility to critically examine the method or
ability is the entrance requirement for any candidate method to
practice, conduct evaluation tests by round robins or other
be considered for standardization by Committee D-22. The
techniques including ruggedness tests, and to recommend it, if
task group to which a candidate method is assigned will review
meritorious, for subcomittee balloting. No method or practice
it for adequacy in this respect, and conduct further tests as
shall be released and recommended for balloting unless the
necessary to evaluate its precision and bias, as technically
precision or accuracy requirements, or both, as set forth in the
feasible. A method may be accepted as a proposed method,
following, have been satisfied.
provided the repeatability is known or has been ascertained and
6.1.1 Collaborative testing by D-22 is the preferred method
provided all other criteria for acceptance have been met.
of validation. Data obtained by collaborative testing by others
Independent tests by at least three laboratories shall be required
may be used in lieu of D-22 testing, provided that such testing
to substantiate the repeatability of a method before it attains the
was equivalent to ASTM approved procedures. In either case,
status of a standard method. Collaborative testing by at least
a copy of the test procedures and data must be filed in a
five laboratories to estimate the interlaboratory bias and, if
research file maintained at ASTM for such purposes.
applicable to evaluate the method’s inherent bias with respect
6.2 Proposed Method—A proposed method is one that has
to the “true” value is needed for all standard methods and must
found favorable usage in a specific laboratory, or has been used
be accomplished within 5 years of its initial issuance as a
by several laboratories, but has not yet been standardized. In
standard, if such testing has not already been done. Failure to
each case, the test method is submitted by its proponents to
subject such methods to appropriate collaborative testing,
Committee D-22 for standardization.
constitutes valid grounds for disallowing its reapproval as a
6.2.1 The minimum requirement for balloting of a proposed
standard.
method shall be the inclusion in it of a single laboratory’s
4.2 Procedures that may be used in collecting the required
statement of single-operator precision, together with support-
data are given with particular emphasis upon the applicability
ing experimental data. Test methods meeting this requirement
to analysis of atmospheres. Documentation requirements are
will be referred to a Task Group, following procedures estab-
established. Terms that are useful in expressing statements of
lished by Committee D-22.
precision and bias are presented.
6.2.2 The experimental data needed to support a proposal
must reflect a test of the method as a whole, that is, sampling,
5. Significance and Use
apparatus, reagents and, calibration, and must use a procedure
5.1 The objective of this standard is to provide guidelines to
that is essentially identical to that described in the proposal.
Committee D-22 for the evaluation of the precision and bias, or
Any significant deviations between the procedure used to
both, of ASTM standard methods and practices at the time of
gather the data and the proposed procedure shall be clearly
their development. Such an evaluation is necessary to assure
identified.
that a cross section of interested laboratories could perform the
6.2.3 If such data are missing or inadequate, but the method
test and achieve satisfactory results, using the method as
itself is considered by consensus of Committee D-22 to be
written. It also provides guidance to the user as to what levels
worthy of further study, a task group may be assigned to
of precision and accuracy may be expected in such usage.
conduct experimental studies or enlist the services of at least
one competent laboratory to obtain the data upon which to base
6 a statement of single-operator precision.
Youden, W. J. and Steiner, G. H., Statistical Manual of the Association of
Offıcial Analytical Chemists, AOAC, P.O. Box 540, Washington, DC 20044, 1975. 6.3 Standard Method–Initial Acceptance—A method that
D 3670
has found favorable acceptance and for which the within- evaluated by simultaneous measurement by participating labo-
laboratory repeatability has been verified by a multilaboratory ratories sampling the same atmosphere at substantially the
test program, shall be examined by the task group for compli- same time. Alternatively, comparison of a candidate method
ance with the following requirements. with a standard method of known precision and bias will
6.3.1 An initial minimum requirement for establishing a constitute an acceptable technique for evaluation of precision
standard method is a statement of within-laboratory precision and accuracy. Such measurements made by several laboratories
based on data from three laboratories similar to that described may be statistically treated to evaluate the reproducibility of
in 6.2.1-6.2.3. the candidate method. In this latter case, the measurements
6.3.2 If the method purports to measure the concentration of need not be made at the same place and time by the collabo-
a substance, an investigation of the bias of the method by rating laboratories.
comparison with a standard must be made by at least one 7.3 A test sample or series of test samples that are stable
laboratory and the results included in an accuracy statement. during the period required to perform a limited series of
6.3.3 A standard can only be carried under the provisions of measurements are adequate for evaluation of single-operator
6.3 for five years. Conditions for reapproval are specified in precision to satisfy the requirements for consideration as a
6.4. proposed method. Three levels of concentration are recom-
6.4 Standard Method–Reapproval—A standard method may mended, with such levels sufficiently well established to
be retained if it has found extensive use and between- determine whether, and to what extent, the repeatability is
laboratory precision data have been obtained. Before doing a dependent or independent of concentration level.
collaborative study, a ruggedness test should be performed by 7.4 A series of test samples of at least three concentration
at least one laboratory. levels, and available in sufficient number, is required for use by
6.4.1 The minimum requirement for retaining a standard collaborating laboratories to evaluate the repeatability and
method shall be a statement of the between-laboratory preci- reproducibility of a candidate method. The samples should be
sion of the method as established in a collaborative test stable during the entire test period, which should include a
including at least five participants. reasonable time following the collaborative test to permit
6.4.2 If a bias statement is appropriate for the method, the resolution of any discrepancies encountered during the evalu-
data supporting the statement should be obtained by at least ation procedures. The compositions of the test samples do not
two laboratories. At least one such test shall include the need to be known accurately, but the samples furnished to each
introduction of potential interferences. collaborator must be sufficiently similar to permit evaluation of
6.5 In all testing, the minimum number of participants measurement errors in excess of compositional inhomogeneity.
should be exceeded to the extent possible. The statistical power The test samples for repeatability and reproducibility should
of collaborative testing is greatly enhanced as such numbers closely simulate actual source or atmospheric air compositions,
are increased. The possibility of invalidation of a test by including the presence of any known interferents. The statisti-
outliers or missing data is also minimized. cal statements must reflect the type of test sample for which the
precision or bias, or both, are specified. The statement should
7. Sample Requirements
include the concentration levels studied and the number of
7.1 The precision and bias of test methods are typically
laboratories participating.
eva
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.