ASTM D3670-91(2022)
(Guide)Standard Guide for Determination of Precision and Bias of Methods of Committee D22
Standard Guide for Determination of Precision and Bias of Methods of Committee D22
SIGNIFICANCE AND USE
5.1 The objective of this standard is to provide guidelines to Committee D22 for the evaluation of the precision and bias, or both, of ASTM standard methods and practices at the time of their development. Such an evaluation is necessary to assure that a cross section of interested laboratories could perform the test and achieve satisfactory results, using the method as written. It also provides guidance to the user as to what levels of precision and accuracy may be expected in such usage.
5.2 The write-up of the method describes the media for which the test method is believed to be appropriate. The collaborative test corroborates the write-up within the limitations of the test design. A collaborative test can only use representative media so that universal applicability cannot be implied from the results.
5.3 The fundamental assumption of the collaborative test is that the media tested, the concentrations used, and the participating laboratories are representative and provide a fair evaluation of the scope and applicability of the test method as written.
SCOPE
1.1 This standard provides guidance to task groups of Committee D22 on Sampling and Analysis of Atmospheres in planning and conducting collaborative testing of candidate methods.
1.2 It is intended for use with other ASTM practices for the determination of precision and bias.
1.3 It is applicable to most manual and automated methods and to most components of monitoring systems. It is recognized that the evaluation of monitoring systems may provide special problems. Practice D3249 should be considered for general guidance in this respect.
1.4 It is directly applicable to chemical methods and in principle to most physical methods, sampling methods, and calibration procedures.
1.5 The processes described are for the general validation of methods of test. A user has the obligation and responsibility to validate any method it uses for a specific application and to demonstrate its own competence in the use of validated methods.
1.6 This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
General Information
Relations
Standards Content (Sample)
This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation: D3670 − 91 (Reapproved 2022)
Standard Guide for
Determination of Precision and Bias of Methods of
Committee D22
This standard is issued under the fixed designation D3670; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope D3249 Practice for General Ambient Air Analyzer Proce-
dures
1.1 This standard provides guidance to task groups of
E177 Practice for Use of the Terms Precision and Bias in
Committee D22 on Sampling and Analysis of Atmospheres in
ASTM Test Methods
planning and conducting collaborative testing of candidate
E180 Practice for Determining the Precision of ASTM
methods.
Methods for Analysis and Testing of Industrial and Spe-
1.2 It is intended for use with otherASTM practices for the 3
cialty Chemicals (Withdrawn 2009)
determination of precision and bias.
E691 Practice for Conducting an Interlaboratory Study to
1.3 It is applicable to most manual and automated methods Determine the Precision of a Test Method
E1169 Practice for Conducting Ruggedness Tests
and to most components of monitoring systems. It is recog-
nized that the evaluation of monitoring systems may provide
3. Terminology
special problems. Practice D3249 should be considered for
general guidance in this respect.
3.1 The terms used in this practice are consistent with those
defined in Practices D2777, E177, E180, and E691.
1.4 It is directly applicable to chemical methods and in
principle to most physical methods, sampling methods, and 3.2 Definitions:
calibration procedures. 3.2.1 accuracy, n—the degree of conformity of a value
generated by a specific procedure to the assumed or accepted
1.5 The processes described are for the general validation of
true value. It includes both precision and bias.
methods of test.Auser has the obligation and responsibility to
3.2.2 bias, n—a systematic (nonrandom) deviation of the
validate any method it uses for a specific application and to
method average value or the measured value from an accepted
demonstrate its own competence in the use of validated
reference value.
methods.
1.6 This international standard was developed in accor- 3.2.3 candidate method, n—an analytical method or mea-
surement process being considered for standardization. A
dance with internationally recognized principles on standard-
ization established in the Decision on Principles for the method is a “candidate” until completion of all phases of the
consensus process specified by ASTM regulations for a
Development of International Standards, Guides and Recom-
mendations issued by the World Trade Organization Technical proposal, an emergency standard, or a standard.
Barriers to Trade (TBT) Committee.
3.2.4 collaborative test, n—an interlaboratory study of a test
method wherein the participants analyze or make measure-
2. Referenced Documents
ments on sub-samples of the same test material. If the test
method includes the sampling of atmospheres, the participants
2.1 ASTM Standards:
should sample the same test atmosphere, as possible.
D2777 Practice for Determination of Precision and Bias of
Applicable Test Methods of Committee D19 on Water
3.2.5 laboratory bias, n—systematic differences between
thetruevalueandavaluereportedbyalaboratoryduetoerrors
of application such as losses, contamination, miscalibration,
1 and faulty manipulations, for example.
This guide is under the jurisdiction of ASTM Committee D22 on Air
Qualityand is the direct responsibility of Subcommittee D22.01 on Quality Control.
3.2.6 method bias, n—systematic departures of the limiting
Current edition approved Sept. 1, 2022. Published September 2022. Originally
meanfromthetruevalueoftheparametermeasured,causedby
approved in 1978. Last previous edition approved in 2014 as D3670 – 91 (2014).
physical or chemical phenomena inherent in the methodology.
DOI: 10.1520/D3670-91R22.
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
Standards volume information, refer to the standard’s Document Summary page on The last approved version of this historical standard is referenced on
the ASTM website. www.astm.org.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
D3670 − 91 (2022)
3.2.7 over-all precision, n—a value including components 5. Significance and Use
of within-laboratory and between-user variability.
5.1 The objective of this standard is to provide guidelines to
3.2.8 precision, n—the degree of mutual agreement between
Committee D22 for the evaluation of the precision and bias, or
individual measurements using an analytical method or mea-
both, of ASTM standard methods and practices at the time of
surement process. In practice, the standard deviation of an
their development. Such an evaluation is necessary to assure
entire array of reviewed and acceptable data is calculated to
that a cross section of interested laboratories could perform the
provide the value to be stated as the precision of the method.
test and achieve satisfactory results, using the method as
written. It also provides guidance to the user as to what levels
3.2.9 ruggedness test, n—a factorial test designed to explore
of precision and accuracy may be expected in such usage.
the sensitivity of the method to variations in the procedure (see
Youden and Steiner, 1987).
5.2 The write-up of the method describes the media for
which the test method is believed to be appropriate. The
3.2.10 single-operator precision, n—a measure of the rep-
collaborative test corroborates the write-up within the limita-
lication of repeated measurements obtained by a single opera-
tions of the test design. A collaborative test can only use
tor on a given sample.
3.2.10.1 Discussion—Other classifications of precision representative media so that universal applicability cannot be
implied from the results.
which are useful in evaluating a method, a measurement, or
performance within a single laboratory are: multioperator
5.3 The fundamental assumption of the collaborative test is
precision, single or multi-apparatus precision, and single or
that the media tested, the concentrations used, and the partici-
multi-day precision.
pating laboratories are representative and provide a fair evalu-
3.2.10.2 Discussion—The terms “repeatability” and “repro-
ation of the scope and applicability of the test method as
ducibility” are not standardized, but have generally come to
written.
mean “single-laboratory-operator-material precision” and
“multi-laboratory-multi-operator-single material precision,”
6. General Policy
respectively. Such usage is maintained in the text of this
6.1 This section describes the general policy to be followed
practice.
by Committee D22, its subcommittees, and task groups in the
3.2.10.3 Discussion—Further classifications of bias which
development of ASTM standard methods and practices. The
are useful in evaluating performance are: operator bias, appa-
objective of Committee D22 is to develop fully evaluated
ratus bias, and day bias.
standard methods and practices as far as possible. In cases
4. Summary of Guide
where this is not expedient, proposed methods, as defined in
6.2, may be developed. In each case, an appropriate task group
4.1 Data supporting a statement of single-operator repeat-
shallhavetheresponsibilitytocriticallyexaminethemethodor
ability is the entrance requirement for any candidate method to
practice, conduct evaluation tests by round robins or other
be considered for standardization by Committee D22. The task
techniques including ruggedness tests, and to recommend it, if
group to which a candidate method is assigned will review it
meritorious,forsubcommitteeballoting.Nomethodorpractice
for adequacy in this respect, and conduct further tests as
shall be released and recommended for balloting unless the
necessary to evaluate its precision and bias, as technically
precision or accuracy requirements, or both, as set forth in the
feasible. A method may be accepted as a proposed method,
following, have been satisfied.
providedtherepeatabilityisknownorhasbeenascertainedand
6.1.1 Collaborative testing by D22 is the preferred method
provided all other criteria for acceptance have been met.
of validation. Data obtained by collaborative testing by others
Independenttestsbyatleastthreelaboratoriesshallberequired
may be used in lieu of D22 testing, provided that such testing
tosubstantiatetherepeatabilityofamethodbeforeitattainsthe
was equivalent to ASTM approved procedures. In either case,
status of a standard method. Collaborative testing by at least
a copy of the test procedures and data must be filed in a
five laboratories to estimate the interlaboratory bias and, if
research file maintained at ASTM for such purposes.
applicable to evaluate the method’s inherent bias with respect
to the “true” value is needed for all standard methods and must
6.2 Proposed Method—A proposed method is one that has
be accomplished within 5 years of its initial issuance as a
foundfavorableusageinaspecificlaboratory,orhasbeenused
standard, if such testing has not already been done. Failure to
by several laboratories, but has not yet been standardized. In
subject such methods to appropriate collaborative testing,
each case, the test method is submitted by its proponents to
constitutes valid grounds for disallowing its reapproval as a
Committee D22 for standardization.
standard.
6.2.1 The minimum requirement for balloting of a proposed
method shall be the inclusion in it of a single laboratory’s
4.2 Procedures that may be used in collecting the required
statement of single-operator precision, together with support-
data are given with particular emphasis upon the applicability
ing experimental data. Test methods meeting this requirement
to analysis of atmospheres. Documentation requirements are
will be referred to a Task Group, following procedures estab-
established. Terms that are useful in expressing statements of
lished by Committee D22.
precision and bias are presented.
6.2.2 The experimental data needed to support a proposal
must reflect a test of the method as a whole, that is, sampling,
Youden, W. J. and Steiner, G. H., Statistical Manual of the Association of
apparatus, reagents and, calibration, and must use a procedure
Offıcial Analytical Chemists, AOAC International, 481 North Frederick Ave., Suite
500, Gaithersburg, MD 20877-2417, 1987. that is essentially identical to that described in the proposal.
D3670 − 91 (2022)
Any significant deviations between the procedure used to test sample preparation. Each method should be tested with
gather the data and the proposed procedure shall be clearly actual samples for which it is applicable, or as close a
identified. simulation as possible. The degree of evaluation will, of
6.2.3 If such data are missing or inadequate, but the method course, depend on the simulation achieved, and the statements
itself is considered by consensus of Committee D22 to be of precision and accuracy must define the test conditions.
worthy of further study, a task group may be assigned to
7.2 The ideal test sample is the actual atmosphere for which
conduct experimental studies or enlist the services of at least
the method is intended. However the use of such offers
onecompetentlaboratorytoobtainthedatauponwhichtobase
complications because the composition may not be known at
a statement of single-operator precision.
the moment of test and furthermore may undergo change
6.3 Standard Method–Initial Acceptance—A method that
during the tests. Because actual atmospheric samples cannot be
has found favorable acceptance and for which the within-
collected and stabilized for long periods of time, two proce-
laboratory repeatability has been verified by a multilaboratory
dures are acceptable. Reproducibility and repeatability may be
test program, shall be examined by the task group for compli-
evaluated by simultaneous measurement by participating labo-
ance with the following requirements.
ratories sampling the same atmosphere at substantially the
6.3.1 An initial minimum requirement for establishing a
same time. Alternatively, comparison of a candidate method
standard method is a statement of within-laboratory precision
with a standard method of known precision and bias will
based on data from three laboratories similar to that described
constitute an acceptable technique for evaluation of precision
in 6.2.1 – 6.2.3.
andaccuracy.Suchmeasurementsmadebyseverallaboratories
6.3.2 If the method purports to measure the concentration of
may be statistically treated to evaluate the reproducibility of
a substance, an investigation of the bias of the method by
the candidate method. In this latter case, the measurements
comparison with a standard must be made by at least one
need not be made at the same place and time by the collabo-
laboratory and the results included in an accuracy statement.
rating laboratories.
6.3.3 Astandard can only be carried under the provisions of
7.3 A test sample or series of test samples that are stable
6.3 for five years. Conditions for reapproval are specified in
during the period required to perform a limited series of
6.4.
measurements are adequate for evaluation of single-operator
6.4 Standard Method–Reapproval—Astandard method may
precision to satisfy the requirements for consideration as a
be retained if it has found extensive use and between-
proposed method. Three levels of concentration are
laboratory precision data have been obtained. Before doing a
recommended, with such levels sufficiently well established to
collaborative study, a ruggedness test should be performed by
determine whether, and to what extent, the repeatability is
at least one laboratory (see Guide E1169).
dependent or independent of concentration level.
6.4.1 The minimum requirement for retaining a standard
7.4 A series of test samples of at least three concentration
method shall be a statement of the between-laboratory preci-
levels, and available in sufficient number, is required for use by
sion of the method as established in a collaborative test
collaborating laboratories to evaluate the repeatability and
including at least five participants.
reproducibility of a candidate method. The samples should be
6.4.2 If a bias statement is appropriate for the method
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.