ASTM D2777-98
(Practice)Standard Practice for Determination of Precision and Bias of Applicable Methods of Committee D-19 on Water
Standard Practice for Determination of Precision and Bias of Applicable Methods of Committee D-19 on Water
SCOPE
1.1 This practice establishes uniform standards for expressing the precision and bias of applicable test methods for Committee D-19 on Water.
1.2 Except as specified in 1.3, 1.4, 1.5, and 1.6, this practice requires that the task group proposing a new test method must carry out a collaborative study from which statements for precision (multiple laboratory operational and single-operator standard deviation estimates) and observed bias can be developed. This practice provides general guidance to task groups in planning and conducting such determinations of precision and bias.
1.3 Where, due to the nature of the test or instability of the sample, a full-scale collaborative test is not technically feasible, a statement of single-operator precision (Section 10) will meet the requirements of this practice. Whenever possible, this statement should be developed from data generated by multiple operators, each doing replicate analyses on independent samples which generally fall within specified concentration ranges. This practice is not applicable to methodology involving continuous sampling and measurement of specific constituents and properties.
1.4 Many of the test methods adopted prior to 1980 do not have adequate precision and bias statements, and there is insufficient information to satisfactorily improve the statements. These test methods are being used, and it is generally agreed that they should be continued as standards until 1985, while a proper collaborative study is conducted. In the interim, a warning must be placed in the scope and precision sections to alert the user that there has not been adequate collaborative testing to validate the test method.
1.5 Existing test methods that do have adequate precision and bias statements based on a minimum of three laboratories for a total of six operators do not require further collaborative testing, and such deficiencies must be listed in the precision and bias statements.
1.6 With the recommendation of the Results Advisor and the concurrence of the Technical Operations Section of the Executive Subcommittee, a statement giving a compelling reason why all or specific points of this practice cannot be fully complied with will meet both ASTM requirements (1) and the related requirements of this practice.
1.7 In principle, all test methods are covered by this practice.
1.8 In Section 11, this practice shows exemplary precision and bias statement formats for: ( ) test methods yielding a numerical measure, ( ) test methods yielding a nonnumerical report of success or failure based on criteria specified in the procedure, and ( ) test methods specifying that procedures in another ASTM method are to be used with only insignificant modifications.
General Information
Relations
Standards Content (Sample)
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Please contact ASTM International (www.astm.org) for the latest information.
Designation: D 2777 – 98
Standard Practice for
Determination of Precision and Bias of Applicable Test
Methods of Committee D-19 on Water
This standard is issued under the fixed designation D 2777; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (ϵ) indicates an editorial change since the last revision or reapproval.
1. Scope the precision and bias statement required in each test method.
Ifthestudydoesnotsatisfythecurrentminimumrequirements
1.1 This practice establishes uniform standards for estimat-
for a collaborative study, a statement listing the study’s
ing and expressing the precision and bias of applicable test
deficiencies and a reference to this paragraph shall be included
methods for Committee D-19 on Water.
in the precision and bias statement as the basis for an
1.2 Except as specified in 1.3, 1.4, and 1.5, this practice
exemption from the current requirements.
requires the task group proposing a new test method to carry
1.5 This paragraph relates to special exemptions not clearly
out a collaborative study from which statements for precision
acceptable under 1.3 or 1.4. With the approval of Committee
(overall and single-operator standard deviation estimates) and
D-19 on the recommendation of the Results Advisor and the
biascanbedeveloped.Thispracticeprovidesgeneralguidance
Technical Operations Section of the Executive Subcommittee
to task groups in planning and conducting such determinations
of Committee D-19, a statement giving a compelling reason
of precision and bias.
why compliance with all or specific points of this practice
1.3 If a full-scale collaborative study is not technically
cannot be achieved will meet both ASTM requirements (1)
feasible, due to the nature of the test method or instability of
andtherelatedrequirementsofthispractice.Precisionandbias
samples, the largest feasible scaled-down collaborative study
statements authorized by this paragraph shall include the date
shallbeconductedtoprovidethebestpossiblelimitedbasisfor
of approval by Committee D-19.
estimating the overall and single-operator standard deviations.
1.6 In principle, all test methods are covered by this
1.3.1 Examples of acceptable scaled-down studies are the
practice.
local-area studies conducted by Subcommittee D19.24 on
1.7 In Section 11 this practice shows exemplary precision
microbiological methods because of inherent sample instabil-
and bias statement formats for: (1) test methods yielding a
ity. These studies involve six or more completely independent
numerical measure, (2) test methods yielding a non-numerical
local-area analysts who can begin analysis of uniform samples
report of success or failure based on criteria specified in the
at an agreed upon time.
procedure, and (3) test methods specifying that procedures in
1.3.2 If uniform samples are not feasible under any circum-
anotherASTM test method are to be used with only insignifi-
stances, a statement of single-operator precision will meet the
cant modifications.
requirements of this practice. Whenever possible, this state-
1.8 All studies, even those exempt from some requirements
mentshouldbedevelopedfromdatageneratedbyindependent
under 1.3 or 1.5, shall receive approval from the Results
multiple operators, each doing replicate analyses on indepen-
Advisor before being conducted (see Section 8) and after
dent samples of a specific matrix type, which generally fall
completion (see Section 12).
within specified concentration ranges (see 7.2.5.2( 3)).
1.3.3 Thispracticeisnotapplicabletomethodologyinvolv-
2. Referenced Documents
ing continuous sampling or measurement, or both, of specific
2.1 ASTM Standards:
constituents and properties.
D1129 Terminology Relating to Water
1.3.4 This practice is also not applicable to open-channel
D1141 Specification for Substitute Ocean Water
flow measurements.
D1193 Specification for Reagent Water
1.4 A collaborative study that satisfied the requirements of
D4375 Terminology for Basic Statistics in Committee
the version of this practice in force when the study was
D-19 on Water
conducted will continue to be considered an adequate basis for
D5790 TestMethodforMeasurementofPurgeableOrganic
Compounds in Water by Capillary Column Gas
This practice is under the jurisdiction ofASTM Committee D-19 onWater and
is the direct responsibility of Subcommittee D19.02 on General Specifications,
Technical Resources, and Statistical Methods. The boldface numbers in parentheses refer to the list of standards at the end of
Current edition approved Jan. 10, 1998. Published October 1998. Originally this practice.
published as D2777–69T. Last previous edition D2777–96. Annual Book of ASTM Standards, Vol 11.01.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Please contact ASTM International (www.astm.org) for the latest information.
D 2777 – 98
Chromatography/Mass Spectrometry the Results Advisor for concurrence before balloting. This
D5905 Specification for Substitute Wastewater assures having an acceptable copy of the collaborative study
E177 Practice for Use of the Terms Precision and Bias in results to send to ASTM for items on the main committee
ASTM Test Methods ballot. In most instances, the collaborative study shall be
E178 Practice for Dealing with Outlying Observations complete before a subcommittee ballot. If the collaborative
E456 Terminology Relating to Quality and Statistics study is not complete, the test method may go on the ballot as
E1169 Guide for Conducting Ruggedness Tests a provisional test method rather than a standard test method.
Copies of the test data, approved calculations, and statistical
3. Terminology
results shall be filed at ASTM Headquarters when the test
method is submitted by the subcommittee chairman as an item
3.1 Definitions—For definitions of terms used in this prac-
for the main committee ballot.
tice, refer to Terminologies D 1129, D 4375 and
E456D1129D1193E177, and Practice E177D5790. 4.1.1 The appendix shows an example of “Form
A—Approval of Plans for Interlaboratory Testing,” as Fig.
3.2 Definitions of Terms Specific to This Standard:
3.2.1 accuracy—a measure of the degree of conformity of X1.1.
4.1.2 For an example of a data reporting form, see Fig.
a single test result generated by a specific procedure to the
X2.1.
assumedoracceptedtruevalueandincludesbothprecisionand
4.1.3 In addition, the appendix shows a sample calculation
bias.
of precision and bias from real collaborative test data, the
3.2.2 bias—the persistent positive or negative deviation of
related table of statistics, and the related precision and bias
the average value of a test method from the assumed or
statement.
accepted true value.
3.2.3 laboratory—asingleandcompletelyindependentana-
5. Significance and Use
lytical system with its own specific apparatus, source of
reagents, set of internal standard operating procedures, etc.
5.1 Following this practice should result in precision and
Different laboratories will differ from each other in all of these
bias statements which can be achieved by any laboratory
aspects, regardless of how physically or organizationally close
properly using the test method studied. These precision and
they may be to each other.
bias statements provide the basis for generic limits for use in
3.2.4 operator—usually the individual analyst within each
the Quality Control section of the test method.
laboratory who performs the test method throughout the
5.2 The method specifies the media for which the test
collaborative study. However, for complicated test methods,
method is appropriate. The collaborative test corroborates the
the operator may be a team of individuals, each performing a
write-up within the limitations of the test design.An extensive
specific function throughout the study.
test can only use representative media so that universal
3.2.5 precision—the degree of agreement of repeated mea-
applicability cannot be implied from the results.
surements of the same property, expressed in terms of disper-
5.3 The fundamental assumption of the collaborative study
sion of test results about the arithmetical mean result obtained
is that the media tested, the concentrations tested, and the
by repetitive testing of a homogeneous sample under specified
participating laboratories are a representative and fair evalua-
conditions. The precision of a test method is expressed
tionofthescopeandapplicabilityofthetestmethodaswritten.
quantitatively as the standard deviation computed from the
results of a series of controlled determinations.
6. Preliminary Studies
6.1 Considerable pilot work on a test method must precede
4. Summary of Practice
the determination of its precision and bias (2,3). This pilot
4.1 After the task group has assured itself that the test
work should explore such variables as preservation require-
method has had all preliminary evaluation work completed, it
ments, reaction time, concentration of reagents, interferences,
shouldpreparethetestmethodwrite-upinfinalform.Theplan
calibration, and sample size. Potentially significant factors
for collaborative study is developed in accordance with this
must be investigated and controlled in the written test method
practice and submitted along with the test method write-up to
in advance of the collaborative test. Also, disregard of such
the ResultsAdvisor for concurrence except as specified in 1.3,
factors may introduce so much variation among operators that
1.4,and1.5.Uponreceiptofconcurrence,thecollaborativetest
results are misleading or inconclusive (4) (see 9.3 and 9.4).A
is conducted, data analyzed, and precision and bias statements
ruggedness study conducted in a single laboratory is particu-
formulated by the task group. The final precision and bias
larly useful for such investigations and should be conducted to
statistics must be based on retained data from at least six
prove a test method is ready for interlaboratory testing (see
independent laboratories. The statements, with backup data
Guide E1169E178 for details).
including the reported results summary, the calculations lead-
6.2 Only after a proposed test method has been tried,
ing up to the statements, and the test method write-up with
proved, and reduced to unequivocal written form should a
precision and bias statements included are submitted to the
determination of its precision and bias be attempted.
subcommittee vice-chairman who in turn sends a copy of it to
7. Planning the Collaborative Test
7.1 Baseduponthetaskgroup’sknowledgeofatestmethod
Annual Book of ASTM Standards, Vol 11.02.
Annual Book of ASTM Standards, Vol 14.02. and having the unequivocal write-up, several factors must be
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Please contact ASTM International (www.astm.org) for the latest information.
D 2777 – 98
considered in planning the collaborative test to properly assess Analytes may be supplied separately as concentrates for
theprecisionofthetestmethod.Thetestingvariablesthatmust addition to this matrix by each laboratory or the reference
be considered in planning are discussed below. It is generally matrix containing the analyte(s) may be supplied to each
notacceptabletocontrolsignificantsourcesofvariabilityinthe participant. Information on how the reference matrix was
collaborative study which cannot be controlled in routine use prepared in the study shall be clear in the precision and bias
of the test method, because this leads to false estimates of the statementofthetestmethodsouserscanreproduceitproperly.
testmethodprecisionandbias.Inaddition,thetaskgroupmust
7.2.5.2 Additional collaborative testing should also be con-
determine within the resources available how to best estimate
ducted using other matrices specified in the scope of the test
the bias of the test method.
method. Since these matrices must be the same for each study
7.2 Testing Variables: participant, they may have to be prepared (or obtained from a
singlesource),preserved,anddistributedtoalllaboratories.As
7.2.1 It is desirable to develop a statement of precision of a
with the reference matrix, analytes may be supplied in a
test method that indicates the contribution to overall variation
separate spiking solution or already added to the matrix. A
of selected causes such as laboratory, operator, sample matrix,
particularly attractive matrix might be a standard material
analyte concentration, and other factors that may or have been
availablefromanorganizationsuchastheNationalInstituteof
shown to have strong effects on the results. Since any test
Standards and Technology (NIST). Use of uniform sample
method can be tried in only a limited number of applications,
matrices is necessary in these studies since they enable a more
thestandarddeviationcalculatedfromtheresultsofastudycan
certain comparison with the reference matrix than is possibly
beonlyanestimateoftheuniversestandarddeviation.Forthis
with matrices supplied separately by each participant.
reason, the symbol s (sample standard deviation) is used
herein. The precision estimates generated from the study data (1) Use of matrices with naturally occurring, non-zero
will usually be the overall standard deviation (s ) and the background levels of the analyte(s) being studied will result in
T
pooled single-operator standard deviation (s ) for each sample precisionandbiasestimatesthatwillbemuchmoredifficultto
o
matrix and concentration studied. properly compare with estimates from the reference matrix.
7.2.2 Laboratories, operators, sample matrices, and analyte (2) Any matrix spiking that may be necessary shall not
concentrationsaretheonlysourcesofvariabilityrepresentedin significantly change the natural characteristics of the matrix.
the precision and bias statements resulting from the usual
(3) With the exception of the kind of limited study
collaborative study. They may not represent the additional
described in 1.3.2, the matrix-of-choice approach, in which
influence that can arise from differences in sample splitting,
each participant is expected to acquire their own sample of a
field preservation, transportation, etc., all of which may influ-
designated type, should not be used. Such studies are basically
enceroutineanalyticalresultsasshowninthegeneralprecision
incompatible with the statistical approaches employed in this
definitions in Terminology D1129D1129.
practice;boththerankingtestandtheindivi
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.