ASTM D2777-21

Standard Practice for Determination of Precision and Bias of Applicable Test Methods of Committee D19 on Water

SIGNIFICANCE AND USE
5.1 Following this practice should result in precision-and-bias statements that can be achieved by any laboratory properly using the test method studied. These precision-and-bias statements provide the basis for generic limits for use in the Quality Control section of the test method. Optionally, the detection and quantitation values provide estimates of the level at which most laboratories should be able to achieve confident detection and meet the minimum precision (expressed as relative standard deviation) expected.  
5.2 The method specifies the matrices for which the test method is appropriate. The collaborative test corroborates the write-up within the limitations of the test design. An extensive test can only use representative matrices so that universal applicability cannot be implied from the results.  
5.3 The fundamental assumption of the collaborative study is that the matrices tested, the concentrations tested, and the participating laboratories are a representative and fair evaluation of the scope and applicability of the test method as written.
SCOPE
1.1 This practice establishes uniform standards for estimating and expressing the precision and bias of applicable test methods for Committee D19 on Water. Statements of precision and bias in test methods are required by the Form and Style for ASTM Standards, “Section A21. Precision and Bias (Mandatory).” In principle, all test methods are covered by this practice. However, the variability equations provided in this standard are applicable only to test methods that yield continuous function values.  
1.2 Except as specified in 1.4, 1.5, and 1.6, this practice requires the task group proposing a new test method to carry out a collaborative study from which statements for precision (overall and single-operator standard-deviation estimates) and bias can be developed. This practice provides general guidance to task groups in planning and conducting such determinations of precision and bias.  
1.3 This practice requires that a task group making a substantive revision to a test method also perform a limited-scale collaborative study (known as a ― comparability study) to evaluate the effect of the revision on the precision and bias statement. This practice provides guidance to task groups for conducting such limited-scale collaborative studies. Examples of substantive modifications may include, but are not limited to, changes in mandatory or allowable instrumentation, reagents, reaction times, etc.  
1.3.1 Changes to applicable water matrices in the Scope of a method may constitute a substantive modification under this provision. Only matrices that have been evaluated in an approved collaborative study may be listed in the Scope of a method. It is recognized that the term “matrix” is generally vague. Terms specifying matrix types can cover significantly different chemical constituents, unless the matrix is synthesized to be of a standardized makeup. Substitute Wastewater (Practice D5905) is one such defined matrix. For purposes of this practice, the importance of this requirement is to assist the user of a D19 standard in determining the applicability of the method to their samples. Evaluated matrices should be described with as much detail as possible to minimize misapplication.  
1.3.2 A method's concentration-range extension that is deemed to merit additional collaborative testing (even without a method modification that would otherwise be considered substantive) shall require a full collaborative study, as described in 7.1 through 7.5, but only at concentrations representative of the extended range. Note that such a collaborative study could involve as little as a single concentration study in a single reproducible matrix.  
1.3.3 Whether a revision to a test method includes substantive modification shall be determined by consensus of the Committee.  
1.4 If a full-scale collaborative study is not technically feasible, because...

General Information

• Status: Published
• Publication Date: 14-Dec-2021
• ICS: 13.060.50 - Examination of water for chemical substances
• Technical Committee: D19 - Water
• Drafting Committee: D19.02 - Quality Systems, Specification, and Statistics

Relations

• Refers: ASTM E456-13a(2022)e1 - Standard Terminology Relating to Quality and Statistics
• Effective Date: 01-Apr-2022
• Refers: ASTM D1129-13(2020)e2 - Standard Terminology Relating to Water
• Effective Date: 01-May-2020
• Refers: ASTM D5790-18 - Standard Test Method for Measurement of Purgeable Organic Compounds in Water by Capillary Column Gas Chromatography/Mass Spectrometry
• Effective Date: 15-Dec-2018
• Refers: ASTM E1169-18 - Standard Practice for Conducting Ruggedness Tests
• Effective Date: 01-Sep-2018
• Refers: ASTM E1169-17 - Standard Practice for Conducting Ruggedness Tests
• Effective Date: 01-Oct-2017
• Refers: ASTM E1169-17e1 - Standard Practice for Conducting Ruggedness Tests
• Effective Date: 01-Oct-2017
• Refers: ASTM E456-13A(2017)e1 - Standard Terminology Relating to Quality and Statistics
• Effective Date: 01-Oct-2017
• Refers: ASTM E456-13A(2017)e3 - Standard Terminology Relating to Quality and Statistics
• Effective Date: 01-Oct-2017
• Refers: ASTM E178-16 - Standard Practice for Dealing With Outlying Observations
• Effective Date: 01-Jun-2016
• Refers: ASTM E1169-14 - Standard Practice for Conducting Ruggedness Tests
• Effective Date: 01-May-2014
• Refers: ASTM E177-14 - Standard Practice for Use of the Terms Precision and Bias in ASTM Test Methods
• Effective Date: 01-May-2014
• Refers: ASTM E456-13ae1 - Standard Terminology Relating to Quality and Statistics
• Effective Date: 15-Nov-2013
• Refers: ASTM E456-13a - Standard Terminology Relating to Quality and Statistics
• Effective Date: 15-Nov-2013
• Refers: ASTM E456-13ae3 - Standard Terminology Relating to Quality and Statistics
• Effective Date: 15-Nov-2013
• Refers: ASTM E456-13ae2 - Standard Terminology Relating to Quality and Statistics
• Effective Date: 15-Nov-2013

Overview

ASTM D2777-21, “Standard Practice for Determination of Precision and Bias of Applicable Test Methods of Committee D19 on Water,” establishes uniform guidelines for assessing and expressing the precision and bias of analytical test methods used in water analysis. Developed by ASTM Committee D19 on Water, this standard practice is crucial for laboratories and organizations involved in water quality measurement, ensuring that test results are both reliable and comparable across different laboratories and applications.

ASTM D2777-21 provides the framework required for developing and updating standardized test methods and supports quality control processes through defined collaborative studies. The practice covers the planning, execution, and statistical evaluation of precision and bias, enabling test method users and developers to generate meaningful statements of method performance.

Key Topics

• Precision and Bias Evaluation: The standard details procedures to estimate and report the precision (repeatability and reproducibility) and bias of water testing methods.
• Collaborative Studies: Provides protocols for interlaboratory studies that form the basis for precision and bias statements in test method documentation.
• Quality Control Integration: The results guide the creation of generic quality control limits used in standardized water test methods.
• Applicability of Matrices: Discusses the importance of clearly specifying water sample matrices applicable to a given method, based on the collaborative studies performed.
• Substantive Method Changes: Outlines requirements for comparability or validation studies when test methods undergo significant modification, such as changes in reagents, instrumentation, or sample type.
• Minimum Requirements: Sets minimum participation thresholds for laboratories and operators to ensure statistically valid outcomes for precision and bias estimates.
• Types of Evaluations: Supports development of statements for methods that yield numerical and non-numerical results, including procedures referencing other standards.

Applications

ASTM D2777-21 plays a pivotal role for:

• Analytical Laboratories: Assisting in the development and validation of new water analysis methods, and in maintaining accreditation through reliable, comparable test results.
• Test Method Development: Guiding task groups in planning robust collaborative and comparability studies for both new and substantively revised water test methods.
• Regulatory Compliance: Supporting the establishment of quality control procedures and performance specifications, often required for regulatory approval and reporting.
• Water Quality Assurance: Ensuring water analysis results, such as those produced by municipal, industrial, or environmental laboratories, are precise, unbiased, and consistent.
• Standard Revisions: Mandating that substantive updates to test methods (e.g., concentration ranges or matrix modifications) are accompanied by appropriate re-evaluations of method performance.

By establishing clear protocols for multi-laboratory studies, ASTM D2777-21 helps organizations build confidence in their analytical data, minimize misapplication of methods to inappropriate matrices, and meet international standards for test method validation.

Related Standards

ASTM D2777-21 references and is complemented by several related ASTM standards, including:

• ASTM D1129: Terminology Relating to Water
• ASTM D1193: Specification for Reagent Water
• ASTM D1141: Practice for Preparation of Substitute Ocean Water
• ASTM D5905: Practice for Preparation of Substitute Wastewater
• ASTM D5847: Practice for Writing Quality Control Specifications for Water Analysis
• ASTM D6091: Practice for 99%/95% Interlaboratory Detection Estimate (IDE)
• ASTM D6512: Practice for Interlaboratory Quantitation Estimate
• ASTM E177: Practice for Use of the Terms Precision and Bias in ASTM Test Methods
• ASTM E691: Practice for Conducting an Interlaboratory Study to Determine the Precision of a Test Method

Together, these standards support robust water analysis and ensure alignment with global best practices for laboratory quality assurance and method validation.

ASTM D2777-21 is critical for any organization striving for consistent, transparent, and high-quality water testing outcomes.

Standards Content (Sample)

This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation: D2777 − 21
Standard Practice for
Determination of Precision and Bias of Applicable Test
Methods of Committee D19 on Water
This standard is issued under the fixed designation D2777; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope this practice, the importance of this requirement is to assist the
user of a D19 standard in determining the applicability of the
1.1 This practice establishes uniform standards for estimat-
method to their samples. Evaluated matrices should be de-
ing and expressing the precision and bias of applicable test
scribed with as much detail as possible to minimize misappli-
methods for Committee D19 onWater. Statements of precision
cation.
and bias in test methods are required by the Form and Style for
ASTM Standards, “Section A21. Precision and Bias (Manda- 1.3.2 A method’s concentration-range extension that is
tory).” In principle, all test methods are covered by this deemed to merit additional collaborative testing (even without
practice. However, the variability equations provided in this a method modification that would otherwise be considered
standard are applicable only to test methods that yield continu- substantive) shall require a full collaborative study, as de-
ous function values.
scribed in 7.1 through 7.5, but only at concentrations represen-
tative of the extended range. Note that such a collaborative
1.2 Except as specified in 1.4, 1.5, and 1.6, this practice
study could involve as little as a single concentration study in
requires the task group proposing a new test method to carry
a single reproducible matrix.
out a collaborative study from which statements for precision
(overall and single-operator standard-deviation estimates) and 1.3.3 Whether a revision to a test method includes substan-
bias can be developed.This practice provides general guidance tive modification shall be determined by consensus of the
to task groups in planning and conducting such determinations
Committee.
of precision and bias.
1.4 If a full-scale collaborative study is not technically
1.3 This practice requires that a task group making a
feasible, because of the nature of the test method or instability
substantive revision to a test method also perform a limited-
of samples, the most complete collaborative study that is
scale collaborative study (known as a — comparability study)
technically feasible shall be conducted to provide the best
to evaluate the effect of the revision on the precision and bias
possible limited basis for estimating the overall and single-
statement. This practice provides guidance to task groups for
operator standard deviations. In some situations, an intermedi-
conducting such limited-scale collaborative studies. Examples
ate collaborative study as described in Guide D7847 may
of substantive modifications may include, but are not limited
provide an appropriate approach. It is recognized that there
to, changes in mandatory or allowable instrumentation,
may be circumstances when even a limited collaborative study
reagents, reaction times, etc.
is not feasible.Any collaborative study plan that does not meet
1.3.1 Changes to applicable water matrices in the Scope of
all the requirements spelled out in this practice will require a
a method may constitute a substantive modification under this
review and recommendation by the Results Advisor and an
provision. Only matrices that have been evaluated in an
approval by the D19 Technical Operations Section of the
approved collaborative study may be listed in the Scope of a
Executive Subcommittee.
method. It is recognized that the term “matrix” is generally
1.4.1 Examples of acceptable studies are the local-area
vague. Terms specifying matrix types can cover significantly
intermediate studies conducted by Subcommittee D19.24 on
different chemical constituents, unless the matrix is synthe-
microbiological methods because of inherent sample perish-
sized to be of a standardized makeup. Substitute Wastewater
ability. Such intermediate collaborative studies meet the same
(Practice D5905) is one such defined matrix. For purposes of
degrees of freedom and participant requirements as full col-
laborative studies. They involve six or more completely
1 independent local-area analysts who can begin analysis of
This practice is under the jurisdiction ofASTM Committee D19 on Water and
is the direct responsibility of Subcommittee D19.02 on Quality Systems,
uniform samples at an agreed upon time. Guide D7847 can
Specification, and Statistics.
provide guidance to the task group, the Results Advisor, and
Current edition approved Dec. 15, 2021. Published June 2022. Originally
theTechnical Operations Section of the Executive Subcommit-
approved in 1969. Last previous edition approved in 2013 as D2777 – 13. DOI:
10.1520/D2777-21. tee of Committee D19 on the appropriate design of an
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
D2777 − 21
acceptable intermediate collaborative study for test methods ization established in the Decision on Principles for the
that measure highly perishable parameters. Development of International Standards, Guides and Recom-
1.4.2 If providing homogenous samples with a sufficiently mendations issued by the World Trade Organization Technical
stable analyte concentration is not feasible under any Barriers to Trade (TBT) Committee.
circumstances, a statement of single-operator precision may
2. Referenced Documents
meet the requirements of this practice.Whenever possible, this
statementshouldbedevelopedfromdatageneratedbymultiple
2.1 ASTM Standards:
independent operators, each doing replicate analyses on inde-
D1129 Terminology Relating to Water
pendent samples (of a specific matrix type), which generally
D1141 Practice for Preparation of Substitute Ocean Water
fall within specified concentration ranges (see 7.2.5.2 (3)).
D1193 Specification for Reagent Water
D4375 Practice for Basic Statistics in Committee D19 on
1.5 A collaborative study that satisfied the requirements of
Water (Withdrawn 2018)
the version of this practice in force when the study was
D5790 Test Method for Measurement of Purgeable Organic
conducted will continue to be considered an adequate basis for
Compounds in Water by Capillary Column Gas
the precision-and-bias statement required in each test method.
Chromatography/Mass Spectrometry
If the study does not satisfy the current minimum requirements
D5847 Practice for Writing Quality Control Specifications
for a collaborative study, a statement listing the study’s
for Standard Test Methods for Water Analysis
deficiencies and a reference to this paragraph shall be included
D5905 Practice for the Preparation of Substitute Wastewater
in the precision-and-bias statement as the basis for an exemp-
D6091 Practice for 99 %/95 % Interlaboratory Detection
tion from the current requirements.
Estimate (IDE) for Analytical Methods with Negligible
1.6 Committee D19, through a Main Committee ballot, may
Calibration Error
approve publication of a “Preliminary” Standard Method for a
D6512 Practice for Interlaboratory Quantitation Estimate
period not to exceed 5 years. Preliminary Standards must
D7847 GuideforInterlaboratoryStudiesforMicrobiological
contain a minimum of a single-operator precision-and-bias
Test Methods
statement and a Quality Control section based on the single
E177 Practice for Use of the Terms Precision and Bias in
operator data. Publication of a Preliminary Standard is condi-
ASTM Test Methods
tional on the approval of a full D2777 collaborative study
E178 Practice for Dealing With Outlying Observations
design for the standard. Precision-and-bias statements autho-
E456 Terminology Relating to Quality and Statistics
rized by this paragraph shall include the date of approval by
E691 Practice for Conducting an Interlaboratory Study to
Committee D19.
Determine the Precision of a Test Method
1.7 Per Section A21.2.3 of the ASTM Form and Style E1169 Practice for Conducting Ruggedness Tests
Manual the committee may delay an interlaboratory study for
2.2 ASTM Adjuncts:
a new method and include a temporary statement in the
DQCALC Microsoft Excel-based software for the Interlabo-
Precision and Bias Section that addresses only single operator
ratory Quantitation Estimate (IQE)
precision (“repeatability”). This statement is valid for five
3. Terminology
years from the initial publication date. In this case, a single
laboratory study shall be conducted in accordance with 7.6.
3.1 Definitions—For definitions of terms used in this
practice, refer to Terminologies D1129, D4375 and E456, and
1.8 In Section 12, this practice shows exemplary precision-
Practice E177.
and-bias-statement formats for: (1) test methods yielding a
3.2 Definitions of Terms Specific to This Standard:
numerical measure, (2) test methods yielding a non-numerical
3.2.1 accuracy, n—a measure of the degree of conformity
report of success or failure based on criteria specified in the
of a single test result generated by a specific procedure to the
procedure, and (3) test methods specifying that procedures in
assumed or accepted true value, and includes both precision
another ASTM test method are to be used with only insignifi-
and bias.
cant modifications.
3.2.2 bias, n—the persistent positive or negative deviation
1.9 All studies, even those exempt from some requirements
of the average value of a test method from the assumed or
under previous sections, shall receive approval from the
accepted true value.
Results Advisor before being conducted (see Section 8) and
after completion (see Section 13). 3.2.3 comparability study, n—a collaborative study that
incorporates side-by-side evaluation of the test method before
1.10 This practice satisfies the QC requirements of Practice
and after a substantive modification to a test method.
D5847.
1.11 It is the intent of this practice that task groups make
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
every effort to retain all the data from their collaborative
contact ASTM Customer Service at [e-mail]. For Annual Book of ASTM
studies. Values should not be eliminated unless solid evidence
Standards volume information, refer to the standard’s Document Summary page on
exists for their exclusion. The Results Advisor should work
the ASTM website.
The last approved version of this historical standard is referenced on
closely with the task groups to effect this goal.
www.astm.org.
1.12 This international standard was developed in accor-
Available fromASTM International Headquarters. OrderAdjunct No.ADJDQ-
dance with internationally recognized principles on standard- CALC. Original adjunct produced in 2007.
D2777 − 21
3.2.4 degrees of freedom, n—the total number of replicates collaborative-study results to send to ASTM for items on the
analyzed across all laboratories/analysts minus the number of main-committee ballot. In most instances, the collaborative
laboratories/analysts. study shall be complete before a subcommittee ballot. If the
collaborative study is not complete, the test method may go on
3.2.5 laboratory, n—a single and completely independent
the ballot as a provisional test method rather than a standard
analytical system with its own specific apparatus, source of
test method. Copies of the test data, approved calculations, and
reagents, set of internal standard-operating procedures, etc.
statisticalresultsshallbefiledatASTMHeadquarterswhenthe
3.2.5.1 Discussion—Different laboratories will differ from
test method is submitted by the subcommittee chairman as an
each other in all of these aspects, regardless of how physically
item for the main-committee ballot.
or organizationally close they may be to each other.
4.1.1 The appendix shows an example of “Form
3.2.6 limited validation study, n—in a test method, a vali-
A—Approval of Plans for Interlaboratory Testing,” as Fig.
dation study that does not fulfill all D2777 requirements for a
X1.1.
full-scale collaborative study, but that can be used for re-
4.1.2 For examples of data-reporting forms, see Appendix
validation of revised methods.
X3, 6.0.
3.2.7 operator,n—usuallytheindividualanalystwithineach
4.1.3 In addition, the appendix shows a sample calculation
laboratory who performs the test method throughout the
of precision and bias from real collaborative-test data, the
collaborative study.
related table of statistics, and the related precision-and-bias
3.2.7.1 Discussion—However, for complicated test
statement.
methods, the operator may be a team of individuals, each
performing a specific function throughout the study.
5. Significance and Use
3.2.8 precision, n—the degree of agreement of repeated
5.1 Following this practice should result in precision-and-
measurements of the same property, expressed in terms of
biasstatementsthatcanbeachievedbyanylaboratoryproperly
dispersion of test results about the arithmetical-mean result
using the test method studied. These precision-and-bias state-
obtained by repetitive testing of a homogeneous sample under
mentsprovidethebasisforgenericlimitsforuseintheQuality
specified conditions.
Control section of the test method. Optionally, the detection
3.2.8.1 Discussion—The precision of a test method is ex-
and quantitation values provide estimates of the level at which
pressedquantitativelyasthestandarddeviationcomputedfrom
most laboratories should be able to achieve confident detection
the results of a series of controlled determinations.
and meet the minimum precision (expressed as relative stan-
3.2.9 substantive modification, n—in a test method, a
dard deviation) expected.
change (or changes) that is deemed by the Committee to be of
5.2 The method specifies the matrices for which the test
such magnitude that the change might affect the precision-and-
method is appropriate. The collaborative test corroborates the
bias data published with the original method.
write-up within the limitations of the test design.An extensive
3.3 Acronyms:
test can only use representative matrices so that universal
3.3.1 MDL, n—method detection limit
applicability cannot be implied from the results.
5.3 The fundamental assumption of the collaborative study
4. Summary of Practice
is that the matrices tested, the concentrations tested, and the
4.1 After the task group has assured itself that the test
participating laboratories are a representative and fair evalua-
methodhashadallpreliminaryevaluationworkcompleted,the
tionofthescopeandapplicabilityofthetestmethodaswritten.
task group should prepare the test-method write-up in final
form. The plan for collaborative study is developed in accor-
6. Preliminary Studies
dance with this practice and submitted along with the test-
6.1 Considerable pilot work on a test method must precede
method write-up to the ResultsAdvisor for concurrence except
the determination of its precision and bias (1, 2). This pilot
as specified in 1.3, 1.5, and 1.6. As noted in 1.4, any
work should explore such variables as preservation
collaborative study plan not meeting all requirements in this
requirements, reaction time, concentration of reagents,
practice will also require D19 Technical Operations Section
interferences, calibration, and sample size. Potentially signifi-
approval. Upon receipt of the ResultsAdvisor concurrence and
cant factors must be investigated and controlled in the written
approval of theTechnical Operations Section, the collaborative
testmethodinadvanceofthecollaborativetest.Also,disregard
test is conducted, data analyzed, and precision-and-bias state-
of such factors may introduce so much variation among
ments formulated by the task group. Estimates of the lower
operators that results are misleading or inconclusive (3) (see
limitsofquantitationanddetectionmayalsobedeveloped.The
9.3 and 9.4). A ruggedness study conducted in a single
final precision-and-bias statistics must be based on usable data
laboratory is particularly useful for such investigations and
fromatleastsixindependentlaboratories.Thestatements,with
should be conducted to prove a test method is ready for
backup data including the reported-results summary, the cal-
interlaboratory testing (see Guide E1169 for details).
culations leading up to the statements, and the test method
write-up with precision-and-bias statements included are sub-
mitted to the subcommittee vice-chairman, who in turn sends a
copy to the Results Advisor for concurrence before balloting.
The boldface numbers in parentheses refer to a list of references at the end of
This procedure assures having an acceptable copy of the this standard.
D2777 − 21
6.2 Only after a proposed test method has been tried, additional influence that can arise from differences in sample
proved, and reduced to unequivocal written form should a splitting, field preservation, transportation, etc., all of which
determination of its precision and bias be attempted.
mayinfluenceroutineanalyticalresultsasshowninthegeneral
precision definitions in Terminology D1129.
6.3 If the task group intends to evaluate the method char-
7.2.3 Laboratories—The final precision-and-bias statistics
acteristics of detection and quantification, Practice D6091
for each analyte, matrix, and concentration must be based on
(Interlaboratory Detection Estimate) or Practice D6512 (Inter-
data from at least six laboratories that passed any outlier tests
laboratory Quantitation Estimate), or both, should be evaluated
(see 10.3) (that is, usable data). To be assured of meeting this
and the recommendations for study designs incorporated.
Determining detection capability and absolute bias for mea- requirement, it is recommended that usable data be obtained
from a minimum of eight independent laboratories. To guar-
surements at background is especially critical for methods
(such as radiochemical methods) that do not censor measure- antee eight will provide usable data, it will often be necessary
ment results (for example, against a critical level, MDL or to get ten or more laboratories to agree to participate, because
reporting level).
some may not provide data and others may not provide usable
6.3.1 To minimize the number of samples required, data data. Maximizing the number of participating laboratories is
would be gathered in two phases:
often the most important thing that can be done to guarantee a
6.3.1.1 Phase I—Single-laboratory characterization. In this
successful study.
phase, the method developer would run a sufficient number of
7.2.4 Even if the single-operator standard deviation is the
samplesatasufficientnumberofconcentrationstocharacterize
only statistic to be estimated in the study (see 1.4.2), there
fully response vs. concentration, as well as error vs. concen-
should be a minimum of eight operators who provide usable
tration. The lowest concentration would be the level of the
data, so there is assurance of data from six operators after all
blank or the lowest concentration that could be measured; the
outlier removal.
highest concentration would be at the upper limit of the
7.2.5 Sample Matrices—The collaborative study shall be
analytical range.
conducted with at least one representative sample matrix,
6.3.1.2 Phase II—Collaborative study. Using the results of
which should be reproducible by subsequent user-laboratories
Phase I, the method developer would estimate the minimum
so that they can compare their results with the results of the
number and the magnitude of concentrations necessary to meet
collaborative study.
the requirements of the documents of interest.
7.2.5.1 Typically, a reagent water prepared according to
7. Planning the Collaborative Test
Specification D1193 or a synthetic medium, such as the
substitute wastewater described in Practice D5905 or the
7.1 Baseduponthetaskgroup’sknowledgeofatestmethod
substitute ocean water described in Specification D1141,is
and the unequivocal write-up, several factors must be consid-
used as the reference matrix. Analytes and matrix may be
ered in planning the collaborative test to assess the precision of
supplied separately, with the analytes supplied as concentrates
the test method properly. The testing variables that must be
for addition to this matrix by each laboratory; alternatively, the
considered in planning are discussed below. In the collabora-
reference matrix containing the analyte(s) may be supplied to
tive study, it is generally not acceptable to control significant
each participant. Information on how the reference matrix was
sourcesofvariabilitythatcannotbecontrolledinroutineuseof
prepared in the study shall be clear in the precision-and-bias
the test method, because this control leads to false estimates of
statement of the test method so users can reproduce the study
the test-method precision and bias. In addition, the task group
conditions properly.
must determine within the resources available how best to
estimate the precision and bias of the test method. 7.2.5.2 Additional collaborative testing should also be con-
ducted using other matrices specified in the scope of the test
7.2 Testing Variables:
method. Since these matrices must be the same for each study
7.2.1 It is desirable to develop a test method’s precision
participant, they may have to be prepared (or obtained from a
statement that indicates the contribution to overall variation of
single source), preserved, and distributed to all laboratories.As
selected causes such as laboratory, operator, sample matrix,
with the reference matrix, analytes may be supplied in a
analyte concentration, and other factors that may or have been
separate spiking solution or already added to the matrix. A
shown to have strong effects on the results. Since any test
particularly attractive matrix might be a standard material
method can be tried in only a limited number of applications,
available from an organization such as the National Institute of
thestandarddeviationcalculatedfromtheresultsofastudycan
Standards and Technology (NIST). In a collaborative test, use
be only an estimate of the universe standard deviation. For this
of uniform sample matrices is necessary since they enable a
reason, the symbol s (sample standard deviation) is used
more certain comparison with the reference matrix than is
herein. The precision estimates generated from the study data
possible with matrices supplied separately by each participant.
will usually be the overall standard deviation (s ) at any one
T
(1) Use of matrices with naturally occurring, non-zero
concentration and the pooled single-operator standard devia-
background levels of the analyte(s) being studied will result in
tion (s ) for each sample matrix and concentration studied.
o
precision-and-bias estimates that will be much more difficult to
7.2.2 Laboratories, operators, sample matrices, and analyte
concentrationsaretheonlysourcesofvariabilityrepresentedin compare properly with estimates from the reference matrix.
(2) Any matrix spiking that may be necessary shall not
the precision-and-bias statements resulting from the usual
collaborative study. These sources may not represent the significantly change the natural characteristics of the matrix.
D2777 − 21
the data included in the study, so care should be taken to assure that trace
(3) With the exception of the kind of limited study de-
concentrations and upper bound of the linear range are considered in
scribed in 1.4.2, the matrix-of-choice approach, in which each
establishing the study concentrations.
participant is expected to acquire his or her own sample of a
designated type, should not be used. Such studies are basically 7.2.6.1 Study samples with concentrations at or near a
incompatible with the statistical approaches employed in this
detection limit can produce non-quantitative results from
practice. In addition, the presence of variable background participating laboratories if participants are permitted to use
concentrations prevents the assignment of a proper mean-
their detection limit to censor their results. Zeroes or ’less
concentration level to each precision estimate produced in the thans’ that result from this censoring process are non-
study.
quantitative results and cannot be included in the statistical
analysis of study results specified later in this practice. Con-
7.2.5.3 Thesamestudydesignshouldbeusedforallsample
matrices. A separate precision-and-bias statement should be ducting the specified statistical analysis on remaining quanti-
generatedforeachsamplematrixwithabriefdescriptionofthe tative data (that is, eliminating the non-numeric data) under
matrix tested. such circumstances can produce misleading precision-and-bias
estimates. In general, if at a single concentration or Youden
7.2.5.4 When studies are available indicating the applicabil-
pair, more than ⁄3 of the data are non-numeric, the
ity of the test method for matrices untested in 7.2.5.1 and
concentration/pair should be excluded from the precision and
7.2.5.2 and not meeting the other requirements of this practice,
at the discretion of the task group responsible for the test bias determination. Therefore, when designing the study,
carefully consider instructions to laboratories on censoring
method and the Results Advisor, and providing the data are
analyzed in accordance with Section 10 of this practice, these practices, typical levels of detection and minimum calibration
and consider including more concentrations in the study than
supporting data may be included in a separate section of the
precision-and-biasstatement.Includedshallbeaclearbutbrief theminimumrequired.Resultsfromanalysesofconcentrations
at or near the detection limit can be included in this traditional
description of the matrices and the study protocol employed. It
is the intent of this practice that ultimately, data concerning the statistical analysis (and thus the working range of the method
precision and bias of the test method in the full range of extended) if it turns out that most laboratories report quantified
matrices covered in the scope and analyzed in accordance with results.
this practice, will be made available to the users of the test
7.2.7 Since the order of analyses should not be a source of
method.
systematic variability in the study, each participant should
7.2.6 Analyte Concentrations—If pilot work has shown that
either be told to randomize the order of study-sample analyses
precision is linear with increasing analyte concentrations, at
or be given a specific random order for the analyses.
least three concentration levels covering the desired range of
7.2.7.1 Whenever the time of analyses has been shown to
the test method should be included for each matrix. Each
influence the analytical results, close control over the time of
concentration level shall consist of samples prepared at either
analyses will be essential.
identical concentrations in replicate or of two similar Youden
7.2.8 Ifpilotworkhasshownthatthesamplecontainermust
pair concentrations, and be presented to each laboratory for
be of a specific material prepared in a specific manner prior to
analysis. Each Youden pair of concentrations shall consist of a
use, the variation in containers obtained and prepared by the
pair of samples containing the same analyte at two different
participants will be a random variable and should be treated as
concentrations that differ from each other by up to 10 %, the
such in the planning of the study and in the statistical analysis
percentage calculation being based on the average of the two
of the data.
samples in the pair. At a given concentration level the true
7.2.9 The manner of preservation or other treatment of the
concentrations, whether replicates or Youden pairs, shall be
sample prior to typical use of the test method (if known to
identical for each laboratory. If the pilot work suggests that the
affect the precision or bias, or both, of results) shall be
precision should be other than constant or linear, more con-
incorporated into the collaborative-study design.
centration levels should be analyzed, especially in the non-
linear portions of the expected relationship between precision
7.3 Measurement of Precision:
and concentration. Also, if the desired uses of the method
7.3.1 Every interlaboratory study done to provide precision-
include comparisons (for example, either among laboratories
and-bias estimates for a D19 test method must use a blind
orwitharegulatorystandard)atorneartheestimateddetection
design consisting of Youden-pairs duplicate (or, in general,
level of the method, sufficient concentrations should be in-
replicate) samples, or a combination of the two designs. The
cludedinthedesiredmatrixtocomplywiththerequirementsof
designusedwithineachconcentrationlevelmustbeconsistent,
the IDE. Similarly, if it is desired to know the level of
thatis,eitherentirelyYoudenpair(atthesametwoYoudenpair
quantitationofthemethodfordatatobeusedininterlaboratory
concentrations) or entirely replicate at the same concentration.
comparisons,concentrationsshouldbeselectedtocomplywith
In either design, each participant receives (or prepares from a
the requirements of the IQE. Study concentrations, except
concentrate and a matrix, both of which are furnished by the
additional concentrations needed in the trace range to charac-
study) a separate sample for each analysis required in the
terizedetectionbelowtherangeofcalibrationshouldgenerally
study.
be rather uniformly distributed over the range of the test
7.3.2 Calculations may be found in Youden and Steiner (4).
method.
Once developed, these mean and standard-deviation estimates
NOTE 1—The precision and bias statement is only valid for the range of are treated the same as statistics from a study with the usual
D2777 − 21
replicate design. A detailed example with and without raw 7.5.1 The Quality Control section to appear in the test
experimental data is given in Refs (5) and (6), respectively. method must be drafted before the collaborative-study design
is made final, and the study design must assure that the
7.3.3 The advantage of the Youden-pair design is that the
collaborative study will produce any background data not
single-operator standard-deviation estimates are free of any
otherwise available to complete the final Quality Control
conscious or unconscious analyst bias. The procedures for
section properly. Each part of the draft Quality Control section
calculating overall and single-operator standard deviations
must be used during the collaborative study, unless insufficient
from Youden-pair designs are given in Section 10 and illus-
background data exist to establish credible interim required
trated in Appendix X2. Advantages of the blind duplicate
performance criteria for that part.
design are that it is free of linearity and homogeneity assump-
7.5.2 All quality control data/information produced to meet
tions inherent to Youden pairs and that it may be easier to
the requirements of 7.5.1 shall be reported to the task-group
prepare samples. The procedures for calculating overall and
chair, along with results from analyses on the study samples.
single-operator standard deviations for blind duplicates are
given in Section 11.
7.6 AtemporaryPrecisionandBiasstatementthataddresses
7.3.4 For a replicate sample design, the minimum require-
only repeatability (see 1.6) shall follow the procedures of 7.1
ment is a duplicate sample design, as assumed throughout the
through 7.5 to the extent possible.
remaining sections of this practice. Calculations used to
7.6.1 Repeatability for each concentration level shall be
estimate the means and standard deviations for more than two
based on a minimum of seven retained replicate determina-
replicate samples can be found in Practice E691.
tions. Adequate replicate concentrations should be used to
insure that there are at least seven values will be usable after
7.4 Measurement of Bias:
eliminating outliers.
7.4.1 The concept of accuracy comprises both precision and
7.6.2 The analyst should be provided no information with
bias(seeTerminologyD1129andPracticeE177).Asdiscussed
respect to the true concentration at each level.
in Practice E177, there is not a single form that can be
7.6.3 At least three concentrations covering the range of the
universallyrecommendedforstatementsofaccuracy.Sincethe
test method shall be included for each matrix tested.
accuracy of a measurement process is affected by both random
7.6.4 The temporary Precision and Bias statement com-
andsystematicsourcesoferror,measuresofbothkindsoferror
monly will be based on results from a single laboratory.
are needed. The standard deviation is a universal measure of
However,twoormorelaboratoriesoranalystsmaybeusedand
random sources of error (or precision). Bias is a measure of the
their results pooled to form the repeatability estimate. Fewer
systematic errors of a test method.
than seven replicates may be analyzed within each laboratory
7.4.2 A collaborative-study evaluation of bias for a specific
or analyst as long as there are at least six “degrees of freedom”
matrix produces a set of analyte/sample means. The difference
for repeatability (the single operator standard deviation). De-
between a true value (however defined) and the related mean is
grees of freedom are calculated as the total number of
an estimate of the average systematic error (that is, bias of the
replicates analyzed across all laboratories/analysts minus the
test method).
numberoflaboratories/analysts.Interlaboratoryresults,suchas
7.4.3 There are three major approaches commonly used to
interlaboratory standard deviation (“reproducibility”) may be
test a measurement procedure: (1) measurement of known
reported with a clear statement that the results are based on
materials, (2) comparison with other measurement procedures,
data not in accordance with D2777 and are to be considered
and (3) comparison with modifications of the procedure itself
only illustrative of potential results. A description of the data
(7). The third approach may involve the standard-addition
actually used (number of laboratories and/or analysts) shall be
technique or the simultaneous analysis of several aliquots of
included in any case.
different sizes (for example, 0.5, 1, 1.5, 2, 2.5 units). The task
7.6.5 All other requirements for collaborative studies shall
group will select the approach that best suits its needs within
apply to the repeatability study. References to interlaboratory
the resources available to it.
results or comparisons shall be ignored.
7.4.4 Themostlikelytask-groupapproachwillbetheuseof
7.7 In cases where a test method has been revised and the
known materials. Since reference standards are unlikely to be
revisionisdeemedtohaveincludedsubstantivemodificationto
available, the task group will prepare its samples with added
the method (see Section 1.3), a comparability study must be
(thereforeknowntothem)quantitiesoftheconstituent(s)being
completed whenever it is feasible to do so. At any rate, a
tested. The best available chemical and analytical techniques
limited validation study must be completed when a full
for preparing, stabilizing (if necessary), storing and shipping
comparability study is not feasible.
the prepared samples should be known within the task group
7.7.1 The comparability study shall follow the general
and will not be addressed in this practice. However, if the
principles of the full-scale Collaborative Test, as outlined in
sample-preparation and handling techniques used for the study
Sections 7.1 through 7.5, above, with following exceptions,
are different from those expected to be used for samples during
which describe the minimum requirements for the comparabil-
routine application of the test method, those differences shall
ity study.
bepointedoutintheprecision-and-biasstatement.Futureusers
7.7.1.1 Three laboratories providing usable data shall par-
of the test method may decide that these differences had an
ticipate in the comparability study.
effect on the precision or bias results, or both, from the study.
7.7.1.2 One representative, reproducible matrix shall be
7.5 Quality Control During the Study: tested, as described in 7.2.5.
D2777 − 21
7.7.1.3 Two concentrations of the analyte(s) shall be tested, the requirements of this practice. If the proposed collaborative
representing low and high levels with respect to the method’s studyplandoesnotmeetalltherequirementsspelledoutinthis
intended range. Triplicates of each concentration level, com- practice, it will require a review and recommendation by the
prising true replicate concentrations, notYouden pairs, shall be ResultsAdvisor and approval by the D19Technical Operations
prepared for each laboratory. Concentrations used shall be Section of the Executive Subcommittee.
identical for each laboratory and no information provided with
8.3 Upon receipt of approval of the collaborative-test plan
respect to the true concentration values.
by the Results Advisor and as needed, the D19 Technical
7.7.1.4 Each laboratory shall conduct analysis of the
Operations Section of the Executive Subcommittee, the task-
samples using both the original method as written before
group chairman (or designee) will conduct the collaborative
revision and the proposed revised method. Each sample shall
test.
be processed through the method, original or revised, in its
entirety prior to analysis of another test sample, even when the
9. Conducting the Collaborative Study
methods are identical with regard to their initial or final steps.
9.1 A single entity, acting for the task group, will prepare
Ideally, laboratories should alternate in their use of the original
the samples for the collaborative study and ship them to the
and revised methods, or randomize their order, although this
participants with: (1) instructions for the study; (2) a copy of
may not be possible in many cases due to limited space,
the exact test method (if not already supplied); and, (3) the
availability of equipment, or other limiting factors.
participant reporting form (or reporting instructions).
7.7.2 In cases where it is not feasible to perform a compa-
9.1.1 The instructions for the collaborative study shall
rability study, such as when it is not possible or practical to
require sufficient preliminary work by potential collaborators
expect a laboratory to operate the two different procedures,
to familiarize them adequately with the test method prior to
different instruments, etc., a limited validation study shall be
study measurements. This preliminary familiarization is nec-
performed in lieu.
essarytoensurethateachcollaborativestudyismadebyapeer
7.7.2.1 Three laboratories providing usable data shall par-
group and that a learning experience is not included in the
ticipate in the study.
statistics of the collaborative study. The task group may also
7.7.2.2 One representative, reproducible matrix shall be
develop procedures to qualify prospective collaborators, and
tested, as described in 7.2.5.
this approach is strongly recommended.
7.7.2.3 Three concentrations of the analyte(s) shall be
9.1.2 Each laboratory should usually supply its own cali-
tested, covering the method’s intended range. Triplicates of
bration materials, as independent calibration materials are a
each concentration level, comprising true replicate
significantsourceofinterlaboratoryvariability.However,ifthe
concentrations, not Youden pairs, shall be prepared for each
cost of availability of calibration materials is judged to be a
laboratory. Concentrations used shall be identical for each
significant deterrent to participation, or if currently available
laboratory,andnoinformationprovidedwithrespecttothetrue
materials are inadequate and not considered typical for subse-
concentration values.
quent routine use of the test method, these materials may be
7.7.3 All other requirements for collaborative studies shall
distributed with the study samples. If calibration standards are
apply to the comparability or limited validation study.
provided, the precision-and-bias section of the test method
should so note, including the concentrations and matrix of the
8. Collaborative Study Design Approval
standards and any specific instructions for their use.
8.1 After design approval by the task group, the task-group
9.1.3 As an aid, the task-group chairman may use the
chair (or designee) will summarize the proposed design of the
“Sample Template for a Collaborative Study Workplan,” as in
collaborative study. This summary will include: (1) the test
Appendix X3.
method (in ASTM format and as approved by the task group)
to be tested; (2) the analytes to be included in the study; (3) the 9.2 Each sample should be clearly marked with a common
uniquecode,informativetothedistributorsbutnotinformative
number of samples in accordance with the paired-sample plan
of 7.3.1;(4) the approach for determining the bias of the test to the study participants with regard to concentration level,
method as exemplified in the collaborative study; (5) the range duplicate concentration, or Youden pairing relative to other
of concentrations covered, and approximate concentration of samples in the batch. Samples should be sized to supply more
material in each sample or set; (6) the approximate number of than the minimum amount necessary to participate in the study
laboratories and analysts; (7) the matrices and QC samples (with reasonable allowance for pipetting, rinsing, etc.) to allow
being tested; (8) plans for developing study samples; and (9)a for trial runs and analytical restarts that may be necessary. A
copy of the instruction and data-reporting package to be given separate set of samples shall be provided for each operator.
to each study participant.This summary should be presented to Sampleconcentrationsshouldnotbeeasilysurmisedvalues(1,
the Results Advisor in the form of a letter. 5, etc.). The assignment of samples to the participating
8.1.1 As an aid, the task group chairman may use, “Form laboratories should be randomized within each concentration
A-Approval of Plans for Interlaboratory Testing,” and in level. The above recommendations should help assure statisti-
Appendix X1 (a completed example is shown in Fig. X1.1). cal independence of results.
8.2 Uponreviewoftheplan,theResultsAdvisorwilladvise 9.3 A copy of the test method under investigation, the
the task-group chairman whether the plan meets the require- written instructions for carrying out his/her part of the
ments of this practice or what changes are necessary to meet program, and the necessary study samples should be supplied
D2777 − 21
to each operator. No supplementary instructions or explana- 10.2 Tabulation of Data—The data reported by the labora-
tions (such as by telephone or from a task-group member tories shall be made consistent in reporting units and, if
within a cooperating laboratory) should be supplied to one possible, in the number of reported values per operator or
participant if not to all. Study materials should be distributed laboratory (8). Before data tabulation is begun, any unusable
from one location, and the operator’s reports should be data sets (that is, sets generated by laboratories that did not
returned to one location. follow significant study instructions or used an unacceptable
variation of the test method being studied) shall be removed.
9.4 The written instructions should cover such items as: (1)
Unless each laboratory used its own matrix with a unique
directives for storing and subdividing the sample; (2) prepara-
background concentration, all bias and precision estimates are
tion of sample prior to using the test method; (3) order of
to be based on the concentration reported, rather than on
analyses of samples (random order within each laboratory is
background-corrected results.
often best); (4) details regarding the reporting of study results
10.2.1 Sometimes, looking at the histogram of a set of data
on the reporting form; and (5) the time limit for return of the
will help one recognize or understand, or both, the cause of
reporting form.
unusual d
...

This document is not an ASTM standard and is intended only to provide the user of an ASTM standard an indication of what changes have been made to the previous version. Because
it may not be technically possible to adequately depict all changes accurately, ASTM recommends that users consult prior editions as appropriate. In all cases only the current version
of the standard as published by ASTM is to be considered the official document.
Designation: D2777 − 13 D2777 − 21
Standard Practice for
Determination of Precision and Bias of Applicable Test
Methods of Committee D19 on Water
This standard is issued under the fixed designation D2777; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope
1.1 This practice establishes uniform standards for estimating and expressing the precision and bias of applicable test methods for
Committee D19 on Water. Statements of precision and bias in test methods are required by the Form and Style for ASTM
Standards, “Section A21. Precision and Bias (Mandatory).” In principle, all test methods are covered by this practice. However,
the variability equations provided in this standard are applicable only to test methods that yield continuous function values.
1.2 Except as specified in 1.4, 1.5, and 1.6, this practice requires the task group proposing a new test method to carry out a
collaborative study from which statements for precision (overall and single-operator standard-deviation estimates) and bias can be
developed. This practice provides general guidance to task groups in planning and conducting such determinations of precision and
bias.
1.3 This practice also provides guidance to task groups for conducting limited-scale collaborative studies (known as
“comparability studies”) for test methods that have been revised, when such revision includes substantive modifications. requires
that a task group making a substantive revision to a test method also perform a limited-scale collaborative study (known as
a — comparability study) to evaluate the effect of the revision on the precision and bias statement. This practice provides guidance
to task groups for conducting such limited-scale collaborative studies. Examples of substantive modifications may include, but are
not limited to, changes in mandatory or allowable instrumentation, reagents, reaction times, etc.
1.3.1 Changes to applicable water matrices in the Scope of a method may constitute a substantive modification under this
provision. However, recognize that even the original Only matrices that have been evaluated in an approved collaborative study
may not have used all the various matrix types specified be listed in the Scope of a method. It is recognized that the term “matrix”
is generally vague. Terms specifying matrix types can cover significantly different chemical constituents, unless the matrix is
synthesized to be of a standardized makeup. Substitute Wastewater (Practice D5905in the method’s original Scope.) is one such
defined matrix. For purposes of this practice, the importance of this requirement is to assist the user of a D19 standard in
determining the applicability of the method to their samples. Evaluated matrices should be described with as much detail as
possible to minimize misapplication.
1.3.2 A method’s concentration-range extension that is deemed to merit additional collaborative testing (even without a method
modification that would otherwise be considered substantive) shall require a full collaborative study, as described in Sections7.1
7.1-through 7.5, but only at Youden-pair concentrations representative of the extended range. Note that such a collaborative study
could involve as little as a single-sample Youden-pairsingle concentration study in a single reproducible matrix.
This practice is under the jurisdiction of ASTM Committee D19 on Water and is the direct responsibility of Subcommittee D19.02 on Quality Systems, Specification,
and Statistics.
Current edition approved Jan. 1, 2013Dec. 15, 2021. Published February 2013 June 2022. Originally approved in 1969. Last previous edition approved in 20122013 as
D2777 – 12.D2777 – 13. DOI: 10.1520/D2777-13.10.1520/D2777-21.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
D2777 − 21
1.3.3 Whether a revision to a test method includes substantive modification shall be determined by consensus of the Committee.
1.4 If a full-scale collaborative study is not technically feasible, due tobecause of the nature of the test method or instability of
samples, the largest feasible scaled-down most complete collaborative study that is technically feasible shall be conducted to
provide the best possible limited basis for estimating the overall and single-operator standard deviations. In some situations, an
intermediate collaborative study as described in Guide D7847 may provide an appropriate approach. It is recognized that there may
be circumstances when even a limited collaborative study is not feasible. Any collaborative study plan that does not meet all the
requirements spelled out in this practice will require a review and recommendation by the Results Advisor and an approval by the
D19 Technical Operations Section of the Executive Subcommittee.
1.4.1 Examples of acceptable scaled-down studies are the local-area intermediate studies conducted by Subcommittee D19.24 on
microbiological methods because of inherent sample instability. These studies perishability. Such intermediate collaborative studies
meet the same degrees of freedom and participant requirements as full collaborative studies. They involve six or more completely
independent local-area analysts who can begin analysis of uniform samples at an agreed upon time. Guide D7847 can provide
guidance to the task group, the Results Advisor, and the Technical Operations Section of the Executive Subcommittee of
Committee D19 on the appropriate design of an acceptable intermediate collaborative study for test methods that measure highly
perishable parameters.
1.4.2 If uniform samples are providing homogenous samples with a sufficiently stable analyte concentration is not feasible under
any circumstances, a statement of single-operator precision willmay meet the requirements of this practice. Whenever possible, this
statement should be developed from data generated by independent multiple independent operators, each doing replicate analyses
on independent samples (of a specific matrix type), which generally fall within specified concentration ranges (see 7.2.5.2 (3)).
1.4.3 This practice is not applicable to methodology involving continuous sampling or continuous measurement, or both, of
specific constituents and properties.
1.4.4 This practice is also not applicable to open-channel flow measurements.
1.5 A collaborative study that satisfied the requirements of the version of this practice in force when the study was conducted will
continue to be considered an adequate basis for the precision-and-bias statement required in each test method. If the study does
not satisfy the current minimum requirements for a collaborative study, a statement listing the study’s deficiencies and a reference
to this paragraph shall be included in the precision-and-bias statement as the basis for an exemption from the current requirements.
1.6 This paragraph relates to special exemptions not clearly acceptable under 1.4 or 1.5. With the approval of Committee D19 on
the recommendation of the Results Advisor and the Technical Operations Section of the Executive Subcommittee of Committee
D19, a statement giving a compelling reason why compliance with all or specific points of this practice cannot be achieved will
meet both ASTM requirements Committee D19, (1) and the related requirements of this practice. In addition, Committee D19,
through a Main Committee ballot, may approve publication of a “Preliminary” Standard Method for a period not to exceed 5 years.
Preliminary Standards must contain a minimum of a single-operator precision-and-bias statement and a Quality Control section
based on the single operator data. Publication of a Preliminary Standard is conditional on the approval of a full D2777 collaborative
study design for the standard. Precision-and-bias statements authorized by this paragraph shall include the date of approval by
Committee D19.
1.7 Per Section A21.2.3 of the ASTM Form and Style Manual the committee may delay an interlaboratory study for a new method
and include a temporary statement in the Precision and Bias Section that addresses only single operator precision (“repeatability”).
This statement is valid for five years from the initial publication date. In this case, a single laboratory study shall be conducted
in accordance with 7.6Section 7.6.
1.8 In Section 1212, this practice shows exemplary precision-and-bias-statement formats for: (1) test methods yielding a numerical
measure, (2) test methods yielding a non-numerical report of success or failure based on criteria specified in the procedure, and
(3) test methods specifying that procedures in another ASTM test method are to be used with only insignificant modifications.
1.9 All studies, even those exempt from some requirements under Sections 1.4 through 1.8, previous sections, shall receive
approval from the Results Advisor before being conducted (see Section 8) and after completion (see Section 13).
D2777 − 21
1.10 This practice satisfies the QC requirements of Practice D5847.
1.11 It is the intent of this practice that task groups make every effort to retain all the data from their round-robincollaborative
studies. Values should not be eliminated unless solid evidence exists for their exclusion. The Results Advisor should work closely
with the task groups to effect this goal.
1.12 This international standard was developed in accordance with internationally recognized principles on standardization
established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued
by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
2. Referenced Documents
2.1 ASTM Standards:
D1129 Terminology Relating to Water
D1141 Practice for Preparation of Substitute Ocean Water
D1193 Specification for Reagent Water
D4375 Practice for Basic Statistics in Committee D19 on Water (Withdrawn 2018)
D5790 Test Method for Measurement of Purgeable Organic Compounds in Water by Capillary Column Gas Chromatography/
Mass Spectrometry
D5847 Practice for Writing Quality Control Specifications for Standard Test Methods for Water Analysis
D5905 Practice for the Preparation of Substitute Wastewater
D6091 Practice for 99 %/95 % Interlaboratory Detection Estimate (IDE) for Analytical Methods with Negligible Calibration
Error
D6512 Practice for Interlaboratory Quantitation Estimate
D7847 Guide for Interlaboratory Studies for Microbiological Test Methods
E177 Practice for Use of the Terms Precision and Bias in ASTM Test Methods
E178 Practice for Dealing With Outlying Observations
E456 Terminology Relating to Quality and Statistics
E691 Practice for Conducting an Interlaboratory Study to Determine the Precision of a Test Method
E1169 Practice for Conducting Ruggedness Tests
2.2 ASTM Adjuncts:
DQCALC Microsoft Excel-based software for the Interlaboratory Quantitation Estimate (IQE)
3. Terminology
3.1 Definitions—For definitions of terms used in this practice, refer to Terminologies D1129, D4375 and E456, and Practice
E177.
3.2 Definitions of Terms Specific to This Standard:
3.2.1 accuracy, n—a measure of the degree of conformity of a single test result generated by a specific procedure to the assumed
or accepted true value, and includes both precision and bias.
3.2.2 bias, n—the persistent positive or negative deviation of the average value of a test method from the assumed or accepted
true value.
3.2.3 comparability study, n—a collaborative study that incorporates side-by-side evaluation of the test method before and after
a substantive modification to a test method.
3.2.4 degrees of freedom, n—the total number of replicates analyzed across all laboratories/analysts minus the number of
laboratories/analysts.
3.2.5 laboratory, n—a single and completely independent analytical system with its own specific apparatus, source of reagents,
set of internal standard-operating procedures, etc.
For referenced ASTM standards, visit the ASTM website, www.astm.org, or contact ASTM Customer Service at [e-mail]. For Annual Book of ASTM Standards
volume information, refer to the standard’s Document Summary page on the ASTM website.
The last approved version of this historical standard is referenced on www.astm.org.
Available from ASTM International Headquarters. Order Adjunct No.ADJDQCALC. Original adjunct produced in 2007.
D2777 − 21
3.2.5.1 Discussion—
Different laboratories will differ from each other in all of these aspects, regardless of how physically or organizationally close they
may be to each other.
3.2.6 limited validation study, n—in a test method, a validation study that does not fulfill all D2777 requirements for a full-scale
collaborative study, but that can be used for re-validation of revised methods.
3.2.7 operator, n—usually the individual analyst within each laboratory who performs the test method throughout the collaborative
study.
3.2.7.1 Discussion—
However, for complicated test methods, the operator may be a team of individuals, each performing a specific function throughout
the study.
3.2.8 precision, n—the degree of agreement of repeated measurements of the same property, expressed in terms of dispersion of
test results about the arithmetical-mean result obtained by repetitive testing of a homogeneous sample under specified conditions.
3.2.8.1 Discussion—
The precision of a test method is expressed quantitatively as the standard deviation computed from the results of a series of
controlled determinations.
3.2.9 substantive modification, n—in a test method, a change (or changes) that is deemed by the Committee to be of such
magnitude that the change might affect the precision-and-bias data published with the original method.
3.3 Acronyms:
3.3.1 MDL, n—method detection limit
4. Summary of Practice
4.1 After the task group has assured itself that the test method has had all preliminary evaluation work completed, the task group
should prepare the test-method write-up in final form. The plan for collaborative study is developed in accordance with this practice
and submitted along with the test-method write-up to the Results Advisor for concurrence except as specified in 1.41.3, 1.5, and
1.6. As noted in 1.4, any collaborative study plan not meeting all requirements in this practice will also require D19 Technical
Operations Section approval. Upon receipt of concurrence, the the Results Advisor concurrence and approval of the Technical
Operations Section, the collaborative test is conducted, data analyzed, and precision-and-bias statements formulated by the task
group. Estimates of the lower limits of quantitation and detection may also be developed. The final precision-and-bias statistics
must be based on usable data from at least six independent laboratories. The statements, with backup data including the
reported-results summary, the calculations leading up to the statements, and the test method write-up with precision-and-bias
statements included are submitted to the subcommittee vice-chairman, who in turn sends a copy to the Results Advisor for
concurrence before balloting. This procedure assures having an acceptable copy of the collaborative-study results to send to ASTM
for items on the main-committee ballot. In most instances, the collaborative study shall be complete before a subcommittee ballot.
If the collaborative study is not complete, the test method may go on the ballot as a provisional test method rather than a standard
test method. Copies of the test data, approved calculations, and statistical results shall be filed at ASTM Headquarters when the
test method is submitted by the subcommittee chairman as an item for the main-committee ballot.
4.1.1 The appendix shows an example of “Form A—Approval of Plans for Interlaboratory Testing,” as Fig. X1.1.
4.1.2 For examples of data-reporting forms, see Appendix X3, 6.0.
4.1.3 In addition, the appendix shows a sample calculation of precision and bias from real collaborative-test data, the related table
of statistics, and the related precision-and-bias statement.
5. Significance and Use
5.1 Following this practice should result in precision-and-bias statements that can be achieved by any laboratory properly using
the test method studied. These precision-and-bias statements provide the basis for generic limits for use in the Quality Control
D2777 − 21
section of the test method. Optionally, the detection and quantitation values provide estimates of the level at which most
laboratories should be able to achieve confident detection and meet the minimum precision (expressed as relative standard
deviation) expected.
5.2 The method specifies the matrices for which the test method is appropriate. The collaborative test corroborates the write-up
within the limitations of the test design. An extensive test can only use representative matrices so that universal applicability cannot
be implied from the results.
5.3 The fundamental assumption of the collaborative study is that the matrices tested, the concentrations tested, and the
participating laboratories are a representative and fair evaluation of the scope and applicability of the test method as written.
6. Preliminary Studies
6.1 Considerable pilot work on a test method must precede the determination of its precision and bias (21, 32). This pilot work
should explore such variables as preservation requirements, reaction time, concentration of reagents, interferences, calibration, and
sample size. Potentially significant factors must be investigated and controlled in the written test method in advance of the
collaborative test. Also, disregard of such factors may introduce so much variation among operators that results are misleading or
inconclusive (43) (see 9.3 and 9.4). A ruggedness study conducted in a single laboratory is particularly useful for such
investigations and should be conducted to prove a test method is ready for interlaboratory testing (see Guide E1169 for details).
6.2 Only after a proposed test method has been tried, proved, and reduced to unequivocal written form should a determination of
its precision and bias be attempted.
6.3 If the task group intends to evaluate the method characteristics of detection and quantification, Practice D6091 (Interlaboratory
Detection Estimate) or Standard Practice D6512 (Interlaboratory Quantitation Estimate), or both, should be evaluated and the
recommendations for study designs incorporated. Determining detection capability and absolute bias for measurements at
background is especially critical for methods (such as radiochemical methods) that do not censor measurement results (for
example, against a critical level, MDL or reporting level).
6.3.1 To minimize the number of samples required, data would be gathered in two phases:
6.3.1.1 Phase I—Single-laboratory characterization. In this phase, the method developer would run a sufficient number of samples
at a sufficient number of concentrations to characterize fully response vs. concentration, as well as error vs. concentration. The
lowest concentration would be the level of the blank or the lowest concentration that could be measured; the highest concentration
would be at the upper limit of the analytical range.
6.3.1.2 Phase II—Collaborative study. Using the results of Phase I, the method developer would estimate the minimum number
and the magnitude of concentrations necessary to meet the requirements of the documents of interest.
7. Planning the Collaborative Test
7.1 Based upon the task group’s knowledge of a test method and the unequivocal write-up, several factors must be considered in
planning the collaborative test to assess the precision of the test method properly. The testing variables that must be considered
in planning are discussed below. In the collaborative study, it is generally not acceptable to control significant sources of variability
that cannot be controlled in routine use of the test method, because this control leads to false estimates of the test-method precision
and bias. In addition, the task group must determine within the resources available how best to estimate the precision and bias of
the test method.
7.2 Testing Variables:
7.2.1 It is desirable to develop a test method’s precision statement that indicates the contribution to overall variation of selected
causes such as laboratory, operator, sample matrix, analyte concentration, and other factors that may or have been shown to have
strong effects on the results. Since any test method can be tried in only a limited number of applications, the standard deviation
calculated from the results of a study can be only an estimate of the universe standard deviation. For this reason, the symbol s
The boldface numbers in parentheses refer to a list of references at the end of this standard.
D2777 − 21
(sample standard deviation) is used herein. The precision estimates generated from the study data will usually be the overall
standard deviation (s ) at any one concentration and the pooled single-operator standard deviation (s ) for each sample matrix and
T o
concentration studied.
7.2.2 Laboratories, operators, sample matrices, and analyte concentrations are the only sources of variability represented in the
precision-and-bias statements resulting from the usual collaborative study. These sources may not represent the additional influence
that can arise from differences in sample splitting, field preservation, transportation, etc., all of which may influence routine
analytical results as shown in the general precision definitions in Terminology D1129.
7.2.3 Laboratories—The final precision-and-bias statistics for each analyte, matrix, and concentration must be based on data from
at least six laboratories that passed any outlier tests (see 10.3) (that is, usable data). To be assured of meeting this requirement,
it is recommended that usable data be obtained from a minimum of eight independent laboratories. To guarantee eight will provide
usable data, it will often be necessary to get ten or more laboratories to agree to participate, because some may not provide data
and others may not provide usable data. Maximizing the number of participating laboratories is often the most important thing that
can be done to guarantee a successful study.
7.2.4 Even if the single-operator standard deviation is the only statistic to be estimated in the study (see 1.4.2), there should be
a minimum of eight operators who provide usable data, so there is assurance of data from six operators after all outlier removal.
7.2.5 Sample Matrices—The collaborative study shall be conducted with at least one representative sample matrix, which should
be reproducible by subsequent user-laboratories so that they can compare their results with the results of the collaborative study.
7.2.5.1 Typically, a reagent water prepared according to Specification D1193 or a synthetic medium, such as the substitute
wastewater described in Practice D5905 or the substitute ocean water described in Specification D1141, is used as the reference
matrix. Analytes and matrix may be supplied separately, with the analytes supplied as concentrates for addition to this matrix by
each laboratory; alternatively, the reference matrix containing the analyte(s) may be supplied to each participant. Information on
how the reference matrix was prepared in the study shall be clear in the precision-and-bias statement of the test method so users
can reproduce the study conditions properly.
7.2.5.2 Additional collaborative testing should also be conducted using other matrices specified in the scope of the test method.
Since these matrices must be the same for each study participant, they may have to be prepared (or obtained from a single source),
preserved, and distributed to all laboratories. As with the reference matrix, analytes may be supplied in a separate spiking solution
or already added to the matrix. A particularly attractive matrix might be a standard material available from an organization such
as the National Institute of Standards and Technology (NIST). In a collaborative test, use of uniform sample matrices is necessary
since they enable a more certain comparison with the reference matrix than is possible with matrices supplied separately by each
participant.
(1) Use of matrices with naturally occurring, non-zero background levels of the analyte(s) being studied will result in
precision-and-bias estimates that will be much more difficult to compare properly with estimates from the reference matrix.
(2) Any matrix spiking that may be necessary shall not significantly change the natural characteristics of the matrix.
(3) With the exception of the kind of limited study described in 1.4.2, the matrix-of-choice approach, in which each participant
is expected to acquire his or her own sample of a designated type, should not be used. Such studies are basically incompatible with
the statistical approaches employed in this practice. In addition, the presence of variable background concentrations prevents the
assignment of a proper mean-concentration level to each precision estimate produced in the study.
7.2.5.3 The same study design should be used for all sample matrices. A separate precision-and-bias statement should be generated
for each sample matrix with a brief description of the matrix tested.
7.2.5.4 When studies are available indicating the applicability of the test method for matrices untested in 7.2.5.1 and 7.2.5.2 and
not meeting the other requirements of this practice, at the discretion of the task group responsible for the test method and the
Results Advisor, and providing the data are analyzed in accordance with Section 10 of this practice, these supporting data may be
included in a separate section of the precision-and-bias statement. Included shall be a clear but brief description of the matrices
and the study protocol employed. It is the intent of this practice that ultimately, data concerning the precision and bias of the test
method in the full range of matrices covered in the scope and analyzed in accordance with this practice, will be made available
to the users of the test method.
7.2.6 Analyte Concentrations—If pilot work has shown that precision is linear with increasing analyte concentrations, at least three
Youden pairsconcentration levels (5) (that is, six concentrations) covering the desired range of the test method should be included
for each matrix. Each concentration level shall consist of samples prepared at either identical concentrations in replicate or of two
D2777 − 21
similar Youden pair concentrations, and be presented to each laboratory for analysis. Each Youden pair of concentrations shall
consist of a pair of samples containing the same analyte at two different concentrations that differ from each other by up to 10 %,
the percentage calculation being based on the average of the two samples in the pair. At a given concentration level the true
concentrations, whether replicates or Youden pairs, shall be identical for each laboratory. If the pilot work suggests that the
precision should be other than constant or linear, more concentration levels should be analyzed, especially in the non-linear
portions of the expected relationship between precision and concentration. Also, if the desired uses of the method include
comparisons (for example, either among laboratories or with a regulatory standard) at or near the estimated detection level of the
method, sufficient concentrations should be included in the desired matrix to comply with the requirements of the IDE. Similarly,
if it is desired to know the level of quantitation of the method for data to be used in interlaboratory comparisons, concentrations
should be selected to comply with the requirements of the IQE. Study concentrations, except additional concentrations needed in
the trace range to characterize detection below the range of calibration should generally be rather uniformly distributed over the
range of the test method.
NOTE 1—The precision and bias statement is only valid for the range of the data included in the study, so care should be taken to assure that trace
concentrations and upper bound of the linear range are considered in establishing the study concentrations.
7.2.6.1 Study samples with concentrations at or near a detection limit can produce non-quantitative results from participating
laboratories if participants are permitted to use their detection limit to censor their results. Zeroes or ’less thans’ that result from
this censoring process are non-quantitative results and cannot be included in the statistical analysis of study results specified later
in this practice. Conducting the specified statistical analysis on remaining quantitative data (that is, eliminating the non-numeric
data) under such circumstances can produce misleading precision-and-bias estimates. In general, if at a single concentration or
Youden pair, more thatthan ⁄3 of the data are non-numeric, the concentration/pair should be excluded from the precision and bias
determination. Therefore, when designing the study, carefully consider instructions to laboratories on censoring practices, typical
levels of detection and minimum calibration and consider including more concentrations in the study than the minimum required.
Results from analyses of Youden pairs concentrations at or near the detection limit can be included in this traditional statistical
analysis (and thus the working range of the method extended) if it turns out that most laboratories report quantified results.
7.2.7 Since the order of analyses should not be a source of systematic variability in the study, each participant should either be
told to randomize the order of study-sample analyses or be given a specific random order for the analyses.
7.2.7.1 Whenever the time of analyses has been shown to influence the analytical results, close control over the time of analyses
will be essential.
7.2.8 If pilot work has shown that the sample container must be of a specific material prepared in a specific manner prior to use,
the variation in containers obtained and prepared by the participants will be a random variable and should be treated as such in
the planning of the study and in the statistical analysis of the data.
7.2.9 The manner of preservation or other treatment of the sample prior to typical use of the test method (if known to affect the
precision or bias, or both, of results) shall be incorporated into the collaborative-study design.
7.3 Measurement of Precision:
7.3.1 Every interlaboratory study done to provide precision-and-bias estimates for a D19 test method must use a Youden-pair
design rather than a replicate-sample design. Justifiable exceptions to this requirement shall be approved through the process
provided inblind design consisting of Youden-pairs duplicate (or, in general, replicate) samples, or a combination of the two
designs. The design used within each concentration level must be consistent, that is, either entirely Youden pair (at the same two
Youden 1.6. In a Youden-pair pair concentrations) or entirely replicate at the same concentration. In either design, each participant
receives (or prepares from a concentrate and a matrix, both of which are furnished by the study) a separate sample for each analysis
required in the study. Among the set of samples each laboratory analyzes for a specific matrix, there are pairs of samples containing
similar but usually different analyte concentrations that differ from each other by up to 20 %; the percentage calculation is based
on the average of the two samples in the pair. As a matter of convenience in preparing the samples or spiking concentrates, up to
half the concentration levels may be represented by blind duplicates rather than Youden pairs, but the participants must have no
basis for comparing their single–test results from analyses of different study samples.
7.3.2 The only difference in treatment of data from a Youden-pair study is the calculation used to estimate the means and standard
deviations; these calculations Calculations may be found in Youden and Steiner (64). Once developed, these mean and
standard-deviation estimates are treated the same as statistics from a study with the usual replicate design. A detailed example with
and without raw experimental data is given in Refs.Refs (75) and (86), respectively.
D2777 − 21
7.3.3 The valueadvantage of the nonreplicateYouden-pair design is that the single-operator standard-deviation estimates are free
of any conscious or unconscious analyst bias. The procedures for calculating overall and single-operator standard deviations from
Youden-pair designs are given in Section 1110 and illustrated in Appendix X2. Advantages of the blind duplicate design are that
it is free of linearity and homogeneity assumptions inherent to Youden pairs and that it may be easier to prepare samples. The
procedures for calculating overall and single-operator standard deviations for blind duplicates are given in Section 11.
7.3.4 For a replicate sample design, the minimum requirement is a duplicate sample design, as assumed throughout the remaining
sections of this practice. Calculations used to estimate the means and standard deviations for more than two replicate samples can
be found in Practice E691.
7.4 Measurement of Bias:
7.4.1 The concept of accuracy comprises both precision and bias (see Terminology D1129 and Practice E177). As discussed in
Practice E177, there is not a single form that can be universally recommended for statements of accuracy. Since the accuracy of
a measurement process is affected by both random and systematic sources of error, measures of both kinds of error are needed.
The standard deviation is a universal measure of random sources of error (or precision). Bias is a measure of the systematic errors
of a test method.
7.4.2 A collaborative-study evaluation of bias for a specific matrix produces a set of analyte/sample means. The difference between
a true value (however defined) and the related mean is an estimate of the average systematic error (that is, bias of the test method).
7.4.3 There are three major approaches commonly used to test a measurement procedure: (1) measurement of known materials,
(2) comparison with other measurement procedures, and (3) comparison with modifications of the procedure itself (97). The third
approach may involve the standard-addition technique or the simultaneous analysis of several aliquots of different sizes (for
example, 0.5, 1, 1.5, 2, 2.5 units). The task group will select the approach that best suits its needs within the resources available
to it.
7.4.4 The most likely task-group approach will be the use of known materials. Since reference standards are unlikely to be
available, the task group will prepare its samples with added (therefore known to them) quantities of the constituent(s) being tested.
The best available chemical and analytical techniques for preparing, stabilizing (if necessary), storing and shipping the prepared
samples should be known within the task group and will not be addressed in this practice. However, if the sample-preparation and
handling techniques used for the study are different from those expected to be used for samples during routine application of the
test method, those differences shall be pointed out in the precision-and-bias statement. Future users of the test method may decide
that these differences had an effect on the precision or bias results, or both, from the study.
7.5 Quality Control During the Study:
7.5.1 The Quality Control section to appear in the test method must be drafted before the collaborative-study design is made final,
and the study design must assure that the collaborative study will produce any background data not otherwise available to complete
the final Quality Control section properly. Each part of the draft Quality Control section must be used during the collaborative
study, unless insufficient background data exist to establish credible interim required performance criteria for that part.
7.5.2 All quality control data/information produced to meet the requirements of 7.5.1 shall be reported to the task-group chair,
along with results from analyses on the study samples.
7.6 A temporary Precision and Bias statement that addresses only repeatability (see 1.6) shall follow the procedures of 7.1 through
7.5 to the extent possible.
7.6.1 Repeatability for each concentration level shall be based on a minimum of seven retained replicate determinations. Adequate
replicate concentrations should be used to insure that there are at least seven values will be usable after eliminating outliers.
7.6.2 Replicates of each concentration shall be true replicate concentrations, not comprised of Youden pairs.
7.6.2 The analyst should be blinded provided no information with respect to the true concentration at each level.
7.6.3 At least three concentrations covering the range of the test method shall be included for each matrix tested.
D2777 − 21
7.6.4 The temporary Precision and Bias statement commonly will be based on results from a single laboratory. However, two or
more laboratories or analysts may be used and their results pooled to form the repeatability estimate. Fewer than seven replicates
may be analyzed within each laboratory or analyst as long as there are at least six “degrees of freedom” for repeatability (the single
operator standard deviation). Degrees of freedom are calculated as the total number of replicates analyzed across all
laboratories/analysts minus the number of laboratories/analysts. Interlaboratory results, such as interlaboratory standard deviation
(“reproducibility”) may be reported with a clear statement that the results are based on data not in accordance with D2777D2777
and are to be considered only illustrative of potential results. A description of the data actually used (number of laboratories and/or
analysts) shall be included in any case.
7.6.5 All other requirements for collaborative studies shall apply to the repeatability study. References to Youden pairs shall be
construed as references to replicate samples. References to interlaboratory results or comparisons shall be ignored.
7.7 In cases where a test method has been revised and the revision is deemed to have included substantive modification to the
method (see Section 1.3), a comparability study must be completed whenever it is feasible to do so. At any rate, a limited validation
study must be completed when a full comparability study is not feasible.
7.7.1 The comparability study shall follow the general principles of the full-scale Collaborative Test, as outlined in Sections 7.1
through 7.5, above, with following exceptions, which describe the minimum requirements for the comparability study.
7.7.1.1 Three laboratories providing usable data shall participate in the comparability study.
7.7.1.2 One representative, reproducible matrix shall be tested, as described in 7.2.5Section 7.2.5.
7.7.1.3 Two concentrations of the analyte(s) shall be tested, representing low and high levels with respect to the method’s intended
range. Triplicates of each concentration level, comprising true replicate concentrations, not Youden pairs, shall be prepared for each
laboratory. Concentrations used shall be identical for each laboratory, which shall be blindlaboratory and no information provided
with respect to the true concentration values.
7.7.1.4 Each laboratory shall conduct analysis of the samples using both the original method as written before revision and the
proposed revised method. Each sample shall be processed through the method, original or revised, in its entirety prior to analysis
of another test sample, even when the methods are identical with regard to their initial or final steps. Ideally, laboratories should
alternate in their use of the original and revised methods, or randomize their order, although this may not be possible in many cases
due to limited space, availability of equipment, or other limiting factors.
7.7.2 In cases where it is not feasible to perform a comparability study, such as when it is not possible or practical to expect a
laboratory to operate the two different procedures, different instruments, etc., a limited validation study shall be performed in lieu.
7.7.2.1 Three laboratories providing usable data shall participate in the study.
7.7.2.2 One representative, reproducible matrix shall be tested, as described in 7.2.5.
7.7.2.3 Three concentrations of the analyte(s) shall be tested, covering the method’s intended range. Triplicates of each
concentration level, comprising true replicate concentrations, not Youden pairs, shall be prepared for each laboratory.
Concentrations used shall be identical for each laboratory, which shall be blindand no information provided with respect to the true
concentration values.
7.7.3 All other requirements for collaborative studies shall apply to the comparability or limited validation study. References to
Youden pairs shall be construed as references to replicate samples.
8. Collaborative Study Design Approval
8.1 After design approval by the task group, the task-group chair (or designee) will summarize the proposed design of the
collaborative study. This summary will include: (1) the test method (in ASTM format and as approved by the task group) to be
tested; (2) the analytes to be included in the study; (3) the number of samples in accordance with the paired-sample plan of 7.3.1;
(4) the approach for determining the bias of the test method as exemplified in the collaborative study; (5) the range of
concentrations covered, and approximate concentration of material in each sample or set; (6) the approximate number of
D2777 − 21
laboratories and analysts; (7) the matrices and QC samples being tested; (8) plans for developing study samples; and (9) a copy
of the instruction and data-reporting package to be given to each study participant. This summary should be presented to the
Results Advisor in the form of a letter.
8.1.1 As an aid, the task group chairman may use, “Form A-Approval of Plans for Interlaboratory Testing,” and in Appendix X1
(a completed example is shown in Fig. X1.1).
8.2 Upon review of the plan, the Results Advisor will advise the task-group chairman whether the plan meets the requirements
of this practice or what changes are necessary to meet the requirements of this practice. If the proposed collaborative study plan
does not meet all the requirements spelled out in this practice, it will require a review and recommendation by the Results Advisor
and approval by the D19 Technical Operations Section of the Executive Subcommittee.
8.3 Upon receipt of approval of the collaborative-test plan by the Results Advisor, the Advisor and as needed, the D19 Technical
Operations Section of the Executive Subcommittee, the task-group chairman (or designee) will conduct the collaborative test.
9. Conducting the Collaborative Study
9.1 A single entity, acting for the task group, will prepare the samples for the collaborative study and ship them to the participants
with: (1)(1) instructions for the study; (2)(2) a copy of the exact test method (if not already supplied); and, (3)(3) the participant
reporting form (or reporting instructions).
9.1.1 The instructions for the collaborative study shall require sufficient preliminary work by potential collaborators to familiarize
them adequately with the test method prior to study measurements. This preliminary familiarization is necessary to ensure that each
collaborative study is made by a peer group and that a learning experience is not included in the statistics of the collaborative study.
The task group may also develop procedures to qualify prospective collaborators, and this approach
...