ASTM E1601-98(2003)e1
(Practice)Standard Practice for Conducting an Interlaboratory Study to Evaluate the Performance of an Analytical Method
Standard Practice for Conducting an Interlaboratory Study to Evaluate the Performance of an Analytical Method
SIGNIFICANCE AND USE
Ideally, interlaboratory testing of a method is carried out by a randomly chosen group of laboratories that typifies the kind of laboratory that is likely to use the method. In actuality, this ideal is only approximated by the laboratories that are available and willing to undertake the test work. The coordinator of the program must ensure that every participating laboratory has appropriate facilities and personnel and performs the method exactly as written. If this goal is achieved, the statistics developed during the ILS will be adequate for determining if the method is capable of producing satisfactory precision in actual use. If the program includes certified reference materials, the test data also provide information concerning the accuracy of the method. The statistics provide a general guide to the expected performance of the method in the laboratories of those who will use it.
SCOPE
1.1 This practice presents procedures and statistics for an interlaboratory study (ILS) of the performance of an analytical method. The study provides statistical values which are useful in determining if a method is satisfactory for the purposes for which it was developed. These statistical values may be incorporated in the method's precision and bias section. This practice discusses the meaning of the statistics and what users of analytical methods may learn from them.
1.2 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility of the user of this standard to establish appropriate safety and health practices and determine the applicability of regulatory limitations prior to use.
General Information
Relations
Standards Content (Sample)
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Contact ASTM International (www.astm.org) for the latest information
´1
Designation: E1601 – 98 (Reapproved 2003)
Standard Practice for
Conducting an Interlaboratory Study to Evaluate the
Performance of an Analytical Method
This standard is issued under the fixed designation E1601; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
´ NOTE—Caution notes were moved into the text editorially in November 2003.
1. Scope 3.2.1 interlaboratory study (ILS)—study undertaken to as-
certainifatestmethodissuitableforitsintendeduse.TheILS
1.1 This practice covers procedures and statistics for an
includes preparation, testing, and evaluation phases.
interlaboratory study (ILS) of the performance of an analytical
3.2.2 interlaboratory test—measures the variability of re-
method. The study provides statistical values which are useful
sults when a test method is applied many times in a number of
in determining if a method is satisfactory for the purposes for
laboratories.
which it was developed. These statistical values may be
3.2.3 replicate results—results obtained by applying a test
incorporated in the method’s precision and bias section. This
method a specified number of times to a material.
practice discusses the meaning of the statistics and what users
3.2.4 result—the numerical value obtained by applying a
of analytical methods may learn from them.
test method once to a material.
1.2 This standard does not purport to address all of the
3.2.5 test method—gives directions for producing a single
safety concerns, if any, associated with its use. It is the
result.
responsibility of the user of this standard to establish appro-
3.2.6 test protocol—gives instructions to each participating
priate safety and health practices and determine the applica-
laboratory, detailing the way it is to conduct its part of the
bility of regulatory limitations prior to use.
interlaboratory test program.
2. Referenced Documents 3.3 Unless the test method destroys the test portion each
time it is applied, the protocol for a Plan A test specifies, if
2.1 ASTM Standards:
possible, replicate results on a single test portion (which may
E135 Terminology Relating to Analytical Chemistry for
be in solution). The protocol for a Plan B test specifies the
Metals, Ores, and Related Materials
number of test portions of a material and requires duplicate
E691 Practice for Conducting an Interlaboratory Study to
results (2 only) on each portion (which may be in solution).
Determine the Precision of a Test Method
E1169 Practice for Conducting Ruggedness Tests
4. Summary of Practice
E1763 Guide for Interpretation and Use of Results from
4.1 Instructions are provided for planning and conducting a
Interlaboratory Testing of Chemical Analysis Methods
cooperative evaluation of a proposed analytical method.
3. Terminology 4.2 The following list describes the organization of this
practice:
3.1 Definitions—For definitions of terms used in this prac-
4.2.1 Sections 1-5 define the scope, significance and use,
tice, refer to Terminology E135.
referenced documents, and terms used in this practice.
3.2 Definitions of Terms Specific to This Standard:
4.2.2 Section6helpsusersofanalyticalmethodsunderstand
andusethestatisticsfoundinthePrecisionandBiassectionof
This practice is under the jurisdiction ofASTM Committee E01 onAnalytical methods.
ChemistryforMetals,Ores,andRelatedMaterialsandisthedirectresponsibilityof
4.2.3 Sections 7 and 8 instruct the ILS coordinator and
Subcommittee E01.22 on Statistics and Quality Control.
members of the task group on how to plan and conduct the
Current edition approved Oct. 1, 2003. Published November 2003. Originally
experimental phase of the study.
approved in 1994. Last previous edition approved in 1998 as E1601–98. DOI:
10.1520/E1601-98R03E01.
4.2.4 Section 9 discusses the procedures for collecting,
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
evaluating,anddisseminatingthedatafromtheinterlaboratory
contactASTM Customer Service at service@astm.org. ForAnnual Book ofASTM
test.
Standards volume information, refer to the standard’s Document Summary page on
the ASTM website. 4.2.5 Section 10 presents the statistical calculations.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.
´1
E1601 – 98 (2003)
4.2.6 Sections 11 and 12 discuss the use of statistics to analytical instruments and equipment must be in good condi-
evaluate a test method and the means of incorporating the ILS tion; and (4) the method must be performed exactly as written.
statistics into Precision and Bias statements. 6.2.1 Reproducibility Index, R—This statistic estimates the
4.2.7 TheAnnexA1 gives the rationale for the calculations expected range of differences in results reported from two
laboratories, a range that is not exceeded in more than 5% of
in Section 10.
such comparisons. Use R to predict how well your results
should agree with those from another laboratory: First, obtain
5. Significance and Use
a result under the conditions stated in 6.2, then add R to, and
5.1 Ideally,interlaboratorytestingofamethodiscarriedout
subtract R from, this result to form a concentration confidence
by a randomly chosen group of laboratories that typifies the
interval.Suchanintervalhasa95%probabilityofincludinga
kind of laboratory that is likely to use the method. In actuality,
result obtainable by the method should another laboratory
this ideal is only approximated by the laboratories that are
analyzethesamesample.Forexample,aresultof46.57%was
available and willing to undertake the test work. The coordi-
obtained. If R for the method at about 45% is 0.543, the 95%
nator of the program must ensure that every participating
confidence interval for the result (that is, one expected to
laboratory has appropriate facilities and personnel and per-
includetheresultobtainedinanotherlaboratory19timesoutof
forms the method exactly as written. If this goal is achieved,
20) extends from 46.03 to 47.11%.
the statistics developed during the ILS will be adequate for
determining if the method is capable of producing satisfactory
NOTE 1—For those not conversant with statistical concepts, it is
importanttorealizethatinmostsuchcomparisons,thedifferenceswillbe
precision in actual use. If the program includes certified
much smaller than the confidence interval implies. The 50% confidence
reference materials, the test data also provide information
interval is only about one third (34.6%) as wide. Thus, the “average”
concerning the accuracy of the method. The statistics provide
interval for the above result (one expected to include the result obtained
a general guide to the expected performance of the method in
by another laboratory half the time) extends from 46.4 to 46.8%. The
the laboratories of those who will use it.
obviousimplicationisthat,althoughhalfthedifferenceswillbemorethan
0.2%, half will be less than 0.2%.
6. Statistical Guide for the Users of Analytical Methods
6.2.2 Repeatability Index, r—This statistic is given in the
Evaluated in Accordance With This Practice
methodonlyiftheinterlaboratorytestwasdesignedtomeasure
6.1 Standard Deviations:
s. It estimates the expected range of results reported in the
r
6.1.1 Minimum Standard Deviation of Method, s —This
same laboratory on different days, a range that is not exceeded
M
statistic measures the precision of test results under conditions
in more than 5% of such comparisons.
ofminimumvariability.Becauseitisimprobablethatamethod
7. Interlaboratory Test Planning
in ordinary use will exhibit precision this good, no predictive
indexiscalculatedfors .Usersadeptinstatisticsmaywishto
7.1 Analytical test methods start from a perceived need to
M
compare s and the short-term standard deviation of the
service one or more material specifications.
M
method measured in their laboratory. For most methods,
7.1.1 Develop a performance requirement for a method
short-term variability refers to results obtained within several
from the material specification(s). Include the following fac-
minutes.(Caution—Thestandarddeviationofresultsobtained
tors:expectedrangesofchemicalcompositionsofthematerials
on different occasions, even in the same laboratory, probably
tobecovered(method’sgeneralscope);specifiedelementsand
will exceed s .)
their concentrations (determination concentration ranges); and
M
6.1.2 Between-Laboratory Standard Deviation, s —This
the precision required.
R
statistic is a measure of the precision expected for results
7.1.2 Prepare a table of the elements and concentration
obtained in different laboratories. It reflects all sources of
rangestocoverthecriticalvaluesinthematerialspecifications.
variability that operate during the interlaboratory test (except
Use this information together with knowledge of the charac-
test material inhomogeneity in tests designed to eliminate that teristics of the candidate analytical method to select test
effect). It is used to calculate the reproducibility index, R. Use
materials for the interlaboratory program.
s for evaluating the precision of methods. It represents the 7.2 Draft Method—The process of developing methods and
R
expected variability of results when a method is used in
testing them in a preliminary way is beyond the scope of this
different laboratories. practice. All analytical skill and experience available to the
6.1.3 Within-Laboratory Standard Deviation, s—This sta-
taskgroupmustbeexertedtoensurethatthemethodwillmeet
r
tistic cannot be calculated in a normal interlaboratory test. It is the project requirements in 7.1 and that it is free of technical
determinedonlyintestsdesignedtomeasurevariabilitywithin
faults.Apreliminary,informaltestofamethodmustbecarried
laboratories.Whenthisstatisticisgiveninamethod,itreflects out in several laboratories before the final draft is prepared.
all variability that may occur from day-to-day within a labo-
Individuals responsible for selecting the method may find
ratory (for example, from calibration, standardization, or envi- helpful information in Practice E691 and Guide E1169. The
ronmental changes). It is used to calculate the repeatability
formal interlaboratory test must not start until the task group
index, r. The user is cautioned that additional sources of reaches consensus on a clearly written, explicitly stated, and
variation may affect results obtained in other laboratories.
unambiguously worded draft of the method in ASTM format,
6.2 Predictive Indexes—For the following indexes to apply, which has completed editorial review.
these conditions must be met: (1) the test materials must be 7.3 Test Materials—Appropriate test materials are essential
homogeneous;(2)analystsmustbecompetentanddiligent;(3) for a successful ILS. The larger the number of test materials
´1
E1601 – 98 (2003)
NOTE 2—If all reasonable effort fails to recruit at least six cooperating
included in the test program, the better the statistical informa-
laboratories, up to two of the recruited laboratories may each volunteer to
tiongenerated.Ontheotherhand,theburdenofrunningavery
submit two independent sets of test data as an expedient to provide a total
large number of materials may reduce the number of labora-
of at least six sets of data. Minimum requirements for independence are
tories willing to participate. A method must cover a concen-
that two typical analysts, who do not consult with each other about the
tration range extending both above and below the specified
method, perform the test protocol on different days. They should use
value(s). If possible, provide test materials near each limit.
separate equipment if possible and must not share calibration solutions or
Concentration ranges covering several orders of magnitude calibration curves.
should be tested with three or more materials.
8. Conducting the Interlaboratory Test
7.3.1 Material composition and form must be within the
general scope of the method. If possible, include all material 8.1 ProgramCoordinator—Oneindividual(presumablythe
types the scope is expected to cover. Often, only limited
task group chairman) will coordinate the entire ILS. A pro-
numbers of certified reference materials are available. Use spectiveILSprogramcoordinatorwillfindhelpfulinformation
thosethatbestmeetthecriteriaforthetest.Iftheydonotcover
on conducting the program in Practice E691. One way to
allconcentrationlevels,findorprepareothermaterialstofillin organize the work to provide close control while moving the
missing values.
program steadily to its conclusion is as follows:
7.3.2 The quantity of the material must be sufficient to 8.1.1 Prepare a draft of the method to be tested.
distribute to all laboratories participating in the test with about 8.1.2 Recruit a task group of participating laboratories.
50% held in reserve to cover unforeseen eventualities.
8.1.3 Select a set of test materials and assemble them into
kits, one for each laboratory.
7.3.3 Materials should be homogeneous on the scale of the
8.1.4 Write the test protocol to instruct each laboratory how
test portion consumed in each determination as well as among
the portions sent to different laboratories. Usually certified to run the test.
reference materials have been tested for homogeneity, but test
8.1.5 Prepare a report form.
materials from other sources may have had only a minimal
8.1.6 Establish a realistic time schedule for each part of the
examination. The use of laboratory-scale melting and casting
test program.
to produce test materials can sometimes lead to segregation of
8.1.7 Assemble and deliver to each participating laboratory
one or more components in an alloy. Unless specially gathered
everything needed to run the test: the draft method; the test
or prepared materials have been subjected to a thorough
materials and a document which describes them; the test
homogeneity test, they require the use of Test Plan B. It
protocol;thereportforms;andacoverletterwhichincludesthe
statistically removes the effect of moderate test material
deadlineforreturnofresults;andthename,address,telephone,
inhomogeneity from the estimates of the ILS statistics.
andFAXnumbersofthepersonwhowillhandleproblemsand
7.3.4 Test material sent to each laboratory must be perma- receive the completed report forms.
nently marked with its identity in such a manner that the 8.1.8 Expedite the laboratory testing. Follow up to be sure
identification is not likely to be lost or obliterated.
that the laboratories receive the test materials and understand
what is expected of them. Encourage laborato
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.