ASTM D7778-12
(Guide)Standard Guide for Conducting an Interlaboratory Study to Determine the Precision of a Test Method
Standard Guide for Conducting an Interlaboratory Study to Determine the Precision of a Test Method
SIGNIFICANCE AND USE
5.1 All ASTM standard test methods are required to include statements on precision and bias. To obtain such data it is necessary to conduct an interlaboratory study. This document is designed to provide a brief overview of the steps necessary in an ILS and to suggest an appropriate sequence in carrying out these steps.
5.1.1 Qualitative tests such as Test Method D130 copper corrosion may not require an ILS.
SCOPE
1.1 This guide describes the procedures for planning and conducting an interlaboratory study (ILS) of a test method used in Petroleum Products and Lubricants Committee D02 of ASTM for the purpose of estimating repeatability and reproducibility of the test method in accordance with ASTM Form and Style requirements.
1.2 This guide is concerned with the management of the ILS and intended to provide guidance for the planning stage and ensure the process, logistics, and tools are identified and agreed upon in advance.
1.2.1 Selection of the samples and the impact of sample selection on the final scope of the test method—both the range of materials covered in the scope and precision sections and the measurement range covered in the precision section—are important, and careful consideration needs to be given to these aspects before an ILS is launched.
1.3 This guide does not concern itself with the development of test methods but rather with gathering the information needed for a test method precision statement.
1.4 This guide is concerned with test methods which yield a single numerical figure as the test result, although the single figure may be the result of a calculation from a set of measurements.
1.5 This guide is designed for tests of properties that are stable, such that bulk samples can be homogenized and sub-samples can be prepared that will be identical and stable for the duration of the ILS testing period. This guide is not designed for unstable properties or for properties that make it difficult to obtain stable, representative samples for an ILS conducted at multiple laboratory sites.
1.6 This guide represents the suggested practices for a typical precision study. Individual subcommittees may modify this approach as their expertise directs.
1.7 This guide is voluntary, and thus, is not required for an approved precision study.
General Information
Relations
Standards Content (Sample)
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Contact ASTM International (www.astm.org) for the latest information
Designation: D7778 − 12 AnAmerican National Standard
Standard Guide for
Conducting an Interlaboratory Study to Determine the
Precision of a Test Method
This standard is issued under the fixed designation D7778; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope 1.7 This guide is voluntary, and thus, is not required for an
approved precision study.
1.1 This guide describes the procedures for planning and
conductinganinterlaboratorystudy(ILS)ofatestmethodused
2. Referenced Documents
in Petroleum Products and Lubricants Committee D02 of
2.1 ASTM Standards:
ASTM for the purpose of estimating repeatability and repro-
D130 Test Method for Corrosiveness to Copper from Petro-
ducibility of the test method in accordance with ASTM Form
and Style requirements. leum Products by Copper Strip Test
D4306 Practice for Aviation Fuel Sample Containers for
1.2 ThisguideisconcernedwiththemanagementoftheILS
Tests Affected by Trace Contamination
and intended to provide guidance for the planning stage and
D5854 Practice for Mixing and Handling of Liquid Samples
ensuretheprocess,logistics,andtoolsareidentifiedandagreed
of Petroleum and Petroleum Products
upon in advance.
D6300 Practice for Determination of Precision and Bias
1.2.1 Selection of the samples and the impact of sample
Data for Use in Test Methods for Petroleum Products and
selection on the final scope of the test method—both the range
Lubricants
ofmaterialscoveredinthescopeandprecisionsectionsandthe
D6708 Practice for StatisticalAssessment and Improvement
measurement range covered in the precision section—are
of Expected Agreement Between Two Test Methods that
important, and careful consideration needs to be given to these
Purport to Measure the Same Property of a Material
aspects before an ILS is launched.
D6792 Practice for Quality System in Petroleum Products
1.3 This guide does not concern itself with the development
and Lubricants Testing Laboratories
of test methods but rather with gathering the information
E177 Practice for Use of the Terms Precision and Bias in
needed for a test method precision statement.
ASTM Test Methods
E178 Practice for Dealing With Outlying Observations
1.4 This guide is concerned with test methods which yield a
E456 Terminology Relating to Quality and Statistics
single numerical figure as the test result, although the single
E691 Practice for Conducting an Interlaboratory Study to
figure may be the result of a calculation from a set of
Determine the Precision of a Test Method
measurements.
E1169 Practice for Conducting Ruggedness Tests
1.5 This guide is designed for tests of properties that are
2.2 ISO/IEC Standards:
stable, such that bulk samples can be homogenized and
ISO/IEC Guide 30 Terms and Definitions Used in Connec-
sub-samples can be prepared that will be identical and stable
tion with Reference Materials
for the duration of the ILS testing period. This guide is not
ISO/IEC Guide 17025 General Requirements for the Com-
designed for unstable properties or for properties that make it
petence of Testing and Calibration Laboratories
difficult to obtain stable, representative samples for an ILS
conducted at multiple laboratory sites.
3. Terminology
1.6 This guide represents the suggested practices for a
3.1 Definitions:
typical precision study. Individual subcommittees may modify
this approach as their expertise directs.
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
This test method is under the jurisdiction of ASTM Committee D02 on Standards volume information, refer to the standard’s Document Summary page on
Petroleum Products and Lubricants and is the direct responsibility of Subcommittee the ASTM website.
D02.94.01 on Task Group A: Quality Assurance and Quality Control. Available from Available from Aerospace Industries Association of America,
Current edition approved April 15, 2012. Published June 2012. DOI: 10.1520/ Inc. (AIA), 1000 Wilson Blvd., Suite 1700, Arlington, VA 22209-3928, http://
D7778–12. www.aia-aerospace.org.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
D7778 − 12
3.1.1 accuracy, n—closeness of agreement between an in- 5.1.1 Qualitative tests such as Test Method D130 copper
dividual test result and an accepted reference value. E456 corrosion may not require an ILS.
3.1.2 bias, n—difference between the population mean of
6. Planning an Interlaboratory Study
the test results and an accepted reference value. E456
6.1 Suggested steps in an ILS are given in Table 1.
3.1.3 outlier, n—observation that appears to deviate mark-
edly from the other observations of the sample (also referred to
6.2 Support for conducting the ILS is available fromASTM
as extreme result, outlying or doubtful observation, or aberrant
through its Interlaboratory Studies Office (ILS@astm.org).
value. E178
Individuals involved in conducting an ILS are listed below.
3.1.4 precision, n—closeness of agreement among test re- 6.2.1 “Support” does not include financial support for new
sults obtained under prescribed conditions. E456 or alternate equipment or procedure beyond sampling and
shipping materials to the participants.
3.1.5 reference material, n—material or substance, one or
6.2.2 A Task Force should be formed consisting at a
more of whose property values are sufficiently homogenous
minimum of the method developers, users and ASTM statisti-
and well established to be used for the calibration of an
cian. Based on the objectives of the ILS, this task force should
apparatus, the assessment of a measurement method, or for
have the overall responsibility for the design, disclosure, and
assigning values to other materials. E177
execution of the ILS. The Task Force is charged with the
3.1.6 repeatability, n—quantitative expression of the ran-
development of an ILS plan that includes scope and purpose,
dom error associated with a single operator in a given
funding, staffing, participants (equipment structure), sample
laboratoryobtainingrepetitiveresultsbyapplyingthesametest
types, tests required, special instructions, reporting form, and
method with the same apparatus under constant operating
sample distribution, participant protocol, and a questionnaire
conditions on identical test material within a short interval of
and statistical tools to be used, and criteria for acceptance of
time on the same day. It is defined as the difference between
results.
two such results at the 95% confidence level. D6792, E177
6.2.3 An ILS Coordinator should be appointed as an overall
3.1.7 reproducibility, n—quantitative expression of the ran-
person responsible for the distribution of materials and proto-
dom error associated with different operators from different
cols to the laboratories, and receive the test results from the
laboratories, using different apparatus, each obtaining a single
laboratories. Eventually this person should be responsible for
resultbyapplyingthesamemethodonanidenticaltestsample.
all aspects of conducting the ILS, and writing the research
It is defined as the 95% confidence limit for the difference
report.
betweentwosuchsingleandindependentresults. D6792, E177
6.2.3.1 A running activity record may be useful to be
3.1.8 trueness, n—closeness of agreement between the maintained by the ILS Coordinator to overview the progress of
population mean of the measurements or the test results and an
the ILS activity.
accepted reference value. E456
6.2.4 Statistician—The Task Force should obtain the ser-
vices of a statistician who is familiar with the protocols used in
4. Summary of Guide
ASTM for precision calculations during the planning stage for
4.1 The procedure presented in this guide consists of five
input towards the design of the ILS, material selection,
basic steps:
establishment of the statistical tools to be used, acceptance
(1) Planning an interlaboratory study (including objectives
criteria, and testing protocol. This person may be a member of
and expected outcome).
theTask Force.This person should also assist the task group in
(2) Disclosure of plan details (participants, equipment,
interpreting the results of the data analysis.
samples, logistics).
(3) Approval by membership.
(4) Guiding the testing phase of the study and collecting
test data results.
TABLE 1 Sequence of an Interlaboratory Study
(5) PostvalidationthattheILSwasexecutedinaccordance
Sequence Procedure Section Number in this
to the agreed plan.
Guide
4.1.1 This guide does not address the data analysis tech-
1 Select ILS membership 6.2.2
2 Prepare Basic Design 6.3
niques. For details of such analysis techniques refer to Practice
3 Specify Test Method(s) 6.4
D6300 and E691, or other technically equivalent documents.
4 List Participating 6.5
4.1.2 The relevant subcommittee is the sole arbiter of Laboratories
5 List Materials 6.6
technical equivalence.
6 Number of Test Results 6.7
per Test Material per
5. Significance and Use
Lab
7 Provide Protocol 6.8
5.1 AllASTM standard test methods are required to include
8 Ruggedness Study 7 and 8
statements on precision and bias. To obtain such data it is
(Pilot Run)
necessarytoconductaninterlaboratorystudy.Thisdocumentis 9 Full Scale Run 9
10 Data Handling 10
designed to provide a brief overview of the steps necessary in
11 Data Presentation 11
an ILS and to suggest an appropriate sequence in carrying out
12 Prepare Study Report 12
these steps.
D7778 − 12
6.2.4.1 The statistical software available from D02 Com- 6.5 Laboratories—An ILS should ideally include enough
mittee can help in these calculations, but it is highly recom- laboratories to produce a reproducibility estimate with at least
mendedthatthisworkbedoneby,orundertheoversightofthe 30 degrees of freedom. See Practice D6300 for further guid-
statistician engaged in 6.2.4. ance. It is important that a reasonable cross section of the
population of qualified laboratories be represented in the ILS.
6.3 Basic Design—The design should be kept as simple as
Special attention is required to ensure that at least 75% of the
possible in order to obtain estimates of within- and between-
participants represent product manufacturer laboratories or
laboratory variability that are free of secondary effects. The
third party commercial independent testing laboratories, or
basic design is represented by a two-way classification table in
both.Take care to ensure that the population is random and not
which the rows represent the laboratories, the columns repre-
biased towards special interest participants such as equipment
sent the materials, and each cell (that is, the intersection of a
vendors. Vendor participation shall be managed carefully so as
row with a column) contains the test results obtained by a
not to bias the data population.
particular laboratory on a particular material. See Table 2 for
6.5.1 The final statement of precision of a test method shall
illustration where x and y are the test results obtained on the
not be based on fewer than six laboratories with acceptable test
same materials by the same operator from each laboratory.
results for each material. More than 7 or 8 laboratories should
6.3.1 Samples to be tested shall be presented as a single
participate, preferably a minimum of 10 laboratories in case
series of test samples, requiring only a single analysis per
some data is unacceptable and data from one or more labora-
sample. Both the material identification and the corresponding
tories has to be excluded from the data evaluation.
duplicateshallbeblindedsuchthatthetesterdoesnotknowthe
6.5.2 Any laboratory considered qualified to run the test
material identification of the test sample, nor its corresponding
routinely should be encouraged to participate in the ILS. In
duplicate run. An example of a randomized series for the ILS
order to obtain an adequate number of participating
design in Table 2 is presented in Table 3. If this single
laboratories, the ILS study should be publicized through trade
randomized series approach is not possible, then blind repeats
magazines (if feasible), meetings, broadcast through pertinent
of the same material(s) should be embedded among the
ASTM committees, etc.
samples provided. The blind samples should represent the
6.5.3 “Qualified” above implies proper laboratory facilities
materials used in the scope of the test method and concentra-
and testing equipment, competent operators, familiar with the
tion range for which the method is applicable.At the least, the
test method, a reputation for reliable testing work, and suffi-
blind duplicates should include a range of easy and difficult
cient time and interest to perform the task.
samples and include the full sample matrix in the sample set.
By including only easy samples in blind testing or the full ILS 6.5.3.1 A good gauge of a laboratories competence can be
for that matter, the resulting precision may demonstrate better estimatedbyitscompliancewithISO17025orPracticeD6792
repeatability than otherwise would be found in practice. laboratory quality management standards.
6.3.2 The randomness of testing samples is so important
6.5.4 If a laboratory is not familiar with the test method, it
that this has to be a requirement, not just a good practice. The
should do several runs on available materials so all the
actual duplicates run for each test material shall be randomized
difficulties are worked out before participating in the actual
within the series.
ILS.
6.5.4.1 Analysts and laboratories new to the Test Method
6.4 Test Method—–Avalid test method is required that has
should be encouraged to document any aspects of the Test
been developed in one or more competent laboratories and
Method that are unclear or ambiguous so the wording of the
studied in a pilot ILS (see Section 8).This draft method should
Test Method, especially the procedure, can be clarified to
describe the test procedure in terms that the steps can be
remove ambiguities.
unequivocally followed in a typical laboratory by competent
personnelwiththeknowledgeofthematerialsandtheproperty 6.5.5 The ILS should not be restricted to a group of
to be tested. The test conditions that affect the test results laboratories judged to be exceptionally qualified and equipped
appreciably should be identified and the proper degree of for the ILS. This can lead to extremely precise data which
control of sample handling and the test conditions be specified cannot be re
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.