ASTM D7778-15(2022)e1
(Guide)Standard Guide for Conducting an Interlaboratory Study to Determine the Precision of a Test Method
Standard Guide for Conducting an Interlaboratory Study to Determine the Precision of a Test Method
SIGNIFICANCE AND USE
5.1 All ASTM standard test methods are required to include statements on precision and bias. To obtain such data it is necessary to conduct an interlaboratory study. This document is designed to provide a brief overview of the steps necessary in an ILS and to suggest an appropriate sequence in carrying out these steps.
5.1.1 Qualitative tests such as Test Method D130 copper corrosion may not require an ILS.
SCOPE
1.1 This guide describes the procedures for planning and conducting an interlaboratory study (ILS) of a test method used in Petroleum Products and Lubricants Committee D02 of ASTM for the purpose of estimating repeatability and reproducibility of the test method in accordance with ASTM Form and Style requirements.
1.2 This guide is concerned with the management of the ILS and intended to provide guidance for the planning stage and ensure the process, logistics, and tools are identified and agreed upon in advance.
1.2.1 Selection of the samples and the impact of sample selection on the final scope of the test method—both the range of materials covered in the scope and precision sections and the measurement range covered in the precision section—are important, and careful consideration needs to be given to these aspects before an ILS is launched.
1.3 This guide does not concern itself with the development of test methods but rather with gathering the information needed for a test method precision statement.
1.4 This guide is concerned with test methods which yield a single numerical figure as the test result, although the single figure may be the result of a calculation from a set of measurements.
1.5 This guide is designed for tests of properties that are stable, such that bulk samples can be homogenized and sub-samples can be prepared that will be identical and stable for the duration of the ILS testing period. This guide is not designed for unstable properties or for properties that make it difficult to obtain stable, representative samples for an ILS conducted at multiple laboratory sites.
1.6 This guide represents the suggested practices for a typical precision study. Individual subcommittees may modify this approach as their expertise directs.
1.7 This guide is voluntary, and thus, is not required for an approved precision study.
1.8 This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
General Information
Relations
Standards Content (Sample)
This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
´1
Designation: D7778 − 15 (2022) An American National Standard
Standard Guide for
Conducting an Interlaboratory Study to Determine the
Precision of a Test Method
This standard is issued under the fixed designation D7778; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
ε NOTE—Editorially corrected 6.2.2 in May 2022.
1. Scope 1.6 This guide represents the suggested practices for a
typical precision study. Individual subcommittees may modify
1.1 This guide describes the procedures for planning and
this approach as their expertise directs.
conductinganinterlaboratorystudy(ILS)ofatestmethodused
1.7 This guide is voluntary, and thus, is not required for an
in Petroleum Products and Lubricants Committee D02 of
approved precision study.
ASTM for the purpose of estimating repeatability and repro-
ducibility of the test method in accordance with ASTM Form
1.8 This international standard was developed in accor-
and Style requirements.
dance with internationally recognized principles on standard-
ization established in the Decision on Principles for the
1.2 ThisguideisconcernedwiththemanagementoftheILS
Development of International Standards, Guides and Recom-
and intended to provide guidance for the planning stage and
mendations issued by the World Trade Organization Technical
ensuretheprocess,logistics,andtoolsareidentifiedandagreed
Barriers to Trade (TBT) Committee.
upon in advance.
1.2.1 Selection of the samples and the impact of sample
2. Referenced Documents
selection on the final scope of the test method—both the range
2.1 ASTM Standards:
ofmaterialscoveredinthescopeandprecisionsectionsandthe
measurement range covered in the precision section—are D130 Test Method for Corrosiveness to Copper from Petro-
leum Products by Copper Strip Test
important, and careful consideration needs to be given to these
aspects before an ILS is launched. D4306 Practice for Aviation Fuel Sample Containers for
Tests Affected by Trace Contamination
1.3 This guide does not concern itself with the development
D5854 Practice for Mixing and Handling of Liquid Samples
of test methods but rather with gathering the information
of Petroleum and Petroleum Products
needed for a test method precision statement.
D6300 Practice for Determination of Precision and Bias
1.4 This guide is concerned with test methods which yield a
Data for Use in Test Methods for Petroleum Products,
single numerical figure as the test result, although the single
Liquid Fuels, and Lubricants
figure may be the result of a calculation from a set of
D6708 Practice for StatisticalAssessment and Improvement
measurements.
of Expected Agreement Between Two Test Methods that
Purport to Measure the Same Property of a Material
1.5 This guide is designed for tests of properties that are
D6792 Practice for Quality Management Systems in Petro-
stable, such that bulk samples can be homogenized and
leum Products, Liquid Fuels, and Lubricants Testing
sub-samples can be prepared that will be identical and stable
Laboratories
for the duration of the ILS testing period. This guide is not
E177 Practice for Use of the Terms Precision and Bias in
designed for unstable properties or for properties that make it
ASTM Test Methods
difficult to obtain stable, representative samples for an ILS
E178 Practice for Dealing With Outlying Observations
conducted at multiple laboratory sites.
E456 Terminology Relating to Quality and Statistics
E691 Practice for Conducting an Interlaboratory Study to
Determine the Precision of a Test Method
This test method is under the jurisdiction of ASTM Committee D02 on
Petroleum Products, Liquid Fuels, and Lubricants and is the direct responsibility of
Subcommittee D02.94 on Coordinating Subcommittee on Quality Assurance and
Statistics. For referenced ASTM standards, visit the ASTM website, www.astm.org, or
Current edition approved May 1, 2022. Published June 2022. Originally contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
approved in 2012. Last previous edition approved in 2015 as D7778 – 15. DOI: Standards volume information, refer to the standard’s Document Summary page on
10.1520/D7778-15R22E01. the ASTM website.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
´1
D7778 − 15 (2022)
E1169 Practice for Conducting Ruggedness Tests 4.1.1 This guide does not address the data analysis tech-
2.2 ISO/IEC Standards: niques. For details of such analysis techniques refer to Practice
ISO/IEC Guide 30 Terms and Definitions Used in Connec- D6300 and E691, or other technically equivalent documents.
tion with Reference Materials 4.1.2 The relevant subcommittee is the sole arbiter of
ISO/IEC Guide 17025 General Requirements for the Com- technical equivalence.
petence of Testing and Calibration Laboratories
5. Significance and Use
3. Terminology
5.1 AllASTM standard test methods are required to include
3.1 Definitions: statements on precision and bias. To obtain such data it is
3.1.1 accuracy, n—closeness of agreement between an in-
necessarytoconductaninterlaboratorystudy.Thisdocumentis
dividual test result and an accepted reference value. E456 designed to provide a brief overview of the steps necessary in
an ILS and to suggest an appropriate sequence in carrying out
3.1.2 bias, n—difference between the population mean of
these steps.
the test results and an accepted reference value. E456
5.1.1 Qualitative tests such as Test Method D130 copper
3.1.3 outlier, n—observation that appears to deviate mark-
corrosion may not require an ILS.
edly from the other observations of the sample (also referred to
as extreme result, outlying or doubtful observation, or aberrant
6. Planning an Interlaboratory Study
value. E178
6.1 Suggested steps in an ILS are given in Table 1.
3.1.4 precision, n—closeness of agreement among test re-
6.2 Support for conducting the ILS is available fromASTM
sults obtained under prescribed conditions. E456
through its Interlaboratory Studies Office (ILS@astm.org).
3.1.5 reference material, n—material or substance, one or
Individuals involved in conducting an ILS are listed below.
more of whose property values are sufficiently homogenous
6.2.1 “Support” does not include financial support for new
and well established to be used for the calibration of an
or alternate equipment or procedure beyond sampling and
apparatus, the assessment of a measurement method, or for
shipping materials to the participants.
assigning values to other materials. E177
6.2.2 A Task Force should be formed consisting at a
3.1.6 repeatability, n—quantitative expression of the ran-
minimum of the method developers, users, and statistician.
dom error associated with a single operator in a given
Based on the objectives of the ILS, this task force should have
laboratoryobtainingrepetitiveresultsbyapplyingthesametest
the overall responsibility for the design, disclosure, and execu-
method with the same apparatus under constant operating
tion of the ILS. The Task Force is charged with the develop-
conditions on identical test material within a short interval of
ment of an ILS plan that includes scope and purpose, funding,
time on the same day. It is defined as the difference between
staffing, participants (equipment structure), sample types, tests
two such results at the 95% confidence level. D6792, E177
required, special instructions, reporting form, and sample
distribution, participant protocol, and a questionnaire and
3.1.7 reproducibility, n—quantitative expression of the ran-
dom error associated with different operators from different statistical tools to be used, and criteria for acceptance of
laboratories, using different apparatus, each obtaining a single results.
resultbyapplyingthesamemethodonanidenticaltestsample. 6.2.3 An ILS Coordinator should be appointed as an overall
It is defined as the 95% confidence limit for the difference person responsible for the distribution of materials and proto-
cols to the laboratories, and receive the test results from the
betweentwosuchsingleandindependentresults. D6792, E177
laboratories. Eventually this person should be responsible for
3.1.8 trueness, n—closeness of agreement between the
all aspects of conducting the ILS, and writing the research
population mean of the measurements or the test results and an
report.
accepted reference value. E456
4. Summary of Guide
TABLE 1 Sequence of an Interlaboratory Study
4.1 The procedure presented in this guide consists of five
Sequence Procedure Section Number in this
basic steps: Guide
(1) Planning an interlaboratory study (including objectives 1 Select ILS membership 6.2.2
2 Prepare Basic Design 6.3
and expected outcome).
3 Specify Test Method(s) 6.4
(2) Disclosure of plan details (participants, equipment,
4 List Participating 6.5
Laboratories
samples, logistics).
5 List Materials 6.6
(3) Approval by membership.
6 Number of Test Results 6.7
(4) Guiding the testing phase of the study and collecting
per Test Material per
test data results. Lab
7 Provide Protocol 6.8
(5) PostvalidationthattheILSwasexecutedinaccordance
8 Ruggedness Study 7 and 8
to the agreed plan.
(Pilot Run)
9 Full Scale Run 9
10 Data Handling 10
Available from Available from Aerospace Industries Association of America,
11 Data Presentation 11
Inc. (AIA), 1000 Wilson Blvd., Suite 1700, Arlington, VA 22209-3928, http:// 12 Prepare Study Report 12
www.aia-aerospace.org.
´1
D7778 − 15 (2022)
6.2.3.1 A running activity record may be useful to be 6.4 Test Method—To establish initial precision statements
maintained by the ILS Coordinator to overview the progress of foranewtestmethod,adraftofthetestmethodisrequiredthat
the ILS activity. has been developed in one or more competent laboratories and
6.2.4 Statistician—The Task Force should obtain the ser- studied in a pilot ILS (see Section 8).This draft method should
vices of a statistician who is familiar with the protocols used in describe the test procedure in terms that the steps can be
ASTM for precision calculations during the planning stage for unequivocally followed in a typical laboratory by competent
input towards the design of the ILS, material selection, personnel with the knowledge of the materials and the property
establishment of the statistical tools to be used, acceptance to be tested. The test conditions that affect the test results
criteria, and testing protocol. This person may be a member of appreciably should be identified and the proper degree of
theTask Force.This person should also assist the task group in control of sample handling and the test conditions be specified
interpreting the results of the data analysis. in the description of the test procedure.The test method should
6.2.4.1 The statistical software available from D02 Com- specify calibration procedure and its frequency and the format
mittee can help in these calculations, but it is highly recom- for reporting the test results.
mendedthatthisworkbedoneby,orundertheoversightofthe 6.4.1 Reevaluation of Existing Precision Statements—To
statistician engaged in 6.2.4. reevaluate existing precision statements for test methods, all
aspects of the existing test method that are not intended to be
6.3 Basic Design—The design should be kept as simple as
revised shall be followed. Intended revisions to the test method
possible in order to obtain estimates of within- and between-
shall be highlighted in the test method and explicitly stated to
laboratory variability that are free of secondary effects. The
the participants.
basic design is represented by a two-way classification table in
6.4.1.1 ILSExecutionProtocol—Theproceduresandspecial
which the rows represent the laboratories, the columns repre-
instructions provided in the ILS protocol for participation shall
sent the materials, and each cell (that is, the intersection of a
not introduce restrictions or controls in the execution of the
row with a column) contains the test results obtained by a
method that are not in the normal execution of the method
particular laboratory on a particular material. See Table 2 for
being evaluated.
illustration where x and y are the test results obtained on the
6.4.1.2 Variables—All variables in the test method that are
same materials by the same operator from each laboratory.
intended to be variables shall be left as variables in the ILS.To
6.3.1 Samples to be tested shall be presented as a single
remove, control, or set limits on factors that are not intended to
series of test samples, requiring only a single analysis per
be controlled in the normal usage of the test method in the
sample. Both the material identification and the corresponding
execution of the ILS will result in the precision being overly
duplicateshallbeblindedsuchthatthetesterdoesnotknowthe
optimistic,andassuchprecisionstatementsthusgeneratedwill
material identification of the test sample, nor its corresponding
likely be unattainable by users in the normal usage of the
duplicate run. An example of a randomized series for the ILS
method.
design in Table 2 is presented in Table 3. If this single
randomized series approach is not possible, then blind repeats 6.5 Laboratories—An ILS should ideally include enough
of the same material(s) should be embedded among the laboratories to produce a reproducibility estimate with at least
samples provided. The blind samples should represent the 30 degrees of freedom. See Practice D6300 for further guid-
materials used in the scope of the test method and concentra- ance. It is important that a reasonable cross section of the
tion range for which the method is applicable.At the least, the population of qualified laboratories be represented in the ILS.
blind duplicates should include a range of easy and difficult Special attention is required to ensure that at least 75 % of the
samples and include the full sample matrix in the sample set. participants represent product manufacturer laboratories or
By including only easy samples in blind testing or the full ILS third party commercial independent testing laboratories, or
for that matter, the resulting precision may demonstrate better both.Take care to ensure that the population is random and not
repeatability than otherwise would be found in practice. biased towards special interest participants such as equipment
6.3.2 The randomn
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.