ASTM E1488-12(2023)
(Guide)Standard Guide for Statistical Procedures to Use in Developing and Applying Test Methods
Standard Guide for Statistical Procedures to Use in Developing and Applying Test Methods
ABSTRACT
This guide identifies statistical procedures for use in developing new test methods or revising or evaluating existing test methods, or both. It also cites statistical procedures especially useful in the application of test methods. This standard recommends what approaches may be taken and indicates which standards may be used to perform such assessments.
SIGNIFICANCE AND USE
4.1 The creation of a standardized test method generally follows a series of steps from inception to approval and ongoing use. In all such stages there are questions of how well the test method performs.
4.1.1 Assessments of a new or existing test method generally involve statistical planning and analysis. This standard recommends what approaches may be taken and indicates which standards may be used to perform such assessments.
4.2 This standard introduces a series of phases which are recommended to be considered during the life cycle of a test method as depicted in Fig. 1. These begin with a design phase where the standard is initially prepared. A development phase involves a variety of experiments that allow further refinement and understanding of how the test method performs within a laboratory. In an evaluation phase the test method is then examined by way of interlaboratory studies resulting in precision and bias statistics which are published in the standard. Finally, the test method is subject to a monitoring phase.
FIG. 1 Sequence of Steps
4.3 All ASTM test methods are required to include statements on precision and bias.3
4.4 Since ASTM began to require all test methods to have precision and bias statements that are based on interlaboratory test methods, there has been increased concern regarding what statistical experiments and procedures to use during the development of the test methods. Although there exists a wide range of statistical procedures, there is a small group of generally accepted techniques that are beneficial to follow. This guide is designed to provide a brief overview of these procedures and to suggest an appropriate sequence of carrying out these procedures.
4.5 Statistical procedures often result in interpretations that are not absolutes. Sometimes the information obtained may be inadequate or incomplete, which may lead to additional questions and the need for further experimentation. Information outside the data is al...
SCOPE
1.1 This guide identifies statistical procedures for use in developing new test methods or revising or evaluating existing test methods, or both.
1.2 This guide also cites statistical procedures especially useful in the application of test methods.
1.3 This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
General Information
Relations
Buy Standard
Standards Content (Sample)
This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation: E1488 − 12 (Reapproved 2023) An American National Standard
Standard Guide for
Statistical Procedures to Use in Developing and Applying
Test Methods
This standard is issued under the fixed designation E1488; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope E2655 Guide for Reporting Uncertainty of Test Results and
Use of the Term Measurement Uncertainty in ASTM Test
1.1 This guide identifies statistical procedures for use in
Methods
developing new test methods or revising or evaluating existing
test methods, or both.
3. Terminology
1.2 This guide also cites statistical procedures especially
3.1 Definitions—For a more extensive list of terms in E11
useful in the application of test methods.
standards, see Terminology E456.
1.3 This international standard was developed in accor-
3.1.1 bias, n—the difference between the expectation of the
dance with internationally recognized principles on standard-
test results and an accepted reference value. E177
ization established in the Decision on Principles for the
3.1.1.1 Discussion—Statistical procedures include the sam-
Development of International Standards, Guides and Recom-
pling considerations or the experiment design for the collection
mendations issued by the World Trade Organization Technical
of data, or both, and the numerical and graphical approaches to
Barriers to Trade (TBT) Committee.
summarize and analyze the collected data.
2. Referenced Documents
3.1.2 coeffıcient of variation, CV, n—for a nonnegative
characteristic, the ratio of the standard deviation to the mean
2.1 ASTM Standards:
for a population or sample. E2586
E177 Practice for Use of the Terms Precision and Bias in
ASTM Test Methods
3.1.3 component of variance, n—a part of a total variance
E178 Practice for Dealing With Outlying Observations
identified with a specified source of variability.
E456 Terminology Relating to Quality and Statistics
3.1.4 control chart, n—chart on which are plotted a statis-
E691 Practice for Conducting an Interlaboratory Study to
tical measure of a subgroup versus time of sampling along with
Determine the Precision of a Test Method
limits based on the statistical distribution of that measure so as
E1169 Practice for Conducting Ruggedness Tests
to indicate how much common, or chance, cause variation is
E1402 Guide for Sampling Design
inherent in the process or product. E2587
E2282 Guide for Defining the Test Result of a Test Method
3.1.5 observation, n—the process of obtaining information
E2489 Practice for Statistical Analysis of One-Sample and
Two-Sample Interlaboratory Proficiency Testing Programs regarding the presence or absence of an attribute of a test
specimen, or of making a reading on a characteristic or
E2554 Practice for Estimating and Monitoring the Uncer-
tainty of Test Results of a Test Method Using Control dimension of a test specimen. E2282
Chart Techniques
3.1.6 observed value, n—the value obtained by making an
E2586 Practice for Calculating and Using Basic Statistics
observation. E2282
E2587 Practice for Use of Control Charts in Statistical
3.1.7 precision, n—the closeness of agreement between
Process Control
independent test results obtained under stipulated conditions.
E177
This guide is under the jurisdiction of ASTM Committee E11 on Quality and
3.1.8 proficiency testing, n—determination of laboratory
Statistics and is the direct responsibility of Subcommittee E11.20 on Test Method
testing performance by means of interlaboratory comparisons.
Evaluation and Quality Control.
E2489
Current edition approved April 1, 2023. Published April 2023. Originally
approved in 1992. Last previous edition approved in 2018 as E1488 – 12 (2018).
3.1.9 repeatability, n—precision under repeatability condi-
DOI: 10.1520/E1488-12R23.
For referenced ASTM standards, visit the ASTM website, www.astm.org, or tions. E177
contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
3.1.10 repeatability conditions, n—conditions where inde-
Standards volume information, refer to the standard’s Document Summary page on
the ASTM website. pendent test results are obtained with the same method on
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E1488 − 12 (2023)
identical test items in the same laboratory by the same operator 4.1.1 Assessments of a new or existing test method gener-
using the same equipment within short intervals of time. E177 ally involve statistical planning and analysis. This standard
recommends what approaches may be taken and indicates
3.1.11 repeatability limit r, n—the value below which the
which standards may be used to perform such assessments.
absolute difference between two individual test results obtained
under repeatability conditions may be expected to occur with a
4.2 This standard introduces a series of phases which are
probability of approximately 0.95 (95 %). E177
recommended to be considered during the life cycle of a test
method as depicted in Fig. 1. These begin with a design phase
3.1.12 repeatability standard deviation, s , n—the standard
r
where the standard is initially prepared. A development phase
deviation of test results obtained under repeatability condi-
involves a variety of experiments that allow further refinement
tions. E177
and understanding of how the test method performs within a
3.1.13 reproducibility, n—precision under reproducibility
laboratory. In an evaluation phase the test method is then
conditions. E177
examined by way of interlaboratory studies resulting in preci-
3.1.14 reproducibility conditions, n—conditions where test
sion and bias statistics which are published in the standard.
results are obtained with the same method on identical test
Finally, the test method is subject to a monitoring phase.
items in different laboratories with different operators using
4.3 All ASTM test methods are required to include state-
different equipment. E177
ments on precision and bias.
3.1.15 reproducibility limit, R, n—the value below which
4.4 Since ASTM began to require all test methods to have
the absolute difference between two test results obtained under
precision and bias statements that are based on interlaboratory
reproducibility conditions may be expected to occur with a
test methods, there has been increased concern regarding what
probability of approximately 0.95 (95 %). E177
statistical experiments and procedures to use during the devel-
3.1.16 reproducibility standard deviation, s , n—the stan-
R
opment of the test methods. Although there exists a wide range
dard deviation of test results obtained under reproducibility
of statistical procedures, there is a small group of generally
conditions. E177
accepted techniques that are beneficial to follow. This guide is
3.1.17 ruggedness, n—insensitivity of a test method to
designed to provide a brief overview of these procedures and to
departures from specified test or environmental conditions.
suggest an appropriate sequence of carrying out these proce-
E1169
dures.
3.1.18 ruggedness test, n—a planned experiment in which
4.5 Statistical procedures often result in interpretations that
environmental factors or test conditions are deliberately varied
are not absolutes. Sometimes the information obtained may be
in order to evaluate the effects of such variation. E1169
inadequate or incomplete, which may lead to additional ques-
3.1.19 standard deviation, n—of a population, σ, the square tions and the need for further experimentation. Information
root of the average or expected value of the squared deviation
outside the data is also important in establishing standards and
of a variable from its mean – of a sample x¯, the square root of in the interpretation of numerical results.
the sum of the squared deviations of the observed values in the
5. Summary of Guide
sample divided by the sample size minus 1. E2586
5.1 Outlined below is a suggested sequence of four phases
3.1.20 state of statistical control, n—process condition
useful in the development of a test method. A flowchart is
when only common causes are operating on the process. E2587
provided in Fig. 1. Such a sequence of analyses may need to be
3.1.21 statistical procedures, n—the organized techniques
modified in specific situations. The assistance of a qualified
and methods used to collect, analyze, and interpret data.
statistician is recommended at each review phase.
3.1.21.1 Discussion—Statistical procedures include the
sampling considerations or the experiment design for the 5.2 Design Phase:
5.2.1 This phase includes the formalization of the scope and
collection of data, or both, and the numerical and graphical
approaches to summarize and analyze the collected data. the significance and use sections. It may include determining
the purpose and describing a general approach to the test
3.1.22 test determination, n—the value of a characteristic or
method but usually does not involve statistical studies.
dimension of a single test specimen derived from one or more
5.2.2 Studies may be conducted to evaluate the basic
observed values. E2282
performance of the method. The draft test method is prepared
3.1.23 test method, n—a definitive procedure that produces
and sampling requirements and the test result (see Guide
a test result. E2282
E2282) are clearly defined.
3.1.24 test observation, n—see observation. E2282
5.2.3 A flow chart is extremely valuable to identify the
sequence of operations involved in a test method, for example,
3.1.25 test result, n—the value of a characteristic obtained
the sampling steps required to obtain the test specimens,
by carrying out a specified test method. E2282
definition of the test determination, how a test result is to be
4. Significance and Use
computed, and running the tests on the specimens.
4.1 The creation of a standardized test method generally
follows a series of steps from inception to approval and
See the Form and Style Manual for ASTM Standards that specifies, when
ongoing use. In all such stages there are questions of how well
possible, precision statements shall be estimated based on the results of an
the test method performs. interlaboratory test program.
E1488 − 12 (2023)
FIG. 1 Sequence of Steps
5.3 Development Phase: estimates of repeatability standard deviations should be devel-
5.3.1 The test method is examined for such concerns as its
oped and published in this test method. Until an interlaboratory
stability, ruggedness, statistical control and the contributions to study is performed, these estimates generally are considered to
variability. The completion of this phase should result in
be provisional. Information on how a lab should develop
preliminary estimates of precision and the identification and
uncertainty estimates should also be provided.
suggested ways to estimate potential contributors to uncer-
5.3.6 Statistical Control—A test method must show capa-
tainty.
bility of performing in a consistent way over time. The use of
5.3.2 Evaluation of Short Term Control of Test Method—A
control charts (see Guide E2655) to monitor a proposed, or
test method must exhibit an ability to provide consistent results
existing, test method over time is one recommended way to
at least over short time periods. Preliminary studies or a pilot
examine the controllability or stability of a test method. This
test should be conducted to evaluate the short term stability of
statistical control should be demonstrated in one or two
the test method. A small series of repeated tests should be
laboratories using homogeneous material (test specimen).
conducted.
5.3.3 Analysis of Variability—Statistically designed experi-
5.4 Evaluation Phase:
ments conducted in one or two laboratories can be used to
5.4.1 The test method is subjected to interlaboratory studies
assess the relative magnitudes of different sources or potential
to provide estimates of within-laboratory repeatability and
contributors to variability of the test results. Such studies can
between-laboratory reproducibility. Additional information is
provide estimates of intermediate measures of precision.
supplied from proficiency studies when conducted.
5.3.4 Ruggedness Test—A ruggedness test (see Practice
5.4.2 Interlaboratory Study (ILS)—In accordance with
E1169) is a statistically designed experiment that helps identify
ASTM Form and Style Manual, whenever feasible, an interla-
problems in running the test method, clarifies errors, and points
boratory study must be conducted. This procedure will provide
out possible environmental conditions, which may adversely
specific estimates of variation anticipated when using the test
affect the test method or point out need for tightening require-
method.
ments. The ruggedness test can assist in locating ways of
reducing variability in the test method. 5.4.3 Protocol for the ILS, Practice E691, provides a guide
for developing the ILS for the test method. A first step is the
5.3.5 Preliminary Estimates of Precision—From the various
studies conducted in accordance with 5.3.2 – 5.3.4, preliminary writing of an ILS Protocol, which will set out what needs to be
E1488 − 12 (2023)
done before the test specimens (or test materials) are distrib- 6.3.1 Control charting, ruggedness tests, and variability
uted to the participating laboratories. analyses will be useful, especially if they have not previously
been conducted. Such tests may provide better information
5.4.4 Precision Statements—Using the estimates of varia-
tion obtained in the interlaboratory test, one may prepare about variation and necessary tolerances than has previously
been available.
precision statements using Practices E691 and E177 or equiva-
lent procedures. 6.3.2 If precision estimates have not been established
through an actual interlaboratory test program, then such a
5.5 Monitoring Phase:
program should be initiated.
5.5.1 After a test method is approved and in use it is
important to ensure that the published precision and bias
7. Data and Sampling
statistics for the test method remain achievable and consistent
7.1 Sample Determination:
over time or amongst different groups conducting the tests.
7.1.1 The sampling section of a standard should indicate
5.5.2 Monitoring Within a Single Location—It is important
...
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Contact ASTM International (www.astm.org) for the latest information
Designation: E1488 − 12 (Reapproved 2023) An American National Standard
Standard Guide for
Statistical Procedures to Use in Developing and Applying
Test Methods
This standard is issued under the fixed designation E1488; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope E2655 Guide for Reporting Uncertainty of Test Results and
Use of the Term Measurement Uncertainty in ASTM Test
1.1 This guide identifies statistical procedures for use in
Methods
developing new test methods or revising or evaluating existing
test methods, or both.
3. Terminology
1.2 This guide also cites statistical procedures especially
3.1 Definitions—For a more extensive list of terms in E11
useful in the application of test methods.
standards, see Terminology E456.
1.3 This international standard was developed in accor-
3.1.1 bias, n—the difference between the expectation of the
dance with internationally recognized principles on standard-
test results and an accepted reference value. E177
ization established in the Decision on Principles for the
3.1.1.1 Discussion—Statistical procedures include the sam-
Development of International Standards, Guides and Recom-
pling considerations or the experiment design for the collection
mendations issued by the World Trade Organization Technical
of data, or both, and the numerical and graphical approaches to
Barriers to Trade (TBT) Committee.
summarize and analyze the collected data.
2. Referenced Documents
3.1.2 coeffıcient of variation, CV, n—for a nonnegative
characteristic, the ratio of the standard deviation to the mean
2.1 ASTM Standards:
for a population or sample. E2586
E177 Practice for Use of the Terms Precision and Bias in
ASTM Test Methods
3.1.3 component of variance, n—a part of a total variance
E178 Practice for Dealing With Outlying Observations
identified with a specified source of variability.
E456 Terminology Relating to Quality and Statistics
3.1.4 control chart, n—chart on which are plotted a statis-
E691 Practice for Conducting an Interlaboratory Study to
tical measure of a subgroup versus time of sampling along with
Determine the Precision of a Test Method
limits based on the statistical distribution of that measure so as
E1169 Practice for Conducting Ruggedness Tests
to indicate how much common, or chance, cause variation is
E1402 Guide for Sampling Design
inherent in the process or product. E2587
E2282 Guide for Defining the Test Result of a Test Method
E2489 Practice for Statistical Analysis of One-Sample and 3.1.5 observation, n—the process of obtaining information
regarding the presence or absence of an attribute of a test
Two-Sample Interlaboratory Proficiency Testing Programs
E2554 Practice for Estimating and Monitoring the Uncer- specimen, or of making a reading on a characteristic or
dimension of a test specimen. E2282
tainty of Test Results of a Test Method Using Control
Chart Techniques
3.1.6 observed value, n—the value obtained by making an
E2586 Practice for Calculating and Using Basic Statistics
observation. E2282
E2587 Practice for Use of Control Charts in Statistical
3.1.7 precision, n—the closeness of agreement between
Process Control
independent test results obtained under stipulated conditions.
E177
This guide is under the jurisdiction of ASTM Committee E11 on Quality and
3.1.8 proficiency testing, n—determination of laboratory
Statistics and is the direct responsibility of Subcommittee E11.20 on Test Method
testing performance by means of interlaboratory comparisons.
Evaluation and Quality Control.
E2489
Current edition approved April 1, 2023. Published April 2023. Originally
approved in 1992. Last previous edition approved in 2018 as E1488 – 12 (2018).
3.1.9 repeatability, n—precision under repeatability condi-
DOI: 10.1520/E1488-12R23.
tions. E177
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
3.1.10 repeatability conditions, n—conditions where inde-
Standards volume information, refer to the standard’s Document Summary page on
the ASTM website. pendent test results are obtained with the same method on
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E1488 − 12 (2023)
identical test items in the same laboratory by the same operator 4.1.1 Assessments of a new or existing test method gener-
using the same equipment within short intervals of time. E177 ally involve statistical planning and analysis. This standard
recommends what approaches may be taken and indicates
3.1.11 repeatability limit r, n—the value below which the
which standards may be used to perform such assessments.
absolute difference between two individual test results obtained
under repeatability conditions may be expected to occur with a
4.2 This standard introduces a series of phases which are
probability of approximately 0.95 (95 %). E177
recommended to be considered during the life cycle of a test
method as depicted in Fig. 1. These begin with a design phase
3.1.12 repeatability standard deviation, s , n—the standard
r
where the standard is initially prepared. A development phase
deviation of test results obtained under repeatability condi-
involves a variety of experiments that allow further refinement
tions. E177
and understanding of how the test method performs within a
3.1.13 reproducibility, n—precision under reproducibility
laboratory. In an evaluation phase the test method is then
conditions. E177
examined by way of interlaboratory studies resulting in preci-
3.1.14 reproducibility conditions, n—conditions where test
sion and bias statistics which are published in the standard.
results are obtained with the same method on identical test
Finally, the test method is subject to a monitoring phase.
items in different laboratories with different operators using
4.3 All ASTM test methods are required to include state-
different equipment. E177
ments on precision and bias.
3.1.15 reproducibility limit, R, n—the value below which
4.4 Since ASTM began to require all test methods to have
the absolute difference between two test results obtained under
precision and bias statements that are based on interlaboratory
reproducibility conditions may be expected to occur with a
test methods, there has been increased concern regarding what
probability of approximately 0.95 (95 %). E177
statistical experiments and procedures to use during the devel-
3.1.16 reproducibility standard deviation, s , n—the stan-
R
opment of the test methods. Although there exists a wide range
dard deviation of test results obtained under reproducibility
of statistical procedures, there is a small group of generally
conditions. E177
accepted techniques that are beneficial to follow. This guide is
3.1.17 ruggedness, n—insensitivity of a test method to
designed to provide a brief overview of these procedures and to
departures from specified test or environmental conditions.
suggest an appropriate sequence of carrying out these proce-
E1169
dures.
3.1.18 ruggedness test, n—a planned experiment in which
4.5 Statistical procedures often result in interpretations that
environmental factors or test conditions are deliberately varied
are not absolutes. Sometimes the information obtained may be
in order to evaluate the effects of such variation. E1169
inadequate or incomplete, which may lead to additional ques-
3.1.19 standard deviation, n—of a population, σ, the square
tions and the need for further experimentation. Information
root of the average or expected value of the squared deviation outside the data is also important in establishing standards and
of a variable from its mean – of a sample x¯, the square root of
in the interpretation of numerical results.
the sum of the squared deviations of the observed values in the
5. Summary of Guide
sample divided by the sample size minus 1. E2586
5.1 Outlined below is a suggested sequence of four phases
3.1.20 state of statistical control, n—process condition
useful in the development of a test method. A flowchart is
when only common causes are operating on the process. E2587
provided in Fig. 1. Such a sequence of analyses may need to be
3.1.21 statistical procedures, n—the organized techniques
modified in specific situations. The assistance of a qualified
and methods used to collect, analyze, and interpret data.
statistician is recommended at each review phase.
3.1.21.1 Discussion—Statistical procedures include the
5.2 Design Phase:
sampling considerations or the experiment design for the
collection of data, or both, and the numerical and graphical 5.2.1 This phase includes the formalization of the scope and
the significance and use sections. It may include determining
approaches to summarize and analyze the collected data.
the purpose and describing a general approach to the test
3.1.22 test determination, n—the value of a characteristic or
method but usually does not involve statistical studies.
dimension of a single test specimen derived from one or more
5.2.2 Studies may be conducted to evaluate the basic
observed values. E2282
performance of the method. The draft test method is prepared
3.1.23 test method, n—a definitive procedure that produces
and sampling requirements and the test result (see Guide
a test result. E2282
E2282) are clearly defined.
3.1.24 test observation, n—see observation. E2282
5.2.3 A flow chart is extremely valuable to identify the
sequence of operations involved in a test method, for example,
3.1.25 test result, n—the value of a characteristic obtained
the sampling steps required to obtain the test specimens,
by carrying out a specified test method. E2282
definition of the test determination, how a test result is to be
4. Significance and Use
computed, and running the tests on the specimens.
4.1 The creation of a standardized test method generally
follows a series of steps from inception to approval and
See the Form and Style Manual for ASTM Standards that specifies, when
ongoing use. In all such stages there are questions of how well
possible, precision statements shall be estimated based on the results of an
the test method performs. interlaboratory test program.
E1488 − 12 (2023)
FIG. 1 Sequence of Steps
5.3 Development Phase: estimates of repeatability standard deviations should be devel-
5.3.1 The test method is examined for such concerns as its oped and published in this test method. Until an interlaboratory
stability, ruggedness, statistical control and the contributions to
study is performed, these estimates generally are considered to
variability. The completion of this phase should result in
be provisional. Information on how a lab should develop
preliminary estimates of precision and the identification and
uncertainty estimates should also be provided.
suggested ways to estimate potential contributors to uncer-
5.3.6 Statistical Control—A test method must show capa-
tainty.
bility of performing in a consistent way over time. The use of
5.3.2 Evaluation of Short Term Control of Test Method—A 3
control charts (see Guide E2655) to monitor a proposed, or
test method must exhibit an ability to provide consistent results
existing, test method over time is one recommended way to
at least over short time periods. Preliminary studies or a pilot
examine the controllability or stability of a test method. This
test should be conducted to evaluate the short term stability of
statistical control should be demonstrated in one or two
the test method. A small series of repeated tests should be
laboratories using homogeneous material (test specimen).
conducted.
5.3.3 Analysis of Variability—Statistically designed experi-
5.4 Evaluation Phase:
ments conducted in one or two laboratories can be used to
5.4.1 The test method is subjected to interlaboratory studies
assess the relative magnitudes of different sources or potential
to provide estimates of within-laboratory repeatability and
contributors to variability of the test results. Such studies can
between-laboratory reproducibility. Additional information is
provide estimates of intermediate measures of precision.
supplied from proficiency studies when conducted.
5.3.4 Ruggedness Test—A ruggedness test (see Practice
5.4.2 Interlaboratory Study (ILS)—In accordance with
E1169) is a statistically designed experiment that helps identify
ASTM Form and Style Manual, whenever feasible, an interla-
problems in running the test method, clarifies errors, and points
boratory study must be conducted. This procedure will provide
out possible environmental conditions, which may adversely
specific estimates of variation anticipated when using the test
affect the test method or point out need for tightening require-
method.
ments. The ruggedness test can assist in locating ways of
5.4.3 Protocol for the ILS, Practice E691, provides a guide
reducing variability in the test method.
5.3.5 Preliminary Estimates of Precision—From the various for developing the ILS for the test method. A first step is the
studies conducted in accordance with 5.3.2 – 5.3.4, preliminary writing of an ILS Protocol, which will set out what needs to be
E1488 − 12 (2023)
done before the test specimens (or test materials) are distrib- 6.3.1 Control charting, ruggedness tests, and variability
uted to the participating laboratories. analyses will be useful, especially if they have not previously
5.4.4 Precision Statements—Using the estimates of varia- been conducted. Such tests may provide better information
about variation and necessary tolerances than has previously
tion obtained in the interlaboratory test, one may prepare
precision statements using Practices E691 and E177 or equiva- been available.
6.3.2 If precision estimates have not been established
lent procedures.
through an actual interlaboratory test program, then such a
5.5 Monitoring Phase:
program should be initiated.
5.5.1 After a test method is approved and in use it is
important to ensure that the published precision and bias
7. Data and Sampling
statistics for the test method remain achievable and consistent
7.1 Sample Determination:
over time or amongst different groups conducting the tests.
7.1.1 The sampling section of a standard should indicate
5.5.2 Monitoring Within a Single Location—It is important
clearly what constitutes the primary sampling unit, how that
for any laboratory or organization that will use a particular test
...
This document is not an ASTM standard and is intended only to provide the user of an ASTM standard an indication of what changes have been made to the previous version. Because
it may not be technically possible to adequately depict all changes accurately, ASTM recommends that users consult prior editions as appropriate. In all cases only the current version
of the standard as published by ASTM is to be considered the official document.
Designation: E1488 − 12 (Reapproved 2018) E1488 − 12 (Reapproved 2023)An American National Standard
Standard Guide for
Statistical Procedures to Use in Developing and Applying
Test Methods
This standard is issued under the fixed designation E1488; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope
1.1 This guide identifies statistical procedures for use in developing new test methods or revising or evaluating existing test
methods, or both.
1.2 This guide also cites statistical procedures especially useful in the application of test methods.
1.3 This international standard was developed in accordance with internationally recognized principles on standardization
established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued
by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
2. Referenced Documents
2.1 ASTM Standards:
E177 Practice for Use of the Terms Precision and Bias in ASTM Test Methods
E178 Practice for Dealing With Outlying Observations
E456 Terminology Relating to Quality and Statistics
E691 Practice for Conducting an Interlaboratory Study to Determine the Precision of a Test Method
E1169 Practice for Conducting Ruggedness Tests
E1402 Guide for Sampling Design
E2282 Guide for Defining the Test Result of a Test Method
E2489 Practice for Statistical Analysis of One-Sample and Two-Sample Interlaboratory Proficiency Testing Programs
E2554 Practice for Estimating and Monitoring the Uncertainty of Test Results of a Test Method Using Control Chart Techniques
E2586 Practice for Calculating and Using Basic Statistics
E2587 Practice for Use of Control Charts in Statistical Process Control
E2655 Guide for Reporting Uncertainty of Test Results and Use of the Term Measurement Uncertainty in ASTM Test Methods
3. Terminology
3.1 Definitions—For a more extensive list of terms in E11 standards, see Terminology E456.
3.1.1 bias, n—the difference between the expectation of the test results and an accepted reference value. E177
3.1.1.1 Discussion—
This guide is under the jurisdiction of ASTM Committee E11 on Quality and Statistics and is the direct responsibility of Subcommittee E11.20 on Test Method Evaluation
and Quality Control.
Current edition approved April 1, 2018April 1, 2023. Published May 2018April 2023. Originally approved in 1992. Last previous edition approved in 20122018 as
ɛ1
E1488 – 12 (2018). . DOI: 10.1520/E1488-12R18.10.1520/E1488-12R23.
For referenced ASTM standards, visit the ASTM website, www.astm.org, or contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM Standards
volume information, refer to the standard’s Document Summary page on the ASTM website.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E1488 − 12 (2023)
Statistical procedures include the sampling considerations or the experiment design for the collection of data, or both, and the
numerical and graphical approaches to summarize and analyze the collected data.
3.1.2 coeffıcient of variation, CV, n—for a nonnegative characteristic, the ratio of the standard deviation to the mean for a
population or sample. E2586
3.1.3 component of variance, n—a part of a total variance identified with a specified source of variability.
3.1.4 control chart, n—chart on which are plotted a statistical measure of a subgroup versus time of sampling along with limits
based on the statistical distribution of that measure so as to indicate how much common, or chance, cause variation is inherent in
the process or product. E2587
3.1.5 observation, n—the process of obtaining information regarding the presence or absence of an attribute of a test specimen,
or of making a reading on a characteristic or dimension of a test specimen. E2282
3.1.6 observed value, n—the value obtained by making an observation. E2282
3.1.7 precision, n—the closeness of agreement between independent test results obtained under stipulated conditions. E177
3.1.8 proficiency testing, n—determination of laboratory testing performance by means of interlaboratory comparisons. E2489
3.1.9 repeatability, n—precision under repeatability conditions. E177
3.1.10 repeatability conditions, n—conditions where independent test results are obtained with the same method on identical test
items in the same laboratory by the same operator using the same equipment within short intervals of time. E177
3.1.11 repeatability limit r, n—the value below which the absolute difference between two individual test results obtained under
repeatability conditions may be expected to occur with a probability of approximately 0.95 (95 %). E177
3.1.12 repeatability standard deviation, s , n—the standard deviation of test results obtained under repeatability conditions. E177
r
3.1.13 reproducibility, n—precision under reproducibility conditions. E177
3.1.14 reproducibility conditions, n—conditions where test results are obtained with the same method on identical test items in
different laboratories with different operators using different equipment. E177
3.1.15 reproducibility limit, R, n—the value below which the absolute difference between two test results obtained under
reproducibility conditions may be expected to occur with a probability of approximately 0.95 (95 %). E177
3.1.16 reproducibility standard deviation, s , n—the standard deviation of test results obtained under reproducibility conditions.
R
E177
3.1.17 ruggedness, n—insensitivity of a test method to departures from specified test or environmental conditions. E1169
3.1.18 ruggedness test, n—a planned experiment in which environmental factors or test conditions are deliberately varied in order
to evaluate the effects of such variation. E1169
3.1.19 standard deviation, n—of a population, σ, the square root of the average or expected value of the squared deviation of a
variable from its mean – of a sample x¯, the square root of the sum of the squared deviations of the observed values in the sample
divided by the sample size minus 1. E2586
3.1.20 state of statistical control, n—process condition when only common causes are operating on the process. E2587
E1488 − 12 (2023)
3.1.21 statistical procedures, n—the organized techniques and methods used to collect, analyze, and interpret data.
3.1.21.1 Discussion—
Statistical procedures include the sampling considerations or the experiment design for the collection of data, or both, and the
numerical and graphical approaches to summarize and analyze the collected data.
3.1.22 test determination, n—the value of a characteristic or dimension of a single test specimen derived from one or more
observed values. E2282
3.1.23 test method, n—a definitive procedure that produces a test result. E2282
3.1.24 test observation, n—see observation. E2282
3.1.25 test result, n—the value of a characteristic obtained by carrying out a specified test method. E2282
4. Significance and Use
4.1 The creation of a standardized test method generally follows a series of steps from inception to approval and ongoing use. In
all such stages there are questions of how well the test method performs.
4.1.1 Assessments of a new or existing test method generally involve statistical planning and analysis. This standard recommends
what approaches may be taken and indicates which standards may be used to perform such assessments.
4.2 This standard introduces a series of phases which are recommended to be considered during the life cycle of a test method
as depicted in Fig. 1. These begin with a design phase where the standard is initially prepared. A development phase involves a
FIG. 1 Sequence of Steps
E1488 − 12 (2023)
variety of experiments that allow further refinement and understanding of how the test method performs within a laboratory. In
an evaluation phase the test method is then examined by way of interlaboratory studies resulting in precision and bias
statistics which are published in the standard. Finally, the test method is subject to a monitoring phase.
4.3 All ASTM test methods are required to include statements on precision and bias.
4.4 Since ASTM began to require all test methods to have precision and bias statements that are based on interlaboratory test
methods, there has been increased concern regarding what statistical experiments and procedures to use during the development
of the test methods. Although there exists a wide range of statistical procedures, there is a small group of generally accepted
techniques that are beneficial to follow. This guide is designed to provide a brief overview of these procedures and to suggest an
appropriate sequence of carrying out these procedures.
4.5 Statistical procedures often result in interpretations that are not absolutes. Sometimes the information obtained may be
inadequate or incomplete, which may lead to additional questions and the need for further experimentation. Information outside
the data is also important in establishing standards and in the interpretation of numerical results.
5. Summary of Guide
5.1 Outlined below is a suggested sequence of four phases useful in the development of a test method. A flowchart is provided
in Fig. 1. Such a sequence of analyses may need to be modified in specific situations. The assistance of a qualified statistician is
recommended at each review phase.
5.2 Design Phase:
5.2.1 This phase includes the formalization of the scope and the significance and use sections. It may include determining the
purpose and describing a general approach to the test method but usually does not involve statistical studies.
5.2.2 Studies may be conducted to evaluate the basic performance of the method. The draft test method is prepared and sampling
requirements and the test result (see Guide E2282) are clearly defined.
5.2.3 A flow chart is extremely valuable to identify the sequence of operations involved in a test method, for example, the
sampling steps required to obtain the test specimens, definition of the test determination, how a test result is to be computed, and
running the tests on the specimens.
5.3 Development Phase:
5.3.1 The test method is examined for such concerns as its stability, ruggedness, statistical control and the contributions to
variability. The completion of this phase should result in preliminary estimates of precision and the identification and suggested
ways to estimate potential contributors to uncertainty.
5.3.2 Evaluation of Short Term Control of Test Method—A test method must exhibit an ability to provide consistent results at least
over short time periods. Preliminary studies or a pilot test should be conducted to evaluate the short term stability of the test
method. A small series of repeated tests should be conducted.
5.3.3 Analysis of Variability—Statistically designed experiments conducted in one or two laboratories can be used to assess the
relative magnitudes of different sources or potential contributors to variability of the test results. Such studies can provide estimates
of intermediate measures of precision.
5.3.4 Ruggedness Test—A ruggedness test (see Practice E1169) is a statistically designed experiment that helps identify problems
in running the test method, clarifies errors, and points out possible environmental conditions, which may adversely affect the test
method or point out need for tightening requirements. The ruggedness test can assist in locating ways of reducing variability in
the test method.
5.3.5 Preliminary Estimates of Precision—From the various studies conducted in accordance with 5.3.2 – 5.3.4, preliminary
See the Form and Style Manual for ASTM Standards that specifies, when possible, precision statements shall be estimated based on the results of an interlaboratory test
program.
E1488 − 12 (2023)
estimates of repeatability standard deviations should be developed and published in this test method. Until an interlaboratory study
is performed, these estimates generally are considered to be provisional. Information on how a lab should develop uncertainty
estimates should also be provided.
5.3.6 Statistical Control—A test method must show capability of performing in a consistent way over time. The use of control
charts (see Guide E2655) to monitor a proposed, or existing, test method over time is one recommended way to examine the
controllability or stability of a test method. This statistical control should be demonstrated in one or two laboratories using
homogeneous material (test specimen).
5.4 Evaluation Phase:
5.4.1 The test method is subjected to interlaboratory studies to provide estimates of within-laboratory repeatability and
between-laboratory reproducibility. Additional information is supplied from proficiency studies when conducted.
5.4.2 Interlaboratory Study (ILS)—In accordance with ASTM Form and Style Manual, whenever feasible, an interlaboratory study
must be conducted. This procedure will provide specific estimates of variation anticipated when using the test method.
5.4.3 Protocol for the ILS, Practice E691, provides a guide for developing the ILS for the test method. A first step is the writing
of an ILS Protocol, which will set out what needs to be done before the test specimens (or test materials) are distributed to the
participating laboratories.
5.4.4 Precision Statements—Using the estimates of variation obtained in the interlaboratory test, one may prepare precision
statements using Practices E691 and E177 or equivalent procedures.
5.5 Monitoring Phase:
5.5.1 After a test method is approved and in use it is important to ensure that the published precision and bias statistics for the
test method remain achievable and consistent over time or amongst different groups conducting the tests.
5.5.2 Monitoring Within a Single Location—It is important for any laboratory or organization that will use a particular test method
over time that a means of monitoring to ensure the method results using quality control samples are stable and in control. Regular
evaluation of the uncertainty (Practice E2554) or use of a control ch
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.