ASTM D8146-22

Standard Guide for Evaluating Test Method Capability and Fitness for Use

SIGNIFICANCE AND USE
4.1 This guide provides the developers of test methods with systematic approaches to evaluate the inherent adequacy (capability) of the precision and the adequacy of between-laboratory standardization as determined by interlaboratory testing in accordance with Practice D6300 for a new or modified test method.  
4.1.1 The tools presented in this guide can be used to assess the adequacy of the repeatability and reproducibility determined for a given test method relative to their intended applications. For those test methods with less than optimal precision, the responsible subcommittees would have an opportunity to investigate, suggest improvements in the procedure, and design an interlaboratory study as necessary. It is important to note that non-optimal precision does not necessarily imply that the test method is invalid, but that the test method may be less capable for an intended application.  
4.2 This guide is applicable to test method users to evaluate the relative performance of different test methods that purport to measure the same property as applied to a specific application of the test methods.  
4.3 This guide can be used to evaluate the fitness for use of a test method reproducibility relative to its intended use in specifications.  
4.3.1 When test method results are intended to be compared against specification limits, this guide provides the tools to assess whether the test method’s reproducibility is expected to be fit for use relative to the stated limits.  
4.4 This guide is most effective when used in conjunction with sufficient knowledge of the chemistry, instrumental technology, metrology principles, sample characteristics, and a fundamental understanding of statistical meaning and implications associated with repeatability and reproducibility of the test method being evaluated.  
4.5 This guide does not cover the qualitative and business factors that might also be considered in judging a test method’s capability or fitness for use versus...
SCOPE
1.1 This guide covers techniques for evaluating the statistical capability and fitness for use of standard test methods used for measuring properties of petroleum products, liquid fuels, and lubricants. Specifically, this guide provides strategies for evaluating the capability of a test method to provide a sufficiently precise estimate of the intended parameter versus a given level or value of that parameter and for assessing, with sufficient confidence, the fitness for use of a test method for determining the acceptability of products versus specification, regulatory, or manufacturing limits.  
1.2 The assessment of capability in this guide is applicable to test methods that always yield non-negative numerical results and have a hard, fixed zero lower limit or fixed upper limit (for example, 100 %). Detailed knowledge of a test method and professional judgement may be required in determining the applicability of this guide to certain test methods.  
1.3 In accordance with this guide, the assessment of fitness for use of a test method versus specification, regulatory, or manufacturing limits is generally applicable to test methods that provide numerical results and have applicable reproducibility precision values.  
1.4 This guide is intended for use by test method developers, specification setters, and laboratories selecting test methods to evaluate products for specific purposes or versus specific limits.  
1.5 This guide is not applicable to test methods that measure temperature to determine properties such as, but not limited to, freeze point, pour point, flash point, and distillations. This guide is not intended for use by laboratories for evaluating laboratory test method execution capability, nor is it intended to evaluate manufacturing process capability.  
1.6 The expressions of capability and fitness for use derived from use of this guide should not be the only criteria for selection and use of a given tes...

General Information

• Status: Published
• Publication Date: 31-Oct-2022
• ICS: 75.080 - Petroleum products in general
• Technical Committee: D02 - Petroleum Products, Liquid Fuels, and Lubricants
• Drafting Committee: D02.94 - Coordinating Subcommittee on Quality Assurance and Statistics

Relations

• Refers: ASTM D6300-24 - Standard Practice for Determination of Precision and Bias Data for Use in Test Methods for Petroleum Products, Liquid Fuels, and Lubricants
• Effective Date: 01-Mar-2024
• Refers: ASTM D6300-23a - Standard Practice for Determination of Precision and Bias Data for Use in Test Methods for Petroleum Products, Liquid Fuels, and Lubricants
• Effective Date: 01-Dec-2023
• Refers: ASTM D6299-23a - Standard Practice for Applying Statistical Quality Assurance and Control Charting Techniques to Evaluate Analytical Measurement System Performance
• Effective Date: 01-Dec-2023
• Refers: ASTM D6792-23c - Standard Practice for Quality Management Systems in Petroleum Products, Liquid Fuels, and Lubricants Testing Laboratories
• Effective Date: 01-Nov-2023
• Refers: ASTM D6792-23b - Standard Practice for Quality Management Systems in Petroleum Products, Liquid Fuels, and Lubricants Testing Laboratories
• Effective Date: 01-Oct-2023
• Refers: ASTM D6300-19a - Standard Practice for Determination of Precision and Bias Data for Use in Test Methods for Petroleum Products and Lubricants
• Effective Date: 01-Dec-2019
• Refers: ASTM D6299-17b - Standard Practice for Applying Statistical Quality Assurance and Control Charting Techniques to Evaluate Analytical Measurement System Performance
• Effective Date: 15-Dec-2017
• Refers: ASTM D6299-17a - Standard Practice for Applying Statistical Quality Assurance and Control Charting Techniques to Evaluate Analytical Measurement System Performance
• Effective Date: 15-Nov-2017
• Refers: ASTM D6299-17 - Standard Practice for Applying Statistical Quality Assurance and Control Charting Techniques to Evaluate Analytical Measurement System Performance
• Effective Date: 01-Jan-2017
• Refers: ASTM D6300-16 - Standard Practice for Determination of Precision and Bias Data for Use in Test Methods for Petroleum Products and Lubricants
• Effective Date: 01-Apr-2016
• Refers: ASTM D6300-15 - Standard Practice for Determination of Precision and Bias Data for Use in Test Methods for Petroleum Products and Lubricants
• Effective Date: 01-Jun-2015
• Refers: ASTM D6300-14ae1 - Standard Practice for Determination of Precision and Bias Data for Use in Test Methods for Petroleum Products and Lubricants
• Effective Date: 01-Jun-2014
• Refers: ASTM D6300-14a - Standard Practice for Determination of Precision and Bias Data for Use in Test Methods for Petroleum Products and Lubricants
• Effective Date: 01-Jun-2014
• Refers: ASTM D6300-14 - Standard Practice for Determination of Precision and Bias Data for Use in Test Methods for Petroleum Products and Lubricants
• Effective Date: 01-May-2014
• Refers: ASTM D6300-13a - Standard Practice for Determination of Precision and Bias Data for Use in Test Methods for Petroleum Products and Lubricants
• Effective Date: 01-Dec-2013

Overview

ASTM D8146-22: Standard Guide for Evaluating Test Method Capability and Fitness for Use offers a systematic approach for developers and users of test methods to assess the statistical capability and overall fitness of test methods, especially those used for measuring properties of petroleum products, liquid fuels, and lubricants. The guide provides practical strategies for evaluating whether a test method is sufficiently precise and reliable for its intended application, particularly in contexts where test results must conform to specific regulatory, specification, or manufacturing limits.

Key Topics

• Test Method Capability: Methods for evaluating the statistical precision of a test method by calculating Analytical Performance Values (APV), Precision Ratio (PR), and Test Performance Index (TPI). These metrics compare repeatability, reproducibility, and laboratory performance to expected standards.
• Fitness for Use: Techniques for assessing if a test method’s reproducibility aligns with specification or regulatory limits, ensuring the method's results can reliably determine product acceptability.
• Applicability: The guide applies to test methods yielding non-negative numerical results and having fixed upper or lower limits, typical for many petroleum and fuel analyses.
• Interpretation of Results: Recommendations on comparing method metrics to application requirements, identifying when further test method improvement or alternative methods may be needed.
• Limitations: The guide does not cover temperature-based methods (e.g., freeze point, flash point) or laboratory execution capability; it focuses on test method design and selection.

Applications

ASTM D8146-22 is valuable for:

• Test Method Development: Assisting developers in systematically evaluating new or modified methods for precision and suitability across intended application ranges.
• Quality Assurance and Compliance: Helping laboratories and specification setters confirm that test methods are capable and fit for determining product conformity to standards.
• Comparative Assessments: Enabling users to evaluate and select the most effective analytical methods among alternatives measuring the same property, by comparing their statistical performance indices.
• Specification and Limit Setting: Informing the establishment or revision of product specifications in line with realistic measurement capability, improving decision confidence for specification conformance.
• Continuous Improvement: Providing a basis for ongoing review of test methods, supporting method refinement, and identifying needs for broader interlaboratory studies.

Related Standards

ASTM D8146-22 references several key international and ASTM standards critical for laboratories and test method developers:

• ASTM D3244: Practice for Utilization of Test Data to Determine Conformance with Specifications.
• ASTM D6259: Practice for Determination of a Pooled Limit of Quantitation for a Test Method.
• ASTM D6299: Practice for Applying Statistical Quality Assurance and Control Charting.
• ASTM D6300: Determination of Precision and Bias Data for Test Methods in Petroleum Products.
• ASTM D6792: Practice for Quality Management in Petroleum Testing Laboratories.
• ISO 4259: Petroleum products-Determination and application of precision data in relation to methods of test.

Practical Value

Implementing ASTM D8146-22 ensures that test methods used in petroleum, fuels, and lubricants testing are statistically sound and fit-for-purpose, leading to:

• Improved confidence in product acceptance decisions
• Enhanced comparability and reliability across laboratories
• Data-driven support for method selection, development, and improvement
• Reduced risk of non-conformity or misinterpretation of test results relative to specification limits

By standardizing the evaluation of test method capability and fitness for use, ASTM D8146-22 supports quality assurance, regulatory compliance, and technical excellence in laboratory testing environments.

Standards Content (Sample)

This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation: D8146 − 22 An American National Standard
Standard Guide for
Evaluating Test Method Capability and Fitness for Use
This standard is issued under the fixed designation D8146; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope* to complete an analysis, typical product results relative to the
regulatory or manufacturing specification limit, closeness of
1.1 This guide covers techniques for evaluating the statisti-
the results to zero, and cost factors may contribute more
cal capability and fitness for use of standard test methods used
significantly to the decision to use a given test method for a
for measuring properties of petroleum products, liquid fuels,
specific application.
and lubricants. Specifically, this guide provides strategies for
evaluating the capability of a test method to provide a
1.7 This guide draws on statistical approaches covered in
sufficiently precise estimate of the intended parameter versus a
other standards supporting petroleum products, liquid fuels,
given level or value of that parameter and for assessing, with
and lubricants, including Practices D3244, D6259, D6299,
sufficient confidence, the fitness for use of a test method for
D6300, D6792 and ISO 4259.
determining the acceptability of products versus specification,
1.8 This standard does not purport to address all of the
regulatory, or manufacturing limits.
safety concerns, if any, associated with its use. It is the
1.2 The assessment of capability in this guide is applicable
responsibility of the user of this standard to establish appro-
to test methods that always yield non-negative numerical
priate safety, health, and environmental practices and deter-
results and have a hard, fixed zero lower limit or fixed upper
mine the applicability of regulatory limitations prior to use.
limit (for example, 100%). Detailed knowledge of a test
1.9 This international standard was developed in accor-
method and professional judgement may be required in deter-
dance with internationally recognized principles on standard-
mining the applicability of this guide to certain test methods.
ization established in the Decision on Principles for the
1.3 In accordance with this guide, the assessment of fitness
Development of International Standards, Guides and Recom-
for use of a test method versus specification, regulatory, or
mendations issued by the World Trade Organization Technical
manufacturing limits is generally applicable to test methods
Barriers to Trade (TBT) Committee.
that provide numerical results and have applicable reproduc-
ibility precision values.
2. Referenced Documents
1.4 This guide is intended for use by test method
2.1 ASTM Standards:
developers, specification setters, and laboratories selecting test
D3244Practice for Utilization of Test Data to Determine
methods to evaluate products for specific purposes or versus
Conformance with Specifications
specific limits.
D6259Practice for Determination of a Pooled Limit of
1.5 Thisguideisnotapplicabletotestmethodsthatmeasure
Quantitation for a Test Method
temperaturetodeterminepropertiessuchas,butnotlimitedto,
D6299Practice for Applying Statistical Quality Assurance
freeze point, pour point, flash point, and distillations. This
and Control Charting Techniques to Evaluate Analytical
guide is not intended for use by laboratories for evaluating
Measurement System Performance
laboratory test method execution capability, nor is it intended
D6300Practice for Determination of Precision and Bias
to evaluate manufacturing process capability.
Data for Use in Test Methods for Petroleum Products,
1.6 Theexpressionsofcapabilityandfitnessforusederived
Liquid Fuels, and Lubricants
from use of this guide should not be the only criteria for
D6792Practice for Quality Management Systems in Petro-
selection and use of a given test method. Other factors such as
leum Products, Liquid Fuels, and Lubricants Testing
thelaboratory’sexperiencewiththetestmethod,lengthoftime
Laboratories
This guide is under the jurisdiction of ASTM Committee D02 on Petroleum
Products, Liquid Fuels, and Lubricants and is the direct responsibility of Subcom-
mittee D02.94 on Coordinating Subcommittee on QualityAssurance and Statistics. For referenced ASTM standards, visit the ASTM website, www.astm.org, or
Current edition approved Nov. 1, 2022. Published November 2022. Originally contact ASTM Customer Service at [e-mail]. For Annual Book of ASTM
approved in 2018. Last previous edition approved in 2018 as D8146–18. DOI: Standards volume information, refer to the standard’s Document Summary page on
10.1520/D8146-22. the ASTM website.
*A Summary of Changes section appears at the end of this standard
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
D8146 − 22
2.2 ISO Standard: 3.1.6 site precision (R'), n—the value below which the
ISO 4259Petroleum products—Determination and applica- absolutedifferencebetweentwoindividualtestresultsobtained
tion of precision data in relation to methods of test under site precision conditions may be expected to occur with
2.3 ASTM Standards referenced only in Appendix X2, a probability of approximately 0.95 (95%). It is calculated as
Worked Examples, are listed in X2.3. 2.77 times the standard deviation (σ ) of results obtained
R'
under site precision conditions. D6299
3. Terminology
3.1.6.1 Discussion—Site precision is often estimated in the
setup of a quality control chart, and the TPI can be used to
3.1 Definitions:
evaluatetheperformanceofthetestmethodinthelaboratoryas
3.1.1 precision ratio (PR), n—an estimate of relative mag-
described in D6792.
nitude of repeatability and reproducibility. The PR for a given
standard test method can provide information on the relative
3.1.7 site precision conditions, n—for a single analytical
significance between variation caused by different operators
measurement system, conditions under which test results are
and laboratories compared to a single operator in a laboratory
obtained by one or more operators in a single site location
performing the standard test method. D6792
practicing the same test method on a single measurement
3.1.1.1 Discussion—The PR for a published test method
system, using test specimens taken at random from the same
estimatestheinfluencethatnon-site-specificvariationshaveon
sampleofmaterialoveranextendedperiodoftimespanningat
the published precision; the PR metric essentially judges the
least a 20 day interval.
adequacy of between-laboratory standardization, or the agree-
3.1.7.1 Discussion—Foranin-depthdiscussion,seePractice
ment between laboratories relative to within-lab repeatability
D6299. D6299
precision.
3.1.8 testperformanceindex(TPI),n—anapproximatemea-
3.1.2 repeatability (r), n—thequantitativeexpressionforthe
sure of a laboratory’s testing capability, defined as the ratio of
random error associated with the difference between two
test method reproducibility to site precision. D6792
independent results obtained under repeatability conditions
3.1.8.1 Discussion—The term capability as used in this
thatwouldbeexceededwithanapproximateprobabilityof5%
definition refers to ability of the laboratory performing the test
(1 case in 20 in the long run) in the normal and correct
method relative to the published reproducibility and not to the
operation of the test method. D6300
capability of the test method with respect to application at a
3.1.2.1 Discussion—This definition is applicable to preci-
given level or value.
sion values obtained following the practices described in
3.1.8.2 Discussion—The TPI term is similar to precision
D6300. Repeatability determined using different or modified
ration (PR = R/r), but the use of site precision instead of
practices may not have the same statistical significance as that
repeatability provides a direct evaluation of the laboratory’s
inferred when following D6300.
performance of the test method versus reproducibility. Since
3.1.3 repeatability conditions, n—conditions where inde-
there is a general expectation thatR>R'>r, then generally it
pendent test results are obtained with the same method on
would be expected that TPI < PR.
identicaltestitemsinthesamelaboratorybythesameoperator
3.2 Definitions of Terms Specific to This Standard:
using the same equipment within short intervals of time.
3.2.1 analytical performance value (APV), n—a metric that
D6300
assesses the precision of a test method relative to applicable
3.1.4 reproducibility (R), n—a quantitative expression for
specification limits. APV is the ratio of the repeatability or
the random error associated with the difference between two
reproducibility of a test method to specific limits, expressed as
independent results obtained under reproducibility conditions
a percent.
would be exceeded with an approximate probability of5%(1
3.2.1.1 Discussion—TheAPVis the percent relative repeat-
case in 20 in the long run) in the normal and correct operation
ability or reproducibility.
of the test method. D6300
3.1.4.1 Discussion—This definition is applicable to preci-
3.2.2 capability, n—as applied to this guide, an estimate of
sion values obtained following the practices described in
the ability of a test method to provide a precise estimate of the
D6300. Repeatability determined using different or modified
intended parameter at a given level or value of that parameter.
practices may not have the same statistical significance as that
3.2.2.1 Discussion—APV is the metric used to express
inferred when following D6300.
capability.
3.1.5 reproducibility conditions, n—conditions where inde-
3.2.3 fit for use, n—asappliedtothisguide,anexpressionof
pendent test results are obtained with the same method on
the adequacy of a test method to provide a precise estimate of
identical test items in different laboratories with different
theintendedparameterwithadesiredlevelofconfidenceatthe
operators using different equipment. Note that different labo-
level of a corresponding specification, regulatory, or manufac-
ratory by necessity means a different operator, different
turing limit to support compliance decision.
equipment, and different location and under different supervi-
3.2.3.1 Discussion—Fit for use is the effectiveness of an
sory control. D6300
analytical measurement system to adequately meet the preci-
sion requirements for the intended application relative to
specific limits.
Available fromAmerican National Standards Institute (ANSI), 25 W. 43rd St.,
4th Floor, New York, NY 10036, http://www.ansi.org. 3.2.3.2 Discussion—Also known as fitness for use.
D8146 − 22
4. Significance and Use 5.2.2 From a capability perspective, numerically smaller
APVare preferred versus larger ones.TheAPVvalue depends
4.1 This guide provides the developers of test methods with
on the intended use of the test method.AsAPVdecreases (r or
systematic approaches to evaluate the inherent adequacy (ca-
R decrease relative to the target level) the test method would
pability) of the precision and the adequacy of between-
appear to be more capable for the intended application.
laboratory standardization as determined by interlaboratory
Conversely, as theAPV increases (r or R increase to approach
testing in accordance with Practice D6300 for a new or
or exceed the target level) the test method would appear to be
modified test method.
less capable for the intended application.
4.1.1 Thetoolspresentedinthisguidecanbeusedtoassess
5.2.2.1 It is up to the user to determine at what point higher
the adequacy of the repeatability and reproducibility deter-
APV values indicate a potential issue with the use of the test
mined for a given test method relative to their intended
method for the intended application.
applications. For those test methods with less than optimal
5.2.2.2 Other factors such as availability of alternative test
precision, the responsible subcommittees would have an op-
methods, proximity of the target to zero or to a detection limit,
portunity to investigate, suggest improvements in the
critical or non-critical nature of specification parameter, or
procedure, and design an interlaboratory study as necessary. It
other factors may influence this determination.
is important to note that non-optimal precision does not
5.2.3 It is important to evaluate APV and APV over the
r R
necessarily imply that the test method is invalid, but that the
entire range of application of the test method. The precision
test method may be less capable for an intended application.
performance of some test methods tends to degrade when used
4.2 This guide is applicable to test method users to evaluate
at or close to their lower limits. (See Appendix X2.)
the relative performance of different test methods that purport
5.2.4 It is useful to compare theAPV andAPV of new or
r R
to measure the same property as applied to a specific applica-
modified test methods to that for existing similar test methods
tion of the test methods.
todeterminethesignificanceofanyimprovementsinprecision
relative to the intended applications.
4.3 This guide can be used to evaluate the fitness for use of
a test method reproducibility relative to its intended use in
5.3 APVr at Lower Limits:
specifications.
5.3.1 When evaluated at the lower limit of the test method,
4.3.1 Whentestmethodresultsareintendedtobecompared
theAPV metricassessesthecapabilityofthetestmethodtobe
r
against specification limits, this guide provides the tools to
repeatedinanindividuallaboratoryrelativetothatlowerlevel.
assess whether the test method’s reproducibility is expected to
As stated above, when test methods have r values that are less
be fit for use relative to the stated limits.
than or much less than the target level (that is, APV is low),
r
then the assessment of capability is more favorable. Examples
4.4 This guide is most effective when used in conjunction
are shown in Appendix X2.
with sufficient knowledge of the chemistry, instrumental
5.3.2 It is generally expected thatAPV should be less than
technology, metrology principles, sample characteristics, and a
r
about 28 % for a capable test method at its lowest limit.
fundamental understanding of statistical meaning and implica-
tions associated with repeatability and reproducibility of the
NOTE 1—The 28 % APV limit is related to limit of quantitation
r
test method being evaluated.
determinations as described in D6259.
4.5 This guide does not cover the qualitative and business
5.4 Precision Ratio (PR):
factorsthatmightalsobeconsideredinjudgingatestmethod’s
5.4.1 The Precision Ratio (PR) is the ratio of the test
capability or fitness for use versus a specific application.
method’s reproducibility (R) to its repeatability
...

This document is not an ASTM standard and is intended only to provide the user of an ASTM standard an indication of what changes have been made to the previous version. Because
it may not be technically possible to adequately depict all changes accurately, ASTM recommends that users consult prior editions as appropriate. In all cases only the current version
of the standard as published by ASTM is to be considered the official document.
Designation: D8146 − 18 D8146 − 22 An American National Standard
Standard Guide for
Evaluating Test Method Capability and Fitness for Use
This standard is issued under the fixed designation D8146; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope Scope*
1.1 This guide covers techniques for evaluating the statistical capability and fitness for use of standard test methods used for
measuring properties of petroleum products, liquid fuels, and lubricants. Specifically, this guide provides strategies for evaluating
the capability of a test method to provide a sufficiently precise estimate of the intended parameter versus a given level or value
of that parameter and for assessing, with sufficient confidence, the fitness for use of a test method for determining the acceptability
of products versus specification, regulatory, or manufacturing limits.
1.2 The assessment of capability in this guide is applicable to test methods that always yield non-negative numerical results and
have a hard, fixed zero lower limit or fixed upper limit (for example, 100 %). Detailed knowledge of a test method and professional
judgement may be required in determining the applicability of this guide to certain test methods.
1.3 In accordance with this guide, the assessment of fitness for use of a test method versus specification, regulatory, or
manufacturing limits is generally applicable to test methods that provide numerical results and have applicable reproducibility
precision values.
1.4 This guide is intended for use by test method developers, specification setters, and laboratories selecting test methods to
evaluate products for specific purposes or versus specific limits.
1.5 This guide is not applicable to test methods that measure temperature to determine properties such as, but not limited to, freeze
point, pour point, flash point, and distillations. This guide is not intended for use by laboratories for evaluating laboratory test
method execution capability, nor is it intended to evaluate manufacturing process capability.
1.6 The expressions of capability and fitness for use derived from use of this guide should not be the only criteria for selection
and use of a given test method. Other factors such as the laboratory’s experience with the test method, length of time to complete
an analysis, typical product results relative to the regulatory or manufacturing specification limit, closeness of the results to zero,
and cost factors may contribute more significantly to the decision to use a given test method for a specific application.
1.7 This guide draws on statistical approaches covered in other standards supporting petroleum products, liquid fuels, and
lubricants, including Practices D3244, D6259, D6299, D6300, D6792 and ISO 4259.
1.8 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility
of the user of this standard to establish appropriate safety, health, and environmental practices and determine the applicability of
regulatory limitations prior to use.
This guide is under the jurisdiction of ASTM Committee D02 on Petroleum Products, Liquid Fuels, and Lubricants and is the direct responsibility of Subcommittee
D02.94 on Coordinating Subcommittee on Quality Assurance and Statistics.
Current edition approved Oct. 1, 2018Nov. 1, 2022. Published November 2018November 2022. Originally approved in 2018. Last previous edition approved in 2018 as
D8146 – 18. DOI: 10.1520/D8146-18.10.1520/D8146-22.
*A Summary of Changes section appears at the end of this standard
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
D8146 − 22
1.9 This international standard was developed in accordance with internationally recognized principles on standardization
established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued
by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
2. Referenced Documents
2.1 ASTM Standards:
D3244 Practice for Utilization of Test Data to Determine Conformance with Specifications
D6259 Practice for Determination of a Pooled Limit of Quantitation for a Test Method
D6299 Practice for Applying Statistical Quality Assurance and Control Charting Techniques to Evaluate Analytical Measure-
ment System Performance
D6300 Practice for Determination of Precision and Bias Data for Use in Test Methods for Petroleum Products, Liquid Fuels, and
Lubricants
D6792 Practice for Quality Management Systems in Petroleum Products, Liquid Fuels, and Lubricants Testing Laboratories
2.2 ISO Standard:
ISO 4259 Petroleum products—Determination and application of precision data in relation to methods of test
2.3 ASTM Standards referenced only in Appendix X2, Worked Examples, are listed in X2.3.
3. Terminology
3.1 Definitions:
3.1.1 precision ratio (PR), n—an estimate of relative magnitude of repeatability and reproducibility. The PR for a given standard
test method can provide information on the relative significance between variation caused by different operators and laboratories
compared to a single operator in a laboratory performing the standard test method. D6792
3.1.1.1 Discussion—
The PR for a published test method estimates the influence that non-site-specific variations have on the published precision; the
PR metric essentially judges the adequacy of between-laboratory standardization, or the agreement between laboratories relative
to within-lab repeatability precision.
3.1.2 repeatability (r), n—the quantitative expression for the random error associated with the difference between two independent
results obtained under repeatability conditions that would be exceeded with an approximate probability of 5 % (1 case in 20 in
the long run) in the normal and correct operation of the test method. D6300
3.1.2.1 Discussion—
This definition is applicable to precision values obtained following the practices described in D6300. Repeatability determined
using different or modified practices may not have the same statistical significance as that inferred when following D6300.
3.1.3 repeatability conditions, n—conditions where independent test results are obtained with the same method on identical test
items in the same laboratory by the same operator using the same equipment within short intervals of time. D6300
3.1.4 reproducibility (R), n—a quantitative expression for the random error associated with the difference between two
independent results obtained under reproducibility conditions would be exceeded with an approximate probability of 5 % (1 case
in 20 in the long run) in the normal and correct operation of the test method. D6300
3.1.4.1 Discussion—
This definition is applicable to precision values obtained following the practices described in D6300. Repeatability determined
using different or modified practices may not have the same statistical significance as that inferred when following D6300.
3.1.5 reproducibility conditions, n—conditions where independent test results are obtained with the same method on identical test
items in different laboratories with different operators using different equipment. Note that different laboratory by necessity means
a different operator, different equipment, and different location and under different supervisory control. D6300
3.1.6 site precision (R'), n—the value below which the absolute difference between two individual test results obtained under site
For referenced ASTM standards, visit the ASTM website, www.astm.org, or contact ASTM Customer Service at [e-mail]. For Annual Book of ASTM Standards
volume information, refer to the standard’s Document Summary page on the ASTM website.
Available from American National Standards Institute (ANSI), 25 W. 43rd St., 4th Floor, New York, NY 10036, http://www.ansi.org.
D8146 − 22
precision conditions may be expected to occur with a probability of approximately 0.95 (95 %). It is calculated as 2.77 times the
standard deviation (σ ) of results obtained under site precision conditions. D6299
R'
3.1.6.1 Discussion—
Site precision is often estimated in the setup of a quality control chart, and the TPI can be used to evaluate the performance of
the test method in the laboratory as described in D6792.
3.1.7 site precision conditions, n—for a single analytical measurement system, conditions under which test results are obtained by
one or more operators in a single site location practicing the same test method on a single measurement system, which may
comprise multiple instruments, using test specimens taken at random from the same sample of material over an extended period
of time spanning at least a 15-day 20 day interval.
3.1.7.1 Discussion—
For an in-depth discussion, see Practice D6299. D6299
3.1.8 test performance index (TPI), n—an approximate measure of a laboratory’s testing capability, defined as the ratio of test
method reproducibility to site precision. D6792
3.1.8.1 Discussion—
The term capability as used in this definition refers to ability of the laboratory performing the test method relative to the published
reproducibility and not to the capability of the test method with respect to application at a given level or value.
3.1.8.2 Discussion—
The TPI term is similar to precision ration (PR = R/r), but the use of site precision instead of repeatability provides a direct
evaluation of the laboratory’s performance of the test method versus reproducibility. Since there is a general expectation that R
> R' > r, then generally it would be expected that TPI < PR.
3.2 Definitions of Terms Specific to This Standard:
3.2.1 analytical performance value (APV), n—a metric that assesses the precision of a test method relative to applicable
specification limits. APV is the ratio of the repeatability or reproducibility of a test method to specific limits, expressed as a percent.
3.2.1.1 Discussion—
The APV is the percent relative repeatability or reproducibility.
3.2.2 capability, n—as applied to this guide, an estimate of the ability of a test method to provide a precise estimate of the intended
parameter at a given level or value of that parameter.
3.2.2.1 Discussion—
APV is the metric used to express capability.
3.2.3 fit for use, n—as applied to this guide, an expression of the adequacy of a test method to provide a precise estimate of the
intended parameter with a desired level of confidence at the level of a corresponding specification, regulatory, or manufacturing
limit to support compliance decision.
3.2.3.1 Discussion—
Fit for use is the effectiveness of an analytical measurement system to adequately meet the precision requirements for the intended
application relative to specific limits.
3.2.3.2 Discussion—
Also known as fitness for use.
4. Significance and Use
4.1 This guide provides the developers of test methods with systematic approaches to evaluate the inherent adequacy (capability)
of the precision and the adequacy of between-laboratory standardization as determined by interlaboratory testing in accordance
with Practice D6300 for a new or modified test method.
4.1.1 The tools presented in this guide can be used to assess the adequacy of the repeatability and reproducibility determined for
a given test method relative to their intended applications. For those test methods with less than optimal precision, the responsible
subcommittees would have an opportunity to investigate, suggest improvements in the procedure, and design an interlaboratory
study as necessary. It is important to note that non-optimal precision does not necessarily imply that the test method is invalid, but
that the test method may be less capable for an intended application.
4.2 This guide is applicable to test method users to evaluate the relative performance of different test methods that purport to
measure the same property as applied to a specific application of the test methods.
D8146 − 22
4.3 This guide can be used to evaluate the fitness for use of a test method reproducibility relative to its intended use in
specifications.
4.3.1 When test method results are intended to be compared against specification limits, this guide provides the tools to assess
whether the test method’s reproducibility is expected to be fit for use relative to the stated limits.
4.4 This guide is most effective when used in conjunction with sufficient knowledge of the chemistry, instrumental technology,
metrology principles, sample characteristics, and a fundamental understanding of statistical meaning and implications associated
with repeatability and reproducibility of the test method being evaluated.
4.5 This guide does not cover the qualitative and business factors that might also be considered in judging a test method’s
capability or fitness for use versus a specific application.
5. Test Method Capability
5.1 Test method capability can be evaluated using Analytical Performance Values (APV), Precision Ratio (PR), and Test
Performance Index (TPI).
5.2 APV:
5.2.1 Determine the Analytical Performance Value (APV) by dividing the test method precision, either reproducibility (R) or
repeatability (r), by the test result or target level and multiplying by 100. Express the result to the nearest percent as appropriate.
For test methods that do not have a published R, then a laboratory’s site precision (R') can be used to estimate the APV . Report
R
APV with reference to the target value, for example APV = 20 % (at 50 mg/kg).
R
APV-Reproducibility APV 5~R/X! 3100 (1)
~ !
R
APV-Repeatability APV 5~r/X! 3100 (2)
~ !
r
Where X is the target level, specification, regulatory limit, a test result, or average from multiple test results.
5.2.2 From a capability perspective, numerically smaller APV are preferred versus larger ones. The APV value depends on the
intended use of the test method. As APV decreases (r or R decrease relative to the target level) the test method would appear to
be more capable for the intended application. Conversely, as the APV increases (r or R increase to approach or exceed the target
level) the test method would appear to be less capable for the intended application.
5.2.2.1 It is up to the user to determine at what point higher APV values indicate a potential issue with the use of the test method
for the intended application.
5.2.2.2 Other factors such as availability of alternative test methods, proximity of the target to zero or to a detection limit, critical
or non-critical nature of specification parameter, or other factors may influence this determination.
5.2.3 It is important to evaluate APV and APV over the entire range of application of the test method. The precision performance
r R
of some test methods tends to degrade when used at or close to their lower limits. (See Appendix X2.)
5.2.4 It is useful to compare the APV and APV of new or modified test methods to that for existing similar test
...