ASTM C1215-18

Standard Guide for Preparing and Interpreting Precision and Bias Statements in Test Method Standards Used in the Nuclear Industry

SIGNIFICANCE AND USE
4.1 To describe the uncertainties of a standard test method, precision and bias statements are required.3 The formulation of these statements has been addressed from time to time, and at least two standards practices (Practices E177 and E691) have been issued. The 1986  Compilation of ASTM Standard Definitions  (6)4 devotes several pages to these terms. This guide should not be used in cases where small numbers of test results do not support statistical normality.  
4.2 The intent of this guide is to help analysts prepare and interpret precision and bias statements. It is essential that, when the terms are used, their meaning should be clear and easily understood.  
4.3 Appendix X1 provides the theoretical foundation for precision and bias concepts and Practice E691 addresses the problem of sources of variation. To illustrate the interplay between sources of variation and formulation of precision and bias statements, a hypothetical data set is analyzed in Appendix X2. This example shows that depending on how the data was collected, different precision and bias statements are possible. Reference to this example will be found throughout this guide.  
4.4 There has been much debate inside and outside the statistical community on the exact meaning of some statistical terms. Thus, following a number of the terms in Section 3 is a list of several ways in which that term has been used. This listing is not meant to indicate that these meanings are equivalent or equally acceptable. The purpose here is more to encourage clear definition of terms used than to take sides. For example, use of the term systematic error is discouraged by some. If it is to be used, the reader should be told exactly what is meant in the particular circumstance.  
4.5 This guide is intended as an aid to understanding the statistical concepts used in precision and bias statements. There is no intention that this be a self-contained introduction to statistics. Since many analysts have no formal sta...
SCOPE
1.1 This guide covers terminology useful for the preparation and interpretation of precision and bias statements. This guide does not recommend a specific error model or statistical method. It provides awareness of terminology and approaches and options to use for precision and bias statements.  
1.2 In formulating precision and bias statements, it is important to understand the statistical concepts involved and to identify the major sources of variation that affect results. Appendix X1 provides a brief summary of these concepts.  
1.3 To illustrate the statistical concepts and to demonstrate some sources of variation, a hypothetical data set has been analyzed in Appendix X2. Reference to this example is made throughout this guide.  
1.4 It is difficult and at times impossible to ship nuclear materials for interlaboratory testing. Thus, precision statements for test methods relating to nuclear materials will ordinarily reflect only within-laboratory variation.  
1.5 No units are used in this statistical analysis.  
1.6 This guide does not involve the use of materials, operations, or equipment and does not address any risk associated.  
1.7 This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.

General Information

• Status: Published
• Publication Date: 30-Jun-2018
• ICS: 27.120.01 - Nuclear energy in general
• Technical Committee: C26 - Nuclear Fuel Cycle
• Drafting Committee: C26.08 - Quality Assurance, Statistical Applications, and Reference Materials

Relations

• Replaces: ASTM C1215-92(2012)e1 - Standard Guide for Preparing and Interpreting Precision and Bias Statements in Test Method Standards Used in the Nuclear Industry
• Effective Date: 01-Jul-2018
• Refers: ASTM C859-24 - Standard Terminology Relating to Nuclear Materials
• Effective Date: 01-Jan-2024
• Refers: ASTM C859-14a - Standard Terminology Relating to Nuclear Materials
• Effective Date: 15-Jun-2014
• Refers: ASTM E177-14 - Standard Practice for Use of the Terms Precision and Bias in ASTM Test Methods
• Effective Date: 01-May-2014
• Refers: ASTM C859-14 - Standard Terminology Relating to Nuclear Materials
• Effective Date: 15-Jan-2014
• Refers: ASTM C859-13a - Standard Terminology Relating to Nuclear Materials
• Effective Date: 01-Jun-2013
• Refers: ASTM E177-13 - Standard Practice for Use of the Terms Precision and Bias in ASTM Test Methods
• Effective Date: 01-May-2013
• Refers: ASTM C859-13 - Standard Terminology Relating to Nuclear Materials
• Effective Date: 01-May-2013
• Refers: ASTM E691-13 - Standard Practice for Conducting an Interlaboratory Study to Determine the Precision of a Test Method
• Effective Date: 01-May-2013
• Refers: ASTM E691-11 - Standard Practice for Conducting an Interlaboratory Study to Determine the Precision of a Test Method
• Effective Date: 01-Nov-2011
• Refers: ASTM C859-10b - Standard Terminology Relating to Nuclear Materials
• Effective Date: 01-Nov-2010
• Refers: ASTM E177-10 - Standard Practice for Use of the Terms Precision and Bias in ASTM Test Methods
• Effective Date: 01-Oct-2010
• Refers: ASTM C859-10a - Standard Terminology Relating to Nuclear Materials
• Effective Date: 01-Aug-2010
• Refers: ASTM C859-10 - Standard Terminology Relating to Nuclear Materials
• Effective Date: 01-Feb-2010
• Refers: ASTM C859-09 - Standard Terminology Relating to Nuclear Materials
• Effective Date: 15-Feb-2009

Overview

ASTM C1215-18: Standard Guide for Preparing and Interpreting Precision and Bias Statements in Test Method Standards Used in the Nuclear Industry provides a comprehensive framework for describing uncertainties in nuclear test methods. This guide focuses on the important concepts of precision and bias, helping analysts understand, prepare, and interpret statements about measurement accuracy and consistency in the context of nuclear materials testing. It aims to ensure that any reported values clearly convey their reliability and limitations, thereby improving quality assurance and facilitating compliance with international standardization principles.

Key Topics

• Precision and Bias Statements: The guide explains how to prepare precise and meaningful statements regarding the repeatability (precision) and systematic errors (bias) in test methods. Such statements are fundamental for transparency and reproducibility in nuclear industry testing.
• Statistical Terminology: ASTM C1215-18 defines key terms such as accuracy, analysis of variance (ANOVA), bias, confidence interval, error model, repeatability, reproducibility, standard deviation, and uncertainty. This shared terminology is critical for consistent communication of results.
• Sources of Variation: The document highlights typical sources of variation (e.g., analyst, instrumentation, time, laboratory conditions) that can impact measurement results. Understanding these sources is vital for accurate evaluation and reporting.
• Error Models: The guide discusses additive and multiplicative error models and their relevance to expressing how errors affect measurement outcomes.
• Application of Statistical Methods: While ASTM C1215-18 does not prescribe specific statistical approaches, it provides guidance on what information to include and when to consult statistical references or experts.

Applications

ASTM C1215-18 is essential for:

• Developing Test Method Standards: It assists committees and analysts in formulating precision and bias statements when drafting or revising ASTM test method standards for the nuclear sector.
• Quality Assurance in Nuclear Laboratories: By ensuring clarity and accuracy in test documentation, laboratories can demonstrate compliance with quality management systems, regulatory requirements, and international nuclear materials protocols.
• Interpreting Test Results: Nuclear industry professionals use the guide to accurately interpret data, assess test repeatability, and identify sources of measurement uncertainty essential for safety and reliability assessments.
• Training and Reference: The guide serves as an educational resource for analysts, laboratory personnel, and stakeholders who require a solid understanding of statistical terminology and reporting practices tailored to nuclear testing environments.

Related Standards

ASTM C1215-18 references and complements several other standards critical to the nuclear industry:

• ASTM E177 - Practice for Use of the Terms Precision and Bias in ASTM Test Methods: Provides detailed definitions and guidance for using precision and bias statements across various scientific testing fields.
• ASTM E691 - Practice for Conducting an Interlaboratory Study to Determine the Precision of a Test Method: Outlines requirements for organizing and analyzing multi-laboratory studies to assess test method precision.
• ASTM C859 - Terminology Relating to Nuclear Materials: Offers essential definitions for terms used in nuclear materials management and testing.
• ANSI N15.5 - Statistical Terminology and Notation for Nuclear Materials Management: Establishes standard statistical terminology for use in the nuclear materials sector.

Practical Value

Implementing the recommendations in ASTM C1215-18 helps organizations:

• Ensure reliable, transparent reporting of nuclear test data
• Facilitate interlaboratory comparisons and collaborative research
• Improve compliance with international trade and safety requirements
• Enhance the credibility and reproducibility of nuclear measurement results

Utilizing standard definitions and approaches for precision and bias statements strengthens communication and quality control within the nuclear industry, supporting both safety and operational excellence.

Standards Content (Sample)

This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation: C1215 − 18
Standard Guide for
Preparing and Interpreting Precision and Bias Statements in
Test Method Standards Used in the Nuclear Industry
This standard is issued under the fixed designation C1215; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
INTRODUCTION
Test method standards are required to contain precision and bias statements. This guide contains a
glossary that explains various terms that often appear in these statements as well as an example
illustrating such statements for a specific set of data. Precision and bias statements are shown to vary
according to the conditions under which the data were collected.This guide emphasizes that the error
model (an algebraic expression that describes how the various sources of variation affect the
measurement) is an important consideration in the formation of precision and bias statements.
1. Scope Development of International Standards, Guides and Recom-
mendations issued by the World Trade Organization Technical
1.1 Thisguidecoversterminologyusefulforthepreparation
Barriers to Trade (TBT) Committee.
and interpretation of precision and bias statements. This guide
does not recommend a specific error model or statistical
2. Referenced Documents
method. It provides awareness of terminology and approaches
2.1 ASTM Standards:
and options to use for precision and bias statements.
C859Terminology Relating to Nuclear Materials
1.2 In formulating precision and bias statements, it is
E177Practice for Use of the Terms Precision and Bias in
importanttounderstandthestatisticalconceptsinvolvedandto
ASTM Test Methods
identify the major sources of variation that affect results.
E691Practice for Conducting an Interlaboratory Study to
Appendix X1 provides a brief summary of these concepts.
Determine the Precision of a Test Method
1.3 To illustrate the statistical concepts and to demonstrate
some sources of variation, a hypothetical data set has been 3. Terminology
analyzed in Appendix X2. Reference to this example is made
3.1 For definitions of terms used in this guide but not
throughout this guide.
defined herein, see Terminology C859.
1.4 It is difficult and at times impossible to ship nuclear
3.2 Terminology for Precision and Bias Statements
materialsforinterlaboratorytesting.Thus,precisionstatements
3.2.1 accuracy (see bias) —(1) bias. (2) the closeness of a
for test methods relating to nuclear materials will ordinarily
measured value to the true value. (3) the closeness of a
reflect only within-laboratory variation.
measured value to an accepted reference or standard value.
1.5 No units are used in this statistical analysis. 3.2.1.1 Discussion—For many investigators, accuracy is
attained only if a procedure is both precise and unbiased (see
1.6 This guide does not involve the use of materials,
bias). Because this blending of precision into accuracy can
operations, or equipment and does not address any risk
resultoccasionallyinincorrectanalysesandunclearstatements
associated.
of results, ASTM requires statement on bias instead of accu-
1.7 This international standard was developed in accor- 3
racy.
dance with internationally recognized principles on standard-
3.2.2 analysis of variance (ANOVA)—the body of statistical
ization established in the Decision on Principles for the
theory,methods,andpracticesinwhichthevariationinasetof
This guide is under the jurisdiction ofASTM Committee C26 on Nuclear Fuel
Cycle and is the direct responsibility of Subcommittee C26.08 on Quality For referenced ASTM standards, visit the ASTM website, www.astm.org, or
Assurance, Statistical Applications, and Reference Materials. contact ASTM Customer Service at [e-mail]. For Annual Book of ASTM
CurrenteditionapprovedJuly1,2018.PublishedJuly2018.Originallyapproved Standards volume information, refer to the standard’s Document Summary page on
ɛ1
in 1992. Last previous edition approved in 2012 as C1215–92 (2012) . DOI: the ASTM website.
10.1520/C1215-18. Refer to Form and Style for ASTM Standards, 8th Ed., 1989, ASTM.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
C1215 − 18
data is partitioned into identifiable sources of variation. mean. Note that the absence of sample size information
Sources of variation may include analysts, instruments, detracts from the usefulness of the confidence interval. If the
samples, and laboratories. To use the analysis of variance, the interval were based on five observations, a second set of five
data collection method must be carefully designed based on a might produce a very different interval. This would not be the
modelthatincludesallthesourcesofvariationofinterest.(See case if 50 observations were taken.
Example, X2.1.1)
3.2.6 confidence level—theprobability,usuallyexpressedas
3.2.3 bias (see accuracy)—a constant positive or negative a percent, that a confidence interval will contain the parameter
deviation of the method average from the correct value or of interest. (See discussion of confidence interval in Appendix
accepted reference value. X3.)
3.2.3.1 Discussion—Bias represents a constant error as op-
3.2.7 error model—an algebraic expression that describes
posed to a random error.
how a measurement is affected by error and other sources of
(a)A method bias can be estimated by the difference (or
variation. The model may or may not include a sampling error
relative difference) between a measured average and an ac-
term.
cepted standard or reference value. The data from which the
3.2.7.1 Discussion—Ameasurement error is an error attrib-
estimateisobtainedshouldbestatisticallyanalyzedtoestablish
utable to the measurement process. The error may affect the
bias in the presence of random error. A thorough bias investi-
measurement in many ways and it is important to correctly
gation of a measurement procedure requires a statistically
model the effect of the error on the measurement.
designed experiment to repeatedly measure, under essentially
(a) Two common models are the additive and the multi-
thesameconditions,asetofstandardsorreferencematerialsof
plicative error models. In the additive model, the errors are
known value that cover the range of application. Bias often
independent of the value of the item being measured. Thus,
varies with the range of application and should be reported
for example, for repeated measurements under identical
accordingly.
conditions, the additive error model might be
(b)In statistical terminology, an estimator is said to be
X 5 µ1b1ε (1)
i i
unbiased if its expected value is equal to the true value of the
parameter being estimated. (See Appendix X1.)
where:
(c)Thebiasofatestmethodisalsocommonlyindicatedby th
X = the result of the i measurement,
i
analytical chemists as percent recovery. A number of repeti-
µ = the true value of the item,
tions of the test method on a reference material are performed,
b = a bias, and
and an average percent recovery is calculated. This average
ε = a random error usually assumed to have a normal
i
provides an estimate of the test method bias, which is multi-
distribution with mean zero and variance σ .
plicative in nature, not additive. (See Appendix X2.)
Inthemultiplicativemodel,theerrorisproportionaltothe
(d)Use of a single test result to estimate bias is strongly
true value.Amultiplicative error model for percent recovery
discouraged because, even if there were no bias, random error
(see bias) might be:
alone would produce a nonzero bias estimate.
X 5 µbε (2)
i i
3.2.4 coeffıcient of variation—see relative standard devia-
tion. and a multiplicative model for a neutron counter mea-
3.2.5 confidence interval—an interval used to bound the surement might be:
value of a population parameter with a specified degree of
X 5 µ1µb1µ·ε (3)
i i
confidence (this is an interval that has different values for
different random samples).
5µ 11b1ε
~ !
i
3.2.5.1 Discussion—When providing a confidence interval,
(b) Clearly, there are many ways in which errors may
analysts should give the number of observations on which the
affect a final measurement. The additive model is fre-
interval is based. The specified degree of confidence is usually
quently assumed and is the basis for many common statis-
90, 95, or 99%. The form of a confidence interval depends on
tical procedures. The form of the model influences how
underlying assumptions and intentions. Usually, confidence
the error components will be estimated and is very
intervals are taken to be symmetric, but that is not necessarily
important, for example, in the determination of measure-
so, as in the case of confidence intervals for variances.
ment uncertainties. Further discussion of models is given
Construction of a symmetric confidence interval for a popula-
in the Example of Appendix X2 and in Appendix X4.
tion mean is discussed in Appendix X3.
3.2.8 precision—a generic concept used to describe the
It is important to realize that a given confidence-interval
dispersion of a set of measured values.
estimate either does or does not contain the population param-
eter.The degree of confidence is actually in the procedure. For 3.2.8.1 Discussion—It is important that some quantitative
example,iftheinterval(9,13)isa90%confidenceintervalfor measure be used to specify precision. A statement such as,
the mean, we are confident that the procedure (take a sample, “The precision is 1.54 g” is useless. Measures frequently used
construct an interval) by which the interval (9, 13) was to express precision are standard deviation, relative standard
constructed will 90% of the time produce an interval that does deviation, variance, repeatability, reproducibility, confidence
indeed contain the mean. Likewise, we are confident that 10% interval, and range. In addition to specifying the measure and
of the time the interval estimate obtained will not contain the the precision, it is important that the number of repeated
C1215 − 18
measurementsuponwhichtheprecisionestimatedisbasedalso
1.96=2s, and the reproducibility limit is defined as 1.96=2s ,
r R
be given. (See Example, Appendix X2.)
where s is the estimated standard deviation associated with
r
(a) It is strongly recommended that a statement on preci-
repeatability, and s is the estimated standard deviation asso-
R
sion of a measurement procedure include the following:
ciated with reproducibility.Thus, if normality can be assumed,
(1)A description of the procedure used to obtain the data,
these limits represent 95% limits for the difference between
(2)The number of repetitions, n, of the measurement twomeasurementstakenundertherespectiveconditions.Inthe
procedure,
reproducibility case, this means that “approximately 95% of
(3) The sample mean and standard deviation of the all pairs of test results from laboratories similar to those in the
measurements, study can be expected to differ in absolute value by less than
(4)The measure of precision being reported,
=
1.96 2s .” It is important to realize that if a particular s is a
R R
(5)The computed value of that measure, and
poor estimate of σ , the 95% figure may be substantially in
R
(6) The applicable range or concentration.
error. For this reason, estimates should be based on adequate
The importance of items (3) and (4) lies in the fact that
sample sizes.
with these a reader may calculate a confidence interval or
3.2.9 propagation of variance—a procedure by which the
relative standard deviation as desired.
mean and variance of a function of one or more random
(b) Precisionissometimesmeasuredbyrepeatabilityand
variables can be expressed in terms of the mean, variance, and
reproducibility (see Practice E177, and Mandel and Laskof
covariances of the individual random variables themselves
(1)).TheANSI andASTM documents differ slightly in their
(Syn. variance propagation, propagation of error).
usagesoftheseterms.ThefollowingisquotedfromKendall
3.2.9.1 Discussion—There are a number of simple exact
and Buckland (2):
formulas and Taylor series approximations which are useful
“In some situations, especially interlaboratory
here (3, 4).
comparisons, precision is defined by employing two addi-
3.2.10 random error—(1) the chance variation encountered
tional concepts: repeatability and reproducibility. The gen-
in all measurement work, characterized by the random occur-
eralsituationgivingrisetothesedistinctionscomesfromthe
rence of deviations from the mean value. (2) an error that
interest in assessing the variability within several groups of
affects each member of a set of data (measurements) in a
measurements and between those groups of measurements.
different manner.
Repeatability, then, refers to the within-group dispersion of
3.2.11 random sample (measurements)—a set of measure-
the measurements, while reproducibility refers to the
ments taken on a single item or on similar items in such a way
between-group dispersion. In interlaboratory comparison
that the measurements are independent and have the same
studies, for example, the investigation seeks to determine
probability distribution.
how well each laboratory can repeat its measurements
3.2.11.1 Discussion—Some authors refer to this as a simple
(repeatability)andhowwellthelaboratoriesagreewitheach
random sample. One must then be careful to distinguish
other (reproducibility). Similar discussions can apply to the
between a simple random sample from a finite population of N
comparison of laboratory technicians’ skills, the study of
items and a simple random sample from an infinite population.
competing types of equipment, and the use of particular
Intheformercase,asimplerandomsampleisasamplechosen
procedures within a laboratory. An essential feature usually
in such a way that all samples of the same size have the same
required, however, is that repeatability and reproducibility
chance of being selected.An example of the latter case occurs
be measured as variances (or standard deviations in certain
when taking measurements. Any value in an interval is
instances), so that both within- and between-group disper-
considered possible and thus the population is conceptually
sions are modeled as a random variable. The statistical tool
infinite. The definition given in 3.2.11 is then the appropriate
usefulfortheanalysisofsuchcomparisonsistheanalysisof
definition. (See representative sample and Appendix X5.)
variance.”
3.2.12 range—the largest minus the smallest of a set of
(c) In Practice E177 it is recommended that the term
numbers.
repeatabilitybereservedfortheintrinsicvariationduesolely
to the measurement procedure, excluding all variation from
3.2.13 relative standard deviation (percent)—the sample
factors such as analyst, time and laboratory and reserving
standard deviation expressed as a percent of the sample mean.
reproducibility for the variation due to all factors including
The %RSD is calculated using the following equation:
laboratory. Repeatability can be measured by the standard
s
deviation, σ,of n consecutive measurements by the same %RSD 5 100 (4)
r 2
x
operator on the same instrument. Reproducibility can be ? ?
measuredbythestandarddeviation, σ ,of mmeasurements,
R
where:
oneobtainedfromeachof mindependentlaboratories.When
s = sample standard deviation and
interlaboratory testing is not practical, the reproducibility
x¯ = sample mean.
conditions should be described.
(d) Two additional terms are recommended in Practice
3.2.13.1 Discussion—Theuseofthe%RSD(orRSD(%))to
E177. These are repeatability limit and reproducibility limit. describe precision implies that the uncertainty is a function of
These are intended to give estimates of how different two the measurement values. An appropriate error model might
measurements can be. The repeatability limit is defined as thenbe X =µ(1+ b+ ε).(SeeExample,AppendixX2.)Some
i i
C1215 − 18
authors use RSD for the ratio, s/|x|, while others call this the where:
coeffıcient of variation. At times authors use RSD to mean
s = standard deviation of the mean of a set of
x¯
%RSD. Thus, it is important to determine which meaning is
measurements,
intended when RSD without the percent sign is used. The
s = standard deviation of the set, and
recommended practice is %RSD=100 (s/|x¯ |) and RSD= s/
n = number of measurements in the set.
|x¯ |.
3.2.19 systematic error—the term systematic error should
3.2.14 repeatability—see Discussion in 3.2.8.
not be used unless defined carefully.
3.2.15 representative sample—agenerictermindicatingthat 3.2.19.1 Discussion—Some consider systematic error as a
the sample is typical of the population with respect to some synonym for bias and treat it as a constant, whereas others
specified characteristic(s). make a distinction between the two terms. Some publications
haveusedsystematicerrortorefertobothafixedandarandom
3.2.15.1 Discussion—Taken literally, a representative
sample is a sample that represents the population from which error. If the term is used, it should be clearly defined,
preferably by specifying the error model. (See bias and
it is selected. Thus, “representative sample” has gained con-
siderable colloquial acceptance in discussions involving the Example, X2.1.1.)
concepts of sampling. However, its use is avoided by most
3.2.20 uncertainty—a generic term indicating the inability
samplingmethodologistsbecausetheconceptofrepresentative
of a measurement process to measure the correct value.
does not lend itself readily to definition or theoretical treat-
3.2.20.1 Discussion—Uncertainty is a concept which has
ment. In particular, the concept is almost meaningless in
been used to encompass both precision and bias. Thus, one
describing a sample or its method of selection. Kendall and
measurement process (or a set of measurements based on the
Buckland (2) suggest: “On the whole, it seems best to confine
process) is sometimes referred to as “more uncertain” than
the word ’representative’ to samples which turn out to be so,
another process. But, just as with precision, it is important that
however chosen, rather than apply it to those chosen with the
a quantitative measure be used to specify uncertainty. Thus, a
objective of being representative.” “Representative sample” is
phrase like, “The uncertainty is 5.2 units,” should be avoided.
not synonymous with “random sample.” A random sample
Unfortunately,nosinglequantitativemeasuretospecifyuncer-
from a well-mixed material is probably representative; a
tainty is universally accepted. Thus, “the quantification of
random sample from an inhomogeneous material probably is
uncertainty is itself an uncertain undertaking.”
not. It is likely many scientists mean random sample when
See precision and bias for preferred terms and Ku (5) for
using the term representative sample. If so, then the term
additional discussion.
random sample should be used to avoid possible confusion. In
Appendix X5, an example relating to random and representa-
3.2.21 variance (sample)—a measure of the dispersion of a
tive samples is given.
set of results. Variance is the sum of the squares of the
individual deviations from the sample mean divided by one
3.2.16 reproducibility—see Discussion in 3.2.8.
less than the number of results involved.
3.2.17 standard deviation—the positive square root of the
3.2.21.1 Discussion—The equation that expresses this defi-
variance.
nition is as follows:
3.2.17.1 Discussion—The use of the standard deviation to
n
describe precision implies that the uncertainty is independent
2 2
s 5 x 2 x¯ (6)
~ !
( i
n 2 1
of the measurement value.
i51
(a)An appropriate error model might be X =µ+ b+ ε.
i i
where:
(See Example, Appendix X2.)
s = sample variance,
(b)The practice of associating the 6 symbol with standard
n = number of results obtained,
deviation (or RSD) is not recommended. The 6 symbol
x = ith individual result, and
i
denotes an interval. The standard deviation is not an interval
x¯ = sample mean
and it should not be treated as such. If the 6 notation is used
n
as in, “The fraction of uranium was estimated as 0.88 6 0.01,”
x¯ 5 x
S D
( i
n
i51
a footnote should be added to clearly explain what is meant. Is
0.01 one standard deviation, two standard deviations, the
The following is an equation that is sometimes used to
standard deviation of the mean, or something else? Is the
calculate sample variance:
interval a confidence interval?
2 2 2
s 5 @ x 2 nx¯ # (7)
3.2.18 standard deviation of the mean (sample)—thesample ( i
n 2 1
standard deviation divided by the square root of the number of
Although this equation is mathematically exact, in prac-
measurements used in the calculation of the mean (Syn.
tice it can lead to appreciable errors because of computer
standard error of the mean).
round-off problems. This can occur especially if the
3.2.18.1 Discussion—Theequationforstandarddeviationof
%RSD is small. The definition formula is, in general, to
the mean is
be preferred. To be useful, the variance must be based on
s
results that are independent and identically distributed.
s 5 (5)
x¯
=n (See Example, X2.1.1.)
C1215 − 18
4. Significance and Use training, it is advised that a trained statistician be consulted for
further clarification if necessary.
4.1 To de
...

This document is not an ASTM standard and is intended only to provide the user of an ASTM standard an indication of what changes have been made to the previous version. Because
it may not be technically possible to adequately depict all changes accurately, ASTM recommends that users consult prior editions as appropriate. In all cases only the current version
of the standard as published by ASTM is to be considered the official document.
´1
Designation: C1215 − 92 (Reapproved 2012) C1215 − 18
Standard Guide for
Preparing and Interpreting Precision and Bias Statements in
Test Method Standards Used in the Nuclear Industry
This standard is issued under the fixed designation C1215; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
ε NOTE—Changes were made editorially in June 2012.
INTRODUCTION
Test method standards are required to contain precision and bias statements. This guide contains a
glossary that explains various terms that often appear in these statements as well as an example
illustrating such statements for a specific set of data. Precision and bias statements are shown to vary
according to the conditions under which the data were collected. This guide emphasizes that the error
model (an algebraic expression that describes how the various sources of variation affect the
measurement) is an important consideration in the formation of precision and bias statements.
1. Scope
1.1 This guide covers terminology useful for the preparation and interpretation of precision and bias statements. This guide does
not recommend a specific error model or statistical method. It provides awareness of terminology and approaches and options to
use for precision and bias statements.
1.2 In formulating precision and bias statements, it is important to understand the statistical concepts involved and to identify
the major sources of variation that affect results. Appendix X1 provides a brief summary of these concepts.
1.3 To illustrate the statistical concepts and to demonstrate some sources of variation, a hypothetical data set has been analyzed
in Appendix X2. Reference to this example is made throughout this guide.
1.4 It is difficult and at times impossible to ship nuclear materials for interlaboratory testing. Thus, precision statements for test
methods relating to nuclear materials will ordinarily reflect only within-laboratory variation.
1.5 No units are used in this statistical analysis.
1.6 This guide does not involve the use of materials, operations, or equipment and does not address any risk associated.
1.7 This international standard was developed in accordance with internationally recognized principles on standardization
established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued
by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
2. Referenced Documents
2.1 ASTM Standards:
C859 Terminology Relating to Nuclear Materials
E177 Practice for Use of the Terms Precision and Bias in ASTM Test Methods
E691 Practice for Conducting an Interlaboratory Study to Determine the Precision of a Test Method
2.2 ANSI Standard:
ANSI N15.5 Statistical Terminology and Notation for Nuclear Materials Management
This guide is under the jurisdiction of ASTM Committee C26 on Nuclear Fuel Cycle and is the direct responsibility of Subcommittee C26.08 on Quality Assurance,
Statistical Applications, and Reference Materials.
Current edition approved June 1, 2012July 1, 2018. Published June 2012July 2018. Originally approved in 1992. Last previous edition approved in 20062012 as
ɛ1
C1215–92(2006).C1215 – 92 (2012) . DOI: 10.1520/C1215-92R12E01.10.1520/C1215-18.
For referenced ASTM standards, visit the ASTM website, www.astm.org, or contact ASTM Customer Service at [e-mail]. For Annual Book of ASTM Standards
volume information, refer to the standard’s Document Summary page on the ASTM website.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
C1215 − 18
3. Terminology for Precision and Bias Statements
3.1 For definitions of terms used in this guide but not defined herein, see Terminology C859.
3.2 Definitions:Terminology for Precision and Bias Statements
3.2.1 accuracy (seebias) —(1) bias. (2) the closeness of a measured value to the true value. (3) the closeness of a measured value
to an accepted reference or standard value.
3.2.1.1 Discussion—
For many investigators, accuracy is attained only if a procedure is both precise and unbiased (see bias). Because this blending of
precision into accuracy can result occasionally in incorrect analyses and unclear statements of results, ASTM requires statement
on bias instead of accuracy.
3.2.2 analysis of variance (ANOVA)—the body of statistical theory, methods, and practices in which the variation in a set of data
is partitioned into identifiable sources of variation. Sources of variation may include analysts, instruments, samples, and
laboratories. To use the analysis of variance, the data collection method must be carefully designed based on a model that includes
all the sources of variation of interest. (See Example, X2.1.1)
3.2.3 bias (see accuracy)—a constant positive or negative deviation of the method average from the correct value or accepted
reference value.
3.2.3.1 Discussion—
Bias represents a constant error as opposed to a random error.
(a) A method bias can be estimated by the difference (or relative difference) between a measured average and an accepted
standard or reference value. The data from which the estimate is obtained should be statistically analyzed to establish bias in the
presence of random error. A thorough bias investigation of a measurement procedure requires a statistically designed experiment
to repeatedly measure, under essentially the same conditions, a set of standards or reference materials of known value that cover
the range of application. Bias often varies with the range of application and should be reported accordingly.
(b) In statistical terminology, an estimator is said to be unbiased if its expected value is equal to the true value of the parameter
being estimated. (See Appendix X1.)
(c) The bias of a test method is also commonly indicated by analytical chemists as percent recovery. A number of repetitions
of the test method on a reference material are performed, and an average percent recovery is calculated. This average provides an
estimate of the test method bias, which is multiplicative in nature, not additive. (See Appendix X2.)
(d) Use of a single test result to estimate bias is strongly discouraged because, even if there were no bias, random error alone
would produce a nonzero bias estimate.
3.2.4 coeffıcient of variation—see relative standard deviation.
3.2.5 confidence interval—an interval used to bound the value of a population parameter with a specified degree of confidence
(this is an interval that has different values for different random samples).
3.2.5.1 Discussion—
When providing a confidence interval, analysts should give the number of observations on which the interval is based. The
specified degree of confidence is usually 90, 95, or 99 %. The form of a confidence interval depends on underlying assumptions
and intentions. Usually, confidence intervals are taken to be symmetric, but that is not necessarily so, as in the case of confidence
intervals for variances. Construction of a symmetric confidence interval for a population mean is discussed in Appendix X3.
It is When providing a confidence interval, analysts should give the number of observations on which the interval is based.
The specified degree of confidence is usually 90, 95, or 99 %. The form of a confidence interval depends on underlying
assumptions and intentions. Usually, confidence intervals are taken to be symmetric, but that is not necessarily so, as in the case
of confidence intervals for variances. Construction of a symmetric confidence interval for a population mean is discussed in
Appendix X3.
It is important to realize that a given confidence-interval estimate either does or does not contain the population parameter.
The degree of confidence is actually in the procedure. For example, if the interval (9, 13) is a 90 % confidence interval for the
mean, we are confident that the procedure (take a sample, construct an interval) by which the interval (9, 13) was constructed
will 90 % of the time produce an interval that does indeed contain the mean. Likewise, we are confident that 10 % of the time
the interval estimate obtained will not contain the mean. Note that the absence of sample size information detracts from the
Refer to Form and Style for ASTM Standards, 8th Ed., 1989, ASTM.
C1215 − 18
usefulness of the confidence interval. If the interval were based on five observations, a second set of five might produce a very
different interval. This would not be the case if 50 observations were taken.
3.2.6 confidence level—the probability, usually expressed as a percent, that a confidence interval will contain the parameter of
interest. (See discussion of confidence interval in Appendix X3.)
3.2.7 error model—an algebraic expression that describes how a measurement is affected by error and other sources of variation.
The model may or may not include a sampling error term.
3.2.7.1 Discussion—
A measurement error is an error attributable to the measurement process. The error may affect the measurement in many ways and
it is important to correctly model the effect of the error on the measurement.
(a) Two common models are the additive and the multiplicative error models. In the additive model, the errors are
independent of the value of the item being measured. Thus, for example, for repeated measurements under identical conditions,
the additive error model might be
X 5 µ1b1ε (1)
i i
where:
th
X = the result of the i measurement,
i
μ = the true value of the item,
b = a bias, and
ε = a random error usually assumed to have a normal distribution with mean zero and variance σ .
i
In the multiplicative model, the error is proportional to the true value. A multiplicative error model for percent recovery (see
bias) might be:
X 5 µbε (2)
i i
and a multiplicative model for a neutron counter measurement might be:
X 5 μ1μb1μ·ε (3)
i i
5μ 11b1ε
~ !
i
(b) Clearly, there are many ways in which errors may affect a final measurement. The additive model is frequently as-
sumed and is the basis for many common statistical procedures. The form of the model influences how the error compo-
nents will be estimated and is very important, for example, in the determination of measurement uncertainties. Further dis-
cussion of models is given in the Example of Appendix X2 and in Appendix X4.
3.2.8 precision—a generic concept used to describe the dispersion of a set of measured values.
3.2.8.1 Discussion—
It is important that some quantitative measure be used to specify precision. A statement such as, “The precision is 1.54 g” is useless.
Measures frequently used to express precision are standard deviation, relative standard deviation, variance, repeatability,
reproducibility, confidence interval, and range. In addition to specifying the measure and the precision, it is important that the
number of repeated measurements upon which the precision estimated is based also be given. (See Example, Appendix X2.)
(a) It is strongly recommended that a statement on precision of a measurement procedure include the following:
(a) It is strongly recommended that a statement on precision of a measurement procedure include the following:
(1) A description of the procedure used to obtain the data,
(2) The number of repetitions, n, of the measurement procedure,
(3) The sample mean and standard deviation of the measurements,
(4) The measure of precision being reported,
(5) The computed value of that measure, and
(6) The applicable range or concentration.
The importance of items (3) and (4) lies in the fact that with these a reader may calculate a confidence interval or relative
standard deviation as desired.
(b) Precision is sometimes measured by repeatability and reproducibility (see Practice E177, and Mandel and Laskof (1)). The
ANSI and ASTM documents differ slightly in their usages of these terms. The following is quoted from Kendall and Buckland
(2):
“In some situations, especially interlaboratory comparisons, precision is defined by employing two additional concepts:
repeatability and reproducibility. The general situation giving rise to these distinctions comes from the interest in assessing the
C1215 − 18
variability within several groups of measurements and between those groups of measurements. Repeatability, then, refers to the
within-group dispersion of the measurements, while reproducibility refers to the between-group dispersion. In interlaboratory
comparison studies, for example, the investigation seeks to determine how well each laboratory can repeat its measurements
(repeatability) and how well the laboratories agree with each other (reproducibility). Similar discussions can apply to the
comparison of laboratory technicians’ skills, the study of competing types of equipment, and the use of particular procedures
within a laboratory. An essential feature usually required, however, is that repeatability and reproducibility be measured as
variances (or standard deviations in certain instances), so that both within- and between-group dispersions are modeled as a
random variable. The statistical tool useful for the analysis of such comparisons is the analysis of variance.”
(c) In Practice E177 it is recommended that the term repeatability be reserved for the intrinsic variation due solely to the
measurement procedure, excluding all variation from factors such as analyst, time and laboratory and reserving reproducibility
for the variation due to all factors including laboratory. Repeatability can be measured by the standard deviation, σ , of n
r
consecutive measurements by the same operator on the same instrument. Reproducibility can be measured by the standard
deviation, σ , of m measurements, one obtained from each of m independent laboratories. When interlaboratory testing is not
R
practical, the reproducibility conditions should be described.
The importance of items (3) and (4) lies in the fact that with these a reader may calculate a confidence interval or relative
standard deviation as desired.
(b) Precision is sometimes measured by repeatability and reproducibility (see Practice E177, and Mandel and Laskof (1)).
The ANSI and ASTM documents differ slightly in their usages of these terms. The following is quoted from Kendall and
Buckland (2):
“In some situations, especially interlaboratory comparisons, precision is defined by employing two additional concepts:
repeatability and reproducibility. The general situation giving rise to these distinctions comes from the interest in assessing the
variability within several groups of measurements and between those groups of measurements. Repeatability, then, refers to the
within-group dispersion of the measurements, while reproducibility refers to the between-group dispersion. In interlaboratory
comparison studies, for example, the investigation seeks to determine how well each laboratory can repeat its measurements
(repeatability) and how well the laboratories agree with each other (reproducibility). Similar discussions can apply to the
comparison of laboratory technicians’ skills, the study of competing types of equipment, and the use of particular procedures
within a laboratory. An essential feature usually required, however, is that repeatability and reproducibility be measured as
variances (or standard deviations in certain instances), so that both within- and between-group dispersions are modeled as a
random variable. The statistical tool useful for the analysis of such comparisons is the analysis of variance.”
(c) In Practice E177 it is recommended that the term repeatability be reserved for the intrinsic variation due solely to the
measurement procedure, excluding all variation from factors such as analyst, time and laboratory and reserving reproducibility
for the variation due to all factors including laboratory. Repeatability can be measured by the standard deviation, σ , of n
r
consecutive measurements by the same operator on the same instrument. Reproducibility can be measured by the standard
deviation, σ , of m measurements, one obtained from each of m independent laboratories. When interlaboratory testing is not
R
practical, the reproducibility conditions should be described.
(d) Two additional terms are recommended in Practice E177. These are repeatability limit and reproducibility limit. These
are intended to give estimates of how different two measurements can be. The repeatability limit is defined as 1.96=2s , and the
r
reproducibility limit is defined as 1.96=2s , where s is the estimated standard deviation associated with repeatability, and s is
R r R
the estimated standard deviation associated with reproducibility. Thus, if normality can be assumed, these limits represent 95 %
limits for the difference between two measurements taken under the respective conditions. In the reproducibility case, this means
that “approximately 95 % of all pairs of test results from laboratories similar to those in the study can be expected to differ in
absolute value by less than 1.96=2s .” It is important to realize that if a particular s is a poor estimate of σ , the 95 % figure may
R R R
be substantially in error. For this reason, estimates should be based on adequate sample sizes.
3.2.9 propagation of variance—a procedure by which the mean and variance of a function of one or more random variables can
be expressed in terms of the mean, variance, and covariances of the individual random variables themselves (Syn. variance
propagation, propagation of error).
3.2.9.1 Discussion—
There are a number of simple exact formulas and Taylor series approximations which are useful here (3, 4).
3.2.10 random error—(1) the chance variation encountered in all measurement work, characterized by the random occurrence
of deviations from the mean value. (2) an error that affects each member of a set of data (measurements) in a different manner.
3.2.11 random sample (measurements)—a set of measurements taken on a single item or on similar items in such a way that
the measurements are independent and have the same probability distribution.
3.2.11.1 Discussion—
C1215 − 18
Some authors refer to this as a simple random sample. One must then be careful to distinguish between a simple random sample
from a finite population of N items and a simple random sample from an infinite population. In the former case, a simple random
sample is a sample chosen in such a way that all samples of the same size have the same chance of being selected. An example
of the latter case occurs when taking measurements. Any value in an interval is considered possible and thus the population is
conceptually infinite. The definition given in 3.1.113.2.11 is then the appropriate definition. (See representative sample and
Appendix X5.)
3.2.12 range—the largest minus the smallest of a set of numbers.
3.2.13 relative standard deviation (percent)—the sample standard deviation expressed as a percent of the sample mean. The
%RSD is calculated using the following equation:
s
%RSD 5 100 (4)
x
? ?
where:
s = sample standard deviation and
x¯ = sample mean.
3.2.13.1 Discussion—
The use of the %RSD (or RSD(%)) to describe precision implies that the uncertainty is a function of the measurement values. An
appropriate error model might then be X = μ(1 + b + ε ). (See Example, Appendix X2.) Some authors use RSD for the ratio, s/ | x |,
i i
while others call this the coeffıcient of variation. At times authors use RSD to mean %RSD. Thus, it is important to determine
which meaning is intended when RSD without the percent sign is used. The recommended practice is %RSD = 100 (s/|x¯ |) and
RSD = s/ |x¯ |.
3.2.14 repeatability—see Discussion in 3.1.83.2.8.
3.2.15 representative sample—a generic term indicating that the sample is typical of the population with respect to some
specified characteristic(s).
3.2.15.1 Discussion—
Taken literally, a representative sample is a sample that represents the population from which it is selected. Thus, “representative
sample” has gained considerable colloquial acceptance in discussions involving the concepts of sampling. However, its use is
avoided by most sampling methodologists because the concept of representative does not lend itself readily to definition or
theoretical treatment. In particular, the concept is almost meaningless in describing a sample or its method of selection (see ANSI
N15.5). selection. Kendall and Buckland (2) suggest: “On the whole, it seems best to confine the word ’representative’ to samples
which turn out to be so, however chosen, rather than apply it to those chosen with the objective of being representative.”
“Representative sample” is not synonymous with “random sample.” A random sample from a well-mixed material is probably
representative; a random sample from an inhomogeneous material probably is not. It is likely many scientists mean random sample
when using the term representative sample. If so, then the term random sample should be used to avoid possible confusion. In
Appendix X5, an example relating to random and representative samples is given.
3.2.16 reproducibility—see Discussion in 3.1.83.2.8.
3.2.17 standard deviation—the positive square root of the variance.
3.2.17.1 Discussion—
The use of the standard deviation to describe precision implies that the uncertainty is independent of the measurement value.
(a) An appropriate error model might be X = μ + b + ε . (See Example, Appendix X2.)
i i
(b) The practice of associating the 6 symbol with standard deviation (or RSD) is not recommended. The 6 symbol denotes
an interval. The standard deviation is not an interval and it should not be treated as such. If the 6 notation is used as in, “The
fraction of uranium was estimated as 0.88 6 0.01,” a footnote should be added to clearly explain what is meant. Is 0.01 one
standard deviation, two standard deviations, the standard deviation of the mean, or something else? Is the interval a confidence
interval?
3.2.18 standard deviation of the mean (sample)—the sample standard deviation divided by the square root of the number of
measurements used in the calculation of the mean (Syn. standard error of the mean).
3.2.18.1 Discussion—
C1215 − 18
The equation for standard deviation of the mean is
s
s 5 (5)
x¯
=
n
where:
s = standard deviation of the mean of a set of measurements,
x¯
s = standard deviation of the set, and
n = number of measurements in the set.
3.2.19 systematic error—the term systematic error should not be used unless defined carefully.
3.2.19.1 Discussion—
Some consider systematic error as a synonym for bias and treat it as a constant, whereas others make a distinction between the
two terms. Some publications have used systematic error to refer to both a fixed and a random error. If the term is used, it should
be clearly defined, preferably by specifying the error model. (See bias and Example, X2.1.1.)
3.2.20 uncertainty—a generic term indicating the inability of a measurement process to measure the correct value.
3.2.20.1 Discussion—
Uncertainty is a concept which has been used to encompass both precision and bias. Thus, one measurement process (or a set of
measurements based on the process) is sometimes referred to as “more uncertain” than another process. But, just as with precision,
it is important that a quantitative measure be used to specify uncertainty. Thus, a phrase like, “The uncertainty is 5.2 units,” should
be avoided. Unfortunately, no single quantitative measure to specify uncertainty is universally accept
...