ASTM C1215-92(1997)
(Guide)Standard Guide for Preparing and Interpreting Precision and Bias Statements in Test Method Standards Used in the Nuclear Industry
Standard Guide for Preparing and Interpreting Precision and Bias Statements in Test Method Standards Used in the Nuclear Industry
SCOPE
1.1 This guide covers terminology useful for the preparation and interpretation of precision and bias statements.
1.2 In formulating precision and bias statements, it is important to understand the statistical concepts involved and to identify the major sources of variation that affect results. Appendix X1 provides a brief summary of these concepts.
1.3 To illustrate the statistical concepts and to demonstrate some sources of variation, a hypothetical data set has been analyzed in Appendix X2. Reference to this example is made throughout this guide.
1.4 It is difficult and at times impossible to ship nuclear materials for interlaboratory testing. Thus, precision statements for test methods relating to nuclear materials will ordinarily reflect only within-laboratory variation.
General Information
Relations
Standards Content (Sample)
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Contact ASTM International (www.astm.org) for the latest information
Designation:C1215–92 (Reapproved 1997)
Standard Guide for
Preparing and Interpreting Precision and Bias Statements in
Test Method Standards Used in the Nuclear Industry
This standard is issued under the fixed designation C1215; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (e) indicates an editorial change since the last revision or reapproval.
INTRODUCTION
Test method standards are required to contain precision and bias statements. This guide contains a
glossary that explains various terms that often appear in these statements as well as an example
illustrating such statements for a specific set of data. Precision and bias statements are shown to vary
according to the conditions under which the data were collected.This guide emphasizes that the error
model (an algebraic expression that describes how the various sources of variation affect the
measurement) is an important consideration in the formation of precision and bias statements.
1. Scope 3. Terminology for Precision and Bias Statements
1.1 Thisguidecoversterminologyusefulforthepreparation 3.1 Definitions:
and interpretation of precision and bias statements. 3.1.1 accuracy (see bias)—(1) bias. ( 2) the closeness of a
1.2 In formulating precision and bias statements, it is measured value to the true value. ( 3) the closeness of a
importanttounderstandthestatisticalconceptsinvolvedandto measured value to an accepted reference or standard value.
identify the major sources of variation that affect results. 3.1.1.1 Discussion—For many investigators, accuracy is
Appendix X1 provides a brief summary of these concepts. attained only if a procedure is both precise and unbiased (see
1.3 To illustrate the statistical concepts and to demonstrate bias). Because this blending of precision into accuracy can
some sources of variation, a hypothetical data set has been resultoccasionallyinincorrectanalysesandunclearstatements
analyzed in Appendix X2. Reference to this example is made of results, ASTM requires statement on bias instead of accu-
throughout this guide. racy.
1.4 It is difficult and at times impossible to ship nuclear 3.1.2 analysis of variance (ANOVA)—the body of statistical
materialsforinterlaboratorytesting.Thus,precisionstatements theory,methods,andpracticesinwhichthevariationinasetof
for test methods relating to nuclear materials will ordinarily data is partitioned into identifiable sources of variation.
reflect only within-laboratory variation. Sources of variation may include analysts, instruments,
samples, and laboratories. To use the analysis of variance, the
2. Referenced Documents
data collection method must be carefully designed based on a
2.1 ASTM Standards:
modelthatincludesallthesourcesofvariationofinterest.(See
E177 Practice for Use of the Terms Precision and Bias in Example, Appendix X2.1.1)
ASTM Test Methods
3.1.3 bias (see accuracy)—a constant positive or negative
E691 Practice for Conducting an Interlaboratory Study to
deviation of the method average from the correct value or
Determine the Precision of a Test Method accepted reference value.
2.2 ANSI Standard:
3.1.3.1 Discussion—Bias represents a constant error as
ANSI N15.5 Statistical Terminology and Notation for opposed to a random error.
Nuclear Materials Management
(a) (a) A method bias can be estimated by the difference
(or relative difference) between a measured average and an
accepted standard or reference value. The data from which the
This guide is under the jurisdiction ofASTM Committee C26 on Nuclear Fuel
estimateisobtainedshouldbestatisticallyanalyzedtoestablish
CycleandisthedirectresponsibilityofSubcommitteeC26.08onQualityAssurance
bias in the presence of random error. A thorough bias investi-
and Reference Materials.
gation of a measurement procedure requires a statistically
Current edition approved May 15, 1992. Published July 1992.
Annual Book of ASTM Standards, Vol 14.02.
American National Standards Institute, 11 W. 42nd St., 13th Floor, New York,
NY 10036. Refer to Form and Style for ASTM Standards, 8th Ed., 1989, ASTM.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.
C1215
value of the item being measured. Thus, for example, for repeated
designed experiment to repeatedly measure, under essentially
measurements under identical conditions, the additive error model
thesameconditions,asetofstandardsorreferencematerialsof
might be
known value that cover the range of application. Bias often
X 5µ 1 b 1e (1)
varies with the range of application and should be reported
i i
accordingly.
where:
(b) (b) In statistical terminology, an estimator is said to be th
X = the result of the i measurement,
i
unbiased if its expected value is equal to the true value of the
µ = the true value of the item,
parameter being estimated. (See Appendix X1.)
b = a bias, and
(c) (c) The bias of a test method is also commonly
e = a random error usually assumed to have a normal
i
indicatedbyanalyticalchemistsas percent recovery.Anumber
distribution with mean zero and variance s .
of repetitions of the test method on a reference material are
In the multiplicative model, the error is proportional to the true
performed, and an average percent recovery is calculated.This
value. A multiplicative error model for percent recovery (see bias)
average provides an estimate of the test method bias, which is
might be:
multiplicative in nature, not additive. (See Appendix X2.)
X 5µbe (2)
i i
(d) (d) Use of a single test result to estimate bias is
andamultiplicativemodelforaneutroncountermeasurementmight
strongly discouraged because, even if there were no bias,
be:
random error alone would produce a nonzero bias estimate.
X 5 µ 1 µb 1 µ· e
i i
3.1.4 coeffıcient of variation—see relative standard devia-
tion.
5 µ 1 1 b1e (3)
~ !
i
3.1.5 confidence interval—an interval used to bound the ( b) Clearly, there are many ways in which errors may affect a final
measurement. The additive model is frequently assumed and is the
value of a population parameter with a specified degree of
basis for many common statistical procedures. The form of the model
confidence (this is an interval that has different values for
influences how the error components will be estimated and is very
different random samples).
important, for example, in the determination of measurement uncer-
3.1.5.1 Discussion—When providing a confidence interval,
tainties. Further discussion of models is given in the Example of
analysts should give the number of observations on which the Appendix X2 and in Appendix X4.
interval is based. The specified degree of confidence is usually
3.1.8 precision—a generic concept used to describe the
90, 95, or 99%. The form of a confidence interval depends on
dispersion of a set of measured values.
underlying assumptions and intentions. Usually, confidence
3.1.8.1 Discussion—It is important that some quantitative
intervals are taken to be symmetric, but that is not necessarily
measure be used to specify precision. A statement such as,
so, as in the case of confidence intervals for variances.
“The precision is 1.54 g” is useless. Measures frequently used
Construction of a symmetric confidence interval for a popula-
to express precision are standard deviation, relative standard
tion mean is discussed in Appendix X3.
deviation, variance, repeatability, reproducibility, confidence
interval, and range. In addition to specifying the measure and
It is important to realize that a given confidence-interval estimate
eitherdoesordoesnotcontainthepopulationparameter.Thedegreeof the precision, it is important that the number of repeated
confidence is actually in the procedure. For example, if the interval (9,
measurementsuponwhichtheprecisionestimatedisbasedalso
13)isa90%confidenceintervalforthemean,weareconfidentthatthe
be given. (See Example, Appendix X2.)
procedure (take a sample, construct an interval) by which the interval
(a) It is strongly recommended that a statement on precision of a
(9, 13) was constructed will 90% of the time produce an interval that
measurement procedure include the following:
does indeed contain the mean. Likewise, we are confident that 10% of
the time the interval estimate obtained will not contain the mean. Note
(1) Adescription of the procedure used to obtain the data,
thattheabsenceofsamplesizeinformationdetractsfromtheusefulness
(2) The number of repetitions, n, of the measurement
of the confidence interval. If the interval were based on five observa-
procedure,
tions, a second set of five might produce a very different interval. This
(3) The sample mean and standard deviation of the
would not be the case if 50 observations were taken.
measurements,
3.1.6 confidencelevel—theprobability,usuallyexpressedas
(4) The measure of precision being reported,
a percent, that a confidence interval will contain the parameter
(5) The computed value of that measure, and
of interest. (See discussion of confidence interval inAppendix
(6) The applicable range or concentration.
X3.)
The importance of items (3) and (4) lies in the fact that with these a
3.1.7 error model—an algebraic expression that describes
readermaycalculateaconfidenceintervalorrelativestandarddeviation
how a measurement is affected by error and other sources of
as desired.
variation. The model may or may not include a sampling error
(b) Precision is sometimes measured by repeatability and reproduc-
term.
ibility (see Practice E177, and Mandel and Laskof (3)).TheANSI and
3.1.7.1 Discussion—Ameasurement error is an error attrib-
ASTM documents differ slightly in their usages of these terms. The
utable to the measurement process. The error may affect the following is quoted from Kendall and Buckland (2):
“In some situations, especially interlaboratory comparisons, preci-
measurement in many ways and it is important to correctly
sionisdefinedbyemployingtwoadditionalconcepts:repeatabilityand
model the effect of the error on the measurement.
reproducibility. The general situation giving rise to these distinctions
(a) Two common models are the additive and the multiplicative comes from the interest in assessing the variability within several
error models. In the additive model, the errors are independent of the groups of measurements and between those groups of measurements.
C1215
Repeatability, then, refers to the within-group dispersion of the
considered possible and thus the population is conceptually
measurements, while reproducibility refers to the between-group dis-
infinite. The definition given in 3.1.11 is then the appropriate
persion. In interlaboratory comparison studies, for example, the inves-
definition. (See representative sample and Appendix X5.)
tigation seeks to determine how well each laboratory can repeat its
3.1.12 range—the largest minus the smallest of a set of
measurements (repeatability) and how well the laboratories agree with
numbers.
each other (reproducibility). Similar discussions can apply to the
3.1.13 relative standard deviation (percent)— the sample
comparison of laboratory technicians’ skills, the study of competing
types of equipment, and the use of particular procedures within a standard deviation expressed as a percent of the sample mean.
laboratory. An essential feature usually required, however, is that
The %RSD is calculated using the following equation:
repeatability and reproducibility be measured as variances (or standard
s
deviationsincertaininstances),sothatbothwithin-andbetween-group
%RSD 5100 (4)
| x¯ |
dispersionsaremodeledasarandomvariable.Thestatisticaltooluseful
for the analysis of such comparisons is the analysis of variance.”
where:
( c) In Practice E177 it is recommended that the term repeatability
s = sample standard deviation and
be reserved for the intrinsic variation due solely to the measurement
x¯ = sample mean.
procedure, excluding all variation from factors such as analyst, time
3.1.13.1 Discussion—Theuseofthe%RSD(orRSD(%))to
and laboratory and reserving reproducibility for the variation due to all
describe precision implies that the uncertainty is a function of
factors including laboratory. Repeatability can be measured by the
standard deviation, s ,of n consecutive measurements by the same the measurement values. An appropriate error model might
r
operator on the same instrument. Reproducibility can be measured by
then be X =µ(1+ b+ e). (See Example, Appendix X2.)
i i
thestandarddeviation, s ,ofmmeasurements,oneobtainedfromeach
R
Some authors use RSD for the ratio,s/|x|, while others call
of m independent laboratories. When interlaboratory testing is not
this the coeffıcient of variation. At times authors use RSD to
practical, the reproducibility conditions should be described.
mean%RSD.Thus,itisimportanttodeterminewhichmeaning
(d) TwoadditionaltermsarerecommendedinPracticeE177.These
is intended when RSD without the percent sign is used. The
are repeatability limit and reproducibility limit. These are intended to
recommended practice is %RSD=100 (s/| x¯ |) and RSD= s/|
give estimates of how different two measurements can be. The
repeatability limit is defined as 1.96 2 s , and the reproducibility x¯ |.
=
r
limit is defined as 1.96 2 s , where s is the estimated standard
= 3.1.14 repeatability—see 3.1.8.1
R r
deviationassociatedwithrepeatability,ands istheestimatedstandard
R
3.1.15 representative sample—a generic term indicating
deviation associated with reproducibility. Thus, if normality can be
thatthesampleistypicalofthepopulationwithrespecttosome
assumed, these limits represent 95% limits for the difference between
specified characteristic(s).
two measurements taken under the respective conditions. In the
3.1.15.1 Discussion—Taken literally, a representative
reproducibility case, this means that 88approximately 95% of all pairs
sample is a sample that represents the population from which
of test results from laboratories similar to those in the study can be
expected to differ in absolute value by less than 1.96 2 s .” It is it is selected. Thus, 88representative sample” has gained con-
=
R
important to realize that if a particular s is a poor estimate of s , the
R R siderable colloquial acceptance in discussions involving the
95% figure may be substantially in error. For this reason, estimates
concepts of sampling. However, its use is avoided by most
should be based on adequate sample sizes.
samplingmethodologistsbecausetheconceptofrepresentative
does not lend itself readily to definition or theoretical treat-
3.1.9 propagation of variance—a procedure by which the
mean and variance of a function o
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.