ASTM D7366-08(2013)
(Practice)Standard Practice for Estimation of Measurement Uncertainty for Data from Regression-based Methods
Standard Practice for Estimation of Measurement Uncertainty for Data from Regression-based Methods
SIGNIFICANCE AND USE
5.1 Appropriate application of this practice should result in an estimate of the test-method’s uncertainty (at any concentration within the working range), which can be compared with data-quality objectives to see if the uncertainty is acceptable.
5.2 With data sets that compare recovered concentration with true concentration, the resulting regression plot allows the correction of the recovery data to true values. Reporting of such corrections is at the discretion of the user.
5.3 This practice should be used to estimate the measurement uncertainty for any application of a test method where measurement uncertainty is important to data use.
SCOPE
1.1 This practice establishes a standard for computing the measurement uncertainty for applicable test methods in Committee D19 on Water. The practice does not provide a single-point estimate for the entire working range, but rather relates the uncertainty to concentration. The statistical technique of regression is employed during data analysis.
1.2 Applicable test methods are those whose results come from regression-based methods and whose data are intra-laboratory (not inter-laboratory data, such as result from round-robin studies). For each analysis conducted using such a method, it is assumed that a fixed, reproducible amount of sample is introduced.
1.3 Calculation of the measurement uncertainty involves the analysis of data collected to help characterize the analytical method over an appropriate concentration range. Example sources of data include: 1) calibration studies (which may or may not be conducted in pure solvent), 2) recovery studies (which typically are conducted in matrix and include all sample-preparation steps), and 3) collections of data obtained as part of the method’s ongoing Quality Control program. Use of multiple instruments, multiple operators, or both, and field-sampling protocols may or may not be reflected in the data.
1.4 In any designed study whose data are to be used to calculate method uncertainty, the user should think carefully about what the study is trying to accomplish and much variation should be incorporated into the study. General guidance on designing studies (for example, calibration, recovery) is given in Appendix A. Detailed guidelines on sources of variation are outside the scope of this practice, but general points to consider are included in Appendix B, which is not intended to be exhaustive. With any study, the user must think carefully about the factors involved with conducting the analysis, and must realize that the computed measurement uncertainty will reflect the quality of the input data.
1.5 Associated with the measurement uncertainty is a user-chosen level of statistical confidence.
1.6 At any concentration in the working range, the measurement uncertainty is plus-or-minus the half-width of the prediction interval associated with the regression line.
1.7 It is assumed that the user has access to a statistical software package for performing regression. A statistician should be consulted if assistance is needed in selecting such a program.
1.8 A statistician also should be consulted if data transformations are being considered.
1.9 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility of the user of this standard to establish appropriate safety and health practices and determine the applicability of regulatory limitations prior to use.
General Information
Relations
Standards Content (Sample)
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Contact ASTM International (www.astm.org) for the latest information
Designation: D7366 − 08 (Reapproved 2013)
Standard Practice for
Estimation of Measurement Uncertainty for Data from
Regression-based Methods
This standard is issued under the fixed designation D7366; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope analysis, and must realize that the computed measurement
uncertainty will reflect the quality of the input data.
1.1 This practice establishes a standard for computing the
1.5 Associated with the measurement uncertainty is a user-
measurement uncertainty for applicable test methods in Com-
chosen level of statistical confidence.
mittee D19 on Water. The practice does not provide a single-
point estimate for the entire working range, but rather relates
1.6 Atanyconcentrationintheworkingrange,themeasure-
the uncertainty to concentration. The statistical technique of
ment uncertainty is plus-or-minus the half-width of the predic-
regression is employed during data analysis.
tion interval associated with the regression line.
1.2 Applicable test methods are those whose results come
1.7 It is assumed that the user has access to a statistical
from regression-based methods and whose data are intra-
software package for performing regression. A statistician
laboratory (not inter-laboratory data, such as result from
should be consulted if assistance is needed in selecting such a
round-robinstudies).Foreachanalysisconductedusingsucha
program.
method, it is assumed that a fixed, reproducible amount of
1.8 A statistician also should be consulted if data transfor-
sample is introduced.
mations are being considered.
1.3 Calculationofthemeasurementuncertaintyinvolvesthe
1.9 This standard does not purport to address all of the
analysis of data collected to help characterize the analytical
safety concerns, if any, associated with its use. It is the
method over an appropriate concentration range. Example
responsibility of the user of this standard to establish appro-
sources of data include: 1) calibration studies (which may or
priate safety and health practices and determine the applica-
may not be conducted in pure solvent), 2) recovery studies
bility of regulatory limitations prior to use.
(which typically are conducted in matrix and include all
sample-preparation steps), and 3) collections of data obtained
2. Referenced Documents
as part of the method’s ongoing Quality Control program. Use
2.1 ASTM Standards:
of multiple instruments, multiple operators, or both, and
D1129Terminology Relating to Water
field-sampling protocols may or may not be reflected in the
data.
3. Terminology
1.4 In any designed study whose data are to be used to
3.1 Definitions of Terms Specific to This Standard:
calculate method uncertainty, the user should think carefully
3.1.1 confidence level—the probability that the prediction
about what the study is trying to accomplish and much
interval from a regression estimate will encompass the true
variation should be incorporated into the study. General guid-
value of the amount or concentration of the analyte in a
ance on designing studies (for example, calibration, recovery)
subsequent measurement. Typical choices for the confidence
is given in Appendix A. Detailed guidelines on sources of
level are 99% and 95%.
variation are outside the scope of this practice, but general
3.1.2 fitting technique—a method for estimating the param-
points to consider are included in Appendix B, which is not
eters of a mathematical model. For example, ordinary least
intended to be exhaustive. With any study, the user must think
squares is a fitting technique that may be used to estimate the
carefully about the factors involved with conducting the
parameters a,a,a ,… of the polynomial modely=a +a x
0 1 2 0 1
+a x +…, based on observed {x,y} pairs. Weighted least
squares is also a fitting technique.
This practice is under the jurisdiction ofASTM Committee D19 on Water and
is the direct responsibility of Subcommittee D19.02 on Quality Systems,
Specification, and Statistics. For referenced ASTM standards, visit the ASTM website, www.astm.org, or
Current edition approved Jan. 1, 2013. Published January 2013. Originally contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
approved in 2008. Last previous approval in 2008 as D7366 – 08. DOI: 10.1520/ Standards volume information, refer to the standard’s Document Summary page on
D7366-08R13. the ASTM website.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
D7366 − 08 (2013)
3.1.3 lack-of-fit (LOF) test—a statistical technique when 4.1.2 The total number of data points in any designed study
replicate data are available; computes the significance of should be kept high. Blanks may or may not be included,
residual means to replicate y variability, to indicate whether depending on the data-quality objectives of the test method.
deviations from model predictions are reasonably accounted 4.1.3 In applying regression to any applicable data set, the
for by random variability, thus indicating that the model is proper fitting technique (for example, ordinary least squares
adequate; at each concentration, compares the amount of (OLS) or weighted least squares (WLS)) must be determined
residual variation from model prediction with the amount of (for fitting the proposed model to the data).
residual variation from the observed mean. 4.1.4 The residual pattern and the lack-of-fit test are used to
evaluate the adequacy of the chosen model.
3.1.4 least squares—fitting technique that minimizes the
4.1.5 The magnitude of the half-width of the prediction
sum of squared residuals between observed y values and those
interval must be evaluated, remembering that accepting or
predicted by the model.
rejecting the amount of uncertainty is a judgment call, not a
3.1.5 model—mathematical expression (for example,
statistical decision.
straight line, quadratic) relating y (directly measured value) to
x (concentration or amount of analyte).
5. Significance and Use
3.1.6 ordinary least squares (OLS)—leastsquares,whereall
5.1 Appropriate application of this practice should result in
data points are given equal weight.
an estimate of the test-method’s uncertainty (at any concentra-
tion within the working range), which can be compared with
3.1.7 prediction interval—a pair of prediction limits (an
“upper”and“lower”)usedtobracketthe“next”observationat data-quality objectives to see if the uncertainty is acceptable.
a certain level of confidence.
5.2 With data sets that compare recovered concentration
3.1.8 p-value—the statistical significance of a test; the
withtrueconcentration,theresultingregressionplotallowsthe
probability value associated with a statistical test, representing correction of the recovery data to true values. Reporting of
the likelihood that a test statistic would assume or exceed a
such corrections is at the discretion of the user.
certainvaluepurelybychance,assumingthenullhypothesisis
5.3 This practice should be used to estimate the measure-
true(alowp-valueindicatesstatisticalsignificanceatalevelof
ment uncertainty for any application of a test method where
confidence equal to 1.0 minus the p-value).
measurement uncertainty is important to data use.
3.1.9 regression—an analysis technique for fitting a model
6. Procedure
to data; often used as a synonym for OLS.
6.1 Introduction:
3.1.10 residual—error in the fit between observed and
6.1.1 For purposes of this practice, only regression-based
modeled concentration; response minus fit.
methods are applicable. An example of a module that is not
3.1.11 root mean square error (RMSE)—an estimate of the
regression-based is a balance. If an object is placed on a
measurement standard deviation (that is, inherent variation in
balance, the readout is in the desired units; that is, in units of
the measurement system).
mass. No user intervention is required to get to the needed
3.1.12 significance level—the likelihood that a measured or
result. However, for an instrument such as a chromatograph or
observed result came about due to simple random behavior.
a spectrometer, the raw data (for example, peak area or
3.1.13 uncertainty (of a measurement)—the lack of exact- absorbance) must be transformed into meaningful units, typi-
ness in measurement (for example, due to sampling error,
cally concentration. Regression is at the core of this transfor-
measurement variation, and model inexactness); a statistical mation process.
interval within which the measurement error is believed to
6.1.2 One additional distinction will be made regarding the
occur, at some level of confidence. applicability of this protocol. This practice will deal only with
intralaboratory data. In other words, the variability introduced
3.1.14 weight—coefficient assigned to observations in order
by collecting results from more than one lab is not being
to manipulate their relative influence in subsequent calcula-
considered. The examples that are shown here are for one
tions. For example, in weighted least squares, noisy observa-
method with one operator. If the user wishes, additional
tionsareweighteddownwards,whileprecisedataareweighted
operators may be included in the design, to capture multiple-
upwards.
operator variability.
3.1.15 weighted least squares (WLS)—least squares, where
6.1.3 Abrief example will help illustrate the importance of
data points are weighted inversely proportional to their vari-
estimating measurement uncertainty. A sample is to be ana-
ance (“noisiness”).
lyzed to determine if it is under the upper specification limit of
5(theactualunitsofconcentrationdonotmatter).Thefinaltest
4. Summary of Practice
result is 4.5. The question then is whether the sample should
4.1 Key points of the statistical protocol for measurement
pass or fail. Clearly, 4.5 is less than 5. If the numbers are
uncertainty are: treated as being absolute, then the sample will pass. However,
4.1.1 Withintheworkingrangeofthemethod’sdataset,the such a judgment call ignores the variability that always exists
estimate of the method uncertainty at any given concentration with a measurement. The width of any measurement’s uncer-
is calculated to be plus-or-minus the half-width of the predic- tainty interval depends not only on the noisiness of the data,
tion interval. butalsoontheconfidenceleveltheuserwishestoassume.This
D7366 − 08 (2013)
latter consideration is not a statistical decision, but a reasoned square of the actual standard deviation has also been used.
decision that must be based on the needs of the customer, the However, the preferred formula comes from modeling the
intendeduseofthedata,orboth.Oncetheconfidencelevelhas standard deviation. In other words, the actual standard-
been chosen, the interval can be calculated from the data. In deviation values are plotted versus true concentration; an
this example, if the uncertainty is determined to be 61.0, then appropriate model is then fitted to the data. The reciprocal
there is serious doubt as to whether the sample passes or not, square of the equation for the line is then used to calculate the
since the true value could be anywhere between 3.5 and 5.5. weights.Thesimplestmodelisastraightline,butmoreprecise
On the other hand, if the uncertainty is only 60.1, then the modeling should be done if the situation requires it. (In
sample could be passed with a high level of comfort. Only by practice, it is best to normalize the weight formula by dividing
making a sound evaluation of the uncertainty can the user by the sum of all the reciprocal squares. This process assures
determine how to apply the sample estimate he or she has that the root mean square error is correct.)
obtained. The following protocol is designed to answer ques-
6.2.2.3 In sum, two choices, which are independent of each
tions such as: 4.5 6 ?
other, must be made in performing regression. These two
choices are a model and a fitting technique. In practice, the
6.2 Regression Diagnostics for Recovery Data:
optionsforthemodelaretypicallyastraightlineoraquadratic,
6.2.1 Analysts who routinely use chromatographs and spec-
while the customary choices for the fitting technique are
trometersarefamiliarwiththebasicsoftheregressionprocess.
ordinary least squares and weighted least squares.
Thefinalresultsare:1)aplotthatvisuallyrelatestheresponses
6.2.2.4 However, a straight line is not automatically associ-
(on the y-axis) to the true concentrations (on the x-axis) and 2)
ated with OLS, nor is a quadratic automatically paired with
an equation that mathematically relates the two variables.
WLS. The fitting technique depends solely on the behavior of
6.2.2 Underlying these results are two basic choices: (1)a
the response standard deviations (that is, do they trend with
model, such as a straight line or some sort of curved line, and
concentrations). The model choice is not related to these
(2) a fitting technique, which is a version of least squares. The
standarddeviations,butdependsprimarilyonwhetherthedata
modeling choices are generally well known to most analysts,
points exhibit some type of curvature.
but the fitting-technique choices are typically less well under-
6.2.3 Once an appropriate model and fitting technique have
stood.The two most common forms of least-squares fitting are
been chosen, the regression line and plot can be determined.
discussed next.
One other very important feature can also be calculated and
6.2.2.1 Ordinary least squares (OLS) assumes that the
graphed. That feature is the prediction interval, which is an
variance of the responses does not trend with concentration. If
“envelope” around the line itself and which reports the
thevariancedoestrendwithconcentration,thenweightedleast
uncertainty (at the chosen confidence level) in a future mea-
squares(WLS)isneeded.InWLS,dataareweightedaccording
surement predicted from the line.An example is given in Fig.
to how noisy they are. Values that have relatively low uncer-
1.The solid red line is the regression line; the dashed red lines
tainty are considered to be more reliable and are subsequently
form the prediction interval.
afforded higher weights (and therefore more influence on the
regression line) than are the more uncertain values. 6.2.4 While the concept of a model is familiar to most
6.2.2.2 Several formulas have been used for calculating the analysts, the statistically sound process for selecting an ad-
weights. The simplest is 1/x (where x = true concentration), equatemodeltypicallyisnot.A
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.