ASTM D7366-08(2019)
(Practice)Standard Practice for Estimation of Measurement Uncertainty for Data from Regression-based Methods
Standard Practice for Estimation of Measurement Uncertainty for Data from Regression-based Methods
SIGNIFICANCE AND USE
5.1 Appropriate application of this practice should result in an estimate of the test-method’s uncertainty (at any concentration within the working range), which can be compared with data-quality objectives to see if the uncertainty is acceptable.
5.2 With data sets that compare recovered concentration with true concentration, the resulting regression plot allows the correction of the recovery data to true values. Reporting of such corrections is at the discretion of the user.
5.3 This practice should be used to estimate the measurement uncertainty for any application of a test method where measurement uncertainty is important to data use.
SCOPE
1.1 This practice establishes a standard for computing the measurement uncertainty for applicable test methods in Committee D19 on Water. The practice does not provide a single-point estimate for the entire working range, but rather relates the uncertainty to concentration. The statistical technique of regression is employed during data analysis.
1.2 Applicable test methods are those whose results come from regression-based methods and whose data are intra-laboratory (not inter-laboratory data, such as result from round-robin studies). For each analysis conducted using such a method, it is assumed that a fixed, reproducible amount of sample is introduced.
1.3 Calculation of the measurement uncertainty involves the analysis of data collected to help characterize the analytical method over an appropriate concentration range. Example sources of data include: (1) calibration studies (which may or may not be conducted in pure solvent), (2) recovery studies (which typically are conducted in matrix and include all sample-preparation steps), and (3) collections of data obtained as part of the method’s ongoing Quality Control program. Use of multiple instruments, multiple operators, or both, and field-sampling protocols may or may not be reflected in the data.
1.4 In any designed study whose data are to be used to calculate method uncertainty, the user should think carefully about what the study is trying to accomplish and much variation should be incorporated into the study. General guidance on designing studies (for example, calibration, recovery) is given in Appendix X1. Detailed guidelines on sources of variation are outside the scope of this practice, but general points to consider are included in Appendix X2, which is not intended to be exhaustive. With any study, the user must think carefully about the factors involved with conducting the analysis, and must realize that the computed measurement uncertainty will reflect the quality of the input data.
1.5 Associated with the measurement uncertainty is a user-chosen level of statistical confidence.
1.6 At any concentration in the working range, the measurement uncertainty is plus-or-minus the half-width of the prediction interval associated with the regression line.
1.7 It is assumed that the user has access to a statistical software package for performing regression. A statistician should be consulted if assistance is needed in selecting such a program.
1.8 A statistician also should be consulted if data transformations are being considered.
1.9 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility of the user of this standard to establish appropriate safety, health, and environmental practices and determine the applicability of regulatory limitations prior to use.
1.10 This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
General Information
Relations
Standards Content (Sample)
This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation: D7366 − 08 (Reapproved 2019)
Standard Practice for
Estimation of Measurement Uncertainty for Data from
Regression-based Methods
This standard is issued under the fixed designation D7366; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope analysis, and must realize that the computed measurement
uncertainty will reflect the quality of the input data.
1.1 This practice establishes a standard for computing the
1.5 Associated with the measurement uncertainty is a user-
measurement uncertainty for applicable test methods in Com-
chosen level of statistical confidence.
mittee D19 on Water. The practice does not provide a single-
point estimate for the entire working range, but rather relates
1.6 Atanyconcentrationintheworkingrange,themeasure-
the uncertainty to concentration. The statistical technique of
ment uncertainty is plus-or-minus the half-width of the predic-
regression is employed during data analysis.
tion interval associated with the regression line.
1.2 Applicable test methods are those whose results come
1.7 It is assumed that the user has access to a statistical
from regression-based methods and whose data are intra-
software package for performing regression. A statistician
laboratory (not inter-laboratory data, such as result from
should be consulted if assistance is needed in selecting such a
round-robinstudies).Foreachanalysisconductedusingsucha
program.
method, it is assumed that a fixed, reproducible amount of
1.8 A statistician also should be consulted if data transfor-
sample is introduced.
mations are being considered.
1.3 Calculationofthemeasurementuncertaintyinvolvesthe
1.9 This standard does not purport to address all of the
analysis of data collected to help characterize the analytical
safety concerns, if any, associated with its use. It is the
method over an appropriate concentration range. Example
responsibility of the user of this standard to establish appro-
sources of data include: (1) calibration studies (which may or
priate safety, health, and environmental practices and deter-
may not be conducted in pure solvent), (2) recovery studies
mine the applicability of regulatory limitations prior to use.
(which typically are conducted in matrix and include all
1.10 This international standard was developed in accor-
sample-preparation steps), and (3) collections of data obtained
dance with internationally recognized principles on standard-
as part of the method’s ongoing Quality Control program. Use
ization established in the Decision on Principles for the
of multiple instruments, multiple operators, or both, and
Development of International Standards, Guides and Recom-
field-sampling protocols may or may not be reflected in the
mendations issued by the World Trade Organization Technical
data.
Barriers to Trade (TBT) Committee.
1.4 In any designed study whose data are to be used to
2. Referenced Documents
calculate method uncertainty, the user should think carefully
about what the study is trying to accomplish and much
2.1 ASTM Standards:
variation should be incorporated into the study. General guid-
D1129Terminology Relating to Water
ance on designing studies (for example, calibration, recovery)
3. Terminology
is given in Appendix X1. Detailed guidelines on sources of
variation are outside the scope of this practice, but general
3.1 Definitions:
points to consider are included in Appendix X2, which is not
3.1.1 For definitions of terms used in this standard, refer to
intended to be exhaustive. With any study, the user must think
Terminology D1129.
carefully about the factors involved with conducting the
3.2 Definitions of Terms Specific to This Standard:
3.2.1 confidence level, n—theprobabilitythattheprediction
interval from a regression estimate will encompass the true
This practice is under the jurisdiction ofASTM Committee D19 on Water and
is the direct responsibility of Subcommittee D19.02 on Quality Systems,
Specification, and Statistics. For referenced ASTM standards, visit the ASTM website, www.astm.org, or
Current edition approved Dec. 1, 2019. Published January 2020. Originally contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
approved in 2008. Last previous approval in 2013 as D7366 – 08 (2013). DOI: Standards volume information, refer to the standard’s Document Summary page on
10.1520/D7366-08R19. the ASTM website.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
D7366 − 08 (2019)
value of the amount or concentration of the analyte in a 4. Summary of Practice
subsequent measurement. Typical choices for the confidence
4.1 Key points of the statistical protocol for measurement
level are 99% and 95%.
uncertainty are:
3.2.2 fitting technique, n—a method for estimating the
4.1.1 Withintheworkingrangeofthemethod’sdataset,the
parameters of a mathematical model. For example, ordinary
estimate of the method uncertainty at any given concentration
least squares is a fitting technique that may be used to estimate
is calculated to be plus-or-minus the half-width of the predic-
the parameters a , a , a ,… of the polynomial model y = a +
tion interval.
0 1 2 0
a x + a x +…, based on observed {x,y} pairs.Weighted least
1 2 4.1.2 The total number of data points in any designed study
squares is also a fitting technique.
should be kept high. Blanks may or may not be included,
3.2.3 lack-of-fit (LOF) test, n—a statistical technique when depending on the data-quality objectives of the test method.
4.1.3 In applying regression to any applicable data set, the
replicate data are available; computes the significance of
residual means to replicate y variability, to indicate whether proper fitting technique (for example, ordinary least squares
(OLS) or weighted least squares (WLS)) must be determined
deviations from model predictions are reasonably accounted
(for fitting the proposed model to the data).
for by random variability, thus indicating that the model is
4.1.4 The residual pattern and the lack-of-fit (LOF) test are
adequate; at each concentration, compares the amount of
used to evaluate the adequacy of the chosen model.
residual variation from model prediction with the amount of
4.1.5 The magnitude of the half-width of the prediction
residual variation from the observed mean.
interval must be evaluated, remembering that accepting or
3.2.4 least squares, n—fitting technique that minimizes the
rejecting the amount of uncertainty is a judgment call, not a
sum of squared residuals between observed y values and those
statistical decision.
predicted by the model.
3.2.5 model, n—mathematical expression (for example,
5. Significance and Use
straight line, quadratic) relating y (directly measured value) to
5.1 Appropriate application of this practice should result in
x (concentration or amount of analyte).
an estimate of the test-method’s uncertainty (at any concentra-
3.2.6 ordinary least squares (OLS), n—least squares, where
tion within the working range), which can be compared with
all data points are given equal weight.
data-quality objectives to see if the uncertainty is acceptable.
3.2.7 prediction interval, n—a pair of prediction limits (an
5.2 With data sets that compare recovered concentration
“upper”and“lower”)usedtobracketthe“next”observationat
withtrueconcentration,theresultingregressionplotallowsthe
a certain level of confidence.
correction of the recovery data to true values. Reporting of
3.2.8 p-value, n—the statistical significance of a test; the
such corrections is at the discretion of the user.
probability value associated with a statistical test, representing
5.3 This practice should be used to estimate the measure-
the likelihood that a test statistic would assume or exceed a
ment uncertainty for any application of a test method where
certainvaluepurelybychance,assumingthenullhypothesisis
measurement uncertainty is important to data use.
true(alow p-valueindicatesstatisticalsignificanceatalevelof
confidence equal to 1.0 minus the p-value). 6. Procedure
3.2.9 regression, n—an analysis technique for fitting a
6.1 Introduction:
model to data; often used as a synonym for OLS.
6.1.1 For purposes of this practice, only regression-based
3.2.10 residual, n—error in the fit between observed and methods are applicable. An example of a module that is not
modeled concentration; response minus fit. regression-based is a balance. If an object is placed on a
balance, the readout is in the desired units; that is, in units of
3.2.11 root mean square error (RMSE), n—an estimate of
mass. No user intervention is required to get to the needed
the measurement standard deviation (that is, inherent variation
result. However, for an instrument such as a chromatograph or
in the measurement system).
a spectrometer, the raw data (for example, peak area or
3.2.12 significance level, n—the likelihood that a measured
absorbance) must be transformed into meaningful units, typi-
or observed result came about due to simple random behavior.
cally concentration. Regression is at the core of this transfor-
3.2.13 uncertainty (of a measurement), n—the lack of ex-
mation process.
actness in measurement (for example, due to sampling error,
6.1.2 One additional distinction will be made regarding the
measurement variation, and model inexactness); a statistical
applicability of this protocol. This practice will deal only with
interval within which the measurement error is believed to
intralaboratory data. In other words, the variability introduced
occur, at some level of confidence.
by collecting results from more than one lab is not being
3.2.14 weight, n—coefficient assigned to observations in
considered. The examples that are shown here are for one
order to manipulate their relative influence in subsequent
method with one operator. If the user wishes, additional
calculations. For example, in weighted least squares, noisy
operators may be included in the design, to capture multiple-
observations are weighted downwards, while precise data are
operator variability.
weighted upwards.
6.1.3 Abrief example will help illustrate the importance of
3.2.15 weighted least squares (WLS), n—least squares, estimating measurement uncertainty. A sample is to be ana-
where data points are weighted inversely proportional to their lyzed to determine if it is under the upper specification limit of
variance (“noisiness”). 5(theactualunitsofconcentrationdonotmatter).Thefinaltest
D7366 − 08 (2019)
result is 4.5. The question then is whether the sample should choices are a model and a fitting technique. In practice, the
pass or fail. Clearly, 4.5 is less than 5. If the numbers are optionsforthemodelaretypicallyastraightlineoraquadratic,
treated as being absolute, then the sample will pass. However,
while the customary choices for the fitting technique are OLS
such a judgment call ignores the variability that always exists
and WLS.
with a measurement. The width of any measurement’s uncer-
6.2.2.4 However, a straight line is not automatically associ-
tainty interval depends not only on the noisiness of the data,
ated with OLS, nor is a quadratic automatically paired with
butalsoontheconfidenceleveltheuserwishestoassume.This
WLS. The fitting technique depends solely on the behavior of
latter consideration is not a statistical decision, but a reasoned
the response standard deviations (that is, do they trend with
decision that must be based on the needs of the customer, the
concentrations). The model choice is not related to these
intendeduseofthedata,orboth.Oncetheconfidencelevelhas
standarddeviations,butdependsprimarilyonwhetherthedata
been chosen, the interval can be calculated from the data. In
points exhibit some type of curvature.
this example, if the uncertainty is determined to be 61.0, then
6.2.3 Once an appropriate model and fitting technique have
there is serious doubt as to whether the sample passes or not,
been chosen, the regression line and plot can be determined.
since the true value could be anywhere between 3.5 and 5.5.
One other very important feature can also be calculated and
On the other hand, if the uncertainty is only 60.1, then the
graphed. That feature is the prediction interval, which is an
sample could be passed with a high level of comfort. Only by
“envelope” around the line itself and which reports the
making a sound evaluation of the uncertainty can the user
uncertainty (at the chosen confidence level) in a future mea-
determine how to apply the sample estimate he or she has
surement predicted from the line.An example is given in Fig.
obtained. The following protocol is designed to answer ques-
tions such as: 4.5 6 ? 1.The solid red line is the regression line; the dashed red lines
form the prediction interval.
6.2 Regression Diagnostics for Recovery Data:
6.2.4 While the concept of a model is familiar to most
6.2.1 Analysts who routinely use chromatographs and spec-
analysts, the statistically sound process for selecting an ad-
trometersarefamiliarwiththebasicsoftheregressionprocess.
equatemodeltypicallyisnot.Aseriesofregressiondiagnostics
The final results are: (1) a plot that visually relates the
will guide the user. The basic steps are as follows, and can be
responses (on the y-axis) to the true concentrations (on the
carried out with most statistical software packages that are
x-axis) and (2) an equation that mathematically relates the two
commercially available:
variables.
(1)Plot y versus x,
6.2.2 Underlying these results are two basic choices: (1)a
(2)Determine the behavior of y’s standard deviation,
model, such as a straight line or some sort of curved line, and
(3)Fit proposed model,
(2) a fitting technique, which is a version of least squares. The
modeling choices are generally well known to most analysts, (4)Examine residuals,
(5)Conduct LOF test, and
but the fitting-technique choices are typically less well under-
stood.The two most common forms of least-squares fitting are (6)Evaluate prediction interval.
discussed next.
Step 1 generates a scatterplot. This graph is helpful for
6.2.2.1 OLSassumesthatthevarianceoftheresponsesdoes
spotting potential errant data points (which may simply be due
not trend with concentration. If the variance does trend with
to typographical errors in the data table), as well as for getting
concentration,thenWLSisneeded.InWLS,dataa
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.