ASTM E2554-07
(Practice)Standard Practice for Estimating and Monitoring the Uncertainty of Test Results of a Test Method in a Single Laboratory Using a Control Sample Program
Standard Practice for Estimating and Monitoring the Uncertainty of Test Results of a Test Method in a Single Laboratory Using a Control Sample Program
SIGNIFICANCE AND USE
Laboratories accredited under ISO 17025 are required to present uncertainty estimates for their test results. This practice provides procedures that use test results to develop uncertainty estimates for an individual laboratory.
Generally, these test results will be from a single sample of stable and homogeneous material known as a control or check sample.
The true value of the characteristic(s) of the control sample being measured will ordinarily be unknown. However, this methodology may also be used if the control sample is a reference material, in which case the test method bias may also be estimated and incorporated into the uncertainty estimate. Many test methods do not have true reference materials available to provide traceable chains of uncertainty estimation.
This practice also allows for ongoing monitoring of the laboratory uncertainty. As estimates of the level of uncertainty change, possibly as contributions to uncertainty are identified and minimized, revision to the laboratory uncertainty will be possible.
SCOPE
1.1 This practice describes techniques for a laboratory to estimate the uncertainty of a test result using data from test results on a control sample.
1.2 Uncertainty as defined by this practice applies to the capabilities of a single laboratory. Any estimate of uncertainty determined through the use of this practice applies only to the individual laboratory for which the data are presented.
1.3 The laboratory uses a well defined and established test method in determining a series of test results. The uncertainty estimated using this practice only applies when the same test method is followed. The uncertainty only applies for the material types represented by the control samples, and multiple control samples may be needed, especially if the method has different precision for different sample types or response levels.
1.4 The uncertainty estimate determined by this practice represents the intermediate precision of test results. This estimate seeks to quantify the total variation expected within a single laboratory using a single established test method while incorporating as many known sources of variation as possible.
1.5 This practice does not establish error estimates (error budget) attributed to individual factors that could influence uncertainty.
1.6 This practice describes the use of control charts to evaluate the data obtained and presents a special type of control chart to monitor the estimate of uncertainty.
1.7 The system of units for this Standard is not specified. Dimensional quantities in the Standard are presented only as illustrations of calculation methods. The examples are not binding on products or test methods treated.
This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility of the user of this standard to establish appropriate safety and health practices and determine the applicability of regulatory limitations prior to use.
General Information
Relations
Standards Content (Sample)
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Contact ASTM International (www.astm.org) for the latest information
Designation: E2554 − 07 AnAmerican National Standard
Standard Practice for
Estimating and Monitoring the Uncertainty of Test Results
of a Test Method in a Single Laboratory Using a Control
Sample Program
This standard is issued under the fixed designation E2554; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope priate safety and health practices and determine the applica-
bility of regulatory limitations prior to use.
1.1 This practice describes techniques for a laboratory to
estimate the uncertainty of a test result using data from test
2. Referenced Documents
results on a control sample.
2.1 ASTM Standards:
1.2 Uncertainty as defined by this practice applies to the
D5184 Test Methods for Determination of Aluminum and
capabilities of a single laboratory.Any estimate of uncertainty
Silicon in Fuel Oils by Ashing, Fusion, Inductively
determined through the use of this practice applies only to the
Coupled Plasma Atomic Emission Spectrometry, and
individual laboratory for which the data are presented.
Atomic Absorption Spectrometry
1.3 The laboratory uses a well defined and established test E177 Practice for Use of the Terms Precision and Bias in
method in determining a series of test results. The uncertainty
ASTM Test Methods
estimated using this practice only applies when the same test E456 Terminology Relating to Quality and Statistics
method is followed. The uncertainty only applies for the
51707 Guide for Estimating Uncertainties in Dosimetry for
materialtypesrepresentedbythecontrolsamples,andmultiple Radiation Processing
control samples may be needed, especially if the method has
E2282 Guide for Defining the Test Result of a Test Method
differentprecisionfordifferentsampletypesorresponselevels. 2.2 ASTM Publication:
Manual on Presentation of Data and Control Chart Analy-
1.4 The uncertainty estimate determined by this practice
sis, 7th Edition
represents the intermediate precision of test results. This
2.3 ISO Standard:
estimate seeks to quantify the total variation expected within a
ISO 17025 General Requirements for the Competence of
single laboratory using a single established test method while
Testing and Calibration Laboratories
incorporating as many known sources of variation as possible.
1.5 This practice does not establish error estimates (error
3. Terminology
budget) attributed to individual factors that could influence
3.1 Definitions:
uncertainty.
3.1.1 The terminology of Terminology E456 applies to this
1.6 This practice describes the use of control charts to
practice except as modified herein.
evaluatethedataobtainedandpresentsaspecialtypeofcontrol
3.1.2 control sample, n—sample taken from a stable, homo-
chart to monitor the estimate of uncertainty.
geneous material for the purposes of monitoring the perfor-
mance of a test method in a laboratory.
1.7 The system of units for this Standard is not specified.
3.1.2.1 Discussion—The control sample material is repre-
Dimensional quantities in the Standard are presented only as
sentativeoftheproducttypicallytestedinthelaboratorybythe
illustrations of calculation methods. The examples are not
given test method. A control sample is run periodically using
binding on products or test methods treated.
thecompletetestmethodprotocoltodevelopatestresult.Such
1.8 This standard does not purport to address all of the
test results may be statistically evaluated to monitor test
safety concerns, if any, associated with its use. It is the
method performance over time. It is not necessary to have an
responsibility of the user of this standard to establish appro-
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
This practice is under the jurisdiction ofASTM Committee E11 on Quality and contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
Statistics and is the direct responsibility of Subcommittee E11.20 on Test Method Standards volume information, refer to the standard’s Document Summary page on
Evaluation and Quality Control. the ASTM website.
Current edition approved May 1, 2007. Published June 2007. DOI: 10.1520/ Available fromAmerican National Standards Institute (ANSI), 25 W. 43rd St.,
E2554-07. 4th Floor, New York, NY 10036, http://www.ansi.org.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E2554 − 07
accepted reference value assigned to the control sample mate- reference material, in which case the test method bias may also
rial. When the current material is nearly consumed, a replace- be estimated and incorporated into the uncertainty estimate.
ment material should be run in parallel with the current Many test methods do not have true reference materials
material to ensure continuity in the control sample program. available to provide traceable chains of uncertainty estimation.
3.1.3 check sample, n—see control sample.
5.4 This practice also allows for ongoing monitoring of the
laboratory uncertainty.As estimates of the level of uncertainty
3.1.4 uncertainty control chart, n—control chart that in-
change, possibly as contributions to uncertainty are identified
cludes control limits based on the variation attributed to the
and minimized, revision to the laboratory uncertainty will be
uncertainty of the test method.
possible.
3.1.5 intermediate precision, n—the closeness of agreement
between test results obtained under specified intermediate
6. General Considerations
precision conditions. E177
6.1 Materials to be Used:
3.1.5.1 Discussion—The specific measure and the specific
6.1.1 This methodology requires a quantity of stable and
conditions must be specified for each intermediate measure of
precision; thus, “standard deviation of test results among homogeneous material which will serve as the source of
control samples (sometimes called check samples). The mate-
operators in a laboratory,” or “day-to-day standard deviation
within a laboratory for the same operator.” rial shall be similar in composition to the samples of material
routinely analyzed by this test method in this laboratory. By
3.1.5.2 Discussion—Because the training of operators, the
stable it is assumed that the test results obtained from this
agreement of different pieces of equipment in the same
material should be consistent over the time interval that this
laboratory and the variation of environmental conditions with
material will be used. By homogeneous it is assumed that
longer time intervals all depend on the degree of within-
samples taken from the material source will not have a
laboratory control, the intermediate measures of precision are
significant variation in the characteristic measured by the test
likely to vary appreciably from laboratory to laboratory. Thus,
method.
intermediate precisions may be more characteristic of indi-
6.1.2 For destructive testing of control sample materials,
vidual laboratories than of the test method.
provision shall be made for depletion and replacement of the
3.1.6 test result, n—the value of a characteristic obtained by
control sample material.
carrying out a specified test method. E2282
6.1.2.1 In some cases, the test method may be nondestruc-
3.1.7 repeatability, n—precision under repeatability
tive and the same material may be reused indefinitely.
conditions. E177
6.1.2.2 In other cases, the material may be used up,
deteriorate, or otherwise gradually change.
4. Summary of Practice
6.1.3 The test method should describe the best practices for
4.1 A standard material or control sample is measured
preparing and storing the control material and taking the
repeatedly over time. The presumption of this practice is that
control samples.
the variation experienced on this material will be indicative of
6.2 Test Conditions:
the laboratory total expected variation. Incorporation of spe-
6.2.1 An uncertainty estimation program should be de-
cific known or potential sources of variation in the testing
signed to include all known sources of variation, such as
program is encouraged.
operators (analysts), equipment, reagents, and so forth, and
4.2 Acontrol chart is prepared and the results are evaluated
these should be deliberately incorporated into the design of the
to identify short-term variation and longer-term variation.
program. In general, these sources of variation will be defined
These data can then be used to determine an estimate of
(including acceptable tolerances) by the test method.
uncertainty standard deviation.
6.2.2 In cases in which control over such variations is not
4.3 Laboratories already having control chart procedures in
possible or undefined, at least 30 to 50 sampling periods shall
place may use existing data.
be evaluated to permit environmental and other factors to be
incorporated in the overall estimate.
4.4 Ongoing monitoring of the test method is conducted
using an uncertainty control chart.
7. Overall Procedure—Control Charting Methods
5. Significance and Use
7.1 General concepts of control charts are described
5.1 LaboratoriesaccreditedunderISO17025arerequiredto
elsewhere, such as in Manual 7A.
present uncertainty estimates for their test results.This practice
7.2 The general procedure involves two major phases:
provides procedures that use test results to develop uncertainty
Preliminary and Monitoring.
estimates for an individual laboratory.
7.2.1 Preliminary Phase:
5.2 Generally, these test results will be from a single sample
7.2.1.1 This phase begins with an initial collection of test
of stable and homogeneous material known as a control or
results.
check sample.
5.3 The true value of the characteristic(s) of the control
sample being measured will ordinarily be unknown. However,
Manual on Presentation of Data and Control Chart Analysis: 7th Edition,
this methodology may also be used if the control sample is a ASTM International, West Conshohocken, PA, 2001.
E2554 − 07
7.2.1.2 Preliminary control charts are then prepared and less. It is preferred that at least 25 sets of test results be
examined. These charts are evaluated to determine if the obtained before developing the charts.
process is in a state of statistical control. The usual principles
8.1.2 Either a range chart or a standard deviation chart is
of control charting utilize short-term variability to estimate the
prepared. This is examined for special cause variation. If the
limits within which samples of test results should vary. For
variability appears random then an estimate of repeatability is
control sample programs this short-term variability is equiva-
computed. This may be done by pooling the sums of squares,
lent to repeatability precision. It is expected, however, that
using the average standard deviation, or using the average
additional contributions to variation will be present over time
range.
and therefore additional variation, equivalent to intermediate
NOTE 1—If the ranges or standard deviations are zero in most of the
precision, will be encountered.
samples, then this estimate of repeatability standard deviation is suspect
7.2.1.3 An estimate of uncertainty standard deviation is
and probably unusable. This is usually the result of insufficient resolution
developed.
of the measurement system in use or severe rounding. An estimate based
on the minimum interval size should be substituted for the zeros.As a rule
7.2.1.4 An uncertainty control chart is then prepared to
of thumb, consider replacing the zeros when more than about ⁄3 are zeros.
monitor future sample results.
8.1.3 A means chart is used to examine variation among
7.2.2 Monitoring Phase:
time periods. Limits on this chart permit comparison of
7.2.2.1 The proposed uncertainty control chart is used to
variation between time periods using repeatability as the
provide evidence that the estimate of uncertainty is not
estimate of error.
exceeding the estimated value.
8.1.3.1 Ifthecontrolchartshowsastateofstatisticalcontrol
7.2.2.2 The estimate of uncertainty should be periodically
then the uncertainty will be assumed approximately equivalent
re-evaluated.
to the repeatability standard deviation.
7.2.2.3 Where appropriate, it is recommended that a stan-
8.1.3.2 In most cases it will be expected that the variability
dard control chart also be maintained to determine whether the
between means will show an “out of control” condition
variation over time has been reduced to the level of short-term
indicating that there are “special” causes of variation in
variation (repeatability).
addition to repeatability. The between means variation and
7.3 Two types of control charting methods are recom-
within means repeatability estimates are then used to compute
mended to develop estimates of uncertainty. These include:
an estimate of uncertainty standard deviation.
7.3.1 Mean (Xbar) and range or standard deviation charts
8.1.4 Using the estimate of uncertainty standard deviation
are used when multiple test results are conducted in each time
an Uncertainty Control Chart is prepared for future monitoring
period.
of the uncertainty. This chart may include control limits for
7.3.2 Individual charts (IndX) are used when single test
means as a possible lower set of control limits along with the
results are obtained in each time period.
uncertainty control limits based on the estimate of uncertainty.
7.4 Variation Estimates:
8.2 Individual Tests:
7.4.1 Either a range chart or a standard deviation chart may
8.2.1 Single tests are generated at each time period. Varia-
be used to estimate the short-term variability when multiple
tion among these results is evaluated.
assays are conducted under repeatability conditions per time
8.2.2 In some cases, it is possible to incorporate external
period. An estimate from the control chart data can be
estimates of repeatability obtained from prior or concurrent
compared to other estimates of repeatability (within laboratory,
studies.
short-term variation) if available.
7.4.2 Sample averages are examined and may provide
9. Multiple Readings per Time Period
estimates of variation caused by other factors. Such factors
9.1 Example 1—Absorbance of Radiochromic Dosimeters:
may include environmental effects, operator factors, reagents,
9.1.1 Over a period of several days, different sets of three
or instruments.
dosimeters were irradiated to the same nominal dose. The
7.5 Systematic Procedures:
irradiation was conducted under standard conditions at a single
7.5.1 Specifically designed experiments can be used to
irradiator facility. Possible sources of
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.