ASTM D6617-00
(Practice)Standard Practice for Laboratory Bias Detection Using Single Test Result from Standard Material
Standard Practice for Laboratory Bias Detection Using Single Test Result from Standard Material
SCOPE
1.1 This practice covers a methodology for establishing an acceptable tolerance zone for the difference between the result obtained from a single implementation of a test method on a CS and its ARV, based on user-specified Type I error, the user-established test method precision, the standard error of the ARV, and a presumed hypothesis that the laboratory is performing the test method without bias.Note 1
Throughout this practice, the term user refers to the user of this practice; and the term laboratory (see 1.1) refers to the organization or entity that is performing the test method.
1.2 For the tolerance zone established in 1.1, a methodology is presented to estimate the probability that the single test result will fall outside the zone, in the event that there is a bias (positive or negative) of a user-specified magnitude that is deemed to be of practical concern (that is, the presumed hypothesis is not true).
1.3 This practice is intended for ASTM Committee D02 test methods that produce results on a continuous numerical scale.
1.4 This practice assumes that the normal (Gaussian) model is adequate for the description and prediction of measurement system behavior when it is in a state of statistical control.
Note 2--While this practice does not cover scenarios in which multiple results are obtained on the same CS under site precision or repeatability conditions, the statistical concepts presented are applicable. Users wishing to apply these concepts for the scenarios described are advised to consult a statistician and to reference the CS methodology described in Practice D6299.
General Information
Relations
Standards Content (Sample)
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Contact ASTM International (www.astm.org) for the latest information
American National Standard
Designation:D6617–00
Standard Practice for
Laboratory Bias Detection Using Single Test Result from
Standard Material
This standard is issued under the fixed designation D 6617; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (e) indicates an editorial change since the last revision or reapproval.
INTRODUCTION
Due to the inherent imprecision in all test methods, a laboratory cannot expect to obtain the
numerically exact accepted reference value (ARV) of a check standard (CS) material every time one
is tested. Results that are reasonably close to theARV should provide assurance that the laboratory is
performing the test method either without bias, or with a bias that is of no practical concern, hence
requiring no intervention. Results differing from the ARV by more than a certain amount, however,
should lead the laboratory to take corrective action.
1. Scope 2. Referenced Documents
1.1 This practice covers a methodology for establishing an 2.1 ASTM Standards:
acceptable tolerance zone for the difference between the result D 2699 Test Method for Research Octane Number of
obtained from a single implementation of a test method on a Spark-Ignition Engine Fuel
CS and its ARV, based on user-specified Type I error, the D 6299 Practice for Applying Statistical Quality Assurance
user-establishedtestmethodprecision,thestandarderrorofthe Techniques to Evaluate Analytical Measurement System
ARV, and a presumed hypothesis that the laboratory is per- Performance
forming the test method without bias. E 178 Practice for Dealing with Outlying Observations
NOTE 1—Throughout this practice, the term user refers to the user of
3. Terminology
this practice; and the term laboratory (see 1.1) refers to the organization or
3.1 Definitions for accepted reference value (ARV), accu-
entity that is performing the test method.
racy, bias, check standard (CS), in statistical control, site
1.2 For the tolerance zone established in 1.1, a methodology
precision, site precision standard deviation (s ), site preci-
SITE
ispresentedtoestimatetheprobabilitythatthesingletestresult
sion conditions, repeatability conditions, and reproducibility
will fall outside the zone, in the event that there is a bias
conditions can be found in Practice D 6299.
(positive or negative) of a user-specified magnitude that is
3.2 Definitions of Terms Specific to This Standard:
deemed to be of practical concern (that is, the presumed
3.2.1 acceptable tolerance zone, n—a numerical zone
hypothesis is not true).
bounded inclusively by zero 6 k e (k is a value based on a
1.3 This practice is intended forASTM Committee D02 test
user-specifiedTypeIerror; eisdefinedin3.2.7)suchthatifthe
methods that produce results on a continuous numerical scale.
differencebetweentheresultobtainedfromasingleimplemen-
1.4 This practice assumes that the normal (Gaussian) model
tation of a test method for a CS and its ARV falls inside this
is adequate for the description and prediction of measurement
zone, the presumed hypothesis that the laboratory or testing
system behavior when it is in a state of statistical control.
organization is performing the test method without bias is
NOTE 2—Whilethispracticedoesnotcoverscenariosinwhichmultiple
accepted, and the difference is attributed to normal random
results are obtained on the same CS under site precision or repeatability
variation of the test method. Conversely, if the difference falls
conditions,thestatisticalconceptspresentedareapplicable.Userswishing
outside this zone, the presumed hypothesis is rejected.
to apply these concepts for the scenarios described are advised to consult
3.2.2 consensus check standard (CCS), n—aspecialtypeof
a statistician and to reference the CS methodology described in Practice
CS in which the ARV is assigned as the arithmetic average of
D 6299.
at least 16 non-outlying (see Practice E 178 or equivalent) test
This practice is under the jurisdiction of ASTM Committee D02 on Petroleum
Products and Lubricants and is the direct responsibility of Subcommittee CS94 on Annual Book of ASTM Standards, Vol 05.01.
Quality Assurance and Statistics. Annual Book of ASTM Standards, Vol 05.03.
Current edition approved Dec. 10, 2000. Published February 2001. Annual Book of ASTM Standards, Vol 14.02.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.
D6617–00
results obtained under reproducibility conditions, and the method is being performed without bias; or it can be equiva-
results pass the Anderson-Darling normality test in Practice lently stated as: H : bias = 0.
D 6299, or other statistical normality test at the 95 % confi- 3.2.9 type II error, n—in applying the methodology of this
dence level. practice, this refers to the long run probability of accepting
(that is, not rejecting) the presumed hypothesis that the method
3.2.2.1 Discussion—These may be production materials
is performed without bias, when in fact the presumed hypoth-
with unspecified composition, but are compositionally repre-
esis is not true, and the test method is biased by a magnitude of
sentative of material routinely tested by the test method, or
at least D, hence, committing an error in decision.
materialswithspecifiedcompositionsthatarereproducible,but
3.2.9.1 Discussion—Type II error, commonly known as
may not be representative of routinely tested materials.
beta (b) error in classical statistical hypothesis testing, refers to
3.2.3 delta (D), n—asignlessquantity,tobespecifiedbythe
the probability of failure to reject the null hypothesis when it is
user as the minimum magnitude of bias (either positive or
not true, based on statistics generated from relevant data. To
negative) that is of practical concern.
quantify Type II error, the user is required to declare a specific
3.2.4 power of bias detection, n—in applying the method-
alternate hypothesis that is believed to be true. In applying this
ology of this practice, this refers to the long run probability of
practice, the alternate hypothesis will take the form: “The test
being able to correctly detect a bias of a magnitude of at least
method is biased by at least D”, where D is a priori decided by
D; given the acceptance tolerance zone set under the presumed
the user as the minimum amount of bias in either direction
hypothesis, and is defined as (1 – Type II error), for a user-
(positive or negative) that is of practical concern. The alternate
specified D.
hypothesis can be equivalently stated as: H : |bias|$ D.
3.2.4.1 Discussion—The quantity (1 – Type II error), com-
monly known as the power of the test in classical statistical
4. Significance and Use
hypothesis testing, refers to the probability of correctly reject-
4.1 Laboratoriesperformingpetroleumtestmethodscanuse
ing the null hypothesis, given that the alternate hypothesis is
this practice to set an acceptable tolerance zone for infrequent
true. In applying this SP, the power refers to the probability of
testing of CS or CCS material, based on e, and a desired Type
detecting a positive or negative bias of at least D.
I error, for the purpose of ascertaining if the test method is
3.2.5 standardized delta (D ), n—D, expressed in units of
S
being performed without bias.
total uncertainty (e) per the equation:
4.2 This practice can be used to estimate the power of
~D !5D / e (1)
S
correctly detecting bias of different magnitudes, using the
3.2.6 standard error of ARV (SE ), n—a statistic quanti- acceptable tolerance zone set in 4.1, and hence, gain insight
ARV
fying the uncertainty associated with the ARV in which the into the limitation of the true bias detection capability associ-
latter is used as an estimate for the true value of the property ated with this acceptable tolerance zone. With this insight,
of interest. For a CCS, this is defined as: trade-offs can be made between desired Type I error versus
desired bias detection capability to suit specific business needs.
s / N (2)
=
CCS
4.3 The CS testing activities described in this practice are
where: intended to augment and not replace the regular statistical
N = total number of non-outlying results used to estab- monitoringoftestmethodperformanceasdescribedinPractice
lish the ARV, collected under reproducibility con-
D 6299.
ditions, and
5. General Requirement
s = the standard deviation of all the non-outlying
CCS
results.
5.1 Application of the methodology in this practice requires
3.2.6.1 Discussion—Assuming a normal model, a 95 %
the following:
confidence interval that would contain the true value of the
5.1.1 The standard material has an ARV and associated
property of interest can be constructed as follows:
standard error (SE ).
ARV
ARV – 1.96 SE to ARV 1 1.96 SE (3)
ARV ARV
NOTE 3—For a given power of detection, the magnitude of the
associated bias detectable is directly proportional to e5
3.2.7 total uncertainty (e), n—combined quantity of test
2 2
SE 1s . Therefore, efforts should be made to keep the ratio
=
ARV SITE
method s and SE as follows:
SITE ARV
(SE / s ) to as low a value as practical. A ratio of 0.5 or less is
ARV SITE
2 2
considered useful.
e5=s 1 SE (4)
SITE ARV
5.1.2 The user has a s for the test method that is
3.2.8 type I error, n—in applying the methodology of this
SITE
reasonably suited for the standard material.
practice, this refers to the theoretical long run probability of
rejecting the presumed hypothesis that the test method is
NOTE 4—It is recognized that there will be situations in which the CS
performed without bias when in fact the hypothesis is true,
may not be compositionally similar to or have property level similar to, or
hence, committing an error in decision. both, the materials regularly tested. For those situations, the site precision
standard deviation (s ) estimated using regularly tested material at a
3.2.8.1 Discussion—Type I error, commonly known as SITE
property level cl
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.