ASTM E141-91(1997)
(Practice)Standard Practice for Acceptance of Evidence Based on the Results of Probability Sampling
Standard Practice for Acceptance of Evidence Based on the Results of Probability Sampling
SCOPE
1.1 This practice presents rules for accepting or rejecting evidence based on a sample. Statistical evidence for this practice is in the form of an estimate of a proportion, an average, a total, or other numerical characteristic of a finite population or lot. It is an estimate of the result which would have been obtained by investigating the entire lot or population under the same rules and with the same care as was used for the sample.
1.2 One purpose of this practice is to describe straightforward sample selection and data calculation procedures so that courts, commissions, etc. will be able to verify whether such procedures have been applied. The methods may not give least uncertainty at least cost, they should however furnish a reasonable estimate with calculable uncertainty.
1.3 This practice is primarily intended for one-of-a-kind studies. Repetitive surveys allow estimates of sampling uncertainties to be pooled; the emphasis of this practice is on estimation of sampling uncertainty from the sample itself. The parameter of interest for this practice is effectively a constant. Thus, the principal inference is a simple point estimate to be used as if it were the unknown constant, rather than, for example, a forecast or prediction interval or distribution devised to match a random quantity of interest.
General Information
Relations
Standards Content (Sample)
NOTICE: This standard has either been superceded and replaced by a new version or discontinued.
Contact ASTM International (www.astm.org) for the latest information.
Designation: E 141 – 91 (Reapproved 1997) An American National Standard
AMERICAN SOCIETY FOR TESTING AND MATERIALS
100 Barr Harbor Dr., West Conshohocken, PA 19428
Reprinted from the Annual Book of ASTM Standards. Copyright ASTM
Standard Practice for
Acceptance of Evidence Based on the Results of Probability
Sampling
This standard is issued under the fixed designation E 141; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (e) indicates an editorial change since the last revision or reapproval.
of ASTM technical committees and others in the preparation of a sampling
1. Scope
plan for a specific material. Practice E 122 aids in deciding on the required
1.1 This practice presents rules for accepting or rejecting
sample size. Practice E 178 helps insure better behaved estimates.
evidence based on a sample. Statistical evidence for this
Terminology E 456 provides definitions of statistical terms used in this
practice is in the form of an estimate of a proportion, an
standard.
average, a total, or other numerical characteristic of a finite
3. Terminology
population or lot. It is an estimate of the result which would
have been obtained by investigating the entire lot or population
3.1 Definitions:
under the same rules and with the same care as was used for the 3.1.1 Equal Complete Coverage Result, n—the numerical
sample. characteristic (u) of interest calculated from observations made
1.2 One purpose of this practice is to describe straightfor-
by drawing randomly from the frame, all of the sampling units
ward sample selection and data calculation procedures so that covered by the frame.
courts, commissions, etc. will be able to verify whether such
3.1.1.1 Discussion—Locating the units and evaluating them
procedures have been applied. The methods may not give least are supposed to be done in exactly the same way and at the
uncertainty at least cost, they should however furnish a
same time as was done for the sample. The quantity itself is
reasonable estimate with calculable uncertainty. denoted u. The equal complete coverage result is never actually
1.3 This practice is primarily intended for one-of-a-kind
calculated. Its purpose is to serve as the objectively defined
studies. Repetitive surveys allow estimates of sampling uncer- concrete goal of the investigation. The quantity u may be the
¯
tainties to be pooled; the emphasis of this practice is on
population mean, (Y), total (Y), median (M), the proportion
estimation of sampling uncertainty from the sample itself. The (P), or any other such quantity.
parameter of interest for this practice is effectively a constant.
3.1.2 frame, n—a list, compiled for sampling purposes,
Thus, the principal inference is a simple point estimate to be which designates all of the sampling units (items or groups) of
used as if it were the unknown constant, rather than, for
a population or universe to be considered in a specific study.
example, a forecast or prediction interval or distribution 3.1.2.1 Discussion—The list may cover a specific shipment
devised to match a random quantity of interest.
or lot, all households in a county, a state or country; for
1.4 This standard does not purport to address all of the example, any population of interest. Every sampling unit in the
safety concerns, if any, associated with its use. It is the
frame (1) has a unique serial number, which may be preas-
responsibility of the user of this standard to establish appro- signed or determined by some definite rule, (2) has an
priate safety and health practices and determine the applica-
address—a complete and clear instruction (or rules for its
bility of regulatory limitations prior to use.
formulation) as to where and when to make the observation or
evaluation, (3) is based on physically concrete clerical mate-
2. Referenced Documents
rials such as directories, dials of clocks or of meters, ledgers,
2.1 ASTM Standards:
maps, aerial photographs, etc., referred to in the addresses.
E 105 Practice for Probability Sampling of Materials
3.1.3 sample, n—a group of items, observations, test results,
E 122 Practice for Choice of Sample Size to Estimate a
or portions of material, taken from a larger collection of such
Measure of Quality for a Lot or Process
items; it provides information for decisions concerning the
E 178 Practice for Dealing with Outlying Observations
larger collection.
E 456 Terminology for Statistical Methods
3.1.3.1 Discussion—A particular sample is identified by the
set of serial numbers from the randomization device and by the
NOTE 1—Practice E 105 provides a statement of principles for guidance
addresses on the frame generated by those serial numbers.
3.1.4 sampling unit, n—an item, test specimen or portion of
This practice is under the jurisdiction of ASTM Committee E-11 on Quality and
material that is to be subjected to evaluation as part of the
Statistical Methods and is the direct responsibility of Subcommittee E11.10 on
Sampling and Data Analysis.
sampling plan.
Current edition approved August 15, 1991. Published November 1991. Origi-
3.1.4.1 Discussion—If it is not feasible to select test speci-
nally published as E 141 – 59 T. Last previous edition E 141 – 69 (1975).
2 mens or laboratory samples individually, the sampling unit
Annual Book of ASTM Standards, Vol 14.02.
NOTICE: This standard has either been superceded and replaced by a new version or discontinued.
Contact ASTM International (www.astm.org) for the latest information.
E 141
may be a group of items, for example, a row, an entire case of standard error will not be a systematic underestimate. Such
items, or a prescribed area (as in the examination of a finishing subsamples were called interpenetrating in [5] where many of
process).
their basic properties were described. See [2] for further theory
3.1.4.2 By a more expensive method of measurement (fu-
and applications.
ture time, more elaborate frame) it may be possible to define a
5.2.1 Discussion—For some types of material a sample
quantity, u8, as a target parameter or ideal goal of an investi-
selected with uniform spacing along the frame (systematic
gation. Criticism that holds u to be an inappropriate goal
sample) has increased precision over a selection made with
should demonstrate that the numerical difference between u
randomly varying spacings (simple random sample). Examples
and u8 is substantial. Measurements may be imprecise but so
include sampling mineral ore or grain from a conveyor belt or
long as measurement errors are not too biased, a large size of
sampling from a list of households along a street. If the
the lot or population, N, insures that u and u8 are essentially
systematic sample is obtained by a single random start the plan
equal.
is then a probability sampling plan, but it does not permit
calculating the standard error as required by this practice. After
4. Significance and Use
dividing the sample size by an integer k (such as k 54or k
4.1 This practice is designed to permit users of sample
5 10) and using a random start for each of k replicate
survey data to judge the trustworthiness of results from such
subsamples, some of the increased precision of systematic
surveys. Section 5 gives extended definitions of the concepts
sampling (and a standard error on k − 1 degrees of freedom)
basic to survey sampling and the user should verify that such
can be achieved.
concepts were indeed used and understood by those who
5.2.2 Audit Subsample—a small subsample of the survey
conducted the survey. What was the frame? How large (ex-
sample (as few as 10 observations may be adequate) for review
actly) was the quantity N? How was the parameter u estimated
of all procedures from use of the random numbers through
and its standard error calculated? If replicate subsamples were
locating and measurement, to editing, coding, data entry and
not used, why not?
tabulation. Selection of the audit subsample may be done by
4.2 Adequate answers should be given for all questions.
putting the n sample observations in order as they are collected,
There are many acceptable answers to the last question. If the
calculating the nearest integer to n , or some other conve-
=
sample design was relatively simple, such as simple random or
nient integer, and taking this number to be the spacing for
stratified, then good estimates of sampling variance are easily
systematic selection of the audit subsample. The review should
available. If a more complex design was used then methods
uncover any gross departures from prescribed practices or any
such as discussed in [1] may be acceptable. Replicate sub-
samples is the most straightforward way to estimate sampling conceptual misunderstandings in the definitions. If the audit
subsample is large enough (say 30 observations or more) the
variances when the survey design is complex.
4.3 Once the survey procedures that were used satisfy regression of audited values on initial observations may be
Section 5, consult Section 4 to see if any increase in sample used to calibrate the estimate. This technique is the method of
size is needed. The calculations for making it are objectively two-phase sampling as discussed in [1]. Helpful discussion of
described in Section 4. an audit appears in [2].
4.4 Refer to Section 6 to guide in the interpretation of the
5.2.3 Estimate—a quantity calculated on the n sample
uncertainty in the reported value of the parameter estimate, u,
observations in the same way as the equal complete coverage
i.e. the value of its standard error, se(u). The quantity se(u)
result u would have been calculated from the entire set of N
should be reviewed to verify that the risks it entails are
possible observations of the population; the symbol u denotes
commensurate with the size of the sample.
the estimate. (In calculating u, replicate subsample member-
ship is ignored.)
5. Descriptive Terms and Procedures
5.2.3.1 Discussion—An estimate has a sampling distribu-
5.1 Probability Sampling Plans—include instructions for
tion induced from the randomness in sample selection. The
using either:
equal complete coverage result is effectively a constant while
5.1.1 carefully prepared tables of random number,
any estimate is only the value from one particular sample.
5.1.2 computer algorithms, carefully programmed and run
Thus, there is a mean value of the sampling distribution and
on a large computer, to generate pseudo-random numbers or,
there is also a standard deviation of the sampling distribution.
5.1.3 certifiably honest physical devices, such as coin flips,
5.2.4 Standard Error—the quantity computed from the
to select the sample units so that inferences may be drawn from
observations as an estimate of the sampling standard deviation
the test results and decisions may be made with risks correctly
of the estimate; se(u) denotes the standard error.
calculated by probability theory.
5.1.4 Such plans are defined and their relative advantages
5.2.4.1 Example 1—When u is the population average of the
discussed in [1], [2] and [6].
N quantities and a simple random sample of size n was drawn,
5.2 Replicate Subsamples—a number of disjoint samples,
then the sample average y¯ becomes the usual estimate u, where
each one separately drawn from the frame in accord with the
n
same probability sampling plan. When appropriate, separate u5 y¯ 5 y /n. (1)
( i
i 5 1
laboratories should each work on separate replicate subsamples
and teams of investigators should be assigned to separate The quantities y ,y , ., y denote the observations. The
1 2 n
replicate subsamples. This approach insures that the calculated standard error is calculated as:
NOTICE: This standard has either been superceded and replaced by a new version or discontinued.
Contact ASTM International (www.astm.org) for the latest information.
E 141
1 3
n
ratios. For an example with k 5 2, average ⁄3 and ⁄5 and
se ~u! 5 se ~y¯! 5 ~y 2 y¯! /n~n 2 1!. (2)
˛ (
i
compare to (1 + 3)/(3 + 5).
i 5 1
5.2.5 Procedures—must be described in written form and
There are n − 1 degrees of freedom in this standard error.
should cover the following matters; (1) parties interested in
When the observations are:
collecting data should agree on the importance of knowing u
81.6, 78.7, 79.7, 78.3, 80.9, 79.5, 79.8, 80.3, 79.5, 80.7
and its definition including measurement methods, (2) the
then y¯ 5 79.90 and se(y¯) 5 .32. As this example illustrates,
frame shall be carefully and explicitly constructed; N shall be
formula (2) is correct when k replaces n and subsample
well established, (3) random numbers (or a certifiably honest
estimates are used in place of observations.
physical random device) shall dictate selection of the sample.
5.2.4.2 Example 2 on the Finite Population Correction
There will be no substitution of one sampling unit for another.
(fpc)—Multiplying se (y¯) by 1 2 n/N is always correct
=
The method of sample selection shall permit calculation of a
when the goal of the survey is to estimate the finite population
standard error of the estimate (4) the use of replicate sub-
¯
mean (u5 Y). Using the previous data and if N 5 50, then
samples is recommended (see section 5.24.2.2); an audit
se(y¯) becomes se(y¯) 5 .28 after applying the fpc. If random
subsample should be selected and processed and any depar-
measurement error exists in the observations, then u8 based on
tures from prescribed measurement methods and location
a reference measurement method may be a more appropriate
instructions noted (see 5.2.2). A report should list u and its
survey goal than u (see section 4.1.4.1). If so, then se(y¯) would
standard error with the degrees of freedom in the se( u).
be further adjusted upward by an amount somewhat less than
the downward adjustment of the fpc. Both of these adjustments
6. Adequacy of Sample Size
are often numerically so small that these adjustments may be
6.1 Deciding on Increasing Sample Size: Choice of sample
omitted—leaving se(y¯) of (2) as a slight overestimate.
size should be made carefully in accordance with Practice
5.2.4.3 Example 3—If the quantity of interest is (a) a
E 122 or on a comparable basis. Since procedures for setting
proportion or (b) a total and the sample is simple random then
sample size are based on judgements of the variability to be
the above formulas are still applicable. A proportion is the
encountered, there is a possibility that the standard error as
mean of zeroes and ones, while the total is a constant times the
calculated from the data could greatly exceed that anticipated.
mean. Thus:
It may happen that the
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.