ASTM E141-10(2018)
(Practice)Standard Practice for Acceptance of Evidence Based on the Results of Probability Sampling
Standard Practice for Acceptance of Evidence Based on the Results of Probability Sampling
ABSTRACT
This practice presents rules for accepting or rejecting evidence based on a sample. Statistical evidence for this practice is in the form of an estimate of a proportion, an average, a total, or other numerical characteristic of a finite population or lot. This practice is an estimate of the result which would have been obtained by investigating the entire lot or population under the same rules and with the same care as was used for the sample. One purpose of this practice is to describe straightforward sample selection and data calculation procedures so that courts, commissions, etc. will be able to verify whether such procedures have been applied.
This practice includes the concepts and procedures of sampling. Examples include sampling mineral ore or grain from a conveyor belt or sampling from a list of households along a street. If the systematic sample is obtained by a single random start, the plan is then a probability sampling plan, but it does not permit calculating the standard error as required by this practice.
SIGNIFICANCE AND USE
4.1 This practice is designed to permit users of sample survey data to judge the trustworthiness of results from such surveys. Practice E105 provides a statement of principles for guidance of ASTM technical committees and others in the preparation of a sampling plan for a specific material. Guide E1402 describes the principal types of sampling designs. Practice E122 aids in deciding on the required sample size.
4.2 Section 5 gives extended definitions of the concepts basic to survey sampling and the user should verify that such concepts were indeed used and understood by those who conducted the survey. What was the frame? How large (exactly) was the quantity N? How was the parameter θ estimated and its standard error calculated? If replicate subsamples were not used, why not? Adequate answers should be given for all questions. There are many acceptable answers to the last question.
4.3 If the sample design was relatively simple, such as simple random or stratified, then fully valid estimates of sampling variance are easily available. If a more complex design was used then methods such as discussed in Ref (1)3 or in Guide E1402 may be acceptable. Use of replicate subsamples is the most straightforward way to estimate sampling variances when the survey design is complex.
4.4 Once the survey procedures that were used satisfy Section 5, see if any increase in sample size is needed. The calculations for making it objectively are described in Section 6.
4.5 Refer to Section 7 to guide in the interpretation of the uncertainty in the reported value of the parameter estimate, θ^, that is, the value of its standard error, se(θ^). The quantity se(θ^) should be reviewed to verify that the risks it entails are commensurate with the size of the sample.
4.6 When the audit subsample shows that there was reasonable conformity with prescribed procedures and when the known instances of departures from the survey plan can be shown to have no appreciable effect on the est...
SCOPE
1.1 This practice presents rules for accepting or rejecting evidence based on a sample. Statistical evidence for this practice is in the form of an estimate of a proportion, an average, a total, or other numerical characteristic of a finite population or lot. It is an estimate of the result which would have been obtained by investigating the entire lot or population under the same rules and with the same care as was used for the sample.
1.2 One purpose of this practice is to describe straightforward sample selection and data calculation procedures so that courts, commissions, etc. will be able to verify whether such procedures have been applied. The methods may not give least uncertainty at least cost, they should however furnish a reasonable estimate with calculable uncertainty.
1.3 This practice is primarily intended for one-of-a-kind studies. Repetitive surveys allow estimates of sa...
General Information
Relations
Buy Standard
Standards Content (Sample)
This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation: E141 − 10 (Reapproved 2018) An American National Standard
Standard Practice for
Acceptance of Evidence Based on the Results of Probability
Sampling
This standard is issued under the fixed designation E141; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope mendations issued by the World Trade Organization Technical
Barriers to Trade (TBT) Committee.
1.1 This practice presents rules for accepting or rejecting
evidence based on a sample. Statistical evidence for this
2. Referenced Documents
practice is in the form of an estimate of a proportion, an
2.1 ASTM Standards:
average, a total, or other numerical characteristic of a finite
E105Practice for Probability Sampling of Materials
population or lot. It is an estimate of the result which would
E122PracticeforCalculatingSampleSizetoEstimate,With
havebeenobtainedbyinvestigatingtheentirelotorpopulation
Specified Precision, the Average for a Characteristic of a
underthesamerulesandwiththesamecareaswasusedforthe
Lot or Process
sample.
E456Terminology Relating to Quality and Statistics
1.2 One purpose of this practice is to describe straightfor-
E1402Guide for Sampling Design
ward sample selection and data calculation procedures so that
E2586Practice for Calculating and Using Basic Statistics
courts, commissions, etc. will be able to verify whether such
procedures have been applied.The methods may not give least
3. Terminology
uncertainty at least cost, they should however furnish a
3.1 Definitions—Refer to Terminology E456 for definitions
reasonable estimate with calculable uncertainty.
of other statistical terms used in this practice.
1.3 This practice is primarily intended for one-of-a-kind
3.1.1 audit subsample, n—a small subsample of a sample
studies. Repetitive surveys allow estimates of sampling uncer-
selected for review of all sample selection and data collection
tainties to be pooled; the emphasis of this practice is on
procedures.
estimation of sampling uncertainty from the sample itself. The
3.1.2 equal complete coverage result, n—the numerical
parameter of interest for this practice is effectively a constant.
characteristic of interest calculated from observations made by
Thus, the principal inference is a simple point estimate to be
drawing randomly from the frame, all of the sampling units
used as if it were the unknown constant, rather than, for
covered by the frame.
example, a forecast or prediction interval or distribution
3.1.2.1 Discussion—Locating the units and evaluating them
devised to match a random quantity of interest.
are supposed to be done in exactly the same way and at the
1.4 A system of units is not specified in this standard.
same time as was done for the sample. The quantity itself is
denoted θ.Theequalcompletecoverageresultisneveractually
1.5 This standard does not purport to address all of the
calculated. Its purpose is to serve as the objectively defined
safety concerns, if any, associated with its use. It is the
concrete goal of the investigation. The quantity θ may be the
responsibility of the user of this standard to establish appro-
¯
populationmean, (Y),total (Y),median(M),theproportion(P),
priate safety, health, and environmental practices and deter-
or any other such quantity.
mine the applicability of regulatory limitations prior to use.
1.6 This international standard was developed in accor-
3.1.3 frame, n—a list, compiled for sampling purposes,
dance with internationally recognized principles on standard-
which designates all of the sampling units (items or groups) of
ization established in the Decision on Principles for the
a population or universe to be considered in a specific study.
Development of International Standards, Guides and Recom-
E1402
3.1.4 probability sample, n—a sample in which the sam-
pling units are selected by a chance process such that a
ThispracticeisunderthejurisdictionofASTMCommitteeE11onQualityand
Statistics and is the direct responsibility of Subcommittee E11.10 on Sampling /
Statistics. For referenced ASTM standards, visit the ASTM website, www.astm.org, or
Current edition approved April 1, 2018. Published May 2018. Originally contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
approved in 1959. Last previous edition approved in 2010 as E141–10. DOI: Standards volume information, refer to the standard’s Document Summary page on
10.1520/E0141-10R18. the ASTM website.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E141 − 10 (2018)
specified probability of selection can be attached to each 5. Concepts and Procedures of Sampling
possible sample that can be selected. E1402
5.1 Probability sampling is a procedure by which one
3.1.5 replicate subsamples, n—a number of disjoint
obtains a result from a selected set of sampling units that will
samples, each one separately drawn from the frame in accord
agree, within calculable limits of variation, with the equal
with the same probability sampling plan.
complete coverage result. Probability sampling plans include
instructions for using either (1) prepared tables of random
3.1.6 sample, n—a group of observations or test results,
numbers, (2) computer algorithms to generate pseudo-random
taken from a larger collection of observations or test results,
numbers,or(3)certifiablyhonestphysicaldevicestoselectthe
whichservestoprovideinformationthatmaybeusedasabasis
sample units so that inferences may be drawn from the test
for making a decision concerning the larger collection. E2586
results and decisions may be made with risks correctly calcu-
3.1.7 sampling unit, n—an item, group of items, or segment
lated by probability theory.
of material that can be selected as part of a probability
5.1.1 Such plans are defined and their relative advantages
sampling plan. E1402
discussed in Guide E1402 and Refs (1-3).
5.2 Procedures must be described in written form. Parties
4. Significance and Use
interested in collecting data should agree on the importance of
4.1 This practice is designed to permit users of sample
knowing θ and its definition including measurement methods.
survey data to judge the trustworthiness of results from such
The frame shall be carefully and explicitly constructed. Every
surveys. Practice E105 provides a statement of principles for
sampling unit in the frame (1) has a unique serial number,
guidance of ASTM technical committees and others in the
which may be preassigned or determined by some definite rule
preparation of a sampling plan for a specific material. Guide
and (2) has an address—a complete and clear instruction (or
E1402 describes the principal types of sampling designs.
rules for its formulation) as to where and when to make the
Practice E122 aids in deciding on the required sample size.
observation or evaluation.Address instructions should refer to
4.2 Section 5 gives extended definitions of the concepts concrete clerical materials such as directories, dials of clocks
basic to survey sampling and the user should verify that such orofmeters,ledgers,maps,aerialphotographs,etc.Duplicates
concepts were indeed used and understood by those who
in the frame shall be eliminated. N shall be well established.
conducted the survey. What was the frame? How large (ex- Random numbers (or a certifiably honest physical random
actly) was the quantity N? How was the parameter θ estimated
device) shall dictate selection of the sample. There shall be no
and its standard error calculated? If replicate subsamples were substitution of one sampling unit for another. The method of
not used, why not? Adequate answers should be given for all
sample selection shall permit calculation of a standard error of
questions. There are many acceptable answers to the last the estimate. The use of replicate subsamples is recommended
question.
(see5.4).Anauditsubsampleshouldbeselectedandprocessed
and any departures from prescribed measurement methods and
4.3 If the sample design was relatively simple, such as
ˆ
location instructions noted (see 5.5).Areport should list θ and
simple random or stratified, then fully valid estimates of
ˆ
its standard error with the degrees of freedom in the se(θ).
sampling variance are easily available. If a more complex
design was used then methods such as discussed in Ref (1) or
5.3 Parameter Definition—The equal complete coverage
in Guide E1402 may be acceptable. Use of replicate sub- result may or may not be acceptable evidence. Whether it is
samples is the most straightforward way to estimate sampling acceptabledependsonmanyconsiderationssuchasdefinitions,
variances when the survey design is complex. method of test, care exercised in the testing, completeness of
the frame, and on other points not to be settled by statistical
4.4 Once the survey procedures that were used satisfy
theory since these points belong to the subject matter, and are
Section 5, see if any increase in sample size is needed. The
the same whether one uses sampling or not. Mistakes, whether
calculations for making it objectively are described in Section
in testing, counting, or weighing will affect the result of a
6.
complete coverage just as such mistakes will affect the sample
4.5 Refer to Section 7 to guide in the interpretation of the
result. By a more expensive method of measurement or more
ˆ
uncertainty in the reported value of the parameter estimate, θ,
elaborate sampling frame, it may be possible to define a
ˆ ˆ
that is, the value of its standard error, se(θ). The quantity se(θ)
quantity, θ', as a target parameter or ideal goal of an investi-
should be reviewed to verify that the risks it entails are
gation.Criticismthatholds θtobeaninappropriategoalshould
commensurate with the size of the sample.
demonstrate that the numerical difference between θ and θ'is
substantial. Measurements may be imprecise but so long as
4.6 When the audit subsample shows that there was reason-
measurementerrorsarenottoobiased,alargesizeofthelotor
able conformity with prescribed procedures and when the
population, N, insures that θ and θ' are essentially equal.
known instances of departures from the survey plan can be
shown to have no appreciable effect on the estimate, the value
5.4 Replicate Subsamples—When appropriate, separate
ˆ
of θ is appropriate for use.
laboratoriesshouldeachworkonseparatereplicatesubsamples
and teams of investigators should be assigned to separate
replicate subsamples.This approach insures that the calculated
standard error will not be a systematic underestimate. Such
The boldface numbers in parentheses refer to a list of references at the end of
this standard. subsampleswerecalledinterpenetratinginRef (4)wheremany
E141 − 10 (2018)
oftheirbasicpropertiesweredescribed.SeeRef (5)forfurther There are n −1 degrees of freedom in this standard error.
theory and applications.
5.7.1.1 Example—When the observations are:
81.6, 78.7, 79.7, 78.3, 80.9, 79.5, 79.8, 80.3, 79.5, 80.7
5.4.1 For some types of material, a sample selected with
uniform spacing along the frame (systematic sample) has then y¯ =79.90 and se(y¯)=0.32.
increased precision over a selection made with randomly 5.7.2 Finite Population Correction (fpc)—Multiplying se(y¯)
varying spacings (simple random sample). Examples include
by =12n/N isalwayscorrectwhenthegoalofthesurveyisto
samplingmineraloreorgrainfromaconveyorbeltorsampling ¯
estimate the finite population mean (θ=Y). If random mea-
from a list of households along a street. If the systematic
surement error exists in the observations, then θ' based on a
sample is obtained by a single random start the plan is then a
reference measurement method may be a more appropriate
probabilitysamplingplan,butitdoesnotpermitcalculatingthe
survey goal than θ (see 5.3). If so, then se(y¯) would be further
standard error as required by this practice. After dividing the
adjusted upward by an amount somewhat less than the down-
samplesizebyaninteger k(suchas k =4or k =10)andusing
wardadjustmentofthefpc.Bothoftheseadjustmentsareoften
a random start for each of k replicate subsamples, some of the
numerically so small that these adjustments may be omitted—
increased precision of systematic sampling (and a standard
leaving se(y¯) of Eq 2 as a slight overestimate.
error on k−1 degrees of freedom) can be achieved.
5.7.2.1 Example—Using the previous data and if N=50,
5.5 An audit subsample of the survey sample should be
then se(y¯) becomes se(y¯)=0.28 after applying the fpc.
taken for review of all procedures from use of the random
5.7.3 Proportions and Total Counts—If the quantity of
numbersthroughlocatingandmeasurement,toediting,coding,
interest is (a) a proportion or (b) a total and the sample is
dataentryandtabulation.Selectionoftheauditsubsamplemay
simple random then the above formulas are still applicable. A
be done by putting the n sample observations in order as they
proportion is the mean of zeroes and ones, while the total is a
are collected, calculating the nearest integer to =n , or some
constant times the mean.
other convenient integer, and taking this number to be the
5.7.3.1 When θ is taken to be the population proportion
spacingforsystematicselectionoftheauditsubsample.Asfew
(θ= P) then
as 10 observations may be adequate. The review should
ˆ
θ 5 p 5 y /n 5 a/n (3)
i
uncover any gross departures from prescribed practices or any (
conceptual misunderstandings in the definitions. If the audit
where a is the number of units in the sample with the
subsample is large enough (say 30 observations or more) the
attribute, and
regression of audited values on initial observations may be
se p 5 =p 1 2 p / n 2 1 (4)
used to calibrate the estimate. This technique is the method of ~ ! ~ ! ~ !
two-phasesamplingasdiscussedinRef (1).Helpfuldiscussion
5.7.3.2 When θ=the population total (θ= Y) then
of an audit appears in Ref (2).
ˆ ˆ
θ 5 Npand se~θ! 5 N·se p (5)
~ !
5.6 The estimate is a quantity calculated on the n sample
observations in the same way as the equal complete coverage
5.7.3.3 Example—If a simple random sample of size
result θ would have been calculated from the entire set of N
n=200 has a=25 items with the attribute then the conclusion
ˆ
ˆ ˆ
possible observations of the population; the symbol θ denotes
is θ=0.125 and se(θ)=0.023 on 199 degrees of freedom.
ˆ
the estimate. In calculating θ, replic
...
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Contact ASTM International (www.astm.org) for the latest information
Designation: E141 − 10 (Reapproved 2018) An American National Standard
Standard Practice for
Acceptance of Evidence Based on the Results of Probability
Sampling
This standard is issued under the fixed designation E141; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope mendations issued by the World Trade Organization Technical
Barriers to Trade (TBT) Committee.
1.1 This practice presents rules for accepting or rejecting
evidence based on a sample. Statistical evidence for this
2. Referenced Documents
practice is in the form of an estimate of a proportion, an
2.1 ASTM Standards:
average, a total, or other numerical characteristic of a finite
E105 Practice for Probability Sampling of Materials
population or lot. It is an estimate of the result which would
E122 Practice for Calculating Sample Size to Estimate, With
have been obtained by investigating the entire lot or population
Specified Precision, the Average for a Characteristic of a
under the same rules and with the same care as was used for the
Lot or Process
sample.
E456 Terminology Relating to Quality and Statistics
1.2 One purpose of this practice is to describe straightfor-
E1402 Guide for Sampling Design
ward sample selection and data calculation procedures so that
E2586 Practice for Calculating and Using Basic Statistics
courts, commissions, etc. will be able to verify whether such
procedures have been applied. The methods may not give least
3. Terminology
uncertainty at least cost, they should however furnish a
3.1 Definitions—Refer to Terminology E456 for definitions
reasonable estimate with calculable uncertainty.
of other statistical terms used in this practice.
1.3 This practice is primarily intended for one-of-a-kind
3.1.1 audit subsample, n—a small subsample of a sample
studies. Repetitive surveys allow estimates of sampling uncer-
selected for review of all sample selection and data collection
tainties to be pooled; the emphasis of this practice is on
procedures.
estimation of sampling uncertainty from the sample itself. The
3.1.2 equal complete coverage result, n—the numerical
parameter of interest for this practice is effectively a constant.
characteristic of interest calculated from observations made by
Thus, the principal inference is a simple point estimate to be
drawing randomly from the frame, all of the sampling units
used as if it were the unknown constant, rather than, for
covered by the frame.
example, a forecast or prediction interval or distribution
3.1.2.1 Discussion—Locating the units and evaluating them
devised to match a random quantity of interest.
are supposed to be done in exactly the same way and at the
1.4 A system of units is not specified in this standard.
same time as was done for the sample. The quantity itself is
denoted θ. The equal complete coverage result is never actually
1.5 This standard does not purport to address all of the
calculated. Its purpose is to serve as the objectively defined
safety concerns, if any, associated with its use. It is the
concrete goal of the investigation. The quantity θ may be the
responsibility of the user of this standard to establish appro-
¯
population mean, (Y), total (Y), median (M), the proportion (P),
priate safety, health, and environmental practices and deter-
or any other such quantity.
mine the applicability of regulatory limitations prior to use.
1.6 This international standard was developed in accor-
3.1.3 frame, n—a list, compiled for sampling purposes,
dance with internationally recognized principles on standard-
which designates all of the sampling units (items or groups) of
ization established in the Decision on Principles for the
a population or universe to be considered in a specific study.
Development of International Standards, Guides and Recom-
E1402
3.1.4 probability sample, n—a sample in which the sam-
pling units are selected by a chance process such that a
This practice is under the jurisdiction of ASTM Committee E11 on Quality and
Statistics and is the direct responsibility of Subcommittee E11.10 on Sampling /
Statistics. For referenced ASTM standards, visit the ASTM website, www.astm.org, or
Current edition approved April 1, 2018. Published May 2018. Originally contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
approved in 1959. Last previous edition approved in 2010 as E141 – 10. DOI: Standards volume information, refer to the standard’s Document Summary page on
10.1520/E0141-10R18. the ASTM website.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E141 − 10 (2018)
specified probability of selection can be attached to each 5. Concepts and Procedures of Sampling
possible sample that can be selected. E1402
5.1 Probability sampling is a procedure by which one
3.1.5 replicate subsamples, n—a number of disjoint
obtains a result from a selected set of sampling units that will
samples, each one separately drawn from the frame in accord
agree, within calculable limits of variation, with the equal
with the same probability sampling plan.
complete coverage result. Probability sampling plans include
instructions for using either (1) prepared tables of random
3.1.6 sample, n—a group of observations or test results,
numbers, (2) computer algorithms to generate pseudo-random
taken from a larger collection of observations or test results,
numbers, or (3) certifiably honest physical devices to select the
which serves to provide information that may be used as a basis
sample units so that inferences may be drawn from the test
for making a decision concerning the larger collection. E2586
results and decisions may be made with risks correctly calcu-
3.1.7 sampling unit, n—an item, group of items, or segment
lated by probability theory.
of material that can be selected as part of a probability
5.1.1 Such plans are defined and their relative advantages
sampling plan. E1402
discussed in Guide E1402 and Refs (1-3).
5.2 Procedures must be described in written form. Parties
4. Significance and Use
interested in collecting data should agree on the importance of
4.1 This practice is designed to permit users of sample
knowing θ and its definition including measurement methods.
survey data to judge the trustworthiness of results from such
The frame shall be carefully and explicitly constructed. Every
surveys. Practice E105 provides a statement of principles for
sampling unit in the frame (1) has a unique serial number,
guidance of ASTM technical committees and others in the
which may be preassigned or determined by some definite rule
preparation of a sampling plan for a specific material. Guide
and (2) has an address—a complete and clear instruction (or
E1402 describes the principal types of sampling designs.
rules for its formulation) as to where and when to make the
Practice E122 aids in deciding on the required sample size.
observation or evaluation. Address instructions should refer to
4.2 Section 5 gives extended definitions of the concepts concrete clerical materials such as directories, dials of clocks
basic to survey sampling and the user should verify that such
or of meters, ledgers, maps, aerial photographs, etc. Duplicates
concepts were indeed used and understood by those who in the frame shall be eliminated. N shall be well established.
conducted the survey. What was the frame? How large (ex-
Random numbers (or a certifiably honest physical random
actly) was the quantity N? How was the parameter θ estimated device) shall dictate selection of the sample. There shall be no
and its standard error calculated? If replicate subsamples were
substitution of one sampling unit for another. The method of
not used, why not? Adequate answers should be given for all sample selection shall permit calculation of a standard error of
questions. There are many acceptable answers to the last
the estimate. The use of replicate subsamples is recommended
question. (see 5.4). An audit subsample should be selected and processed
and any departures from prescribed measurement methods and
4.3 If the sample design was relatively simple, such as
ˆ
location instructions noted (see 5.5). A report should list θ and
simple random or stratified, then fully valid estimates of
ˆ
its standard error with the degrees of freedom in the se(θ).
sampling variance are easily available. If a more complex
design was used then methods such as discussed in Ref (1) or 5.3 Parameter Definition—The equal complete coverage
in Guide E1402 may be acceptable. Use of replicate sub- result may or may not be acceptable evidence. Whether it is
samples is the most straightforward way to estimate sampling acceptable depends on many considerations such as definitions,
variances when the survey design is complex. method of test, care exercised in the testing, completeness of
the frame, and on other points not to be settled by statistical
4.4 Once the survey procedures that were used satisfy
theory since these points belong to the subject matter, and are
Section 5, see if any increase in sample size is needed. The
the same whether one uses sampling or not. Mistakes, whether
calculations for making it objectively are described in Section
in testing, counting, or weighing will affect the result of a
6.
complete coverage just as such mistakes will affect the sample
4.5 Refer to Section 7 to guide in the interpretation of the
result. By a more expensive method of measurement or more
ˆ
uncertainty in the reported value of the parameter estimate, θ,
elaborate sampling frame, it may be possible to define a
ˆ ˆ
that is, the value of its standard error, se(θ). The quantity se(θ)
quantity, θ', as a target parameter or ideal goal of an investi-
should be reviewed to verify that the risks it entails are
gation. Criticism that holds θ to be an inappropriate goal should
commensurate with the size of the sample.
demonstrate that the numerical difference between θ and θ' is
substantial. Measurements may be imprecise but so long as
4.6 When the audit subsample shows that there was reason-
measurement errors are not too biased, a large size of the lot or
able conformity with prescribed procedures and when the
population, N, insures that θ and θ' are essentially equal.
known instances of departures from the survey plan can be
shown to have no appreciable effect on the estimate, the value
5.4 Replicate Subsamples—When appropriate, separate
ˆ
of θ is appropriate for use.
laboratories should each work on separate replicate subsamples
and teams of investigators should be assigned to separate
replicate subsamples. This approach insures that the calculated
3 standard error will not be a systematic underestimate. Such
The boldface numbers in parentheses refer to a list of references at the end of
this standard. subsamples were called interpenetrating in Ref (4) where many
E141 − 10 (2018)
of their basic properties were described. See Ref (5) for further There are n − 1 degrees of freedom in this standard error.
theory and applications.
5.7.1.1 Example—When the observations are:
5.4.1 For some types of material, a sample selected with 81.6, 78.7, 79.7, 78.3, 80.9, 79.5, 79.8, 80.3, 79.5, 80.7
then y¯ = 79.90 and se(y¯) = 0.32.
uniform spacing along the frame (systematic sample) has
increased precision over a selection made with randomly
5.7.2 Finite Population Correction (fpc)—Multiplying se(y¯)
varying spacings (simple random sample). Examples include
by =12n/N is always correct when the goal of the survey is to
sampling mineral ore or grain from a conveyor belt or sampling ¯
estimate the finite population mean (θ = Y). If random mea-
from a list of households along a street. If the systematic
surement error exists in the observations, then θ' based on a
sample is obtained by a single random start the plan is then a
reference measurement method may be a more appropriate
probability sampling plan, but it does not permit calculating the
survey goal than θ (see 5.3). If so, then se(y¯) would be further
standard error as required by this practice. After dividing the
adjusted upward by an amount somewhat less than the down-
sample size by an integer k (such as k = 4 or k = 10) and using
ward adjustment of the fpc. Both of these adjustments are often
a random start for each of k replicate subsamples, some of the
numerically so small that these adjustments may be omitted—
increased precision of systematic sampling (and a standard
leaving se(y¯) of Eq 2 as a slight overestimate.
error on k − 1 degrees of freedom) can be achieved.
5.7.2.1 Example—Using the previous data and if N = 50,
5.5 An audit subsample of the survey sample should be
then se(y¯) becomes se(y¯) = 0.28 after applying the fpc.
taken for review of all procedures from use of the random
5.7.3 Proportions and Total Counts—If the quantity of
numbers through locating and measurement, to editing, coding,
interest is (a) a proportion or (b) a total and the sample is
data entry and tabulation. Selection of the audit subsample may
simple random then the above formulas are still applicable. A
be done by putting the n sample observations in order as they
proportion is the mean of zeroes and ones, while the total is a
are collected, calculating the nearest integer to =n , or some
constant times the mean.
other convenient integer, and taking this number to be the
5.7.3.1 When θ is taken to be the population proportion
spacing for systematic selection of the audit subsample. As few
(θ = P) then
as 10 observations may be adequate. The review should
ˆ
θ 5 p 5 y /n 5 a/n (3)
( i
uncover any gross departures from prescribed practices or any
conceptual misunderstandings in the definitions. If the audit
where a is the number of units in the sample with the
subsample is large enough (say 30 observations or more) the
attribute, and
regression of audited values on initial observations may be
=
se~p! 5 p~1 2 p!/~n 2 1! (4)
used to calibrate the estimate. This technique is the method of
two-phase sampling as discussed in Ref (1). Helpful discussion
5.7.3.2 When θ = the population total (θ = Y) then
of an audit appears in Ref (2).
ˆ ˆ
θ 5 Np and se~θ! 5 N·se~p! (5)
5.6 The estimate is a quantity calculated on the n sample
observations in the same way as the equal complete coverage
5.7.3.3 Example—If a simple random sample of size
result θ would have been calculated from the entire set of N
n = 200 has a = 25 items with the attribute then the conclusion
ˆ
possible observations of the population; th
...
This document is not an ASTM standard and is intended only to provide the user of an ASTM standard an indication of what changes have been made to the previous version. Because
it may not be technically possible to adequately depict all changes accurately, ASTM recommends that users consult prior editions as appropriate. In all cases only the current version
of the standard as published by ASTM is to be considered the official document.
Designation: E141 − 10 E141 − 10 (Reapproved 2018) An American National Standard
Standard Practice for
Acceptance of Evidence Based on the Results of Probability
Sampling
This standard is issued under the fixed designation E141; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope
1.1 This practice presents rules for accepting or rejecting evidence based on a sample. Statistical evidence for this practice is
in the form of an estimate of a proportion, an average, a total, or other numerical characteristic of a finite population or lot. It is
an estimate of the result which would have been obtained by investigating the entire lot or population under the same rules and
with the same care as was used for the sample.
1.2 One purpose of this practice is to describe straightforward sample selection and data calculation procedures so that courts,
commissions, etc. will be able to verify whether such procedures have been applied. The methods may not give least uncertainty
at least cost, they should however furnish a reasonable estimate with calculable uncertainty.
1.3 This practice is primarily intended for one-of-a-kind studies. Repetitive surveys allow estimates of sampling uncertainties
to be pooled; the emphasis of this practice is on estimation of sampling uncertainty from the sample itself. The parameter of interest
for this practice is effectively a constant. Thus, the principal inference is a simple point estimate to be used as if it were the
unknown constant, rather than, for example, a forecast or prediction interval or distribution devised to match a random quantity
of interest.
1.4 A system of units is not specified in this standard.
1.5 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility
of the user of this standard to establish appropriate safety safety, health, and healthenvironmental practices and determine the
applicability of regulatory limitations prior to use.
1.6 This international standard was developed in accordance with internationally recognized principles on standardization
established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued
by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
2. Referenced Documents
2.1 ASTM Standards:
E105 Practice for Probability Sampling of Materials
E122 Practice for Calculating Sample Size to Estimate, With Specified Precision, the Average for a Characteristic of a Lot or
Process
E456 Terminology Relating to Quality and Statistics
E1402 Guide for Sampling Design
E2586 Practice for Calculating and Using Basic Statistics
3. Terminology
3.1 Definitions—Refer to Terminology E456 for definitions of other statistical terms used in this practice.
3.1.1 audit subsample, n—a small subsample of a sample selected for review of all sample selection and data collection
procedures.
3.1.2 equal complete coverage result, n—the numerical characteristic of interest calculated from observations made by drawing
randomly from the frame, all of the sampling units covered by the frame.
This practice is under the jurisdiction of ASTM Committee E11 on Quality and Statistics and is the direct responsibility of Subcommittee E11.10 on Sampling / Statistics.
Current edition approved May 15, 2010April 1, 2018. Published August 2010May 2018. Originally approved in 1959. Last previous edition approved in 20032010 as
ε1
E141 – 91 (2003)E141 – 10. . DOI: 10.1520/E0141-10.10.1520/E0141-10R18.
For referenced ASTM standards, visit the ASTM website, www.astm.org, or contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM Standards
volume information, refer to the standard’s Document Summary page on the ASTM website.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E141 − 10 (2018)
3.1.2.1 Discussion—
Locating the units and evaluating them are supposed to be done in exactly the same way and at the same time as was done for
the sample. The quantity itself is denoted θ. The equal complete coverage result is never actually calculated. Its purpose is to serve
as the objectively defined concrete goal of the investigation. The quantity θ may be the population mean, (Y¯), total (Y), median
(M), the proportion (P), or any other such quantity.
3.1.3 frame, n—a list, compiled for sampling purposes, which designates all of the sampling units (items or groups) of a
population or universe to be considered in a specific study. E1402
3.1.4 probability sample, n—a sample in which the sampling units are selected by a chance process such that a specified
probability of selection can be attached to each possible sample that can be selected. E1402
3.1.5 replicate subsamples, n—a number of disjoint samples, each one separately drawn from the frame in accord with the same
probability sampling plan.
3.1.6 sample, n—a group of observations or test results, taken from a larger collection of observations or test results, which
serves to provide information that may be used as a basis for making a decision concerning the larger collection. E2586
3.1.7 sampling unit, n—an item, group of items, or segment of material that can be selected as part of a probability sampling
plan. E1402
4. Significance and Use
4.1 This practice is designed to permit users of sample survey data to judge the trustworthiness of results from such surveys.
Practice E105 provides a statement of principles for guidance of ASTM technical committees and others in the preparation of a
sampling plan for a specific material. Guide E1402 describes the principal types of sampling designs. Practice E122 aids in
deciding on the required sample size.
4.2 Section 5 gives extended definitions of the concepts basic to survey sampling and the user should verify that such concepts
were indeed used and understood by those who conducted the survey. What was the frame? How large (exactly) was the quantity
N? How was the parameter θ estimated and its standard error calculated? If replicate subsamples were not used, why not? Adequate
answers should be given for all questions. There are many acceptable answers to the last question.
4.3 If the sample design was relatively simple, such as simple random or stratified, then fully valid estimates of sampling
variance are easily available. If a more complex design was used then methods such as discussed in Ref (1) or in Guide E1402
may be acceptable. Use of replicate subsamples is the most straightforward way to estimate sampling variances when the survey
design is complex.
4.4 Once the survey procedures that were used satisfy Section 5, see if any increase in sample size is needed. The calculations
for making it objectively are described in Section 6.
4.5 Refer to Section 7 to guide in the interpretation of the uncertainty in the reported value of the parameter estimate, θˆ, that
is, the value of its standard error, se(θˆ). The quantity se(θˆ) should be reviewed to verify that the risks it entails are commensurate
with the size of the sample.
4.6 When the audit subsample shows that there was reasonable conformity with prescribed procedures and when the known
instances of departures from the survey plan can be shown to have no appreciable effect on the estimate, the value of θˆ is
appropriate for use.
5. Concepts and Procedures of Sampling
5.1 Probability sampling is a procedure by which one obtains a result from a selected set of sampling units that will agree,
within calculable limits of variation, with the equal complete coverage result. Probability sampling plans include instructions for
using either (1) prepared tables of random numbers, (2) computer algorithms to generate pseudo-random numbers, or (3) certifiably
honest physical devices to select the sample units so that inferences may be drawn from the test results and decisions may be made
with risks correctly calculated by probability theory.
5.1.1 Such plans are defined and their relative advantages discussed in Guide E1402 and Refs (1-3).
5.2 Procedures must be described in written form. Parties interested in collecting data should agree on the importance of
knowing θ and its definition including measurement methods. The frame shall be carefully and explicitly constructed. Every
sampling unit in the frame (1) has a unique serial number, which may be preassigned or determined by some definite rule and (2)
has an address—a complete and clear instruction (or rules for its formulation) as to where and when to make the observation or
evaluation. Address instructions should refer to concrete clerical materials such as directories, dials of clocks or of meters, ledgers,
maps, aerial photographs, etc. Duplicates in the frame shall be eliminated. N shall be well established. Random numbers (or a
The boldface numbers in parentheses refer to a list of references at the end of this standard.
E141 − 10 (2018)
certifiably honest physical random device) shall dictate selection of the sample. There shall be no substitution of one sampling unit
for another. The method of sample selection shall permit calculation of a standard error of the estimate. The use of replicate
subsamples is recommended (see 5.4). An audit subsample should be selected and processed and any departures from prescribed
measurement methods and location instructions noted (see 5.5). A report should list θˆ and its standard error with the degrees of
freedom in the se(θˆ).
5.3 Parameter Definition—The equal complete coverage result may or may not be acceptable evidence. Whether it is acceptable
depends on many considerations such as definitions, method of test, care exercised in the testing, completeness of the frame, and
on other points not to be settled by statistical theory since these points belong to the subject matter, and are the same whether one
uses sampling or not. Mistakes, whether in testing, counting, or weighing will affect the result of a complete coverage just as such
mistakes will affect the sample result. By a more expensive method of measurement or more elaborate sampling frame, it may be
possible to define a quantity, θ', as a target parameter or ideal goal of an investigation. Criticism that holds θ to be an inappropriate
goal should demonstrate that the numerical difference between θ and θ' is substantial. Measurements may be imprecise but so long
as measurement errors are not too biased, a large size of the lot or population, N, insures that θ and θ' are essentially equal.
5.4 Replicate Subsamples—When appropriate, separate laboratories should each work on separate replicate subsamples and
teams of investigators should be assigned to separate replicate subsamples. This approach insures that the calculated standard error
will not be a systematic underestimate. Such subsamples were called interpenetrating in Ref (4) where many of their basic
properties were described. See Ref (5) for further theory and applications.
5.4.1 For some types of material, a sample selected with uniform spacing along the frame (systematic sample) has increased
precision over a selection made with randomly varying spacings (simple random sample). Examples include sampling mineral ore
or grain from a conveyor belt or sampling from a list of households along a street. If the systematic sample is obtained by a single
random start the plan is then a probability sampling plan, but it does not permit calculating the standard error as required by this
practice. After dividing the sample size by an integer k (such as k = 4 or k = 10) and using a random start for each of k replicate
subsamples, some of the increased precision of systematic sampling (and a standard error on k − 1 degrees of freedom) can be
achieved.
5.5 An audit subsample of the survey sample should be taken for review of all procedures from use of the random numbers
through locating and measurement, to editing, coding, data entry and tabulation. Selection of the audit subsample may be done by
putting the n sample observations in order as they are collected, calculating the nearest integer to =n , or some other convenient
integer, and taking this number to be the spacing for systematic selection of the audit subsample. As few as 10 observations may
be adequate. The review should uncover any gross departures from prescribed practices or any conceptual misunderstandings in
the definitions. If the audit subsample is large enough (say 30 observations or more) the regression of audited values on initial
observations may be used to calibrate the estimate. This technique is the method of two-phase sampling as discussed in Ref (1).
Helpful discussion of an audit appears in Ref (2).
5.6 The estimate is a quantity calculated on the n sample observations in the same way as the equal complete coverage result
θ would have been calculated from the entire set of N possible observations of the population; the symbol θˆ denotes the estimate.
In calculating θˆ, replicate subsample membership is ignored.
5.6.1 An estimate has a sampling distribution induced from the randomness in sample selection. The equal complete coverage
result is effectively a constant while any estimate is only the value from one particular sample. Thus, there is a mean value of the
sampling distribution and there is also a standard deviation of the sampling distribution.
5.7 The standard error is the quantity computed from the observations as an estimate of the sampling standard deviation of the
estimate; se(θˆ) denotes the standard error.
5.7.1 When θ is the population average of the N quantities and a simple random sample of size n was drawn, then the sample
average y¯ becomes the usual estimate θˆ, where:
n
ˆ
θ5 yH 5 y /n (1)
i
(
i51
The quantities y , y , ., y denote the observations. The standard error is calculated as:
1 2 n
n
ˆ 2
se~θ! 5 se~yH! 5Œ ~y 2 yH! /n~n 2 1! (2)
( i
i51
There are n − 1 degrees of freedom in this standard error.
5.7.1.1 Example—When the observations are:
81.6, 78.7, 79.7, 78.3, 80.9, 79.5, 79.8, 80.3, 79.5, 80.7
then y¯ = 79.90 and se(y¯) = 0.32.
5.7.2 Finite Population Correction (fpc)—Multiplying se(y¯) by =12n/N is always correct when the
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.