ASTM E178-08
(Practice)Standard Practice for Dealing With Outlying Observations
Standard Practice for Dealing With Outlying Observations
ABSTRACT
This practice covers outlying observations in samples and how to test the statistical significance of them. An outlying observation, or outlier, is one that appears to deviate markedly from other members of the sample in which it occurs. In this connection, the following two alternatives are of interest: (i) an outlying observation may be merely an extreme manifestation of the random variability inherent in the data. If this is true, the value should be retained and processed in the same manner as the other observations in the sample. (ii) An outlying observation may be the result of gross deviation from prescribed experimental procedure or an error in calculating or recording the numerical value. In such cases, it may be desirable to institute an investigation to ascertain the reason for the aberrant value. The observation may even actually be rejected as a result of the investigation, though not necessarily so. At any rate, in subsequent data analysis, the outlier or outliers will be recognized as probably being from a different population than that of the other sample values. Recommended criteria and illustrations for single samples including the Dixon criteria which are based entirely on ratios of differences between the observations, criterion using independent standard deviation, and criterion for known standard deviation are presented.
SCOPE
1.1 This practice covers outlying observations in samples and how to test the statistical significance of them. An outlying observation, or “outlier,” is one that appears to deviate markedly from other members of the sample in which it occurs. In this connection, the following two alternatives are of interest:
1.1.1 An outlying observation may be merely an extreme manifestation of the random variability inherent in the data. If this is true, the value should be retained and processed in the same manner as the other observations in the sample.
1.1.2 On the other hand, an outlying observation may be the result of gross deviation from prescribed experimental procedure or an error in calculating or recording the numerical value. In such cases, it may be desirable to institute an investigation to ascertain the reason for the aberrant value. The observation may even actually be rejected as a result of the investigation, though not necessarily so. At any rate, in subsequent data analysis the outlier or outliers will be recognized as probably being from a different population than that of the other sample values.
1.2 It is our purpose here to provide statistical rules that will lead the experimenter almost unerringly to look for causes of outliers when they really exist, and hence to decide whether alternative 1.1.1 above, is not the more plausible hypothesis to accept, as compared to alternative 1.1.2, in order that the most appropriate action in further data analysis may be taken. The procedures covered herein apply primarily to the simplest kind of experimental data, that is, replicate measurements of some property of a given material, or observations in a supposedly single random sample. Nevertheless, the tests suggested do cover a wide enough range of cases in practice to have broad utility.
General Information
Relations
Buy Standard
Standards Content (Sample)
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Contact ASTM International (www.astm.org) for the latest information
Designation: E178 − 08 AnAmerican National Standard
Standard Practice for
1
Dealing With Outlying Observations
This standard is issued under the fixed designation E178; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope 2. Referenced Documents
2
1.1 This practice covers outlying observations in samples 2.1 ASTM Standards:
andhowtotestthestatisticalsignificanceofthem.Anoutlying E456Terminology Relating to Quality and Statistics
observation, or “outlier,” is one that appears to deviate mark-
3. Terminology
edly from other members of the sample in which it occurs. In
3.1 Definitions: The terminology defined in Terminology
this connection, the following two alternatives are of interest:
E456 applies to this standard unless modified herein.
1.1.1 An outlying observation may be merely an extreme
3.1.1 outlier—see outlying observation.
manifestation of the random variability inherent in the data. If
this is true, the value should be retained and processed in the
3.1.2 outlying observation, n—an observation that appears
same manner as the other observations in the sample.
todeviatemarkedlyinvaluefromothermembersofthesample
1.1.2 Ontheotherhand,anoutlyingobservationmaybethe
in which it appears.
result of gross deviation from prescribed experimental proce-
4. Significance and Use
dureoranerrorincalculatingorrecordingthenumericalvalue.
In such cases, it may be desirable to institute an investigation
4.1 When the experimenter is clearly aware that a gross
to ascertain the reason for the aberrant value. The observation deviation from prescribed experimental procedure has taken
may even actually be rejected as a result of the investigation,
place,theresultantobservationshouldbediscarded,whetheror
though not necessarily so. At any rate, in subsequent data not it agrees with the rest of the data and without recourse to
analysis the outlier or outliers will be recognized as probably
statistical tests for outliers. If a reliable correction procedure,
being from a different population than that of the other sample for example, for temperature, is available, the observation may
values.
sometimes be corrected and retained.
1.2 Itisourpurposeheretoprovidestatisticalrulesthatwill
4.2 In many cases evidence for deviation from prescribed
lead the experimenter almost unerringly to look for causes of
procedure will consist primarily of the discordant value itself.
outliers when they really exist, and hence to decide whether
Insuchcasesitisadvisabletoadoptacautiousattitude.Useof
alternative 1.1.1 above, is not the more plausible hypothesis to
one of the criteria discussed below will sometimes permit a
accept, as compared to alternative 1.1.2, in order that the most
clear-cut decision to be made. In doubtful cases the experi-
appropriate action in further data analysis may be taken. The
menter’s judgment will have considerable influence.When the
procedures covered herein apply primarily to the simplest kind
experimentercannotidentifyabnormalconditions,heshouldat
of experimental data, that is, replicate measurements of some
least report the discordant values and indicate to what extent
property of a given material, or observations in a supposedly
they have been used in the analysis of the data.
single random sample. Nevertheless, the tests suggested do
4.3 Thus, for purposes of orientation relative to the over-all
cover a wide enough range of cases in practice to have broad
problem of experimentation, our position on the matter of
utility.
screening samples for outlying observations is precisely the
following:
4.3.1 Physical Reason Known or Discovered for Outlier(s):
1
This practice is under the jurisdiction ofASTM Committee D19 on Water and
is the direct responsibility of Subcommittee D19.05 on Inorganic Constituents in
2
Water. For referenced ASTM standards, visit the ASTM website, www.astm.org, or
Current edition approved . Published November 2008. Originally approved in contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
1961. Last previous edition approved in 2002 as E178–02. DOI: 10.1520/E0178- Standards volume information, refer to the standard’s Document Summary page on
08. the ASTM website.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
1
---------------------- Page: 1 ----------------------
E178 − 08
4.3.1.1 Reject observation(s). criteria presented may also be used to test the hypothesis of
4.3.1.2 Correct observation(s) on physical grounds. normality or that the random sample taken did come from a
4.3.1.3 Reject it (them) an
...
This document is not anASTM standard and is intended only to provide the user of anASTM standard an indication of what changes have been made to the previous version. Because
it may not be technically possible to adequately depict all changes accurately, ASTM recommends that users consult prior editions as appropriate. In all cases only the current version
of the standard as published by ASTM is to be considered the official document.
An American National Standard
Designation:E 178–94 Designation: E178 – 08
Standard Practice for
1
Dealing With Outlying Observations
This standard is issued under the fixed designation E178; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (ϵ) indicates an editorial change since the last revision or reapproval.
1. Scope
1.1 This practice covers outlying observations in samples and how to test the statistical significance of them. An outlying
observation, or “outlier,” is one that appears to deviate markedly from other members of the sample in which it occurs. In this
connection, the following two alternatives are of interest:
1.1.1 An outlying observation may be merely an extreme manifestation of the random variability inherent in the data. If this
is true, the value should be retained and processed in the same manner as the other observations in the sample.
1.1.2 On the other hand, an outlying observation may be the result of gross deviation from prescribed experimental procedure
or an error in calculating or recording the numerical value. In such cases, it may be desirable to institute an investigation to
ascertain the reason for the aberrant value. The observation may even actually be rejected as a result of the investigation, though
not necessarily so. At any rate, in subsequent data analysis the outlier or outliers will be recognized as probably being from a
different population than that of the other sample values.
1.2 It is our purpose here to provide statistical rules that will lead the experimenter almost unerringly to look for causes of
outlierswhentheyreallyexist,andhencetodecidewhetheralternative1.1.1above,isnotthemoreplausiblehypothesistoaccept,
as compared to alternative 1.1.2, in order that the most appropriate action in further data analysis may be taken. The procedures
covered herein apply primarily to the simplest kind of experimental data, that is, replicate measurements of some property of a
given material, or observations in a supposedly single random sample. Nevertheless, the tests suggested do cover a wide enough
range of cases in practice to have broad utility.
1
This practice is under the jurisdiction ofASTM Committee E-11 on Statistical Methods and is the direct responsibility of Subcommittee E11 on Quality and Statistics
and is the direct responsibility of Subcommittee E11.10 on Sampling and Data Analysis.
ϵ1
CurrenteditionapprovedJuly15,1994.PublishedSeptember1994.OriginallypublishedasE178–61T.LastpreviouseditionE178–80(1989) .onSampling/Statistics.
Current edition approved Oct. 1, 2008. Published November 2008. Originally approved in 1961. Last previous edition approved in 2002 as E178–02. DOI:
10.1520/E0178-08.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.
1
---------------------- Page: 1 ----------------------
E178 – 08
2. GeneralReferenced Documents
2
2.1When 2.1 ASTM Standards:
E456 Terminology Relating to Quality and Statistics
3. Terminology
3.1 Definitions: The terminology defined in Terminology E456 applies to this standard unless modified herein.
3.1.1 outlier—see outlying observation.
3.1.2 outlying observation, n—an observation that appears to deviate markedly in value from other members of the sample in
which it appears.
4. Significance and Use
4.1 When the experimenter is clearly aware that a gross deviation from prescribed experimental procedure has taken place, the
resultant observation should be discarded, whether or not it agrees with the rest of the data and without recourse to statistical tests
for outliers. If a reliable correction procedure, for example, for temperature, is available, the observation may sometimes be
corrected and retained.
2.2In4.2 In many cases evidence for deviation from prescribed procedure will consist primarily of the discordant value itself.
In such cases it is advisable to adopt a cautious attitude. Use of one of the criteria discussed below will sometimes permit a
clear-cut decision to be made. In doubtful cases the experimenter’s judgment will have considerable influence. When the
experimenter cannot identify abnormal conditions, he should at least report the discordant values and indicate to what extent they
have been used in the analysis of the data.
2.3Thus,4.3 Thus, for purposes of orientation relative to the over-all problem of experimentation, our posit
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.