ASTM E178-16
(Practice)Standard Practice for Dealing With Outlying Observations
Standard Practice for Dealing With Outlying Observations
ABSTRACT
This practice covers outlying observations in samples and how to test the statistical significance of them. An outlying observation, or outlier, is one that appears to deviate markedly from other members of the sample in which it occurs. In this connection, the following two alternatives are of interest: (i) an outlying observation may be merely an extreme manifestation of the random variability inherent in the data. If this is true, the value should be retained and processed in the same manner as the other observations in the sample. (ii) An outlying observation may be the result of gross deviation from prescribed experimental procedure or an error in calculating or recording the numerical value. In such cases, it may be desirable to institute an investigation to ascertain the reason for the aberrant value. The observation may even actually be rejected as a result of the investigation, though not necessarily so. At any rate, in subsequent data analysis, the outlier or outliers will be recognized as probably being from a different population than that of the other sample values. Recommended criteria and illustrations for single samples including the Dixon criteria which are based entirely on ratios of differences between the observations, criterion using independent standard deviation, and criterion for known standard deviation are presented.
SCOPE
1.1 This practice covers outlying observations in samples and how to test the statistical significance of outliers.
1.2 The system of units for this standard is not specified. Dimensional quantities in the standard are presented only as illustrations of calculation methods. The examples are not binding on products or test methods treated.
1.3 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility of the user of this standard to establish appropriate safety and health practices and determine the applicability of regulatory requirements prior to use.
General Information
Relations
Buy Standard
Standards Content (Sample)
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Contact ASTM International (www.astm.org) for the latest information
Designation: E178 − 16 AnAmerican National Standard
Standard Practice for
1
Dealing With Outlying Observations
This standard is issued under the fixed designation E178; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope 4. Significance and Use
1.1 This practice covers outlying observations in samples 4.1 An outlying observation, or “outlier,” is an extreme one
and how to test the statistical significance of outliers. in either direction that appears to deviate markedly from other
members of the sample in which it occurs.
1.2 The system of units for this standard is not specified.
4.2 Statistical rules test the null hypothesis of no outliers
Dimensional quantities in the standard are presented only as
illustrations of calculation methods. The examples are not against the alternative of one or more actual outliers. The
procedures covered were developed primarily to apply to the
binding on products or test methods treated.
simplest kind of experimental data, that is, replicate measure-
1.3 This standard does not purport to address all of the
ments of some property of a given material or observations in
safety concerns, if any, associated with its use. It is the
a supposedly random sample.
responsibility of the user of this standard to establish appro-
priate safety and health practices and determine the applica- 4.3 Astatistical test may be used to support a judgment that
bility of regulatory requirements prior to use. a physical reason does actually exist for an outlier, or the
statistical criterion may be used routinely as a basis to initiate
action to find a physical cause.
2. Referenced Documents
2
2.1 ASTM Standards:
5. Procedure
E456Terminology Relating to Quality and Statistics
E2586Practice for Calculating and Using Basic Statistics 5.1 In dealing with an outlier, the following alternatives
should be considered:
3. Terminology
5.1.1 An outlying observation might be the result of gross
deviation from prescribed experimental procedure or an error
3.1 Definitions: The terminology defined in Terminology
in calculating or recording the numerical value. When the
E456 applies to this standard unless modified herein.
experimenter is clearly aware that a deviation from prescribed
3.1.1 order statistic x ,n—value of the kth observed value
(k)
experimental procedure has taken place, the resultant observa-
in a sample after sorting by order of magnitude. (Practice
tion should be discarded, whether or not it agrees with the rest
E2586.)
of the data and without recourse to statistical tests for outliers.
3.1.1.1 Discussion—In this Practice, x is used to denote
k
If a reliable correction procedure is available, the observation
order statistics in place of x , to simplify the notation.
(k)
may sometimes be corrected and retained.
3.1.2 outlier—see outlying observation.
5.1.2 An outlying observation might be merely an extreme
manifestation of the random variability inherent in the data. If
3.1.3 outlying observation, n—an extreme observation in
this is true, the value should be retained and processed in the
either direction that appears to deviate markedly in value from
same manner as the other observations in the sample. Trans-
other members of the sample in which it appears.
formation of data or using methods of data analysis designed
for a non-normal distribution might be appropriate.
5.1.3 Test units that give outlying observations might be of
1
ThispracticeisunderthejurisdictionofASTMCommitteeE11onQualityand
special interest. If this is true, once identified they should be
Statistics and is the direct responsibility of Subcommittee E11.10 on Sampling /
Statistics. segregated for more detailed study.
Current edition approved June 1, 2016. Published June 2016. Originally
5.2 In many cases, evidence for deviation from prescribed
approved in 1961. Last previous edition approved in 2008 as E178–08. DOI:
10.1520/E0178-16.
procedure will consist primarily of the discordant value itself.
2
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
Insuchcasesitisadvisabletoadoptacautiousattitude.Useof
contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
one of the criteria discussed below will sometimes permit a
Standards volume information, refer to the standard’s Document Summary page on
the ASTM website. clearcut decision to be made.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
1
---------------------- Page: 1 ----------------------
E178 − 16
5.2.1 When the experimenter cannot identify abnormal 6.3 Although
...
This document is not an ASTM standard and is intended only to provide the user of an ASTM standard an indication of what changes have been made to the previous version. Because
it may not be technically possible to adequately depict all changes accurately, ASTM recommends that users consult prior editions as appropriate. In all cases only the current version
of the standard as published by ASTM is to be considered the official document.
Designation: E178 − 08 E178 − 16 An American National Standard
Standard Practice for
1
Dealing With Outlying Observations
This standard is issued under the fixed designation E178; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope
1.1 This practice covers outlying observations in samples and how to test the statistical significance of them. An outlying
observation, or “outlier,” is one that appears to deviate markedly from other members of the sample in which it occurs. In this
connection, the following two alternatives are of interest:outliers.
1.1.1 An outlying observation may be merely an extreme manifestation of the random variability inherent in the data. If this
is true, the value should be retained and processed in the same manner as the other observations in the sample.
1.1.2 On the other hand, an outlying observation may be the result of gross deviation from prescribed experimental procedure
or an error in calculating or recording the numerical value. In such cases, it may be desirable to institute an investigation to
ascertain the reason for the aberrant value. The observation may even actually be rejected as a result of the investigation, though
not necessarily so. At any rate, in subsequent data analysis the outlier or outliers will be recognized as probably being from a
different population than that of the other sample values.
1.2 It is our purpose here to provide statistical rules that will lead the experimenter almost unerringly to look for causes of
outliers when they really exist, and hence to decide whether alternative The system of units for this standard 1.1.1 above, is not
the more plausible hypothesis to accept, as compared to alternative is not specified. Dimensional quantities 1.1.2, in order that the
most appropriate action in further data analysis may be taken. The procedures covered herein apply primarily to the simplest kind
of experimental data, that is, replicate measurements of some property of a given material, or observations in a supposedly single
random sample. Nevertheless, the tests suggested do cover a wide enough range of cases in practice to have broad utility.the
standard are presented only as illustrations of calculation methods. The examples are not binding on products or test methods
treated.
1.3 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility
of the user of this standard to establish appropriate safety and health practices and determine the applicability of regulatory
requirements prior to use.
2. Referenced Documents
2
2.1 ASTM Standards:
E456 Terminology Relating to Quality and Statistics
E2586 Practice for Calculating and Using Basic Statistics
3. Terminology
3.1 Definitions: The terminology defined in Terminology E456 applies to this standard unless modified herein.
3.1.1 order statistic x , n—value of the kth observed value in a sample after sorting by order of magnitude. (Practice E2586.)
(k)
1
This practice is under the jurisdiction of ASTM Committee D19E11 on WaterQuality and Statistics and is the direct responsibility of Subcommittee D19.05E11.10 on
Inorganic Constituents in WaterSampling / Statistics.
Current edition approved June 1, 2016. Published November 2008June 2016. Originally approved in 1961. Last previous edition approved in 20022008 as
E178 – 02.E178 – 08. DOI: 10.1520/E0178-08.10.1520/E0178-16.
2
For referenced ASTM standards, visit the ASTM website, www.astm.org, or contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM Standards
volume information, refer to the standard’s Document Summary page on the ASTM website.
3.1.1.1 Discussion—
In this Practice, x is used to denote order statistics in place of x , to simplify the notation.
k (k)
3.1.2 outlier—see outlying observation.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
1
---------------------- Page: 1 ----------------------
E178 − 16
3.1.3 outlying observation, n—an observation extreme observation in either direction that appears to deviate markedly in value
from other members of the sample in which it appears.
4. Significance and Use
4.1 When the experimenter is clearly aware that a gross deviation from prescribed experimental procedu
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.