ASTM E2782-17(2022)e1

Standard Guide for Measurement Systems Analysis (MSA)

ABSTRACT
This guide presents terminology, concepts, and selected methods and formulas useful for measurement systems analysis (MSA). Measurement systems analysis may be broadly described as a body of theory and methodology that applies to the non-destructive measurement of the physical properties of manufactured objects. This guide presents selected concepts and methods useful for describing and understanding the measurement process. This guide is not intended to be a comprehensive survey of this topic.
SIGNIFICANCE AND USE
4.1 Many types of measurements are made routinely in research organizations, business and industry, and government and academic agencies. Typically, data are generated from experimental effort or as observational studies. From such data, management decisions are made that may have wide-reaching social, economic, and political impact. Data and decision making go hand in hand and that is why the quality of any measurement is important—for data originate from a measurement process. This guide presents selected concepts and methods useful for describing and understanding the measurement process. This guide is not intended to be a comprehensive survey of this topic.  
4.2 Any measurement result will be said to originate from a measurement process or system. The measurement process will consist of a number of input variables and general conditions that affect the final value of the measurement. The process variables, hardware and software and their properties, and the human effort required to obtain a measurement constitute the measurement process. A measurement process will have several properties that characterize the effect of the several variables and general conditions on the measurement results. It is the properties of the measurement process that are of primary interest in any such study. The term “measurement systems analysis” or MSA study is used to describe the several methods used to characterize the measurement process.
Note 1: Sample statistics discussed in this guide are as described in Practice E2586; control chart methodologies are as described in Practice E2587.
SCOPE
1.1 This guide presents terminology, concepts, and selected methods and formulas useful for measurement systems analysis (MSA). Measurement systems analysis may be broadly described as a body of theory and methodology that applies to the non-destructive measurement of the physical properties of manufactured objects.  
1.2 Units—The system of units for this guide is not specified. Dimensional quantities in the guide are presented only as illustrations of calculation methods and are not binding on products or test methods treated.  
1.3 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility of the user of this standard to establish appropriate safety, health, and environmental practices and determine the applicability of regulatory limitations prior to use.  
1.4 This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.

General Information

• Status: Published
• Publication Date: 14-May-2022
• ICS: 17.020 - Metrology and measurement in general
• Technical Committee: E11 - Quality and Statistics
• Drafting Committee: E11.50 - Metrology

Relations

• Refers: ASTM E456-13a(2022)e1 - Standard Terminology Relating to Quality and Statistics
• Effective Date: 01-Apr-2022
• Refers: ASTM E2586-19e1 - Standard Practice for Calculating and Using Basic Statistics
• Effective Date: 01-Apr-2019
• Refers: ASTM E456-13A(2017)e1 - Standard Terminology Relating to Quality and Statistics
• Effective Date: 01-Oct-2017
• Refers: ASTM E456-13A(2017)e3 - Standard Terminology Relating to Quality and Statistics
• Effective Date: 01-Oct-2017
• Refers: ASTM E2587-15 - Standard Practice for Use of Control Charts in Statistical Process Control
• Effective Date: 01-Apr-2015
• Refers: ASTM E2587-14 - Standard Practice for Use of Control Charts in Statistical Process Control
• Effective Date: 01-Oct-2014
• Refers: ASTM E2587-14e1 - Standard Practice for Use of Control Charts in Statistical Process Control
• Effective Date: 01-Oct-2014
• Refers: ASTM E2586-14 - Standard Practice for Calculating and Using Basic Statistics
• Effective Date: 01-Jun-2014
• Refers: ASTM E177-14 - Standard Practice for Use of the Terms Precision and Bias in ASTM Test Methods
• Effective Date: 01-May-2014
• Refers: ASTM E456-13ae1 - Standard Terminology Relating to Quality and Statistics
• Effective Date: 15-Nov-2013
• Refers: ASTM E456-13a - Standard Terminology Relating to Quality and Statistics
• Effective Date: 15-Nov-2013
• Refers: ASTM E456-13ae2 - Standard Terminology Relating to Quality and Statistics
• Effective Date: 15-Nov-2013
• Refers: ASTM E456-13ae3 - Standard Terminology Relating to Quality and Statistics
• Effective Date: 15-Nov-2013
• Refers: ASTM E2586-13 - Standard Practice for Calculating and Using Basic Statistics
• Effective Date: 01-Oct-2013
• Refers: ASTM E456-13 - Standard Terminology Relating to Quality and Statistics
• Effective Date: 15-Aug-2013

Overview

ASTM E2782-17(2022)e1: Standard Guide for Measurement Systems Analysis (MSA), developed by ASTM International, provides essential terminology, foundational concepts, and selected methods for conducting measurement systems analysis. MSA is a comprehensive methodology used to evaluate measurement processes and systems, ensuring that the data generated during the non-destructive assessment of physical properties of manufactured objects are accurate, reliable, and fit for decision-making in research, industry, business, and government.

The guide is designed to aid professionals in understanding the sources of variation within measurement processes by defining and analyzing key attributes such as repeatability, reproducibility, bias, linearity, stability, consistency, resolution, and measurement error. While not exhaustive, ASTM E2782-17 equips users with practical approaches and statistical tools to characterize and improve their measurement systems.

Key Topics

• Measurement Systems Analysis (MSA): Focuses on evaluating the entire measurement process, encompassing hardware, software, human influence, and environmental conditions.
• Terminology and Definitions: Clearly defines terms such as repeatability, reproducibility, bias, linearity, resolution, gage R&R (repeatability and reproducibility), and measurement system capability.
• Properties of Measurement Systems:
  • Repeatability: Variation when the same operator measures the same item multiple times under identical conditions.
  • Reproducibility: Variation observed when different operators measure the same item under identical conditions.
  • Bias: The difference between the average measurement and an accepted reference value.
  • Linearity: Change in measurement bias through the expected operating range.
  • Stability: Consistency of measurement bias over time.
  • Consistency: Uniformity in repeatability over time.
  • Resolution: The measurement system's ability to detect and differentiate small variations.
  • Gage R&R Studies: Analyze the combined effect of repeatability and reproducibility to assess overall measurement variation.
• Statistical Methods: Uses analysis of variance (ANOVA), control chart methodologies, and statistical control to evaluate and improve measurement systems.
• Measurement Error Evaluation: Examines all potential sources of error, including equipment variation and human factors.

Applications

Measurement systems analysis according to ASTM E2782-17(2022)e1 is crucial across diverse fields:

• Quality Assurance & Control: Ensures measurement systems are capable of producing data suitable for making reliable management, manufacturing, or regulatory decisions.
• Manufacturing: Validates and improves measurement processes in non-destructive testing of parts, products, and assemblies.
• Research & Development: Supports experimental and observational studies by establishing sound measurement foundations.
• Calibration Laboratories: Assists in benchmarking the accuracy and precision of measurement equipment.
• Regulatory Compliance: Provides methodologies recognized internationally for demonstrating due diligence in measurement reliability, aligning with WTO Technical Barriers to Trade (TBT) Committee principles.

Adopting MSA as outlined in this standard helps organizations identify, quantify, and mitigate sources of measurement variation, reduce risks in decision-making, and improve overall product and process quality.

Related Standards

ASTM E2782-17(2022)e1 builds upon and references several foundational ASTM standards, including:

• ASTM E177: Practice for Use of the Terms Precision and Bias in ASTM Test Methods
• ASTM E456: Terminology Relating to Quality and Statistics
• ASTM E2586: Practice for Calculating and Using Basic Statistics
• ASTM E2587: Practice for Use of Control Charts in Statistical Process Control

These related standards provide detailed definitions, statistical techniques, and control chart methodologies that complement the measurement systems analysis approaches advocated in ASTM E2782-17.

---

Keywords: Measurement Systems Analysis, MSA, ASTM E2782-17, measurement process, repeatability, reproducibility, gage R&R, measurement bias, linearity, resolution, control charts, measurement system evaluation, quality assurance, calibration, non-destructive testing, measurement error, statistical process control, data reliability, international standards.

Standards Content (Sample)

This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
´1
Designation: E2782 − 17 (Reapproved 2022) An American National Standard
Standard Guide for
Measurement Systems Analysis (MSA)
This standard is issued under the fixed designation E2782; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
ε NOTE—Subsection 3.1 was corrected editorially in July 2022.
1. Scope 3. Terminology
3.1 Definitions—Unless otherwise noted in this standard, all
1.1 This guide presents terminology, concepts, and selected
terms relating to quality and statistics are defined in Terminol-
methods and formulas useful for measurement systems analy-
ogy E456.
sis (MSA). Measurement systems analysis may be broadly
described as a body of theory and methodology that applies to 3.1.1 accepted reference value, n—a value that serves as an
the non-destructive measurement of the physical properties of agreed-upon reference for comparison, and which is derived
manufactured objects. as: (1) a theoretical or established value, based on scientific
principles, (2) an assigned or certified value, based on experi-
1.2 Units—The system of units for this guide is not speci-
mental work of some national or international organization, or
fied. Dimensional quantities in the guide are presented only as
(3) a consensus or certified value, based on collaborative
illustrations of calculation methods and are not binding on
experimental work under the auspices of a scientific or
products or test methods treated.
engineering group. E177
1.3 This standard does not purport to address all of the
3.1.2 calibration, n—process of establishing a relationship
safety concerns, if any, associated with its use. It is the
between a measurement device and a known standard value(s).
responsibility of the user of this standard to establish appro-
3.1.3 gage, n—device used as part of the measurement
priate safety, health, and environmental practices and deter-
process to obtain a measurement result.
mine the applicability of regulatory limitations prior to use.
1.4 This international standard was developed in accor- 3.1.4 measurement process, n—process used to assign a
dance with internationally recognized principles on standard-
number to a property of an object or other physical entity.
ization established in the Decision on Principles for the
3.1.4.1 Discussion—The term “measurement system” is
Development of International Standards, Guides and Recom-
sometimes used in place of measurement process. (See 3.1.6.)
mendations issued by the World Trade Organization Technical
3.1.5 measurement result, n—number assigned to a property
Barriers to Trade (TBT) Committee.
of an object or other physical entity being measured.
3.1.5.1 Discussion—Theword“measurement”isusedinthe
2. Referenced Documents
same sense as measurement result.
2.1 ASTM Standards:
3.1.6 measurement system, n—the collection of hardware,
E177 Practice for Use of the Terms Precision and Bias in
software, procedures and methods, human effort, environmen-
ASTM Test Methods
tal conditions, associated devices, and the objects that are
E456 Terminology Relating to Quality and Statistics
measured for the purpose of producing a measurement.
E2586 Practice for Calculating and Using Basic Statistics
3.1.7 measurement systems analysis (MSA), n—any of a
E2587 Practice for Use of Control Charts in Statistical
number of specialized methods useful for studying a measure-
Process Control
ment system and its properties.
3.2 Definitions of Terms Specific to This Standard:
3.2.1 appraiser, n—the person who uses a gage or measure-
This guide is under the jurisdiction of ASTM Committee E11 on Quality and
Statistics and is the direct responsibility of Subcommittee E11.50 on Metrology.
ment system.
Current edition approved May 15, 2022. Published May 2022. Originally
3.2.2 discrimination ratio, n—statistical ratio calculated
approved in 2011. Last previous edition approved in 2017 as E2782 – 17. DOI:
10.1520/E2782-17R22E01.
from the statistics from a gage R&R study that measures the
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
number of 97 % confidence intervals, constructed from gage
contact ASTM Customer Service at [e-mail]. For Annual Book of ASTM
R&R variation, that fit within six standard deviations of true
Standards volume information, refer to the standard’s Document Summary page on
the ASTM website. object variation.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
´1
E2782 − 17 (2022)
3.2.3 distinct product categories, n—alternate meaning of making go hand in hand and that is why the quality of any
the discrimination ratio. measurement is important—for data originate from a measure-
ment process. This guide presents selected concepts and
3.2.4 gage consistency, n—constancy of repeatability vari-
methods useful for describing and understanding the measure-
ance over a period of time.
ment process.This guide is not intended to be a comprehensive
3.2.4.1 Discussion—Consistency means that the variation
survey of this topic.
within measurements of the same object (or group of objects)
under the same conditions by the same appraiser behaves in a
4.2 Any measurement result will be said to originate from a
state of statistical control as judged, for example, using a
measurementprocessorsystem.Themeasurementprocesswill
control chart. See Practice E2587.
consist of a number of input variables and general conditions
that affect the final value of the measurement. The process
3.2.5 gage performance curve, n—curve that shows the
variables, hardware and software and their properties, and the
probability of gage acceptance of an object given its real value
human effort required to obtain a measurement constitute the
or the probability that an object’s real measure meets a
measurement process. A measurement process will have sev-
requirement given the measurement of the object.
eral properties that characterize the effect of the several
3.2.6 gage R&R, n—combined effect of gage repeatability
variables and general conditions on the measurement results. It
and reproducibility.
isthepropertiesofthemeasurementprocessthatareofprimary
3.2.7 gage resolution, n—degree to which a gage can
interest in any such study. The term “measurement systems
discriminate between differing objects.
analysis” or MSAstudy is used to describe the several methods
3.2.7.1 Discussion—The smallest difference between two
used to characterize the measurement process.
objects that a gage is capable of detecting is referred to as its
NOTE 1—Sample statistics discussed in this guide are as described in
finiteresolutionproperty.Forexample,alinearscalegraduated Practice E2586; control chart methodologies are as described in Practice
E2587.
in tenths of an inch is not capable of discriminating between
objects that differ by less than 0.1 in. (0.25 cm).
5. Characteristics of a Measurement System (Process)
3.2.8 gage stability, n—absence of a change, drift, or erratic
5.1 Measurement has been defined as “the assignment of
behavior in bias over a period of time.
numbers to material objects to represent the relations existing
3.2.8.1 Discussion—Stability means that repeated measure-
among them with respect to particular properties. The number
ments of the same object (or average of a set of objects) under
assigned to some particular property serves to represent the
the same conditions by the same appraiser behave in a state of
relative amount of this property associated with the object
statistical control as judged for example by using a control
concerned.” (1)
chart technique. See Practice E2587.
5.2 Ameasurement system may be described as a collection
3.2.9 linearity, n—difference or change in bias throughout
of hardware, software, procedures and methods, human effort,
theexpectedoperatingrangeofagageormeasurementsystem.
environmental conditions, associated devices, and the objects
3.2.10 measurement error, n—error incurred in the process
that are measured for the purpose of producing a measurement.
of measurement.
In the practical working of the measurement system, these
3.2.10.1 Discussion—As used in this guide, measurement
factors combine to cause variation among measurements of the
error includes one or both of R&R types of error.
same object that would not be present if the system were
3.2.11 repeatability conditions, n—in a gage R&R study,
perfect. A measurement system can have varying degrees of
conditionsinwhichindependentmeasurementsareobtainedon
each of these factors, and in some cases, one or more factors
identical objects, or a group of objects, by the same operator
may be the dominant contributor to this variation.
using the same measurement system within short intervals of
5.2.1 A measurement system is like a manufacturing pro-
time.
cess for which the product is a supply of numbers called
3.2.11.1 Discussion—As used in this guide, repeatability is
measurement results. The measurement system uses input
often referred to as equipment variation or EV.
factors and a sequence of steps to produce a result. The inputs
are just varying degrees of the several factors described in 5.2
3.2.12 reproducibility conditions, n— in a gage R&R study,
including the objects being measured. The sequence of process
conditions in which independent test results are obtained with
steps are that which would be described in a method or
the same method, on identical test items by different operators.
procedure for producing the measurement. Taken as a whole,
3.2.12.1 Discussion—As used in this guide, reproducibility
the various factors and the process steps work collectively to
is often referred to as appraiser variation or AV. This term is
form the measurement system/process.
also used in a broader sense in Practice E177.
5.3 An important consideration in analyzing any measure-
4. Significance and Use
ment process is its interaction with time. This gives rise to the
properties of stability and consistency. A system that is stable
4.1 Many types of measurements are made routinely in
and consistent is one that is predictable, within limits, over a
research organizations, business and industry, and government
period of time. Such a system has properties that do not
and academic agencies. Typically, data are generated from
experimentaleffortorasobservationalstudies.Fromsuchdata,
management decisions are made that may have wide-reaching
The boldface numbers in parentheses refer to the list of references at the end of
social, economic, and political impact. Data and decision this standard.
´1
E2782 − 17 (2022)
deteriorate with time (at least within some set time period) and
is said to be in a state of statistical control. Statistical control,
stability and consistency, and predictability have the same
meaning in this sense. Measurement system instability and
inconsistency will cause further added overall variation over a
period of time.
5.3.1 In general, instability is a common problem in mea-
surement systems. Mechanical and electrical components may
wear or degrade with time, human effort may exhibit increas-
ing fatigue with time, software and procedures may change
FIG. 2 Reproducibility Concept
with time, environmental variables will vary with time, and so
forth. Thus, measurement system stability is of primary con-
homogeneous population, among several laboratories or as
cern in any ongoing measurement effort.
measured using several systems.
5.4 There are several basic properties of measurement
5.4.2.1 Reproducibility may include different equipment
systems that are widely recognized among practitioners. These
and measurement conditions. This broader interpretation has
are repeatability, reproducibility, linearity, bias, stability,
attached “reproducibility conditions” and shall be defined and
consistency, and resolution. In studying one or more of these
interpreted by the user of a measurement system. (In Practice
properties,thefinalresultofanysuchstudyissomeassessment
E177, reproducibility includes interlaboratory variation.)
of the capability of the measurement system with respect to the
5.4.3 Bias is the difference between a standard or accepted
property under investigation. Capability may be cast in several
reference value for an object, often called a “master,” and the
ways, and this may also be application dependent. One of the
average value of a sample of measurements of the object(s)
primary objectives in any MSA effort is to assess variation
under a fixed set of conditions (see Fig. 1).
attributabletothevariousfactorsofthesystem.Allofthebasic
5.4.4 Linearity is the change in bias over the operational
properties assess variation in some form.
range of the measurement system. If the bias is changing as a
5.4.1 Repeatabilityisthevariationthatresultswhenasingle
function of the object being measured, we would say that the
object is repeatedly measured in the same way, by the same
system is not linear. Linearity can also be interpreted to mean
appraiser, under the same conditions (see Fig. 1). The term
that an instrument response is linearly related to the character-
“precision” also denotes the same concept, but “repeatability”
istic being measured.
is found more often in measurement applications. The term
5.4.5 Stability is variation in bias with time, usually a drift
“conditions” is sometimes combined with repeatability to
or trend, or erratic behavior.
denote “repeatability conditions” (see Terminology E456).
5.4.6 Consistency is the change in repeatability with time.A
5.4.1.1 The phrase “intermediate precision” is also used (for
system is consistent with time when the standard deviation of
example, see Practice E177). The user of a measurement
the repeatability error remains constant. When a measurement
system shall decide what constitutes “repeatability conditions”
system is stable and consistent, we say that it is a state of
or “intermediate precision conditions” for the given applica-
statistical control.
tion. Typically, repeatability conditions for MSA will be as
5.4.7 The resolution of a measurement system has to do
described in 5.4.1.
with its ability to discriminate between different objects. A
5.4.2 Reproducibility is defined as the variation among
system with high resolution is one that is sensitive to small
average values as determined by several appraisers when
changesfromobjecttoobject.Inadequateresolutionmayresult
measuring the same group of objects using identical measure-
in identical measurements when the same object is measured
ment systems under the same conditions (see Fig. 2). In a
several times under identical conditions. In this scenario, the
broader sense, this may be taken as variation in average values
measurement device is not capable of picking up variation as a
of samples, either identical or selected at random from one
result of repeatability (under the conditions defined). Poor
resolution may also result in identical measurements when
differing objects are measured. In this scenario, the objects
themselves are too close in true magnitude for the system to
distinguish among.
5.4.7.1 Resolution plays an important role in measurement
in general. We can imagine the output of a process that is in
statistical control and follows a normal distribution with mean,
µ, and standard deviation, σ. Based on the normal distribution,
the natural spread of the process is 6σ. Suppose we measure
objects from this process with a perfect gage except for its
finite resolution property. Suppose further that the gage we are
using is “graduated” as some fraction, 1/k, of the 6σ natural
process spread (integer k). For example, if k = 4, then the
natural process tolerance would span four graduations on the
FIG. 1 Repeatability and Bias Concepts gage; if k = 6, then the natural process spread would span six
´1
E2782 − 17 (2022)
TABLE 1 Behavior of the Measurement Variance and Standard
graduations on the gage. It is clear that, as k increases, we
Deviation for Selected Finite Resolution Property, k, True
would have an increasingly better resolution and would be
Process Variance is 1
more likely to distinguish between distinct objects, however
total resolution stdev due to
close their magnitudes; at the opposite extreme, for small k, k
variance component resolution
fewer and fewer distinct objects from the process would be
2 1.36400 0.36400 0.60332
distinguishable. In the limit, for large k, every object from this
3 1.18500 0.18500 0.43012
4 1.11897 0.11897 0.34492
process would be distinguishable.
5 1.08000 0.08000 0.28284
5.4.7.2 In using this perfect gage, the finite resolution
6 1.05761 0.05761 0.24002
property plays a role in repeatability. For very large k, the
8 1.04406 0.04406 0.20990
9 1.03549 0.03549 0.18839
resulting standard deviation of many objects from the process
10 1.01877 0.01877 0.13700
would be nearly the magnitude of the true object standard
12 1.00539 0.00539 0.07342
deviation, σ.As k diminishes, the standard deviation of the
15 1.00447 0.00447 0.06686
measurementswouldincreaseasaresultofthefiniteresolution
property. Fig. 3 illustrates this concept for a process centered at
0 and having σ = 1 for k=4.
5.4.7.5 A common rule of thumb is for a measurement
5.4.7.3 The illustration from Fig. 3 is a system capable of
device to have a resolution no greater than 0.6σ, where σ is the
discriminating objects into groups no smaller than 1.5σ in
true natural process standard deviation. This would give us
widthsothatafrequencydistributionofmeasuredobjectsfrom
k = 10 graduation divisions within the true 6σ natural process
this system will generally have four bins. This means four
limits. In that particular case, the resulting variance of all
distinct product values can be detected. Using Fig. 3 and the
measurements would have increased by approximately 1.9 %
theoretical probabilities from the normal distribution, it is
(Table 1, k = 10).
possible to calculate the variance of the measured values for
5.5 MSAisabroadclassofactivitiesthatstudiestheseveral
various values of k. In this case, the variance of the measured
properties of measurement systems, either individually, or
values is approximately 1.119 or 11.9 % larger than the true
some relevant subset of properties taken collectively. Much of
variance. The standard deviation is, therefore, 1.058 or 5.8 %
this activity uses well known methods of classical statistics,
larger.
most notably experimental design techniques. In classical
5.4.7.4 This illustrates the important role that resolution
statistics, the term variance is used to denote variation in a set
plays in measurement in general and an MSA study in
of numbers. It is the square of the standard deviation. One of
particular. There is a subtle interaction between the degree of
the primary goals in conducting an MSAstudy is to assess the
resolution and more general repeatability and other measure-
several variance components that may be at play. Each factor
ment effects. In extreme cases of poor resolution, an MSA
will have its own variance component that contributes to the
study may not be able to pick up a repeatability effect (all
overall variation. Components of variance for independent
objects measured yield the same value). For an ideal system,
variables are additive. For example, suppose y is the result of
for varying degrees of finite resolution as described in 5.4.7,
a measurement in which three independent factors are at play.
there will be a component of variance as a result of resolution
Suppose that the three independent factors are x , x , and x .A
alone. For positive integer value, k, when the smallest mea-
1 2 3
simple model for the linear sum of the three components is y =
surementunitforadeviceis1/kthofthe6σtruenaturalprocess
x + x + x . The variance of the overall sum, y, given the
range, the standard deviation as a result of the resolution effect
1 2 3
variances of the components is:
may be determined theoretically (assuming a normal distribu-
2 2 2 2
tion). Table 1 shows the effect for selected values of k.
σ 5 σ 1σ 1σ (1)
y 1 2 3
5.5.1 We say that each variance on the right is a component
of the overall variance on the left. This model is theoretical; in
practice, we do not know the true variances and have to
estimate their values from data.
5.5.2 Statistical methods allow one to estimate the several
variancecomponentsinMSA.Sometimestheanalystmayonly
be interested in one of the components, for example, repeat-
ability. In other cases, it may be two or more components that
may be of interest. Depending on how the MSA study is
designed, the variance components may be estimable free and
clear of each other or combined as a composite measure.
Several widely used basic models and associated statistical
techniques are discussed in Section 6.
6. Basic MSA Methods
6.1 Simple Repeatability—Simple repeatability may be
FIG. 3 Finite Resolution Property of a Measurement System
evaluated using at least two measurements of each of several
where Four “Graduations Fit within the Natural 6σ Process
Spread” objects by a single appraiser under identical conditions. The
´1
E2782 − 17 (2022)
simplest such experiment is to use a number of distinct objects, of variance (ANOVA) may be used for the more than two
say n, and two measurements of each object. Let y and y be measurements per object case. The ANOVA technique also
i1 i2
the two measurements of object i. Each measurement is allows for differences from one object to another in the number
contaminated with a repeatability error component, e. This of times each object is measured (see 6.4 for details).
model may be written as:
6.2 Use of the Range Control Chart in Evaluating
y 5 x 1e (2)
Repeatability—Therangecontrolchartmaybeusedtoevaluate
ij i ij
consistency of the measurement system and resolution issues.
6.1.1 In this model, the y values are the observed measure-
ij
In addition, the control chart gives an alternative measure of
ments of object, i, measurement, j; the x values are considered
i
repeatability that, under perfect stability and consistency
the “true” or reference values of the objects being measured;
conditions, should be very close to Eq 4. Suppose there are n
and e is the repeatability error associated with object, i, and
ij
objects to be measured. For each pair of repeated
measurement, j. The difference, d, between two measurements
i
measurements, calculate the range as:
of the same object may be written as:
R 5 y 2 y (7)
i ? i1 i2?
d 5 y 2 y 5 x 1e 2 x 2 e 5 e 2 e (3)
i i1 i2 i i1 i i2 i1 i2
6.2.1 The absolute value bars simply indicate that we are
6.1.2 If the error terms can be considered normally
lookingattheabsolutedifferencebetweenmeasurementsorthe
distributed, then the paired differences, the d’s, will possess a
range in each pair. The average range is:
normaldistribution.Generally,therepeatabilityerrorterm, e,is
n
assumed to have a mean of 0 and some unknown variance σ .
R
i
(
This is the repeatability variance. Under the model assump- i51
¯
R 5 (8)
n
tions and further assuming that the errors are uncorrelated; the
variable, d, will be normally distributed with mean 0 and
i
6.2.2 The range estimate of the standard deviation of
2 2
variance 2σ . The variance σ may be analyzed using standard
repeatability is:
statistical theory as follows. The estimate of σ is formed as:
¯
R
n
σˆ 5 (9)
d d
i 2
(
i51
σˆ 5 (4)
2n 6.2.3 A short table of the constant d appears in Appendix
X6 or see Practice E2587. The constant d converts the
The square root of quantity Eq 4 is the estimate of the
average range into an unbiased estimate of σ. It is a function of
repeatability standard deviation.
the subgroup size, k. Here, k = 2. The range control chart is
6.1.3 Withthepreviousassumptions,thesumofthesquared
constructed as a series of the n sample range values with center
2 2
deviations, the d terms, divided by 2σ will have a chi-square
line equal to the average range and control limits (upper and
distribution with n degrees of freedom. From this fact, a
lower control limits) calculated from the formulas:
confidence interval for σ may be constructed.
¯
UCL 5 D R (10)
n
R 4
d
( i
¯
i51 LCL 5 D R (11)
2 R 3
χ 5 Chi 2 square withndf (5)
2σ
6.2.4 The values D and D may be found in Appendix X6,
3 4
6.1.4 It may be important to check that the mean of the
PracticeE2587,oranytextonstatisticalprocesscontrol.When
variable d is zero. For this purpose, we can use a classical the subgroup size is less than seven, the constant D will be 0.
confidence interval construction for the true mean of the
A sequence of such values, exhibiting good statistical control,
differences. The form of the confidence interval is: will give every indication of a random sequence of observa-
tions with all points falling within the control limits. This kind
tS
d
¯
d6 (6)
of chart is always done first when performing MSAstudies on
=n
repeatability. The principle signs of inconsistency are points
outside of the control limits or other nonrandom patterns such
6.1.5 Here, t is selected from the Student’s t distribution,
as runs above (below) the center line or trends of increasing
using a two-sided 100(1 – α)% confidence level and degrees of
(decreasing) direction. Such signs indicate inconsistency and
freedom n–1,and S istheordinarysamplestandarddeviation
d
out-of-control conditions.
of the differences. If the interval includes 0, then the assump-
6.2.5 When zeros appear on a range control chart, this is a
tion of a mean equal to 0 cannot be refuted at significance
sign of either a resolution problem or that the repeatability
level, α. The normal distribution assumption may be checked
error is small enough to be considered negligible. In any event,
using the several values of d, and a normal probability plotting
i
technique (see Practice E2586).
6.1.6 The paired comparison (two measurements per each
object) scenario is convenient and very common in practice;
Formally,theconstant d isequaltothemathematicalexpectationofthesample
range divided by σ, when sampling from a normal distribution. The value, d,isa
however, it is also possible to modify the methodology using 2
*
function only of the subgroup size, k. Some writers prefer to use the constant d .
more than two measurements per object measured. When this
Dividing the average range (Eq 8) by this constant and squaring makes the resulting
2 *
approach is used, the formulas for the resulting estimates and
number an unbiased estimate of σ . The value, d , is a function of the subgroup
confidence interval formulation will be different. An analysis size, k, as well as the number of subgroups, n. See Appendix X6 for tables.
´1
E2782 − 17 (2022)
it is still resolution that is at issue. Poor resolution in the fall outside of the control limits. Points falling within the
presence of modest repeatability error may yield identical controllimitsaresaidtobeindistinguishablefromoneanother.
values in repeating a measurement. Too many zeros appearing
The region between the control limits is a kind of noise band
in the range chart will reduce the estimate of the repeatability
(noise being a repeatability error) and object averages are like
standard deviation and perhaps underestimate its real effect.
the “signals” of real object-to-object variation. A fair bench-
One way to counteract this problem is to replace zeros with q
mark for this kind of chart is to have at least 50 % of the
as:
average points fall outside of the control limits. Anything less
indicates that repeatability variation is dominant over object
~ = !
q 5 ud / 2 3 (12)
variation. This method is more powerful when the subgroup
6.2.6 The quantity, u, is defined as the smallest unit of sample size is greater than two. Also, if the objects were
resolution the measurement device is capable of discriminating
handpicked and not random samples from a process, interpre-
and d is as previously defined. For example, if one uses a ruler
tation of this type of chart may be incorrect.
graduated in eighths of an inch, then u = 0.125. The reason for
6.4 Repeatability Using More than Two Observations per
this is that the standard deviation of a uniform distribution
Object Measured—When more than two measurements can be
~ = !
between0and uis u/ 2 3 . Postmultiplicationby d givesan
made for each of several sample objects, the analysis of
estimate of the expected range in a sample of size, k (the
variance (ANOVA) with random effects may be used. It is best
subgroup size). An alternative method is to estimate the
to use a fixed number of repeat measurements per object,
expected range based on the subgroup sample size, k. For this
althoughthismethodstillworkswhenthesubgroupsizevaries.
method, we would replace a zero range with u(k – 1)/(k + 1),
Whenever possible, measurements should be made in random-
which is precisely the expected range in a sample of size, k,
ized order. If there are n objects to be measured and m
from a uniform distribution between 0 and u. Still, another
measurements per object, randomize the numbers 1, 2, … mn.
method is to replace zero ranges with simulated ranges from a
The randomized numbers should then form the basis of the
uniform distribution on the interval [0,u]. In each method,
sequence for how the measurements would be obtained. Upon
these pseudo ranges replace zeros on the range chart.
completion,therewillbe nsetsof mrepeatedmeasurements, m
6.3 Use of the Average Control Chart in Evaluating
for each of the n objects measured. Let y be the jth repeated
ij
Repeatability—The averages are formed from each pair of
measurement of object, i, where i varies from 1 to n and j from
repeated measurements (each pair is a subgroup).These can be
1to m. The quantity y¯ represents the average of the measure-
i.
plotted in time order using a control chart for averages. The
th
ments from the i object (the “dot” notation in the subscript
=
center line for such a chart will be the overall average, x,ofthe
signifies that we are averaging over the second index, j). The
several subgroup averages. Note that the subgroup size in this
following statistic is an unbiased estimate of the repeatability
case is two; but this method is general and any subgroup size
variance σ .
maybeused.Therangechart,havingalreadybeenconstructed,
n m
is used in constructing the control limits for the average chart.
~y 2 yH !
( ( ij i.
SSE
The control limits are calculated from the following classical i51 j51
σˆ 5 5 (17)
formulas based on the subgroup average range: n~m 2 1! n~m 2 1!
¯
UCL 5 x%1A R (13)
6.4.1 Table X1.1 in Appendix X1 contains a complete
xH 2
ANOVA table for this type of model. The quantity SSE/σ
¯
LCL 5 x%1A R (14)
Hx 2
possesses a chi-square distribution with n(m – 1) degrees of
freedom, and from this fact, a confidence interval for the
6.3.1 The constant, A , is defined as:
repeatability standard deviation, σ, can be obtained. In the case
where there are varying numbers of repeat measurements for
A 5 (15)
d =n
eachoftheseveralobjectsmeasured,Eq17wouldbemodified.
Suppose for n objects, there are m measurements for the ith
i
6.3.2 For n = 2, the constant A is 1.88. The overall average
=
object. The estimate of repeatability variance becomes:
x is defined as:
n m
i
n
~y 2 yH !
( ( ij i.
xH
i SSE
( i51 j51
i51
σˆ 5 5 (18)
n n
x% 5 (16)
n
~m 2 1! ~m 2 1!
( i ( i
i51 i51
6.3.3 For subgroup sizes other than two, tables of the
6.4.2 Again, the quantity SSE/σ possesses a chi-square
constant, A , can be found in Appendix X6 or Practice E2587.
distribution with degrees of freedom as shown in the
The stability of the system may be assessed from the control
denominator, and from this fact, a confidence interval for the
chart for averages. Individual plotted points should indicate a
repeatability standard deviation, σ, can be obtained. Eq 17 and
random pattern about the center line. The control limits for the
18 come from an ANOVA approach to repeatability analysis.
average chart are constructed from the range chart and the
range chart is measuring repeatability variation only; therefore, Table X1.1 in Appendix X1 contains the details for this model
in which n objects are measured m times each by a single
if the object to object variation is much greater than the
repeatability variation, most points on the average chart will appraiser.
´1
E2782 − 17 (2022)
6.5 Repeatability Using Known or Standard Reference y 5 x 1u 1e (21)
ijk i j ijk
Values—An MSA study may be conducted using several
6.6.2.1 The quantity e continues to play the role of the
ijk
known or standard objects. Let y be the measurement of an
repeatability error term which is assumed to have mean 0 and
object whose standard value is x. The model is:
variance σ . Quantity x represents the standard or “true” value
i
y 5 x 1ε (19)
i i i of the object being measured and quantity u is a random
j
reproducibility term associated with appraiser j. This last
6.5.1 The ε term is assumed to have mean 0 and some
quantity is assumed to come from a distribution having mean
unknown variance σ representing repeatability. The goal is to
0 and some variance θ . If the objects being measured can be
estimate σ. If we have n objects to measure, then form the
considered a random sample from a population of objects, then
paired differences d = y – x. The d values are equivalent to
i i i i
the x are random variables with some mean, the true popula-
the ε values. In this model, we do not have to assume a i
i
tion mean, and variance v .
distribution for the variable, x. We only need one consistent
6.6.2.2 Eq 21 assumes no part-operator interaction term,
distribution for the paired differences. In theory, this type of
which might sometimes be a reasonable assumption in prac-
study could be carried out using a single object measured m
tice. An interaction between part and operator means that for
times (see 6.5.4).
increasing (decreasing) values of some objects, some apprais-
6.5.2 The following quantity is used as the point estimate of
ers follow an opposite trend—that is, they measure smaller
the repeatability variance:
(larger) values. If interaction is to be considered, an additional
n
term, w , would have to be included in Eq 21. The model
d
i ij
(
i51
including interaction is:
σˆ 5 (20)
n
y 5 x 1u 1w 1e (22)
ijk i j ij ijk
6.5.3 If the d terms can be considered normally distributed,
i
6.6.3 Variance Components—For Eq 21, assuming indepen-
the sum of squared differences divided by σ will possess a
dence of the three terms, the variance of the measurements is
chi-square distribution with n degrees of freedom, and from
simply:
this fact, a confidence interval for σ can be constructed.
6.5.4 Ifseveralobjectsareeachmeasuredavariablenumber 2 2 2
var y 5 ν 1θ 1σ (23)
~ !
ijk
of times, say k times for object i, the formulation of Eq 20 is
i
6.6.3.1 For Eq 22, including the interaction term, the vari-
the same. Let N = the total number of paired observations
ance of the measurements becomes:
including all repeated measurements of all n objects. The
2 2 2 2
estimate of repeatability variance is Eq 20 with n replaced by
var~y ! 5 ν 1θ 1α 1σ (24)
ijk
N. The sum of all N squared differences possesses a chi-square
2 2 2 2
6.6.3.2 Each of ν , θ , α , and σ are the components of the
distribution with N degrees of freedom.
overall variance with α playing the role of interaction. One
6.6 Reproducibility—Appraiser Component of Variance—In
principle objective of a measurement systems analysis is to
using a measurement system, it is always possible for different
obtain estimates of these variance components. The combined
people to get different results when identical objects are 2 2
variance components, θ and σ , are often referred to as the
measured in the same manner. Two sources of variation are
gage R&R variance. Many software packages will perform this
responsible for the difference among appraisers: (1) simple
type of analysis.
repeatability error will cause differences among appraisers and
6.6.3.3 When several appraisers each measure a group of
(2) overall differences among appraisers may be due to
objects once only (no repeats), it is still possible to estimate
individual biases on the average. The second component is the
R&R but not interaction. Appendix X2 and Appendix X3 give
subject of reproducibility. This is shown in Fig. 2. The
complete ANOVA tables for Eq 21 and 22, respectively.
difference between means of the two appraisers in Fig. 2 is the
6.7 Bias—Reproducibility variance may be viewed as com-
effect of reproducibility. In practice, it is the difference in
ingfromadistributionoftheappraiser’spersonalmeasurement
sample means of the two groups as measured by differing
bias. In addition, there may be a global bias present in the
appraisers that estimates reproducibility.
measurement system that is shared equally by all appraisers.
6.6.1 Reproducibility can be considered as a random bias
Bias is the difference between the mean of the overall distri-
component assigned to every appraiser. A bias simply means
bution of all measurements by all appraisers and a “true” or
thattheappraisertendstomeasureeveryobjecteitherhigheror
reference average of all objects.Whereas reproducibility refers
lower on average than the “true” measure of the object.We can
to a distribution of appraiser averages, bias refers to a differ-
think of appraisers coming from a population of all such
ence between the average of a set of measurements and a
appraisers, each with a unique and fixed bias. The distribution
known or reference value. The measurement distribution may
of these biases is assumed to be normal with mean 0 and some
itself be composed of measurements from differing appraisers
unknown variance θ . The parameter, θ, is the reproducibility
or it may be a single appraiser that is being evaluated. Thus, it
standard deviation. We may think of the random variable u as
is important to know what the objective is in evaluating bias.
denoting the random bias (reproducibility) component. When
several specific appraisers are used in a measurement systems 6.7.1 Bias may also vary as a function of the reference
study, we are effectively picking several random values of u. value. For example, bias may be larger for “larger objects” and
6.6.2 For several appraisers, the model for the measurement smaller for “smaller objects.” This concept is referred to as
of the ith object by appraiser j at the kth repeat is: linearity. See 6.8 for further details on this concept.
´1
E2782 − 17 (2022)
6.7.2 For single appraiser, and a single object, bias is linearity effect if the bias changes in linear manner over the
evaluated as the difference between the average of several operational range of a set of reference standards. Linearity may
measurements and the known reference value. This is Eq 25, be measured using a linear regression analysis of
where x is the known reference value and b is the observed measurements, y, on reference values, x. The measure of
empirical bias. linearity is the slope of the least squares best fit line or some
function thereof. A simple model for linearity is:
b 5 y¯ 2 x (25)
y 5 mx 1B1ε (31)
ij i ij
6.7.2.1 Eq 25 represents a point estimate for the bias. It
might or might not be significant, because quantity b is also
6.8.1 The concept of linearity is often applied in calibration
contaminated with repeatability error. We can determine if the problems. In such cases, a measurement y is related to a set of
observed bias is significant by constructing a confidence
standard values (x) according to Eq 31. One objective is to
interval for the real bias B. If the confidence interval includes determine the range for y that makes the probability of
0 as a plausible value, then we may conclude that a significant
conforming x values very high, say 90 %.
non-zero bias has not been detected. To understand how the
6.8.2 The y term is the jth measurement of object x, and
ij i
confidence interval is constructed, we shall consider the model
the ε term is the random repeatability error term normally
ij
for this scenario and its assumptions.
distributed with mean 0 and variance σ associated with the jth
6.7.3 The model for simple bias is:
measurement of object, i.The parameter, B, represents a global
bias; the parameter m represents linearity. When m = 1 and
y 5 B1x1ε (26)
i i
B = 0, the measurement model reverts to Eq 2. The system is
6.7.3.1 The value x is the fixed known reference value,
then unbiased and perfectly linear. When m ≠ 1, then the
quantity B is the unknown bias, the ε terms are random
system possesses a linearity effect.
repeatability errors assumed to be normally distributed with
6.8.3 The linear regression proceeds with a selection of
mean0andunknownvariance σ ,andthe ytermsaretheactual
several reference objects (x) to be used for measurement
measurements. A series of n measurements will have an
several times each. It is important that the reference objects
average given by:
represent the range of possible objects that the system may see
y¯. 5 B1x1ε¯. (27)
in practice.Alinear regression analysis of y on x is then carried
out on the data. Typically, when a simple regression analysis is
6.7.3.2 The empirical bias b is therefore equal to:
implemented using a software package, the results will include
b 5 y¯.2x 5 B6ε¯. (28)
point estimates for the model parameters (m and B). Confi-
dence intervals may be constructed to determine if B ≠0orif
6.7.3.3 Quantity b therefore possesses a normal distribution
m ≠1.Theestimateoftherepeatabilityerrorstandarddeviation
with mean B and variance σ / n. If the repeatability variance
σ is also output from any good statistics software package
were known, then we could create a confidence interval for B
when a simple linear regression analysis is performed.
in the usual way using critical values from the standard normal
6.8.3.1 For a set of n (x,y) data pairs, the regression analysis
distribution. Typically, σ is not known and must be estimated
results in a pair of estimates calculated using Eq 32 and Eq 33.
from the sample data. The estimate of the σ under the
Let S and S be the ordinary standard deviations of the x and
assumptions of this model is: x y
y values, respectively; let r be the Pearson correlation coeffi-
n
cient between x and y; and let x¯ and y¯ be the sample averages
~y 2 yH!
( i
i51
for the x and y values. The point estimates of m and B are:
σˆ 5 (29)
n 2 1
rS
y
mˆ 5 (32)
6.7.3.4 A confidence interval for the bias B may then be
S
x
constructed using Student’s t distribution with n – 1 degrees of
ˆ
B 5 y¯ 2 mˆx¯ (33)
freedom. The confidence interval is:
t S 6.8.3.2 Confidence intervals for the model parameters may
α/2 y
y¯.2x6 (30)
be constructed from well-known formulas. See, for example,
=n
Ref (2). The predicted value of y given x is calculated as
6.7.3.5 Here, t is a positive constant chosen using confi- ˆ
yˆ =mˆ x +B. The estimate of the standard error (repeatability
α/2
i
i
dence 1 – α from Student’s t distribution with n – 1 degrees of
standard deviation) is:
freedom such that P(-t ≤ t ≤ t )=1– α. If the confidence
α/2 α/2
n
interval includes 0 as a plausible value, then we cannot
~y 2 yˆ !
( i i
conclude that the bias, B, is non-zero. Note that this does not i51
σˆ55 (34)
!
n 2 2
mean that the bias is 0; it simply indicates that we have not
detected a significant non-zero bias. This may mean that our
6.8.4 In some cases, the bias term may be known to be 0 at
sample size was not adequate to detect the bias.
the outset, and linearity is the main concern. The model then
6.8 Linearity—A closely related concept to bias is linearity.
becomes:
Bias may vary as a function of the reference value. For
y 5 mx 1ε (35)
ij i ij
example,biasmaybelargerfor“largerobjects”andsmallerfor
“smaller objects.” A measurement process has a significant 6.8.4.1 The least squares estimate of the parameter, m, is:
´1
E2782 − 17 (2022)
n
population of all potentials appraisers. There are p appraisers.
x y
( i i
Often, there may only be two or t
...