ASTM E2919-13
(Test Method)Standard Test Method for Evaluating the Performance of Systems that Measure Static, Six Degrees of Freedom (6DOF), Pose
Standard Test Method for Evaluating the Performance of Systems that Measure Static, Six Degrees of Freedom (6DOF), Pose
SIGNIFICANCE AND USE
5.1 Pose measurement systems are used in a wide range of fields including manufacturing, material handling, construction, medicine, and aerospace. The use of pose measurement systems could, for example, replace the need to fix the poses of objects of interest by mechanical means.
5.2 Potential users have difficulty comparing pose measurement systems because of the lack of standard performance specifications and test methods, and must rely on the specifications of a vendor regarding the system’s performance, capabilities, and suitability for a particular application. This standard makes it possible for a user to assess and compare the performance of candidate pose measurement systems, and allows the user to determine if the measured performance results are within the vendor’s claimed specifications with regard to the user’s application. This standard also facilitates the improvement of pose measurement systems by providing a common set of metrics to evaluate system performance.
5.3 The intent of this test method is to allow a user to determine the performance of a vendor’s system under conditions specific to the user’s application, and to determine whether the system still performs in accordance with the vendor’s specifications under those conditions. The intention of this test method is not to validate a vendor’s claims; although, under specific situations, this test method may be adapted for this purpose.
SCOPE
1.1 Purpose—In this test method, metrics and procedures for collecting and analyzing data to determine the performance of a pose measurement system in computing the pose (position and orientation) of a rigid object are provided.
1.2 This test method applies to the situation in which both the object and the pose measurement system are static with respect to each other when measurements are performed. Vendors may use this test method to establish the performance limits for their six degrees of freedom (6DOF) pose measurement systems. The vendor may use the procedures described in Section 9.2 to generate the test statistics, then apply an appropriate margin or scaling factor as desired to generate the performance specifications. This test method also provides a uniform way to report the relative or absolute pose measurement capability of the system, or both, making it possible to compare the performance of different systems.
1.3 Test Location—The methodology defined in this test method shall be performed in a facility in which the environmental conditions are within the pose measurement system’s rated conditions and meet the user’s requirements.
1.4 Units—The values stated in SI units are to be regarded as the standard. No other units of measurement are included in this standard.
1.5 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility of the user of this standard to establish appropriate safety and health practices and determine the applicability of regulatory limitations prior to use.
General Information
Relations
Standards Content (Sample)
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Contact ASTM International (www.astm.org) for the latest information
Designation: E2919 − 13
StandardTest Method for
Evaluating the Performance of Systems that Measure Static,
Six Degrees of Freedom (6DOF), Pose
This standard is issued under the fixed designation E2919; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope E2544Terminology for Three-Dimensional (3D) Imaging
Systems
1.1 Purpose—In this test method, metrics and procedures
2.2 ASME Standard:
forcollectingandanalyzingdatatodeterminetheperformance
ASME B89.4.19Performance Evaluation of Laser-Based
ofaposemeasurementsystemincomputingthepose(position
Spherical Coordinate Measurement Systems
and orientation) of a rigid object are provided.
2.3 ISO/IEC Standards:
1.2 This test method applies to the situation in which both
JCGM 200:2012International Vocabulary of Metrology—
the object and the pose measurement system are static with
BasicandGeneralConceptsandAssociatedTerms(VIM),
respect to each other when measurements are performed.
3rd edition
Vendors may use this test method to establish the performance
JCGM 100:2008Evaluation of Measurement Data—Guide
limits for their six degrees of freedom (6DOF) pose measure-
to the Expression of Uncertainty in Measurement (GUM)
mentsystems.Thevendormayusetheproceduresdescribedin
IEC 60050-300:2001 International Electrotechnical
Section 9.2 to generate the test statistics, then apply an
Vocabulary—Electrical and Electronic Measurements and
appropriate margin or scaling factor as desired to generate the
Measuring Instruments
performance specifications. This test method also provides a
uniform way to report the relative or absolute pose measure- 3. Terminology
ment capability of the system, or both, making it possible to
3.1 Definitions from Other Standards:
compare the performance of different systems.
3.1.1 calibration, n—operation that, under specified
1.3 Test Location—The methodology defined in this test
conditions, in a first step, establishes a relation between the
method shall be performed in a facility in which the environ- quantity values with measurement uncertainties provided by
mental conditions are within the pose measurement system’s
measurement standards and corresponding indications with
rated conditions and meet the user’s requirements.
associated measurement uncertainties and, in a second step,
uses this information to establish a relation for obtaining a
1.4 Units—The values stated in SI units are to be regarded
measurement result from an indication. JCGM 200:2012
as the standard. No other units of measurement are included in
3.1.1.1 Discussion—
this standard.
(1)Acalibration may be expressed by a statement, calibra-
1.5 This standard does not purport to address all of the
tion function, calibration diagram, calibration curve, or cali-
safety concerns, if any, associated with its use. It is the
bration table. In some cases, it may consist of an additive or
responsibility of the user of this standard to establish appro-
multiplicative correction of the indication with associated
priate safety and health practices and determine the applica-
measurement uncertainty.
bility of regulatory limitations prior to use.
(2)Calibration should not be confused with either adjust-
ment of a measuring system, often mistakenly called “self-
2. Referenced Documents
calibration,” or verification of calibration.
2.1 ASTM Standards:
(3)Often, the first step alone in 3.1.1 is perceived as being
E456Terminology Relating to Quality and Statistics
calibration.
3.1.2 maximum permissible measurement error, maximum
This test method is under the jurisdiction of ASTM Committee E57 on 3D
permissible error, and limit of error, n—extreme value of
Imaging Systems and is the direct responsibility of Subcommittee E57.02 on Test
Methods.
Current edition approved May 1, 2013. Published June 2013. DOI: 10.1520/
E2919-13. Available from American Society of Mechanical Engineers (ASME), ASME
For referenced ASTM standards, visit the ASTM website, www.astm.org, or International Headquarters, Three Park Ave., New York, NY 10016-5990, http://
contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM www.asme.org.
Standards volume information, refer to the standard’s Document Summary page on Available fromAmerican National Standards Institute (ANSI), 25 W. 43rd St.,
the ASTM website. 4th Floor, New York, NY 10036, http://www.ansi.org.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E2919 − 13
measurement error, with respect to a known reference quantity that may be used as a basis for making a decision concerning
value, permitted by specifications or regulations for a given the larger collection. E456
measurement, measuring instrument, or measuring system.
3.1.6 measurement uncertainty, uncertainty of
JCGM 200:2012
measurement, and uncertainty, n—non-negative parameter
3.1.2.1 Discussion—
characterizing the dispersion of the quantity values being
(1)Usually, the terms “maximum permissible errors” or
attributed to a measurand based on the information used.
“limits of error” are used when there are two extreme values.
JCGM 200:2012
(2)The term “tolerance” should not be used to designate
3.1.6.1 Discussion—
“maximum permissible error.”
(1)Measurement uncertainty includes components arising
3.1.3 measurand, n—quantity intended to be measured.
from systematic effects, such as components associated with
JCGM 200:2012
corrections and the assigned quantity values of measurement
3.1.3.1 Discussion—
standards, as well as the definitional uncertainty. Sometimes
(1)Thespecificationofameasurandrequiresknowledgeof
estimated systematic effects are not corrected for but, instead,
the kind of quantity; description of the state of the
associated measurement uncertainty components are incorpo-
phenomenon, body, or substance carrying the quantity, includ-
rated.
ing any relevant component; and the chemical entities in-
(2)The parameter may be, for example, a standard devia-
volved.
tion called standard measurement uncertainty (or a specified
(2) In the second edition of the VIM and IEC 60050-300,
multiple of it) or the half width of an interval, having a stated
the measurand is defined as the “quantity subject to mea-
coverage probability.
surement.”
(3)Measurement uncertainty comprises, in general, many
(3)Themeasurement,includingthemeasuringsystemand
components. Some of these may be evaluated by Type A
the conditions under which the measurement is carried out,
evaluation of measurement uncertainty from the statistical
might change the phenomenon, body, or substance such that
distributionofthequantityvaluesfromseriesofmeasurements
the quantity being measured may differ from the measurand
and can be characterized by standard deviations. The other
as defined. In this case, adequate correction is necessary.
components, which may be evaluated by Type B evaluation of
(a) Example 1—The potential difference between the ter-
measurement uncertainty, can also be characterized by stan-
minals of a battery may decrease when using a voltmeter with
dard deviations evaluated from probability density functions
asignificantinternalconductancetoperformthemeasurement.
based on experience or other information.
Theopen-circuitpotentialdifferencecanbecalculatedfromthe
(4)In general, for a given set of information, it is under-
internal resistances of the battery and the voltmeter.
stood that the measurement uncertainty is associated with a
(b) Example 2—The length of a steel rod in equilibrium
stated quantity value attributed to the measurand. A modifica-
with the ambient Celsius temperature of 23°C will be different
tion of this value results in a modification of the associated
from the length at the specified temperature of 20°C, which is
uncertainty.
the measurand. In this case, a correction is necessary.
(4) In chemistry, “analyte,” or the name of a substance or
3.1.7 precision, n—closeness of agreement between inde-
compound, are terms sometimes used for “measurand.”This
pendent test results obtained under stipulated conditions. E456
usage is erroneous because these terms do not refer to
3.1.7.1 Discussion—
quantities.
(1)Precision depends on random errors and does not relate
3.1.4 measurement error, error of measurement, and error,
to the true value or the specified value.
n—measured quantity value minus a reference quantity value.
(2)The measure of precision is usually expressed in terms
JCGM 200:2012
ofimprecisionandcomputedasastandarddeviationofthetest
3.1.4.1 Discussion—
results. Less precision is reflected by a larger standard devia-
(1) The concept of “measurement error” can be used both:
tion.
(a)When there is a single reference quantity value to refer
(3)“Independent test results” means results obtained in a
to, which occurs if a calibration is made by means of a
manner not influenced by any previous result on the same or
measurement standard with a measured quantity value having
similar test object. Quantitative measures of precision depend
a negligible measurement uncertainty or if a conventional
critically on the stipulated conditions. Repeatability and repro-
quantityvalueisgiven,inwhichcasethemeasurementerroris
ducibility conditions are particular sets of extreme stipulated
known, and
conditions.
(b)If a measurand is supposed to be represented by a
3.1.8 rated conditions, n—manufacturer-specified limits on
unique true quantity value or a set of true quantity values of
environmental, utility, and other conditions within which the
negligible range, in which case the measurement error is not
manufacturer’s performance specifications are guaranteed at
known.
the time of installation of the instrument. ASME B89.4.19
(2) Measurement error should not be confused with
production error or mistake.
3.1.9 reference quantity value and reference value,
n—quantityvalueusedasabasisforcomparisonwithvaluesof
3.1.5 measurement sample and sample, n—group of obser-
quantities of the same kind. JCGM 200:2012
vations or test results, taken from a larger collection of
observations or test results, that serves to provide information 3.1.9.1 Discussion—
E2919 − 13
(1)Areference quantity value can be a true quantity value (1)Intheerrorapproachtodescribingmeasurement,atrue
ofameasurand,inwhichcaseitisunknown,oraconventional quantity value is considered unique and, in practice, unknow-
quantity value, in which case it is known. able. The uncertainty approach is to recognize that, owing to
(2)A reference quantity value with associated measure- the inherently incomplete amount of detail in the definition of
ment uncertainty is usually provided with reference to: a quantity, there is not a single true quantity value but rather a
(a)Amaterial,forexample,acertifiedreferencematerial; set of true quantity values consistent with the definition.
(b)A device, for example, a stabilized laser; However, this set of values is, in principle and practice,
(c)A reference measurement procedure; and unknowable. Other approaches dispense altogether with the
(d)A comparison of measurement standards. concept of true quantity value and rely on the concept of
metrological compatibility of measurement results for assess-
3.1.10 registration, n—process of determining and applying
ing their validity.
to two or more datasets the transformations that locate each
(2)In the special case of a fundamental constant, the
dataset in a common coordinate system so that the datasets are
quantity is considered to have a single true quantity value.
aligned relative to each other. E2544
(3)When the definitional uncertainty associated with the
3.1.10.1 Discussion—
measurandisconsideredtobenegligiblecomparedtotheother
(1)A three-dimensional (3D) imaging system generally
components of the measurement uncertainty, the measurand
collectsmeasurementsinitslocalcoordinatesystem.Whenthe
may be considered to have an “essentially unique” true
samesceneorobjectismeasuredfrommorethanoneposition,
quantity value. This is the approach taken by JCGM 100 and
it is necessary to transform the data so that the datasets from
associateddocumentsinwhichtheword“true”isconsideredto
each position have a common coordinate system.
be redundant.
(2)Sometimestheregistrationprocessisperformedontwo
3.2 Definitions of Terms Specific to This Standard:
or more datasets that do not have regions in common. For
example, when several buildings are measured independently, 3.2.1 absolute pose, n—pose of an object in the coordinate
each dataset may be registered to a global coordinate system frame of the system under test.
instead of to each other.
3.2.2 degree of freedom, DOF, n—any of the minimum
(3)In the context of this definition, a dataset may be a
number of translation or rotation components required to
mathematicalrepresentationofsurfacesormayconsistofaset
specify completely the pose of a rigid body.
ofcoordinates,forexample,apointcloud,a3Dimage,control
3.2.2.1 Discussion—
points, survey points, or reference points from a computer-
(1)In a 3D space, a rigid object can have at most 6DOFs,
aideddrafted(CAD)model.Additionally,oneofthedatasetsin
three translation and three rotation.
a registration may be a global coordinate system (as in
(2)The term “degree of freedom” is also used with regard
3.1.10.1(2)).
to statistical testing. It will be clear from the context in which
(4)The process of determining the transformation often
it is used whether the term relates to a statistical test or the
involvestheminimizationofanerrorfunction,suchasthesum
rotation/translation aspect of the object.
ofthesquareddistancesbetweenfeatures(forexample,points,
3.2.3 pose, n—a 6DOF vector whose components represent
lines, curves, and surfaces) in two datasets.
the position and orientation of a rigid object with respect to a
(5)In most cases, the transformations determined from a
coordinate frame.
registrationprocessarerigidbodytransformations.Thismeans
that the distances between points within a dataset do not
3.2.4 pose measurement system, n—a 3-D imaging system
changeafterapplyingthetransformations,thatis,rotationsand
that measures the pose of an object.
translations.
3.2.4.1 Discussion—This system can consist of both hard-
(6)In some cases, the transformations determined from a
ware and software.
registration process are nonrigid body transformations. This
3.2.5 reference system, n—a measurement instrument or
means that the transformation includes a deformation of the
system used to generate a reference value or quantity.
dataset.Onepurposeofthistypeofregistrat
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.