ASTM F3107-14(2023)
(Test Method)Standard Test Method for Measuring Accuracy After Mechanical Disturbances on Reference Frames of Computer Assisted Surgery Systems
Standard Test Method for Measuring Accuracy After Mechanical Disturbances on Reference Frames of Computer Assisted Surgery Systems
SIGNIFICANCE AND USE
5.1 The purpose of this practice is to provide data that can be used for comparison and evaluation of the accuracy of different CAS systems.
5.2 The use of CAS systems and robotic tracking systems is becoming increasingly common and requires a degree of trust by the user that the data provided by the system meets necessary accuracy requirements. In order to evaluate the potential use of these systems, and to make informed decisions about suitability of a system for a given procedure, objective performance data of such systems are necessary. While the end user will ultimately want to know the accuracy parameters of a system under clinical application, the first step must be to characterize the digitization accuracy of the tracking subsystem in a controlled environment under controlled conditions.
5.3 In order to make comparisons within and between systems, a standardized way of measuring and reporting point accuracy is needed. Parameters such as coordinate system, units of measure, terminology, and operational conditions must be standardized.
SCOPE
1.1 This standard will measure the effects on the accuracy of computer assisted surgery (CAS) systems of the environmental influences caused by equipment utilized for bone preparation during the intended clinical application for the system. The environmental vibration effect covered in this standard will include mechanical vibration from: cutting saw (sagittal or reciprocating), burrs, drills, and impact loading. The change in accuracy from detaching and re-attaching or disturbing a restrained connection that does not by design require repeating the registration process of a reference base will also be measured.
1.2 It should be noted that one system may need to undergo multiple iterations (one for each clinical application) of this standard to document its accuracy during different clinical applications since each procedure may have different exposure to outside forces given the surgical procedure variability from one procedure to the next.
1.3 All units of measure will be reported as millimeters for this standard.
1.4 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility of the user of this standard to establish appropriate safety, health, and environmental practices and determine the applicability of regulatory limitations prior to use.
1.5 This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
General Information
Relations
Standards Content (Sample)
This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation: F3107 − 14 (Reapproved 2023)
Standard Test Method for
Measuring Accuracy After Mechanical Disturbances on
Reference Frames of Computer Assisted Surgery Systems
This standard is issued under the fixed designation F3107; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope 2. Referenced Documents
2.1 ASTM Standards:
1.1 This standard will measure the effects on the accuracy of
E456 Terminology Relating to Quality and Statistics
computer assisted surgery (CAS) systems of the environmental
E691 Practice for Conducting an Interlaboratory Study to
influences caused by equipment utilized for bone preparation
Determine the Precision of a Test Method
during the intended clinical application for the system. The
F2554 Practice for Measurement of Positional Accuracy of
environmental vibration effect covered in this standard will
Computer-Assisted Surgical Systems
include mechanical vibration from: cutting saw (sagittal or
reciprocating), burrs, drills, and impact loading. The change in
2.2 ISO Standard:
accuracy from detaching and re-attaching or disturbing a ISO 10360 Geometrical Product Specifications (GPS)—
restrained connection that does not by design require repeating
Acceptance and re-verification tests for coordinate mea-
the registration process of a reference base will also be suring machines (CMM)
measured.
3. Terminology
1.2 It should be noted that one system may need to undergo
3.1 Definition of Terms Specific to Accuracy Reporting:
multiple iterations (one for each clinical application) of this
3.1.1 accuracy, n—the closeness of agreement between a
standard to document its accuracy during different clinical
measurement result and an accepted reference value. E456
applications since each procedure may have different exposure
3.1.1.1 Discussion—The term accuracy, when applied to a
to outside forces given the surgical procedure variability from
set of measurement results, involves a combination of a
one procedure to the next.
random component and of a common systematic error or bias
1.3 All units of measure will be reported as millimeters for
component.
this standard.
3.1.2 bias, n—the difference between the expectation of the
1.4 This standard does not purport to address all of the
measurement results and an accepted reference value. E456
safety concerns, if any, associated with its use. It is the
3.1.2.1 Discussion—Bias is the total systematic error as
responsibility of the user of this standard to establish appro-
contrasted to random error. There may be one or more
priate safety, health, and environmental practices and deter-
systematic error components contributing to the bias. A larger
mine the applicability of regulatory limitations prior to use.
systematic difference from the accepted reference value is
1.5 This international standard was developed in accor-
reflected by a larger bias value.
dance with internationally recognized principles on standard-
3.1.3 maximum error, n—the largest distance between any
ization established in the Decision on Principles for the
measured point and its corresponding reference position for
Development of International Standards, Guides and Recom-
any trial during a testing procedure.
mendations issued by the World Trade Organization Technical
3.1.4 mean, n—the arithmetic mean (or simply the mean) of
Barriers to Trade (TBT) Committee.
a list of numbers is the sum of all the members of the list
1 2
This test method is under the jurisdiction of ASTM Committee F04 on Medical For referenced ASTM standards, visit the ASTM website, www.astm.org, or
and Surgical Materials and Devices and is the direct responsibility of Subcommittee contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
F04.38 on Computer Assisted Orthopaedic Surgical Systems. Standards volume information, refer to the standard’s Document Summary page on
Current edition approved March 1, 2023. Published March 2023. Originally the ASTM website.
approved in 2014. Last previous edition approved in 2014 as F3107 – 14. DOI: Available from American National Standards Institute (ANSI), 25 W. 43rd St.,
10.1520/F3107-14R23. 4th Floor, New York, NY 10036, http://www.ansi.org.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
F3107 − 14 (2023)
divided by the number of items in the list. If one particular 3.2.3 degree of freedom (DOF), n—set of independent
number occurs more times than others in the list, it is called a displacements that specify completely the displaced or de-
mode. The arithmetic mean is what students are taught very formed position of the body or system.
early to call the “average.” If the list is a statistical population,
3.2.4 dynamic reference base, n—a reference element that is
then the mean of that population is called a population mean.
intraoperatively attached to a therapeutic object and allows
If the list is a statistical sample, we call the resulting statistic a
tracking that object. It defines the local coordinate system of
sample mean.
the therapeutic object.
3.1.5 measurement range, n—see measurement volume.
3.2.5 fiducial, n—an artificial object (e.g., screw or sphere)
3.1.6 precision, n—the closeness of agreement between that is implanted into, or a feature created on, a therapeutic
object prior to virtual object acquisition to facilitate registra-
independent measurement results obtained under stipulated
conditions. E456 tion.
3.1.6.1 Discussion—Precision depends on random errors
3.2.6 marker, n—a single indicator on a reference element or
and does not relate to the true value or the specified value. The
dynamic reference base where a collection of these indicators
measure of precision usually is expressed in terms of impreci-
are utilized to define an object, tool, or reference frame in
sion and computed as a standard deviation of the test results.
space.
3.1.6.2 Discussion—Less precision is reflected by a larger
3.2.7 measurement volume, n—measuring range of a
standard deviation. “Independent test results” means results
tracker, stated as simultaneous limits on all spatial coordinates
obtained in a manner not influenced by any previous result on
measured by the tracker. ISO 10360-1
the same or similar test object. Quantitative measures of
3.2.8 navigation system, n—a device consisting of a com-
precision depend critically on the stipulated conditions. Re-
puter with associated software and a localizer that tracks
peatability and reproducibility conditions are particular sets of
reference elements attached to surgical instruments or implants
extreme stipulated conditions.
as well as one or more dynamic reference bases attached to the
3.1.7 range, R, n—the largest observation minus the small-
therapeutic object. It provides real-time feedback of the per-
est observation in a set of values or observations. E456, E2281
formed action by visualizing it within the virtual environment.
3.1.8 repeatability, n—precision under repeatability
3.2.9 reference element, n—a device attached to surgical
conditions. E456
instruments and implants and other devices that enables
3.1.8.1 Discussion—Repeatability is one of the concepts or
determination of position and orientation in 3D space (up to six
categories of the precision of a test method. Measures of
degrees of freedom) of these by means of a tracker. It defines
repeatability defined in this compilation are repeatability,
the local coordinate system of this instrument or implant.
standard deviation, and repeatability limit.
3.2.10 reference point, n—a designated point on the phan-
3.1.9 reproducibility, n—precision under reproducibility
tom or sawbone used to repeat measures and to make com-
conditions. E456
parisons to after each trial is performed within the standard.
3.1.9.1 Discussion—Ability of a test or experiment to be
3.2.11 referencing, n—tracking of a therapeutic object by
accurately reproduced or replicated.
means of a dynamic reference base.
3.1.10 resolution, n—of a devise/sensor, smallest change it
3.2.12 registration, n—the determination of the transforma-
can detect in the quantity that it is measuring. The resolution is
tion between the coordinate spaces of the therapeutic and
related to the precision with which the measurement is made.
virtual objects or between the coordinate spaces of two virtual
objects. A registration is rigid if it consists only of rotations,
3.1.11 standard deviation, n—the most usual measure of the
translations, and scaling; it is non-rigid if it also comprises
dispersion of observed values or results expressed as the
local or global distortions.
positive square root of the variance. E456
3.2.13 robotic positioning system, n—use of an active me-
3.1.12 variance, n—of a random variable, measure of its
chanical (mechatronic) device to position an instrument guide
statistical dispersion, indicating how its possible values are
at a specified location in 3D space (up to six degrees of
spread around the expected value. Where the expected value
freedom).
shows the location of the distribution, the variance indicates
the scale of the values. A more understandable measure is the
3.2.14 stylus, n—a mechanical device consisting of a stylus
square root of the variance, called the standard deviation.
tip and a shaft. The stylus tip is the physical element that
establishes the contact with the workpiece. ISO 10360-1
3.2 Definition of Terms Specific to Surgical Navigation and
Robotic Positioning Systems:
3.2.15 tool calibration, n—the pre- or intraoperative deter-
3.2.1 computer assisted surgery (CAS), n—the use of com- mination of the location of points of interest on a navigated
puters to facilitate or enhance surgical procedures via the use of
instrument (e.g., its tip position, axis) in relation to a reference
three-dimensional space tracking of objects. frame (e.g., the attached reference element for a tracked
instrument).
3.2.2 data integrity, n—condition in which data is identi-
cally maintained during any operation, such as transfer, 3.2.16 tracker, n—a device that measures the spatial loca-
storage, and retrieval. tion and orientation of surgical instruments, implants, or the
F3107 − 14 (2023)
therapeutic object that are instrumented with reference ele- characterize the digitization accuracy of the tracking subsystem
ments or a dynamic reference base respectively. A tracker may in a controlled environment under controlled conditions.
measure based on infrared light (see tracking, active and
5.3 In order to make comparisons within and between
tracking, passive), ultrasound, electromagnetic fields, mechani-
systems, a standardized way of measuring and reporting point
cal linkage, video streams, etc.
accuracy is needed. Parameters such as coordinate system,
3.2.17 tracking, active, n—a tracking technology that uses
units of measure, terminology, and operational conditions must
markers that emit energy (e.g., an infrared light based tracking
be standardized.
technology that uses pulsed LEDs as markers, ultrasound,
electromagnetic fields, etc.).
6. Apparatus
3.2.18 tracking, passive, n—a tracking technology that uses
6.1 Standardized measurement object (phantom). See Fig.
markers that absorb or reflect externally produced energy (e.g.,
1.
a light-based tracking technology that uses reflective spheres or
6.2 System to be evaluated, including tracking system,
similar objects as markers).
stylus or any pointing device, and associated required hardware
and software. While the software may be custom written for the
4. Summary of Test Method
tasks outlined in this standard, it should use exactly the same
4.1 Reference Base Attachment Tolerance—This portion of
algorithms and methodologies being implemented in the
the standard will evaluate the robustness of the reference base
commercial/clinical system to be assessed under this standard.
and tracking array attachment to outside forces and repeated
6.3 A chosen biomechanical test specimen that is anchored
use in an environment and application that is clinically relevant
or rigidly fixed to the phantom and placed on a table will be
to their intended use. These forces will simulate operating
utilized. A standard reference base attachment and manufac-
room incidental contacts from end users and assistants. The
turer application tools will be utilized during the procedure.
results will allow comparison of the stability and repeatability
See Fig. 1.
of the tolerances and strength of the reference base attachment
of the markers used for the CAS system. Utilization of a
7. Hazards
phantom will be used as described in F2554–10 for attachment
7.1 The hazards from performing the standard are from
of the reference base in a clinically applicable manner to the
intended use of the CAS system being tested. handling the instruments and from the saw and drill bits. No
human or biological tissues are utilized so no biohazard exists.
4.2 The second portion of this standard will entail testing
typical situations in the operating room that occur during a
8. Procedure—Reference Base Attachment Test
surgical procedure that can impact the accuracy of the CAS
system that involves a computer marker attachment to a bony 8.1 Complete steps 8.1 to 8.33 in F2554–10, utilizing the
phantom described in Appendix A2 but with utilizing one
landmark. This portion of the standard will involve attachment
of a reference base to the chosen model according to manu- determined point on the phantom. Note that the reference base
must be part of the CAS system being tested (those typical of
facturer’s guidelines and performing procedures using a mate-
rial of similar density and mechanical property of the appli- tracking bony reference landmarks and instruments) and at-
tached to the phantom. It cannot be a permanent part of the
cable bony anatomical region being tested. Measurement of a
marked point on the chosen test specimen which will be phantom and must include a reference base attachment for the
repeated with a pointing device to a set of six points on the test active or passive markers. If the array is not detachable from
specimen with removal of the tracking array from the reference the reference base then it is left alone for all tests in this
base, reattachment and repeat. This procedure will be repeated standard.
for sagittal saw, drilling, and i
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.