ASTM E2853-12(2021)
(Test Method)Standard Test Method for Evaluating Emergency Response Robot Capabilities: Human-System Interaction (HSI): Search Tasks: Random Mazes with Complex Terrain
Standard Test Method for Evaluating Emergency Response Robot Capabilities: Human-System Interaction (HSI): Search Tasks: Random Mazes with Complex Terrain
SIGNIFICANCE AND USE
5.1 A main purpose of using robots in emergency response operations is to enhance the safety and effectiveness of emergency responders operating in hazardous or inaccessible environments. The testing results of the candidate robot shall describe, in a statistically significant way, how reliably the robot is able to perform the specified types of tasks and thus provide emergency responders sufficiently high levels of confidence to determine the applicability of the robot.
5.2 This test method addresses robot performance requirements expressed by emergency responders and representatives from other interested organizations. Robot performance data captured within this test method are indicative of the robotic system’s capabilities. Having available a roster of successfully tested robots with associated performance data to guide procurement and deployment decisions for emergency responders is consistent with the guideline of “Governments at all levels have a responsibility to develop detailed, robust, all-hazards response plans” as stated in National Response Framework.
5.3 This test method is part of a test suite and is intended to provide a capability baseline for the robotic HSI subsystems based on the identified needs of the emergency response community. Adequate performance using this test suite will not ensure successful operation in all emergency response situations due to possible extreme operational difficulties. Rather, this test method is intended to provide a common comparison of technologies against a reasonable simulation of emergency response environments and to provide quantitative performance data to emergency response organizations to aid in choosing appropriate systems. This standard is also intended to encourage development of improved and innovative communications systems for use on emergency response robots.
5.4 The standard apparatus is specified to be easily fabricated to facilitate self-evaluation by robot developers and provide practice...
SCOPE
1.1 Purpose:
1.1.1 The purpose of this test method, as a part of a suite of human-system interactions (HSI) test methods, is to quantitatively evaluate a teleoperated ground robot’s (see Terminology E2521) capability of searching in a maze.
1.1.2 Teleoperated robots shall possess a certain set of HSI capabilities to suit critical operations such as emergency responses, including enabling the operators to search for required targets. A passage that forms on complex terrains and possesses complex and visually similar branches is a type of environments that exists in emergency response and other robotically applicable situations. The complexity often poses challenges for the operators to teleoperate the robots to conduct searches. This test method is based on a standard maze and specifies metrics and a procedure for testing the search capability.
1.1.3 Emergency response robots shall enable the operator to handle many types of tasks. The required HSI capabilities include search and navigation on different types of terrains, passages, and confined spaces. Standard test methods are required to evaluate whether candidate robots meet these requirements.
1.1.4 ASTM E54.08.01 Task Group on Robotics specifies a HSI test suite, which consists of a set of test methods for evaluating these HSI capability requirements. This random maze searching test method is a part of the HSI test suite. The apparatuses associated with the test methods challenge specific robot capabilities in repeatable ways to facilitate comparison of different robot models as well as particular configurations of similar robot models. (See Fig. 1.)
FIG. 1 HSI: Search Tasks: Random Maze Illustration
1.1.5 The test methods quantify elemental HSI capabilities necessary for ground robots intended for emergency response applications. As such, based on their particular capability requirements, users of this test suite can select only the applicable tes...
General Information
Buy Standard
Standards Content (Sample)
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Contact ASTM International (www.astm.org) for the latest information
Designation: E2853 − 12 (Reapproved 2021)
Standard Test Method for
Evaluating Emergency Response Robot Capabilities:
Human-System Interaction (HSI): Search Tasks: Random
1
Mazes with Complex Terrain
This standard is issued under the fixed designation E2853; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope requirements, users of this test suite can select only the
applicable test methods and can individually weight particular
1.1 Purpose:
test methods or particular metrics within a test method. The
1.1.1 The purpose of this test method, as a part of a suite of
testing results should collectively represent a ground robot’s
human-system interactions (HSI) test methods, is to quantita-
overall HSI capability. The test results can be used to guide
tively evaluate a teleoperated ground robot’s (see Terminology
procurement specifications and acceptance testing for robots
E2521) capability of searching in a maze.
intended for emergency response applications.
1.1.2 Teleoperated robots shall possess a certain set of HSI
capabilities to suit critical operations such as emergency
NOTE 1—The teleoperation performance is affected by the robot’s as
responses, including enabling the operators to search for
well as the operator’s capabilities. Among all the standard test methods
required targets.Apassage that forms on complex terrains and thatASTME54.08.01TaskGrouponRoboticshasspecified,somedepend
more on the former while the others on the latter, but it would be
possesses complex and visually similar branches is a type of
extremely hard to totally isolate the two factors. This HSI test suite is
environments that exists in emergency response and other
specified to focus on evaluating the operator’s capabilities of interacting
robotically applicable situations. The complexity often poses
with the robotic system, whereas a separately specified sensor test suite,
challengesfortheoperatorstoteleoperatetherobotstoconduct
including Test Method E2566, focuses on the robots’sensing capabilities.
searches. This test method is based on a standard maze and
NOTE 2—As robotic systems are more widely applied, emergency
specifies metrics and a procedure for testing the search responders might identify additional or advanced HSI capability require-
ments to help them respond to emergency situations. They might also
capability.
desire to use robots with higher levels of autonomy, beyond teleoperation
1.1.3 Emergency response robots shall enable the operator
to help reduce their workload—see NIST Special Publication 1011-II-1.0.
to handle many types of tasks. The required HSI capabilities
Further, emergency responders in expanded emergency response domains
include search and navigation on different types of terrains,
might also desire to apply robotic technologies to their situations, a source
passages, and confined spaces. Standard test methods are
for new sets of requirements. As a result, additional standards within the
suite would be developed. This standard is, nevertheless, standalone and
required to evaluate whether candidate robots meet these
complete.
requirements.
1.1.4 ASTM E54.08.01 Task Group on Robotics specifies a
1.2 Performing Location—This test method shall be per-
HSI test suite, which consists of a set of test methods for
formed in a testing laboratory or the field where the specified
evaluating these HSI capability requirements. This random
apparatus and environmental conditions are implemented.
maze searching test method is a part of the HSI test suite. The
1.3 Units—The values stated in SI units are to be regarded
apparatuses associated with the test methods challenge specific
as the standard.The values given in parentheses are not precise
robotcapabilitiesinrepeatablewaystofacilitatecomparisonof
mathematical conversions to inch-pound units. They are close
different robot models as well as particular configurations of
approximate equivalents for the purpose of specifying material
similar robot models. (See Fig. 1.)
dimensions that are readily available to avoid excessive fabri-
1.1.5 The test methods quantify elemental HSI capabilities
cation costs of test apparatuses while maintaining repeatability
necessary for ground robots intended for emergency response
and reproducibility of the test method results. These values
applications. As such, based on their particular capability
given in parentheses are provided for information only and are
not considered standard.
1
1.4 This standard does not purport to address all of the
This test method is under the jurisdiction of ASTM Committee E54 on
Homeland Security Applica
...
This document is not an ASTM standard and is intended only to provide the user of an ASTM standard an indication of what changes have been made to the previous version. Because
it may not be technically possible to adequately depict all changes accurately, ASTM recommends that users consult prior editions as appropriate. In all cases only the current version
of the standard as published by ASTM is to be considered the official document.
Designation: E2853 − 12 E2853 − 12 (Reapproved 2021)
Standard Test Method for
Evaluating Emergency Response Robot Capabilities:
Human-System Interaction (HSI): Search Tasks: Random
1
Mazes with Complex Terrain
This standard is issued under the fixed designation E2853; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope
1.1 Purpose:
1.1.1 The purpose of this test method, as a part of a suite of human-system interactions (HSI) test methods, is to quantitatively
evaluate a teleoperated ground robot’s (see Terminology E2521) capability of searching in a maze.
1.1.2 Teleoperated robots shall possess a certain set of HSI capabilities to suit critical operations such as emergency responses,
including enabling the operators to search for required targets. A passage that forms on complex terrains and possesses complex
and visually similar branches is a type of environments that exists in emergency response and other robotically applicable
situations. The complexity often poses challenges for the operators to teleoperate the robots to conduct searches. This test method
is based on a standard maze and specifies metrics and a procedure for testing the search capability.
1.1.3 Emergency response robots shall enable the operator to handle many types of tasks. The required HSI capabilities include
search and navigation on different types of terrains, passages, and confined spaces. Standard test methods are required to evaluate
whether candidate robots meet these requirements.
1.1.4 ASTM E54.08.01 Task Group on Robotics specifies a HSI test suite, which consists of a set of test methods for evaluating
these HSI capability requirements. This random maze searching test method is a part of the HSI test suite. The apparatuses
associated with the test methods challenge specific robot capabilities in repeatable ways to facilitate comparison of different robot
models as well as particular configurations of similar robot models. (See Fig. 1.)
1.1.5 The test methods quantify elemental HSI capabilities necessary for ground robots intended for emergency response
applications. As such, based on their particular capability requirements, users of this test suite can select only the applicable test
methods and can individually weight particular test methods or particular metrics within a test method. The testing results should
collectively represent a ground robot’s overall HSI capability. The test results can be used to guide procurement specifications and
acceptance testing for robots intended for emergency response applications.
NOTE 1—The teleoperation performance is affected by the robot’s as well as the operator’s capabilities. Among all the standard test methods that ASTM
E54.08.01 Task Group on Robotics has specified, some depend more on the former while the others on the latter, but it would be extremely hard to totally
isolate the two factors. This HSI test suite is specified to focus on evaluating the operator’s capabilities of interacting with the robotic system, whereas
a separately specified sensor test suite, including Test Method E2566, focuses on the robots’ sensing capabilities.
1
This test method is under the jurisdiction of ASTM Committee E54 on Homeland Security Applications and is the direct responsibility of Subcommittee E54.09 on
Response Robots.
Current edition approved Feb. 1, 2012Jan. 1, 2021. Published April 2012January 2021. Originally approved in 2012. Last previous edition approved in 2012 as E2853 – 12.
DOI: 10.1520/E2853-12.10.1520/E2853-12R21.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
1
---------------------- Page: 1 ----------------------
E2853 − 12 (2021)
FIG. 1 HSI: Search Tasks: Random Maze Illustration
NOTE 2—As robotic systems are more widely applied, emergency responders might identify additional or advanced HSI capability requirements to help
them respond to emergency situations. They might also desire to use robots with higher levels of autonomy, beyond teleoperation to help reduce their
workload—see NIST Special Publication 1011-II-1.0. Further, emergency responders in expanded emergency response domains might also desire to apply
robotic technologies to their situations, a source for new sets of requirements. As a result, additional stand
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.