ASTM E3426/E3426M-24

This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation: E3426/E3426M − 24
Standard Test Method for
Evaluating Aerial Response Robot Endurance
This standard is issued under the fixed designation E3426/E3426M; the number immediately following the designation indicates the year
of original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval.
A superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
INTRODUCTION
The robotics community needs ways to measure whether a particular robot is capable of performing
specific missions in complex, unstructured, and often hazardous environments. These missions require
various combinations of elemental robot capabilities. Each capability can be represented as a test
method with an associated apparatus to provide tangible challenges for various mission requirements
and performance metrics to communicate results. These test methods can then be combined and
sequenced to evaluate essential robot capabilities and remote pilot proficiencies necessary to
successfully perform intended missions.
The ASTM International Standards Committee on Homeland Security Applications (E54) specifies
these standard test methods to facilitate comparisons across different testing locations and dates for
diverse robot sizes and configurations. These standards support robot researchers, manufacturers, and
user organizations in different ways. Researchers use the standards to understand mission
requirements, encourage innovation, and demonstrate break-through capabilities. Manufacturers use
the standards to evaluate design decisions, integrate emerging technologies, and harden systems.
Emergency responders and soldiers use them to guide purchasing decisions and align deployment
expectations. Associated usage guides describe how these standards can be applied to support various
objectives. These standard test methods may be used in concert with Specification F3330 to create
scenario-based training programs.
Several suites of standards address these elemental capabilities including maneuvering, mobility,
dexterity, sensing, endurance, communications, durability, proficiency, autonomy, and logistics.
1. Scope evaluate assistive or autonomous behaviors intended to im-
prove the effectiveness or efficiency of remotely operated
1.1 This test method is intended for remotely operated aerial
systems.
response robots (that is, unmanned aerial systems [UAS],
drones, unmanned aircrafts) operating in complex, 1.3 Different user communities can set their own thresholds
unstructured, and often hazardous environments. It specifies of acceptable performance within this test method for various
the apparatuses, procedures, and performance metrics neces- mission requirements.
sary to measure the mission endurance of an aerial robot while
1.4 Performing Location—This test method may be per-
either station keeping or following an approximate flight path
formed anywhere the specified apparatuses and environmental
defined by obstacles or boundaries, or both, intended to induce
conditions can be implemented. Flying unmanned aircraft
repeated cyclical movement. This test method is one of several
without a comprehensive understanding of the laws and
robot tests that can be used to evaluate overall system
regulations enforced by the relevant jurisdiction poses signifi-
capabilities.
cant safety and legal risks. Failure to comply with these
1.2 The robotic system includes a remote pilot in control of
regulations may result in accidents, injuries, property damage,
most functionality, so an onboard camera and remote pilot and legal consequences. Users of this standard are strongly
display are typically required. This test method can be used to
advised to review and adhere to all applicable ASTM Com-
mittee F38 standards and to ensure full compliance with the
authorities holding jurisdiction.
This test method is under the jurisdiction of ASTM Committee E54 on
1.5 Units—The International System of Units (SI Units) and
Homeland Security Applications and is the direct responsibility of Subcommittee
U.S. Customary Units (Imperial Units) are used throughout this
E54.09 on Response Robots.
document. They are not mathematical conversions. Rather,
Current edition approved Feb. 1, 2024. Published February 2024. DOI: 10.1520/
E3426_E3426M-24. they are approximate equivalents in each system of units to
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E3426/E3426M − 24
enable use of readily available materials in different countries. 3.4.1 apparatus clearance width (W), n—a specification for
The differences between the stated dimensions in each system the apparatus dimensions chosen from one of four possible
of units are insignificant for the purposes of comparing test measurements, based on the intended robot deployment envi-
method results, so each system of units is separately considered ronment:
standard within this test method. 240 cm 6 2.5 cm tolerance [96 in. 6 1 in. tolerance], such
as open and outdoor public spaces;
1.6 This standard does not purport to address all of the
120 cm 6 2.5 cm tolerance [48 in. 6 1 in. tolerance], such
safety concerns, if any, associated with its use. It is the
as indoor spaces in accessibility-compliant buildings;
responsibility of the user of this standard to establish appro-
60 cm 6 1.3 cm tolerance [24 in. 6 0.5 in. tolerance],
priate safety, health, and environmental practices and deter-
residences and aisles of public transportation; or
mine the applicability of regulatory limitations prior to use.
30 cm 6 1.3 cm tolerance [12 in. 6 0.5 in. tolerance],
1.7 This international standard was developed in accor-
cluttered indoor spaces, ductwork, and voids in collapsed
dance with internationally recognized principles on standard-
structures.
ization established in the Decision on Principles for the
3.4.1.1 Discussion—The measures for these scales are
Development of International Standards, Guides and Recom-
nominal and do not represent the measurement of the narrowest
mendations issued by the World Trade Organization Technical
point in the apparatus through which the robot should pass.
Barriers to Trade (TBT) Committee.
Consult Section 6 for the overall measurements and dimen-
sions of the apparatus at each scale.
2. Referenced Documents
3.4.2 remote pilot, n—the remote pilot in command (RPIC)
2.1 ASTM Standards:
or person other than the RPIC who is controlling the flight of
E2521 Terminology for Evaluating Response Robot Capa-
an unmanned aircraft (UA) under the supervision of the RPIC.
bilities
F3341
E2592 Practice for Evaluating Response Robot Capabilities:
Logistics: Packaging for Urban Search and Rescue Task 3.4.3 unmanned aircraft, n—aircraft operated without the
Force Equipment Caches possibility of direct human intervention from within or on the
E3132 Practice for Evaluating Response Robot Logistics: aircraft. F3341
System Configuration
NOTE 1—Due to similarities in characteristics and to maintain consis-
F3330 Specification for Training and the Development of
tency across standards developed through ASTM E54.09, the “unmanned
Training Manuals for the UAS Operator
aircraft” (Terminology F3341) is referred to as the “robot” (Terminology
F3341 Terminology for Unmanned Aircraft Systems E2521) throughout this standard.
2.2 Other Documents:
4. Summary of Test Method
NIST Special Publication 1011-I-2.0 Autonomy Levels for
4.1 This test method is performed by a remote pilot in
Unmanned Systems (ALFUS) Framework Volume I: Ter-
control of an aerial response robot (that is, unmanned aerial
minology
system [UAS], drone, unmanned aircraft). The test administra-
tor and all participants shall ensure compliance with the
3. Terminology
regulations of the authority holding jurisdiction before con-
3.1 Definitions—The following terms are used in this test
ducting any tests. The robot follows one of four defined
method and are defined in Terminology E2521: abstain,
operating profiles in the specified testing area, requiring the
administrator or test administrator, emergency response robot
robot to overcome challenges such as continuous movement,
or response robot, fault condition, operator, operator station,
obstacle avoidance, constant vector adjustment, station
remote control, repetition, robot, teleoperation, test event or
keeping, or dwelling in varied environmental conditions. Four
event, test form, test sponsor, test suite, testing target or target,
tests are defined, one for each operating profile: outdoor
testing task or task, and trial or test trial.
movement endurance (where the robot continuously flies down
3.2 The following terms are used in this test method and are
range, ascends, descends, and returns up range), indoor move-
defined in ALFUS Framework Volume I:3: autonomous,
ment endurance (where the robot continuously flies following
autonomy, level of autonomy, operator control unit (OCU), and
a figure-8 flight path inside a confined space), indoor hovering
semi-autonomous.
endurance (where the robot hovers in place inside a confined
space), and indoor dwelling endurance (where the robot lands
3.3 The following terms are used in this test method and are
on the ground and remains in place inside a confined space).
defined in Terminology F3341: remote pilot and unmanned
The outdoor operating profile is performed in a testing area
aircraft.
measuring at least 15 m [50 ft] wide by 90 m [300 ft] long by
3.4 Definitions of Terms Specific to This Standard:
90 m [300 ft] tall; see Fig. 1. The three indoor operating
profiles are performed in a testing area measuring 2W wide by
7W long (or longer) by 2W tall, defined by physical boundaries
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
and with barrier posts that aid in defining the flight path. See
contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
Standards volume information, refer to the standard’s Document Summary page on Fig. 2.
the ASTM website.
4.2 The outdoor movement test uses a straight, forward
Available from National Institute of Standards and Technology (NIST), 100
Bureau Dr., Stop 1070, Gaithersburg, MD 20899-1070, http://www.nist.gov. flight path followed by an ascending/descending flight path in
E3426/E3426M − 24
FIG. 1 Overview of the Testing Area for the Outdoor Endurance Test
an open, outdoor area. It can be used to demonstrate horizontal able to complete all four of the 90 m [300 ft] flight segments,
and vertical aerial traversal over long distances. See Fig. 3. The the distance of those that were completed are included in its
robot starts in the takeoff area and then proceeds into the flight performance metrics; 0.25 repetitions = 90 m [300 ft], 0.5
area after taking off. Each repetition of the flight path begins
repetitions = 180 m [600 ft], 0.75 repetitions = 270 m [900 ft].
and ends when the robot crosses the start/end line without a
4.3 The indoor movement test uses a figure-8 forward flight
fault, after approximately following the flight path. A line
path through the testing area with alternating left and right
marking on the ground is made to guide the remote pilot in
turns to avoid barriers. It can be used to demonstrate indoor
controlling the robot when flying the horizontal 90 m [300 ft]
aerial traversal over long distances within a relatively small
flight path; the line marking remains visible in the robot’s
apparatus. See Fig. 5; the flight path and available flight area
forward-facing or downward-facing camera such that the robot
are shown in green. With the left-most boundary removed, the
approximately follows the flight path. An upward-facing in-
robot starts in the takeoff area and then proceeds into the flight
spection target is positioned at the end of the horizontal flight
area after taking off, at which point the left-most boundary is
path to guide the remote pilot in controlling the robot when
put back into position. Each repetition of the figure-8 flight
flying the vertical 90 m [300 ft] flight path; the robot’s
path begins and ends when the robot crosses the start/end line
downward-facing camera remains aimed at the target while
without a fault after approximately following the flight path.
ascending/descending such that the robot approximately fol-
The robot will visibly pass in front of the edge of the barrier as
lows the flight path. When properly aligned with the target, the
it crosses its starting point, enabling more accurate data
remote pilot must be able to see the entire colored ring on the
OCU display of the robot’s camera (see Fig. 4 for examples of collection from an outside observation point or from post-flight
camera footage. The distance per repetition is a total of 8W
correct and incorrect alignment). The distance per repetition is
a total of 360 m [1200 ft]. If the test ends before the robot is (two 4W segments of the distance between the outer edges of
E3426/E3426M − 24
Dimensions scale proportionally to the apparatus clearance width (W). Wall and ceiling boundaries in 3D rendering are shown as transparent only for diagrammatic
purposes.
FIG. 2 Overview of the Testing Area for the Indoor Endurance Tests
the barriers). If the test ends before the robot is able to while landed at that location. See Fig. 6; the flight path and
complete both 4W flight segments, but it was able to complete available flight area are shown in green and the area where
one 4W flight segment (that is, 0.5 repetitions), then that 4W dwelling is performed is shown in purple. With the left-most
distance that was completed is included in its performance boundary removed, the robot starts in the takeoff area and then
metrics. proceeds into the flight area after taking off, at which point the
left-most boundary is put back into position. The robot crosses
4.4 The indoor hovering test involves the robot traversing a
the start line and performs a single figure-8 traversal. Once
distance and then hovering in place at a specified location with
completed, it shall stop, land, and dwell in that position for as
the intention of remaining as stationary as possible within that
long as it is able.
location. See Fig. 6; the flight path and available flight area are
shown in green and the area where hovering is performed is 4.6 Potential faults include:
shown in purple. With the left-most boundary removed, the 4.6.1 Any contact by the robot with the walls or barriers that
robot starts in the takeoff area and then proceeds into the flight requires adjustment or repair to return the walls or barriers to
area after taking off, at which point the left-most boundary is the initial condition;
put back into position. The robot crosses the start line and 4.6.2 Any physical interaction with the robot that assists
performs a single figure-8 traversal. Once completed, it shall either the robot or the remote pilot (for example, if the robot
stop and hover in place, remaining in position for as long as it crashes and the remote pilot picks it up to resume testing); and
is able. The remote pilot is allowed to correct minor deviations 4.6.3 Leaving the apparatus during the trial.
in position and height as needed, so long as the robot does not
4.7 Test trials of the outdoor and indoor movement tests
leave the designated flight area.
shall produce enough successful repetitions to demonstrate the
4.5 The indoor dwelling test involves the robot traversing a reliability of the system capability or the remote pilot profi-
distance, landing at a specified location, and then dwelling ciency. The endurance test is unique in that a complete test
E3426/E3426M − 24
FIG. 3 The Outdoor Movement Endurance Test
Correct alignment is defined as when the remote pilot is able to see the entire colored ring, as shown in the two left images.
FIG. 4 Correct Alignment
running until exhaustion of the energy source on the robot is for minor adjustments by the remote pilot as needed to remain
required to calculate the performance metrics of distance
within the defined flight area during the indoor hovering test)
traveled and flight time. During the outdoor and indoor
for the entire duration of the test.
movement tests, the higher the ratio of successful repetitions to
4.8 There are two metrics to consider when calculating the
faults, the more reliable the system or remote pilot, or both. At
results of a test trial. They should be considered in the
least 90 % of the attempted repetitions in the outdoor and
following order of importance: time and distance traveled
indoor movement tests should be successful to consider the
(outdoor and indoor movement test only). The results from the
performance metrics as valid measurements of endurance. For
outdoor movement, indoor movement, indoor hovering, and
the indoor hovering and dwelling tests, no repetitions are
indoor dwelling tests are not comparable because they measure
performed as the robot remains static without moving (except
E3426/E3426M − 24
Wall and ceiling boundaries in 3D rendering are shown as transparent only for diagrammatic purposes.
FIG. 5 The Indoor Movement Endurance Test
different capabilities. The results from test apparatuses of capabilities, but for remaining stationary in air within an
different appearance clearance widths (that is, W values) are outdoor or confined indoor area. The endurance test standard
also not comparable because they represent different clearances provides a method in which the operational endurance of a
and distances. large variety of robot sizes and locomotion system designs may
be compared. The test provides both a measure of the endur-
5. Significance and Use
ance of the robot and a measure of the reliability of the robot
when operating continuously for extended periods of time on
5.1 This test method is part of an overall suite of related test
complex flight paths or continuous use, or both.
methods that provide repeatable measures of robotic system
mobility and remote pilot proficiency. The operational endur-
5.2 The indoor tests with containment walls represent re-
ance of a robot significantly impacts the performance of the
peatable complexity within commercial spaces and residential
robot during a variety of tasks. Robot endurance is a complex
dwellings with hallways and doorways, or warehouses.
function of robot design, control scheme design, and energy
5.3 The test apparatuses are low-cost and easy to fabricate
storage selection. This test method evaluates the endurance of
so they can be widely replicated. The procedure is also simple
a robot through continuous operation. The outdoor and indoor
to conduct. This eases comparisons across various testing
movement tests flight path chosen for endurance testing
locations and dates to determine best-in-class systems and
specifically challenges robotic system locomotion, flight sys-
remote pilots.
tem to maintain position, and remote situational awareness by
the remote pilot. As such, it can be used to represent modest 5.4 Evaluation—This test method can be used in a con-
outdoor flight or indoor flight within confined areas. The trolled environment to measure baseline capabilities. The
indoor hovering and dwelling tests similarly challenge these endurance test apparatus can also be embedded into operational
E3426/E3426M − 24
Wall and ceiling boundaries in 3D rendering are shown as transparent only for diagrammatic purposes.
FIG. 6 The Indoor Hovering or Dwelling Endurance Test
training scenarios to measure degradation due to uncontrolled 6. Apparatus
variables in lighting, weather, radio communications, GPS
6.1 The apparatus required to perform this test method
accuracy, etc.
consists of a flight area (outdoor or indoor), physical barriers to
define the robot path, an inspection target (outdoor movement
5.5 Procurement—This test method can be used to identify
endurance only), ground markings, and a timer.
inherent capability trade-offs in systems, make informed pur-
6.2 Apparatus for Outdoor Test—For the outdoor movement
chasing decisions, and verify performance during acceptance
test, an outdoor environment must be used that allows for the
testing. This aligns requirement specifications and user expec-
robot to traverse downrange at least 90 m [300 ft] and to ascend
tations with existing capability limits.
at least 90 m [300 ft] vertically. A minimum width of 15 m
5.6 Innovation—This test method can be used to inspire
[50 ft] is required to minimize any potential impacts of air flow
technical innovation, demonstrate break-through capabilities,
between the robot and outdoor boundaries such as buildings,
and measure the reliability of systems performing specific tasks
fences, or trees.
within an overall mission sequence. Combining or sequencing
6.2.1 Line Marking to Define the Horizontal Robot Path—In
multiple test methods can guide manufacturers toward imple-
order to ensure the robot traverses approximately 90 m [300 ft]
menting the combinations of capabilities necessary to perform downrange and uprange, an approximately straight line is
essential mission tasks. marked on the ground, spanning this distance between the
E3426/E3426M − 24
start/end line and the target used for vertical ascension and dimensions scale proportionally with the apparatus clearance
descension. This marking should be thick enough or its color width (see Fig. 2). For example, the overall width of the flight
should be high contrast with the ground enough, or both, to be area is 2W and the overall length of the flight area is 6W. The
easily seen by the remote pilot through the robot’s camera(s), takeoff area is 2W wide and at least 1W long; it can be made
such that it aids the remote pilot in traversing the path.
longer for robots needing more space to takeoff (for example,
Example materials to fabricate the line marking include lengths due to larger physical size or operational limitations, or both).
of rope or measure tape.
Robots can also takeoff inside of the flight area if desired. See
6.2.2 Inspection Target to Define the Vertical Robot Fig. 8 for a comparison of the apparatus dimensions at each
Path—In order to ensure the robot traverses approximately
value of W. When choosing a specific apparatus clearance
90 m [300 ft] vertically when ascending and descending, an
width for the apparatus, note the resulting data is not compa-
upward-facing target is positioned at the end of the horizontal
rable to other apparatuses with different apparatus clearance
90 m [300 ft] line marking. The target is a cylinder with an
widths.
open top and an inspection target at the bottom consisting of a
6.3.1 Barriers to Define the Robot Path—For the indoor
circular colored ring; the walls of the cylinder prevent the
tests, the barrier posts placed within the flight path must
target from being visible unless the robot’s downward-facing
provide visual guidance for the remote pilot to correctly
camera is above and approximately in line with it. The cylinder
traverse the defined figure-8 path. The barrier posts can be
measures 60 cm 6 20 cm tolerance [24 in. 6 8 in. tolerance]
made from any solid or porous material that provides visual
in diameter by 90 cm 6 20 cm tolerance [36 in. 6 8 in.
guidance. They should be sturdy and easily repaired or
tolerance] in height (for example, a 55 gal trash can), and the
replaced in case
...