NASA's Human Space Flight program depends heavily on spacewalks performed by pairs of suited human astronauts. These Extra-Vehicular Activities (EVAs) are severely restricted in both duration and scope by consumables and available manpower. An expanded multi-agent EVA team combining the information-gathering and problem-solving skills of humans with the survivability and physical capabilities of robots is proposed and illustrated by example. Such teams are useful for large-scale, complex missions requiring dispersed manipulation, locomotion and sensing capabilities. To study collaboration modalities within a multi-agent EVA team, a 1-g test is conducted with humans and robots working together in various supporting roles

Ambrose, Robert O.

Culbert, Christopher

Currie, Nancy

Rehnmark, Fredrik

NASA Technical Reports Server

Source of Acquisition 
NASA Johnson Space Center 
HUMAN-ROBOT TEAMING IN A MULTI-AGENT SPACE ASSEMBLY TASK 
FREDKIK KEHNMARK, LOCKHEED MARTIN SPACE OPERATIONS, USA 
frehnmar@ems.jsc,nasa.gov 
NANCY CURRIE, NASA JOHNSON SPACE CENTER, USA 
nancy.j.currie@nasa.gov 
ROBERT 0 .  AMBROSE, NASA JOHNSON SPACE CENTER, USA 
robert.o.ambrose@nasa.gov 
CHRISTOPHER CULBERT, NASA JOHNSON SPACE CENTER, USA 
christopher.j.culbert@nasa.gov 
ABSTRACT 
NASA's Human Space Flight program depends heavily on spacewalks perforaed by pairs of 
suited human astronauts. These Extra-Vehicular Activities (EVAs) are severely restricted in both 
duration and scope by consumables and available manpower. An expanded multi-agent EVA 
team combining the information-gathering and problem-solving skills of humans with the 
survivability and pl~ysical capabilities of robots is proposed and illustrated by example. Such 
teams are useful for large-scale, complex missions requiring dispersed manipulation, locomotion 
and sensing capabilities. To study collaboration modalities within a multi-agent EVA team, a l-g 
test is ccnducted with humans and robots working together in various supporting roles. 
KEYWORDS: Robonaut, Mini-AERCam, FAIR, multi-agent, EVA, NASA, DARPA 
I. INTRODUCTION 
The Hubble Space Telescope (HST) was carried into orbit in 1990 aboard the Space Slluttle 
Discovery (STS-3 1). Newer, high focal length designs are too large to fit in the Orbiter payload 
bay as a single unit. Instead, components of these space telescopes could launch separately and 
rendezvous in Low Earth Orbit (LEO) for assembly and final repositioning. These and other 
larger, lighter, more extendable space structures will require greatly expanded EVA and Extra- 
abilities as well as new and innovative structural systems. 
f highly dexterous space robots dramatically increases the 
robots working together in space. These machines can help 
conserve EVA hours by relieving human astronauts of many routine chores and assisting them in E /% 'b 
more complex tasks. Robots can take risks unacceptable to humans, perfonn contingency EVA 
operations in minutes, instead of hours, and setup worksites in preparation for the arrival of 
human astronauts. 
2. THE FAIR TELESCOP 
The proposed Filled-Aperture Infra-Red (FAIR) telescope is one example of the new 
generation of space science platforms requiring expanded EVAIEVR capabilities. Boasting an 
extremely long focal length, the FAIR design is too large and flimsy to be carried into orbit as a 
single pre-integrated assembly. Instead, the spacecraft subassembly and components of the 
telescope truss are launched aboard the Space Shuttle while the propulsion stage is launched <p&Le, 
separately on an expendable vehicle. 
After executing a Low Ealth Orbit (LEO) rendezvous with the propulsion stage, the EVA 3Fs. f- 
assembly team builds telescope truss segments from individual struts, ranging in length from 6 m &S ,& - 
to 13.6 In, which are stored side-by-side in the Shuttle payload bay. The resulting truss segments, 
up to 25 m in length, are, in turn, mated at both ends to attachment nodes on the partially- 5 ~ ~ 3  
https://ntrs.nasa.gov/search.jsp?R=20050220645 2019-08-29T20:57:23+00:00Z
assembled telescope truss. Once complete, the telescope truss is attached to the spacecraft 
subassembly using the Shuttle Remote Manipu!ator System (PMS). Next, the Shuttle approaches 
and berths the propulsion stage, to which the spacecraft is mated (Fig. 1). After final testing and 
inspection, the propulsion stage boosts the FAIR telescope from LEO to its final position at the 
L1 libration point. 
Figure I .  Fully-assembled FAIR telescope in Space SJmttie payload bay (adapfedfion~ [I]). 
The FAIR assembly concept [I] serves to illustrate the need for multi-agent teaming. The 
telescope truss consists of struts and nodes based on a modular EVA construction system 
developed at NASA's Langley Research Center and tested on STS-49 in the ASEM (Assembly of 
Station by EVA Methods) flight experiment. The FAIR telescope strut lengths and truss geometry 
naturally disperse the mechanical interfaces and EVA worksites across a huge volumetric 
workspace growing out of the Shuttle payload bay. In order to reduce point-to-point translation 
requirements between these worksites and simplifjr the handling of long, unwieldy structural 
components, the conventional EVA team consisting of two astronauts and an RMS is expanded to 
include the NASAIDARPA Robonaut [2] (a highly dexterous space robot) and the NASA Mini- 
AERCam [3] (a fiee-flying inspection nanosatellite) (Fig. 2). 
- - 
Figure 2. Men7bers of the expanded multi-agent team (left to rigl?t): astronaut I-iding ZgdS, 
astronaut at ASEMnode, Robonaut cliinbing along EVA handrail, Adini-AERCam fiee-fyer. 
As part of the truss assembly sequence, the Robonaut system (RVl) is positioned in the 
payload bay, unstowing and feeding struts up to an astronaut (EV!) positioned at a central node. 
Together, these two agents integrate the struts into truss segments, which are then mated to the 
central node (Fig. 3). The second astronaut (EV2) rides the RFvlS between peripherai nodes on the 
telescope truss and mates them to free ends of truss segments. The Mini-AERCam robot (RV2), 
meanwhile, provides critical teleoperation and diagnostic views while improving overall 
sitiiational awareness. 
Figure 3. Dispersed assembly agents, remote teain men7bers not shown (adaptedfiorn [I]). 
Over the course of multi-day EVA operations like the FAIR assembly procedure, robotic 
agents can continue working even after all humans have left the worksite. Robonaut could 
reconfigure worksites in preparation for the next EVA workday and perform other single-agent 
tasks. In the meantime, Mini-AERCam could visually inspect truss joints and verify critical 
alignments. 
Coordinating heterogeneous teams of humans and robots is not a trivial undertaking. The 
astronauts direct the robotic agents through physical interaction, indirectly via appropriate 
human-machine interfaces or by communicating with their remote huinan operators. The most 
flexible architecture allows for each agent to have multiple command paths and degrees of 
automation. For example, an astronaut teamed with a large-scale manipulator might want to fly it 
using hand controllers and spoken commands while riding the end-effector and then allow a 
remote huinan to take control when faced with a task that requires two hands. Similarly, control 
of the Mini-AERCain and its roving view could be passed between team inembers as required by 
the task. 
3. ONE-G MULTI-AGENT TESTING ACTIVITIES 
A 1-g hardware test based on the STS-61B ACCESS flight experiment [4] is conducted to 
evaluate human-robot teaming strategies in the context of a sin~ulated EVA assembly task. Four 
human test participants assume prescribed roles as members of a muiti-agent team featuring two 
NASAIDAWA Robonaut systems working side-by-side with a suited human subject to build an 
erectable truss on a rotating assembly fixture. Integrating some of the latest technology advances 
in high-mobility spacesuits, dexterous robots and modular structural systems, the Multi-agent 
Truss Assembly Test demonstrates new EVA capabilities enabling orbital assembly of large 
space structures. 
3.1 Objectives 
Key test objectives include: 
Demonstrate supporting capabilities applicable to in-space assembly using existing 
technoiogy prototypes and 1-g test beds. 
* Develop teaming strategies for EVA astronauts working side-by-side with highly 
dexterous, teleoperated robots. 
* Study the operational trade-offs inherent in human and robot teaming in a space 
assembly context. 
The Multi-agent Test was conducted to exercise collaboration modalities within a multi-agent 
team, not to assess the feasibility of a particular EVA task. The focus of the activity was to 
economically learn as much as possible about how heterogeneous teams of collocated and remote 
humans and robots can work together in space using available 1-g hardware and test beds. 
3.2 Description 
The Multi-agent Test is based on the STS-61B ACCESS flight experiment in which two EVA 
crewmembers built an erectable truss on an assembly fixture in the Space Shuttle payload bay. It 
features two Robonaut systems working side-by-side with astronaut Nancy Currie, wearing the 
experimental I-Suit, to build the Langley Cubic Truss in 1-g (Figure 4). Robonaut is an 
anthropomorphic robotic system capable of using tools and handling mecllanicai interfaces 
designed for humans. The I-Suit is a high field of view, high mobility, lightweight prototype soft 
suit developed by ILC-Dover. The Langley Cubic Truss is a stiff, lightweight structural system 
designed for EVA assembly. Truss assembly is facilitated with a low-fidelity, rotating assembly 
fixture and other supporting equipment. 
Figure 4. Multi-agei7t Test initial conJ'igzlratio~?, as seen by Ground Controller via Mini-kE2Canz 
Working together, the two humanoid class robots (RVI and RV2), controlled by remote 
human teleoperators, serve in various roles supporting the astronaut (EVI), who operates as team 
leader. A ground controller (IVl), stationed at a simulated mission console, is tasked with 
directing the EVA over a wireless colnmunication system and dictates the procedural details to 
each of,the assembly agents. An emulation of the Mini-AERCain (Autonomous Extravehicular 
Robotic Camera), a free-flying nanosatellite outfitted with video cameras, is positioned adjacent 
to'-the worksite, providing the ground controller with a "bird's eye" view of the worksite afid 
assembly team during the course of construction. 
Initially, the truss consists of a single cubic bay, which is translated up along the assembly 
fixture to make room for the addition of a second bay beneath it. The struts forming the truss are 
housed in two quivers located near EVl and RVI while the nodes are contained in trays. These 
two assembly agents unstow struts and nodes from their containers and cooperatively assemble 
them to form segments. EVl then attaches these segments to the truss. The third assembly agent, 
RV2, rotates and stabilizes the assembly fixture as needed (Figure 5). 
Figure 5. Selected elements of the multi-agent iest (lest-io-righQ: RVI attaches a node io a strut 
supported by EVl, RV2 stabilizes the fr.ziss assembly as EVI attaches a segment to the truss, RVI 
passes a sfrut to RV2 as EVl rotates the truss assembly. 
The assembly procedure requires a high tier working position to mate connections along the 
top edges of a truss bay as well as a low tier position to access the bottom edges. This also allows 
the astronaut to interact directly with both robots, though not at the same time. Once all the high 
tier connections are locked, EV1 translates to the low tier position, beginning the second half of 
the task. To complete the truss bay, RVl unstows struts and passes them to RV2, who, in turn, 
transfers them to EV1. EVI then attaches them to the truss (Figure 5). 
Once all the low tier connections are locked, the truss bay is structurally complete and the 
team begins installing an umbilical cable terminating in EVA electrical connectors around the 
exterior of the truss. An EVA cable tie caddy is first unstowed and passed from RV1 to RV2 to 
EV1, who attaches it temporarily to the truss. Next, the umbilical is unstowed and routed around 
the truss to EV1, who plugs the connector into a waiting socket on the assembly fixture. As EVl 
rotates the fixture, RVI and RV2 uncoil the cable and hold it in place against the truss, allowing 
EVl to apply a cable tie at each comer. Once all the cable has been deployed and secured, EV1 
plugs the remaining connector into a second socket on the assembly fixture. At this point, the 
assembly task is complete and task time stops. 
A mix of manipulation and teaming skills are required to complete the truss assembly task. 
Truss assembly agents must not only be capable of mating nodes and struts, they must also be 
able to coordinate cooperative manipulation, hand-offs and other mu!ti-agent interactions in the 
pre-planned assembly sequence. 
The topics explored during this initial investigation include collaboration modalities between 
EVA and remote humans and skill mappings of dexterous robots and human workers. Timeline 
optimization is an important aspect of EVA planning examining both workload distribution 
between crewmembers and the sequence of tasks to be performed. The overarching objective in 
planning the activity is to employ the human EVA crewmember(s) as safely and efficiently as 
possible, thereby conserving consumables and limiting exposure to the hostile space environment. 
3.3 Observations 
in order to compiete the assembly task within the E'VA window (about I hour) aiiowed by 
EVl's breathing air supply, much of the workload fell on the astronaut. Although the robots were 
capable of performing nearly every subtask on their own, they would have taken more time. In 
order to be practical, the multi-agent team must be able to perform its task in less time (as 
measured in EVA astronaut-hours) than the conventional team consisting of two astronauts. An 
interesting exception is the multi-agent team consisting only of collocated robots and remote 
humans. Depending on the particular EVA task in question, there may be other considerations as 
well, such as risk mitigation and physical limitations. 
Spacewalking astronauts have a very limited field-of-view restricted to the window in the 
EMU helmet, which does not swivel with neck motions. In general, two astronauts working side- 
by-side on an EVA cannot see each other. They are unable to communicate tllrough body 
language or gestures and cannot anticipate each other's actions through observation. By necessity, 
EV1 and EV2 communicate almost exclusively by radio, employing very methodical 
handshaking to confirm mutual understanding. For similar reasons, participants in the Multi-agent 
Test developed a wireless headset comlnunication protocol to help them function as a team. 
The EMU encumbers t!le body motions of an EEVA worker. Spacewalking as?ronauts have a 
restricted working envelope dictated by the EMU range of motion. The EMU glove also degrades 
the tactile sensing of the wearer. To a degree, the degraded tactile feedback and manual dexterity 
suffered by the human subject wearing the spacesuit and performing the task adjacently parallels 
the sensorimotor handicap suffered by the human controlling the robot and performing the task 
remotely. 
4. CONCLUSIONS 
The Multi-agent Test should be understood as part of a continuing effort to study how 
l~u~nans and robots can work together effectively in space. At the same time, these experiments 
test the limits of robotics and teleoperation, demonstrating new EVR capabilities and the 
feasibility of performing more tasks telerobotically. Higher-fidelity tests involving Inore 
sophisticated gravity compensation and more realistic mobility, communication time delay, 
lighting conditions, etc. will complicate some operational aspects of EVR but simplify others. 
The multi-agent team featured in this test represents only one particular instance of humans 
and robots working together. It represents a novel integration of existing technology prototypes 
and test beds intended to meet test objectives but should only be interpreted as part of the solution 
to increasing EVA capability and productivity. More diversified multi-agent teams, improved 
EVA technology and new vehicles will all be required to meet future needs. 
5. REFERENCES 
[ l ]  Geffre, Jim et al., "Filled-Aperture Infrared (FAIR) Telescope Assembly," Exploration Office 
Internal Report (NASAIJSC), Houston, Texas, December 2001. 
[2] Ambrose, Robert 0. et al., "Robonaut: NASA's Space Humanoid," IEEE Intelligent Systems 
and Their Applications, Vol. 15, No. 4, pp. 57-63. July, 2000. 
[3] Fredrickson, Steven et al., "Mini AERCarn: Development of a Free-Flying Nanosatellite 
Inspection Robot," Proceedings of SPIE, Val. 5088, pp. 97-1 11, August 2003. 
[4] Heard, Walter and Watson, Judith, "Results of the ACCESS Experiment," Proceedings of the 
Space Construction Conference (NASAILaRC), Hampton, Virginia, August 1986. 


Human-Robot Teaming in a Multi-Agent Space Assembly Task

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server