Deep Reinforcement Learning for Swarm Systems

Recently, deep reinforcement learning (RL) methods have been applied successfully to multi-agent scenarios. Typically, these methods rely on a concatenation of agent states to represent the information content required for decentralized decision making. However, concatenation scales poorly to swarm systems with a large number of homogeneous agents as it does not exploit the fundamental properties inherent to these systems: (i) the agents in the swarm are interchangeable and (ii) the exact number of agents in the swarm is irrelevant. Therefore, we propose a new state representation for deep multi-agent RL based on mean embeddings of distributions. We treat the agents as samples of a distribution and use the empirical mean embedding as input for a decentralized policy. We define different feature spaces of the mean embedding using histograms, radial basis functions and a neural network learned end-to-end. We evaluate the representation on two well known problems from the swarm literature (rendezvous and pursuit evasion), in a globally and locally observable setup. For the local setup we furthermore introduce simple communication protocols. Of all approaches, the mean embedding representation using neural network features enables the richest information exchange between neighboring agents facilitating the development of more complex collective strategies.Comment: 31 pages, 12 figures, version 3 (published in JMLR Volume 20

An intelligent, free-flying robot

The ground based demonstration of the extensive extravehicular activity (EVA) Retriever, a voice-supervised, intelligent, free flying robot, is designed to evaluate the capability to retrieve objects (astronauts, equipment, and tools) which have accidentally separated from the Space Station. The major objective of the EVA Retriever Project is to design, develop, and evaluate an integrated robotic hardware and on-board software system which autonomously: (1) performs system activation and check-out; (2) searches for and acquires the target; (3) plans and executes a rendezvous while continuously tracking the target; (4) avoids stationary and moving obstacles; (5) reaches for and grapples the target; (6) returns to transfer the object; and (7) returns to base

Tele-autonomous systems: New methods for projecting and coordinating intelligent action at a distance

There is a growing need for humans to perform complex remote operations and to extend the intelligence and experience of experts to distant applications. It is asserted that a blending of human intelligence, modern information technology, remote control, and intelligent autonomous systems is required, and have coined the term tele-autonomous technology, or tele-automation, for methods producing intelligent action at a distance. Tele-automation goes beyond autonomous control by blending in human intelligence. It goes beyond tele-operation by incorporating as much autonomy as possible and/or reasonable. A new approach is discussed for solving one of the fundamental problems facing tele-autonomous systems: The need to overcome time delays due to telemetry and signal propagation. New concepts are introduced called time and position clutches, that allow the time and position frames between the local user control and the remote device being controlled, to be desynchronized respectively. The design and implementation of these mechanisms are described in detail. It is demonstrated that these mechanisms lead to substantial telemanipulation performance improvements, including the result of improvements even in the absence of time delays. The new controls also yield a simple protocol for control handoffs of manipulation tasks between local operators and remote systems

New concepts in tele-autonomous systems

Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/76226/1/AIAA-1987-1686-200.pd

Visual detection of point source targets

Visual detection of point source targets in simulated star field backgroun

Abstraction of analytical models from cognitive models of human control of robotic swarms

In order to formally validate cyber-physical systems, analytically tractable models of human control are desirable. While those models can be abstracted directly from human data, limitations on the amount and reliability of data can lead to over-fitting and lack of generalization. We introduce a methodology for deriving formal models of human control of cyberphysical systems based on the use of cognitive models. Analytical models such as Markov models can be derived from an instance-based learning model of the task built using the ACT-R cognitive architecture. The approach is illustrated in the context of a robotic control task involving the choice of two options to control a robotic swarm. The cognitive model and various forms of the analytical model are validated against each other and against human performance data. The current limitations of the approach are discussed as well as its implications for the automated validation of cyber-physical systems

Technology assessment of advanced automation for space missions

Six general classes of technology requirements derived during the mission definition phase of the study were identified as having maximum importance and urgency, including autonomous world model based information systems, learning and hypothesis formation, natural language and other man-machine communication, space manufacturing, teleoperators and robot systems, and computer science and technology

Advancing automation and robotics technology for the space station and for the US economy: Submitted to the United States Congress October 1, 1987

In April 1985, as required by Public Law 98-371, the NASA Advanced Technology Advisory Committee (ATAC) reported to Congress the results of its studies on advanced automation and robotics technology for use on the space station. This material was documented in the initial report (NASA Technical Memorandum 87566). A further requirement of the Law was that ATAC follow NASA's progress in this area and report to Congress semiannually. This report is the fifth in a series of progress updates and covers the period between 16 May 1987 and 30 September 1987. NASA has accepted the basic recommendations of ATAC for its space station efforts. ATAC and NASA agree that the mandate of Congress is that an advanced automation and robotics technology be built to support an evolutionary space station program and serve as a highly visible stimulator affecting the long-term U.S. economy

Space exploration: The interstellar goal and Titan demonstration

Automated interstellar space exploration is reviewed. The Titan demonstration mission is discussed. Remote sensing and automated modeling are considered. Nuclear electric propulsion, main orbiting spacecraft, lander/rover, subsatellites, atmospheric probes, powered air vehicles, and a surface science network comprise mission component concepts. Machine, intelligence in space exploration is discussed

The Flow Rendezvous: Where Flow and Reality Meet

Abstract This paper explores many published journal articles and one outstanding book which reported and conducted research on the phenomenon of flow. Flow captures the positive mental state of being completely absorbed, focused, and involved in your activities at a certain point in time, as well as having a feeling of enjoyment from being engaged in that activity. The articles detail the findings of the research conducted on flow in regards to the validity of flow, if it can be measured, how it could be measured, and various examples of the state. Each of the articles have a similar definition of the zone and describe it in a standardized manner. Most of the articles, as well as the book, The Rise of Superman: Decoding the Science of Ultimate Human Perforce, reference the works of Dr. Mihaly Csikszentmihalyi who pioneered the early research in this field. This paper examines the correlation between mind and body, and what is needed to reach this elusive state in order to produce one’s own and group peek performance. This paper will also point to additional research needed to better understand the phenomenon of flow and how it can be better utilized