Deep Reinforcement Learning for Swarm Systems

Recently, deep reinforcement learning (RL) methods have been applied successfully to multi-agent scenarios. Typically, these methods rely on a concatenation of agent states to represent the information content required for decentralized decision making. However, concatenation scales poorly to swarm systems with a large number of homogeneous agents as it does not exploit the fundamental properties inherent to these systems: (i) the agents in the swarm are interchangeable and (ii) the exact number of agents in the swarm is irrelevant. Therefore, we propose a new state representation for deep multi-agent RL based on mean embeddings of distributions. We treat the agents as samples of a distribution and use the empirical mean embedding as input for a decentralized policy. We define different feature spaces of the mean embedding using histograms, radial basis functions and a neural network learned end-to-end. We evaluate the representation on two well known problems from the swarm literature (rendezvous and pursuit evasion), in a globally and locally observable setup. For the local setup we furthermore introduce simple communication protocols. Of all approaches, the mean embedding representation using neural network features enables the richest information exchange between neighboring agents facilitating the development of more complex collective strategies.Comment: 31 pages, 12 figures, version 3 (published in JMLR Volume 20

Robust Distance-Based Formation Control of Multiple Rigid Bodies with Orientation Alignment

This paper addresses the problem of distance- and orientation-based formation control of a class of second-order nonlinear multi-agent systems in 3D space, under static and undirected communication topologies. More specifically, we design a decentralized model-free control protocol in the sense that each agent uses only local information from its neighbors to calculate its own control signal, without incorporating any knowledge of the model nonlinearities and exogenous disturbances. Moreover, the transient and steady state response is solely determined by certain designer-specified performance functions and is fully decoupled by the agents' dynamic model, the control gain selection, the underlying graph topology as well as the initial conditions. Additionally, by introducing certain inter-agent distance constraints, we guarantee collision avoidance and connectivity maintenance between neighboring agents. Finally, simulation results verify the performance of the proposed controllers.Comment: IFAC Word Congress 201

Advanced transport operating system software upgrade: Flight management/flight controls software description

The Flight Management/Flight Controls (FM/FC) software for the Norden 2 (PDP-11/70M) computer installed on the NASA 737 aircraft is described. The software computes the navigation position estimates, guidance commands, those commands to be issued to the control surfaces to direct the aircraft in flight based on the modes selected on the Advanced Guidance Control System (AGSC) mode panel, and the flight path selected via the Navigation Control/Display Unit (NCDU)

Safe, Remote-Access Swarm Robotics Research on the Robotarium

This paper describes the development of the Robotarium -- a remotely accessible, multi-robot research facility. The impetus behind the Robotarium is that multi-robot testbeds constitute an integral and essential part of the multi-agent research cycle, yet they are expensive, complex, and time-consuming to develop, operate, and maintain. These resource constraints, in turn, limit access for large groups of researchers and students, which is what the Robotarium is remedying by providing users with remote access to a state-of-the-art multi-robot test facility. This paper details the design and operation of the Robotarium as well as connects these to the particular considerations one must take when making complex hardware remotely accessible. In particular, safety must be built in already at the design phase without overly constraining which coordinated control programs the users can upload and execute, which calls for minimally invasive safety routines with provable performance guarantees.Comment: 13 pages, 7 figures, 3 code samples, 72 reference

A Framework for Collaborative Multi-task, Multi-robot Missions

Robotics is a transformative technology that will empower our civilization for a new scale of human endeavors. Massive scale is only possible through the collaboration of individual or groups of robots. Collaboration allows specialization, meaning a multirobot system may accommodate heterogeneous platforms including human partners. This work develops a unified control architecture for collaborative missions comprised of multiple, multi-robot tasks. Using kinematic equations and Jacobian matrices, the system states are transformed into alternative control spaces which are more useful for the designer or more convenient for the operator. The architecture allows multiple tasks to be combined, composing tightly coordinated missions. Using this approach, the designer is able to compensate for non-ideal behavior in the appropriate space using whatever control scheme they choose. This work presents a general design methodology, including analysis techniques for relevant control metrics like stability, responsiveness, and disturbance rejection, which were missing in prior work. Multiple tasks may be combined into a collaborative mission. The unified motion control architecture merges the control space components for each task into a concise federated system to facilitate analysis and implementation. The task coordination function defines task commands as functions of mission commands and state values to create explicit closed-loop collaboration. This work presents analysis techniques to understand the effects of cross-coupling tasks. This work analyzes system stability for the particular control architecture and identifies an explicit condition to ensure stable switching when reallocating robots. We are unaware of any other automated control architectures that address large-scale collaborative systems composed of task-oriented multi-robot coalitions where relative spatial control is critical to mission performance. This architecture and methodology have been validated in experiments and in simulations, repeating earlier work and exploring new scenarios and. It can perform large-scale, complex missions via a rigorous design methodology

Nonlinear control of nonholonomic mobile robot formations

In this thesis, the framework developed to control a single nonholonomic mobile robot is expanded to include the control of formations of multiple nonholonomic mobile robots. A combined kinematic/torque control law is developed for leader-follower based formation control using backstepping in order to accommodate the dynamics of the robots and the formation in contrast with kinematic-based formation controllers typically found in literature --Abstract, page iv

A Low-Cost Experimental Testbed for Multi-Agent System Coordination Control

A multi-agent system can be defined as a coordinated network of mobile, physical agents that execute complex tasks beyond their individual capabilities. Observations of biological multi-agent systems in nature reveal that these ``super-organisms” accomplish large scale tasks by leveraging the inherent advantages of a coordinated group. With this in mind, such systems have the potential to positively impact a wide variety of engineering applications (e.g. surveillance, self-driving cars, and mobile sensor networks). The current state of research in the area of multi-agent systems is quickly evolving from the theoretical development of coordination control algorithms and their computer simulations to experimental validations on proof-of-concept testbeds using small-scale mobile robotic platforms. An in-house testbed would allow for rapid prototyping and validation of control algorithms, and potentially lead to new research directions spawned by experimentally-observed issues. To this end, a custom experimental testbed, TIGER Square, has been designed, developed, built, and tested at Louisiana State University. In this work, the completed design and test results for a centralized testbed is presented. That is, the individual robots follow an overarching control entity and are reliant on a global structure, such as a central processing computer. As part of the validation process, a series of formation control experiments were executed to assess the performance of the testbed. In order to eliminate single-point failures, a multi-agent system must be fully decentralized or distributed. This means that the responsibilities of processing, localization, and communication are distributed to each agent. Therefore, this work concludes with the introduction of a prototype localization module that will be integrated into the existing centralized testbed. This initial step allows for the future decentralization of TIGER Square and opens the path to achieve a fully capable multi-agent system testbed