Local Communication Protocols for Learning Complex Swarm Behaviors with Deep Reinforcement Learning

Swarm systems constitute a challenging problem for reinforcement learning (RL) as the algorithm needs to learn decentralized control policies that can cope with limited local sensing and communication abilities of the agents. While it is often difficult to directly define the behavior of the agents, simple communication protocols can be defined more easily using prior knowledge about the given task. In this paper, we propose a number of simple communication protocols that can be exploited by deep reinforcement learning to find decentralized control policies in a multi-robot swarm environment. The protocols are based on histograms that encode the local neighborhood relations of the agents and can also transmit task-specific information, such as the shortest distance and direction to a desired target. In our framework, we use an adaptation of Trust Region Policy Optimization to learn complex collaborative tasks, such as formation building and building a communication link. We evaluate our findings in a simulated 2D-physics environment, and compare the implications of different communication protocols.Comment: 13 pages, 4 figures, version 2, accepted at ANTS 201

An evolutionary algorithm for multi-robot unsupervised learning

Abstract -Based on evolutionary computation principles, an algorithm is presented for learning safe navigation of multiple robot systems. It is a basic step towards automatic generation of sensorimotor control architectures for completing complex cooperative tasks while using simple reactive mobile robots. Each individual estimates its own performance, without requiring any supervision. When two robots meet each other, the proposed crossover mechanism allows them to improve the mean performance index. In order to accelerate the evolution and prevent the population from staying in a local maximum, an adaptive selfmutation is added: the mutation rate is made dependent on the individual performance. Computer simulations and experiments using a team of real mobile robots have demonstrated the rapidity of convergence to the best-expected solution

A Decentralized Mobile Computing Network for Multi-Robot Systems Operations

Collective animal behaviors are paradigmatic examples of fully decentralized operations involving complex collective computations such as collective turns in flocks of birds or collective harvesting by ants. These systems offer a unique source of inspiration for the development of fault-tolerant and self-healing multi-robot systems capable of operating in dynamic environments. Specifically, swarm robotics emerged and is significantly growing on these premises. However, to date, most swarm robotics systems reported in the literature involve basic computational tasks---averages and other algebraic operations. In this paper, we introduce a novel Collective computing framework based on the swarming paradigm, which exhibits the key innate features of swarms: robustness, scalability and flexibility. Unlike Edge computing, the proposed Collective computing framework is truly decentralized and does not require user intervention or additional servers to sustain its operations. This Collective computing framework is applied to the complex task of collective mapping, in which multiple robots aim at cooperatively map a large area. Our results confirm the effectiveness of the cooperative strategy, its robustness to the loss of multiple units, as well as its scalability. Furthermore, the topology of the interconnecting network is found to greatly influence the performance of the collective action.Comment: Accepted for Publication in Proc. 9th IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conferenc

Cooperative Adaptive Control for Cloud-Based Robotics

This paper studies collaboration through the cloud in the context of cooperative adaptive control for robot manipulators. We first consider the case of multiple robots manipulating a common object through synchronous centralized update laws to identify unknown inertial parameters. Through this development, we introduce a notion of Collective Sufficient Richness, wherein parameter convergence can be enabled through teamwork in the group. The introduction of this property and the analysis of stable adaptive controllers that benefit from it constitute the main new contributions of this work. Building on this original example, we then consider decentralized update laws, time-varying network topologies, and the influence of communication delays on this process. Perhaps surprisingly, these nonidealized networked conditions inherit the same benefits of convergence being determined through collective effects for the group. Simple simulations of a planar manipulator identifying an unknown load are provided to illustrate the central idea and benefits of Collective Sufficient Richness.Comment: ICRA 201