Local Communication Protocols for Learning Complex Swarm Behaviors with Deep Reinforcement Learning

Swarm systems constitute a challenging problem for reinforcement learning (RL) as the algorithm needs to learn decentralized control policies that can cope with limited local sensing and communication abilities of the agents. While it is often difficult to directly define the behavior of the agents, simple communication protocols can be defined more easily using prior knowledge about the given task. In this paper, we propose a number of simple communication protocols that can be exploited by deep reinforcement learning to find decentralized control policies in a multi-robot swarm environment. The protocols are based on histograms that encode the local neighborhood relations of the agents and can also transmit task-specific information, such as the shortest distance and direction to a desired target. In our framework, we use an adaptation of Trust Region Policy Optimization to learn complex collaborative tasks, such as formation building and building a communication link. We evaluate our findings in a simulated 2D-physics environment, and compare the implications of different communication protocols.Comment: 13 pages, 4 figures, version 2, accepted at ANTS 201

Multi-Agent Based Simulation of an Unmanned Aerial Vehicles System

The rapid growth of using Unmanned Aerial Vehicles (UAV) for civilian and military applications has promoted the development of research in many areas. Most of the unmanned aerial vehicles in use are manually controlled. Often, UAVs require highly trained pilot operators. Hence, the main challenge faced by researchers has been to make UAVs autonomous or semiautonomous. The goal of this research project is to develop and implement a simulation for a user-defined environment allowing UAVs to maneuver in free environments and obstacle-laden environments using Boid's algorithm of flocking with obstacle avoidance. The users are permitted to analyze the maneuvering area and coverage efficiency of the UAVs and to dynamically change environments. This project makes use of Boid's flocking algorithm to generate different kinds of movements for the flying agents, enabling the user to analyze the effectiveness of patrolling in that particular scenario. The number of UAVs and the type of environment are set by the user. The set number of UAVs moves as a flock or swarm inside the set environment by using Boid's rules of flocking: cohesion, alignment, and separation. The coverage efficiency of the UAVs in that particular environment is reported based on the ratio between the area covered and the time when the search time reaches a threshold. The advantages and feasibilities of the approach are discussed with the simulation results