Local Communication Protocols for Learning Complex Swarm Behaviors with Deep Reinforcement Learning

Swarm systems constitute a challenging problem for reinforcement learning (RL) as the algorithm needs to learn decentralized control policies that can cope with limited local sensing and communication abilities of the agents. While it is often difficult to directly define the behavior of the agents, simple communication protocols can be defined more easily using prior knowledge about the given task. In this paper, we propose a number of simple communication protocols that can be exploited by deep reinforcement learning to find decentralized control policies in a multi-robot swarm environment. The protocols are based on histograms that encode the local neighborhood relations of the agents and can also transmit task-specific information, such as the shortest distance and direction to a desired target. In our framework, we use an adaptation of Trust Region Policy Optimization to learn complex collaborative tasks, such as formation building and building a communication link. We evaluate our findings in a simulated 2D-physics environment, and compare the implications of different communication protocols.Comment: 13 pages, 4 figures, version 2, accepted at ANTS 201

Subsumption architecture for enabling strategic coordination of robot swarms in a gaming scenario

The field of swarm robotics breaks away from traditional research by maximizing the performance of a group - swarm - of limited robots instead of optimizing the intelligence of a single robot. Similar to current-generation strategy video games, the player controls groups of units - squads - instead of the individual participants. These individuals are rather unintelligent robots, capable of little more than navigating and using their weapons. However, clever control of the squads of autonomous robots by the game players can make for intense, strategic matches. The gaming framework presented in this article provides players with strategic coordination of robot squads. The developed swarm intelligence techniques break up complex squad commands into several commands for each robot using robot formations and path finding while avoiding obstacles. These algorithms are validated through a 'Capture the Flag' gaming scenario where a complex squad command is split up into several robot commands in a matter of milliseconds

Physical interactions in swarm robotics: the hand-bot case study

This paper presents a case-study on the performance achieved by the mechanical interactions of self-assembling mobile robots. This study is based on the hand-bot robot, designed to operate within heterogeneous swarms of robots. The hand-bot is specialized in object manipulation and can improve its performance by exploiting physical collaborations by self-assembling with other hand-bots or with foot-bots (ground robots). The paper analyzes the achieved performance and demonstrates the highly super-linear properties of the accessible volume in respect to the number of robots. These extremely interesting performances are strongly linked to the self-assembling mechanisms and the physical nature of the interaction, and do not scale to a large number of robots. Finally, this study suggests that such interesting properties are more accessible for heterogeneous systems or devices achieving complex tasks

Quality-sensitive foraging by a robot swarm through virtual pheromone trails

Large swarms of simple autonomous robots can be employed to find objects clustered at random locations, and transport them to a central depot. This solution offers system parallelisation through concurrent environment exploration and object collection by several robots, but it also introduces the challenge of robot coordination. Inspired by ants’ foraging behaviour, we successfully tackle robot swarm coordination through indirect stigmergic communication in the form of virtual pheromone trails. We design and implement a robot swarm composed of up to 100 Kilobots using the recent technology Augmented Reality for Kilobots (ARK). Using pheromone trails, our memoryless robots rediscover object sources that have been located previously. The emerging collective dynamics show a throughput inversely proportional to the source distance. We assume environments with multiple sources, each providing objects of different qualities, and we investigate how the robot swarm balances the quality-distance trade-off by using quality-sensitive pheromone trails. To our knowledge this work represents the largest robotic experiment in stigmergic foraging, and is the first complete demonstration of ARK, showcasing the set of unique functionalities it provides

Modelling and Verification of Timed Robotic Controllers

Designing robotic systems can be very challenging, yet controllers are often specified using informal notations with development driven primarily by simulations and physical experiments, without relation to abstract models of requirements. The ability to perform formal analysis and replicate results across different robotic platforms is hindered by the lack of well-defined formal notations. In this paper we present a timed state-machine based formal notation for robotics that is informed by current practice. We motivate our work with an example from swarm robotics and define a compositional CSP-based discrete timed semantics suitable for refinement. Our results support verification and, importantly, enable rigorous connection with sound simulations and deployments.</p

Opinion Dynamics for Decentralized Decision-Making in a Robot Swarm

Swarm Intelligence: 7th International Conference, ANTS 2010 (Brussels, Belgium, September 8-10, 2010)info:eu-repo/semantics/publishe

Four Exercises in Programming Dynamic Reconfigurable Systems: Methodology and Solution in DR-BIP

International audienceDR-BIP is an extension of the BIP component framework intended for programming reconfigurable systems encompassing various aspects of dynamism. A system is built from instances of types of components characterized by their interfaces. The latter consist of sets of ports through which data can be exchanged when interactions take place. DR-BIP allows the description of parametric exogenous interactions and reconfiguration operations. To naturally model self-organization and mobility of components, a system is composed of several architecture motifs, each motif consisting of a set of component instances and coordination rules. The use of motifs allows a disciplined management of dynamically changing coordination rules. The paper illustrates the basic concepts of DR-BIP through a collection of four non-trivial exercises from different application areas: fault-tolerant systems, mobile systems and autonomous systems. The presented solutions show that DR-BIP is both minimal and expressive allowing concise and natural description of non-trivial systems

Improving the Performance of Challenged Networks with Controlled Mobility

International audienceIn this work, we investigate the application of an adapted controlled mobility strategy on self-propelling nodes, which could efficiently provide network resource to users scattered on a designated area. We design a virtual force-based controlled mobility scheme, named VFPc, and evaluate its ability to be jointly used with a dual packet-forwarding and epidemic routing protocol. In particular, we study the possibility for end-users to achieve synchronous communications at given times of the considered scenarios. On this basis, we study the delay distribution for such user traffic and show the advantages of VFPc compared to other packet-forwarding and packet-replication schemes, and highlight that VFPc-enabled applications could take benefit of both schemes to yield a better user experience, despite challenging network conditions