Beyond Reynolds: A Constraint-Driven Approach to Cluster Flocking

In this paper, we present an original set of flocking rules using an ecologically-inspired paradigm for control of multi-robot systems. We translate these rules into a constraint-driven optimal control problem where the agents minimize energy consumption subject to safety and task constraints. We prove several properties about the feasible space of the optimal control problem and show that velocity consensus is an optimal solution. We also motivate the inclusion of slack variables in constraint-driven problems when the global state is only partially observable by each agent. Finally, we analyze the case where the communication topology is fixed and connected, and prove that our proposed flocking rules achieve velocity consensus.Comment: 6 page

Learning for Multi-robot Cooperation in Partially Observable Stochastic Environments with Macro-actions

This paper presents a data-driven approach for multi-robot coordination in partially-observable domains based on Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) and macro-actions (MAs). Dec-POMDPs provide a general framework for cooperative sequential decision making under uncertainty and MAs allow temporally extended and asynchronous action execution. To date, most methods assume the underlying Dec-POMDP model is known a priori or a full simulator is available during planning time. Previous methods which aim to address these issues suffer from local optimality and sensitivity to initial conditions. Additionally, few hardware demonstrations involving a large team of heterogeneous robots and with long planning horizons exist. This work addresses these gaps by proposing an iterative sampling based Expectation-Maximization algorithm (iSEM) to learn polices using only trajectory data containing observations, MAs, and rewards. Our experiments show the algorithm is able to achieve better solution quality than the state-of-the-art learning-based methods. We implement two variants of multi-robot Search and Rescue (SAR) domains (with and without obstacles) on hardware to demonstrate the learned policies can effectively control a team of distributed robots to cooperate in a partially observable stochastic environment.Comment: Accepted to the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2017

Deaf, Dumb, and Chatting Robots, Enabling Distributed Computation and Fault-Tolerance Among Stigmergic Robot

We investigate ways for the exchange of information (explicit communication) among deaf and dumb mobile robots scattered in the plane. We introduce the use of movement-signals (analogously to flight signals and bees waggle) as a mean to transfer messages, enabling the use of distributed algorithms among the robots. We propose one-to-one deterministic movement protocols that implement explicit communication. We first present protocols for synchronous robots. We begin with a very simple coding protocol for two robots. Based on on this protocol, we provide one-to-one communication for any system of n \geq 2 robots equipped with observable IDs that agree on a common direction (sense of direction). We then propose two solutions enabling one-to-one communication among anonymous robots. Since the robots are devoid of observable IDs, both protocols build recognition mechanisms using the (weak) capabilities offered to the robots. The first protocol assumes that the robots agree on a common direction and a common handedness (chirality), while the second protocol assumes chirality only. Next, we show how the movements of robots can provide implicit acknowledgments in asynchronous systems. We use this result to design asynchronous one-to-one communication with two robots only. Finally, we combine this solution with the schemes developed in synchronous settings to fit the general case of asynchronous one-to-one communication among any number of robots. Our protocols enable the use of distributing algorithms based on message exchanges among swarms of Stigmergic robots. Furthermore, they provides robots equipped with means of communication to overcome faults of their communication device

Near-Optimal Adversarial Policy Switching for Decentralized Asynchronous Multi-Agent Systems

A key challenge in multi-robot and multi-agent systems is generating solutions that are robust to other self-interested or even adversarial parties who actively try to prevent the agents from achieving their goals. The practicality of existing works addressing this challenge is limited to only small-scale synchronous decision-making scenarios or a single agent planning its best response against a single adversary with fixed, procedurally characterized strategies. In contrast this paper considers a more realistic class of problems where a team of asynchronous agents with limited observation and communication capabilities need to compete against multiple strategic adversaries with changing strategies. This problem necessitates agents that can coordinate to detect changes in adversary strategies and plan the best response accordingly. Our approach first optimizes a set of stratagems that represent these best responses. These optimized stratagems are then integrated into a unified policy that can detect and respond when the adversaries change their strategies. The near-optimality of the proposed framework is established theoretically as well as demonstrated empirically in simulation and hardware

Reinforcement Learning for Mobile Robot Collision Avoidance in Navigation Tasks

Collision avoidance is fundamental for mobile robot navigation. In general, its solutions include: {\it map-based} and {\it mapless approaches.} In the map-based approach, robots pre-plan collision-free paths based on an environment map and follow their paths during navigation. On the other hand, the mapless approach requires robots to avoid collisions without referencing to an environment map. This thesis first studies the map-based approach for multiple robots to collectively build environment maps. In this study, a robot following a pre-planned path may encounter unexpected obstacles, such as other moving robots and obstacles inaccurately presented on an environment map. This motivates us to study mapless collision avoidance in the second part of the thesis. Mapless collision avoidance requires a robot to infer an optimal action based on sensor data and operate in real time. Inferring an optimal action in a timely manner is computationally expensive, particularly when a robot has limited on-board computing resources. To avoid the expensive online action inferring, this thesis presents a reinforcement learning approach which learns policies for mapless collision avoidance under real-world settings. We first propose a Real-Time Actor-Critic Architecture (RTAC) to support asynchronous reinforcement learning under real-time constraint. Based on RTAC, we propose asynchronous reinforcement learning methods for mapless collision avoidance of various numbers of robots under different environment configurations. Through extensive experiments, we demonstrate that RTAC serves as a solid foundation to support multi-task and multi-agent learning for mapless collision avoidance under asynchronous settings

Coordination of Multirobot Systems Under Temporal Constraints

Multirobot systems have great potential to change our lives by increasing efficiency or decreasing costs in many applications, ranging from warehouse logistics to construction. They can also replace humans in dangerous scenarios, for example in a nuclear disaster cleanup mission. However, teleoperating robots in these scenarios would severely limit their capabilities due to communication and reaction delays. Furthermore, ensuring that the overall behavior of the system is safe and correct for a large number of robots is challenging without a principled solution approach. Ideally, multirobot systems should be able to plan and execute autonomously. Moreover, these systems should be robust to certain external factors, such as failing robots and synchronization errors and be able to scale to large numbers, as the effectiveness of particular tasks might depend directly on these criteria. This thesis introduces methods to achieve safe and correct autonomous behavior for multirobot systems. Firstly, we introduce a novel logic family, called counting logics, to describe the high-level behavior of multirobot systems. Counting logics capture constraints that arise naturally in many applications where the identity of the robot is not important for the task to be completed. We further introduce a notion of robust satisfaction to analyze the effects of synchronization errors on the overall behavior and provide complexity analysis for a fragment of this logic. Secondly, we propose an optimization-based algorithm to generate a collection of robot paths to satisfy the specifications given in counting logics. We assume that the robots are perfectly synchronized and use a mixed-integer linear programming formulation to take advantage of the recent advances in this field. We show that this approach is complete under the perfect synchronization assumption. Furthermore, we propose alternative encodings that render more efficient solutions under certain conditions. We also provide numerical results that showcase the scalability of our approach, showing that it scales to hundreds of robots. Thirdly, we relax the perfect synchronization assumption and show how to generate paths that are robust to bounded synchronization errors, without requiring run-time communication. However, the complexity of such an approach is shown to depend on the error bound, which might be limiting. To overcome this issue, we propose a hierarchical method whose complexity does not depend on this bound. We show that, under mild conditions, solutions generated by the hierarchical method can be executed safely, even if such a bound is not known. Finally, we propose a distributed algorithm to execute multirobot paths while avoiding collisions and deadlocks that might occur due to synchronization errors. We recast this problem as a conflict resolution problem and characterize conditions under which existing solutions to the well-known drinking philosophers problem can be used to design control policies that prevents collisions and deadlocks. We further provide improvements to this naive approach to increase the amount of concurrency in the system. We demonstrate the effectiveness of our approach by comparing it to the naive approach and to the state-of-the-art.PHDElectrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/162921/1/ysahin_1.pd