Fully Distributed Multi-Robot Collision Avoidance via Deep Reinforcement Learning for Safe and Efficient Navigation in Complex Scenarios

In this paper, we present a decentralized sensor-level collision avoidance policy for multi-robot systems, which shows promising results in practical applications. In particular, our policy directly maps raw sensor measurements to an agent's steering commands in terms of the movement velocity. As a first step toward reducing the performance gap between decentralized and centralized methods, we present a multi-scenario multi-stage training framework to learn an optimal policy. The policy is trained over a large number of robots in rich, complex environments simultaneously using a policy gradient based reinforcement learning algorithm. The learning algorithm is also integrated into a hybrid control framework to further improve the policy's robustness and effectiveness. We validate the learned sensor-level collision avoidance policy in a variety of simulated and real-world scenarios with thorough performance evaluations for large-scale multi-robot systems. The generalization of the learned policy is verified in a set of unseen scenarios including the navigation of a group of heterogeneous robots and a large-scale scenario with 100 robots. Although the policy is trained using simulation data only, we have successfully deployed it on physical robots with shapes and dynamics characteristics that are different from the simulated agents, in order to demonstrate the controller's robustness against the sim-to-real modeling error. Finally, we show that the collision-avoidance policy learned from multi-robot navigation tasks provides an excellent solution to the safe and effective autonomous navigation for a single robot working in a dense real human crowd. Our learned policy enables a robot to make effective progress in a crowd without getting stuck. Videos are available at https://sites.google.com/view/hybridmrc

Decentralized Rendezvous of Nonholonomic Robots with Sensing and Connectivity Constraints

A group of wheeled robots with nonholonomic constraints is considered to rendezvous at a common specified setpoint with a desired orientation while maintaining network connectivity and ensuring collision avoidance within the robots. Given communication and sensing constraints for each robot, only a subset of the robots are aware or informed of the global destination, and the remaining robots must move within the network connectivity constraint so that the informed robots can guide the group to the goal. The mobile robots are also required to avoid collisions with each other outside a neighborhood of the common rendezvous point. To achieve the rendezvous control objective, decentralized time-varying controllers are developed based on a navigation function framework to steer the robots to perform rendezvous while preserving network connectivity and ensuring collision avoidance. Only local sensing feedback, which includes position feedback from immediate neighbors and absolute orientation measurement, is used to navigate the robots and enables radio silence during navigation. Simulation results demonstrate the performance of the developed approach.Comment: 9 pages, 5 figures, submitted to Automatic

Realtime Collision Avoidance for Mobile Robots in Dense Crowds using Implicit Multi-sensor Fusion and Deep Reinforcement Learning

We present a novel learning-based collision avoidance algorithm, CrowdSteer, for mobile robots operating in dense and crowded environments. Our approach is end-to-end and uses multiple perception sensors such as a 2-D lidar along with a depth camera to sense surrounding dynamic agents and compute collision-free velocities. Our training approach is based on the sim-to-real paradigm and uses high fidelity 3-D simulations of pedestrians and the environment to train a policy using Proximal Policy Optimization (PPO). We show that our learned navigation model is directly transferable to previously unseen virtual and dense real-world environments. We have integrated our algorithm with differential drive robots and evaluated its performance in narrow scenarios such as dense crowds, narrow corridors, T-junctions, L-junctions, etc. In practice, our approach can perform real-time collision avoidance and generate smooth trajectories in such complex scenarios. We also compare the performance with prior methods based on metrics such as trajectory length, mean time to goal, success rate, and smoothness and observe considerable improvement.Comment: 8 pages, 7 figure

3-D Reciprocal Collision Avoidance on Physical Quadrotor Helicopters with On-Board Sensing for Relative Positioning

In this paper, we present an implementation of 3-D reciprocal collision avoidance on real quadrotor helicopters where each quadrotor senses the relative position and velocity of other quadrotors using an on-board camera. We show that using our approach, quadrotors are able to successfully avoid pairwise collisions in GPS and motion-capture denied environments, without communication between the quadrotors, and even when human operators deliberately attempt to induce collisions. To our knowledge, this is the first time that reciprocal collision avoidance has been successfully implemented on real robots where each agent independently observes the others using on-board sensors. We theoretically analyze the response of the collision-avoidance algorithm to the violated assumptions by the use of real robots. We quantitatively analyze our experimental results. A particularly striking observation is that at times the quadrotors exhibit "reciprocal dance" behavior, which is also observed when humans move past each other in constrained environments. This seems to be the result of sensing uncertainty, which causes both robots involved to have a different belief about the relative positions and velocities and, as a result, choose the same side on which to pass.Comment: 8 pages, 9 Figures, 1 Tabl

Navigation Function Based Decentralized Control of A Multi-Agent System with Network Connectivity Constraints

A wide range of applications require or can benefit from collaborative behavior of a group of agents. The technical challenge addressed in this chapter is the development of a decentralized control strategy that enables each agent to independently navigate to ensure agents achieve a collective goal while maintaining network connectivity. Specifically, cooperative controllers are developed for networked agents with limited sensing and network connectivity constraints. By modeling the interaction among the agents as a graph, several different approaches to address the problems of preserving network connectivity are presented, with the focus on a method that utilizes navigation function frameworks. By modeling network connectivity constraints as artificial obstacles in navigation functions, a decentralized control strategy is presented in two particular applications, formation control and rendezvous for a system of autonomous agents, which ensures global convergence to the unique minimum of the potential field (i.e., desired formation or desired destination) while preserving network connectivity. Simulation results are provided to demonstrate the developed strategy.Comment: 16 pages, 9 figures, submitted to NATO Science for Peace and Security Series by IOS Press. arXiv admin note: substantial text overlap with arXiv:1402.563

Particle Swarm Optimization Based Source Seeking

Signal source seeking using autonomous vehicles is a complex problem. The complexity increases manifold when signal intensities captured by physical sensors onboard are noisy and unreliable. Added to the fact that signal strength decays with distance, noisy environments make it extremely difficult to describe and model a decay function. This paper addresses our work with seeking maximum signal strength in a continuous electromagnetic signal source with mobile robots, using Particle Swarm Optimization (PSO). A one to one correspondence with swarm members in a PSO and physical Mobile robots is established and the positions of the robots are iteratively updated as the PSO algorithm proceeds forward. Since physical robots are responsive to swarm position updates, modifications were required to implement the interaction between real robots and the PSO algorithm. The development of modifications necessary to implement PSO on mobile robots, and strategies to adapt to real life environments such as obstacles and collision objects are presented in this paper. Our findings are also validated using experimental testbeds.Comment: 13 pages, 12 figure

A Hierarchical Collision Avoidance Architecture for Multiple Fixed-Wing UAVs in an Integrated Airspace

This paper studies the collision avoidance problem for autonomous multiple fixedwing UAVs in the complex integrated airspace. By studying and combining the online path planning method, the distributed model predictive control algorithm, and the geometric reactive control approach, a three-layered collision avoidance system integrating conflict detection and resolution procedures is developed for multiple fixed-wing UAVs modeled by unicycle kinematics subject to input constraints. The effectiveness of the proposed methodology is evaluated and validated via test results of comparative simulations under both deterministic and probabilistic sensing conditions.Comment: 6 pages, 3 figures, 21st IFAC World Congress 202

A Distributed Control Framework of Multiple Unmanned Aerial Vehicles for Dynamic Wildfire Tracking

Wild-land fire fighting is a hazardous job. A key task for firefighters is to observe the "fire front" to chart the progress of the fire and areas that will likely spread next. Lack of information of the fire front causes many accidents. Using Unmanned Aerial Vehicles (UAVs) to cover wildfire is promising because it can replace humans in hazardous fire tracking and significantly reduce operation costs. In this paper we propose a distributed control framework designed for a team of UAVs that can closely monitor a wildfire in open space, and precisely track its development. The UAV team, designed for flexible deployment, can effectively avoid in-flight collisions and cooperate well with neighbors. They can maintain a certain height level to the ground for safe flight above fire. Experimental results are conducted to demonstrate the capabilities of the UAV team in covering a spreading wildfire.Comment: arXiv admin note: substantial text overlap with arXiv:1704.0263

Navigation in the Presence of Obstacles for an Agile Autonomous Underwater Vehicle

Navigation underwater traditionally is done by keeping a safe distance from obstacles, resulting in "fly-overs" of the area of interest. Movement of an autonomous underwater vehicle (AUV) through a cluttered space, such as a shipwreck or a decorated cave, is an extremely challenging problem that has not been addressed in the past. This paper proposes a novel navigation framework utilizing an enhanced version of Trajopt for fast 3D path-optimization planning for AUVs. A sampling-based correction procedure ensures that the planning is not constrained by local minima, enabling navigation through narrow spaces. Two different modalities are proposed: planning with a known map results in efficient trajectories through cluttered spaces; operating in an unknown environment utilizes the point cloud from the visual features detected to navigate efficiently while avoiding the detected obstacles. The proposed approach is rigorously tested, both on simulation and in-pool experiments, proven to be fast enough to enable safe real-time 3D autonomous navigation for an AUV.Comment: ICRA 202

A survey on unmanned aerial vehicle collision avoidance systems

Collision avoidance is a key factor in enabling the integration of unmanned aerial vehicle into real life use, whether it is in military or civil application. For a long time there have been a large number of works to address this problem; therefore a comparative summary of them would be desirable. This paper presents a survey on the major collision avoidance systems developed in up to date publications. Each collision avoidance system contains two main parts: sensing and detection, and collision avoidance. Based on their characteristics each part is divided into different categories; and those categories are explained, compared and discussed about advantages and disadvantages in this paper.Comment: This is only a draf