Decentralized MPC based Obstacle Avoidance for Multi-Robot Target Tracking Scenarios

In this work, we consider the problem of decentralized multi-robot target tracking and obstacle avoidance in dynamic environments. Each robot executes a local motion planning algorithm which is based on model predictive control (MPC). The planner is designed as a quadratic program, subject to constraints on robot dynamics and obstacle avoidance. Repulsive potential field functions are employed to avoid obstacles. The novelty of our approach lies in embedding these non-linear potential field functions as constraints within a convex optimization framework. Our method convexifies non-convex constraints and dependencies, by replacing them as pre-computed external input forces in robot dynamics. The proposed algorithm additionally incorporates different methods to avoid field local minima problems associated with using potential field functions in planning. The motion planner does not enforce predefined trajectories or any formation geometry on the robots and is a comprehensive solution for cooperative obstacle avoidance in the context of multi-robot target tracking. We perform simulation studies in different environmental scenarios to showcase the convergence and efficacy of the proposed algorithm. Video of simulation studies: \url{https://youtu.be/umkdm82Tt0M

Combining Subgoal Graphs with Reinforcement Learning to Build a Rational Pathfinder

In this paper, we present a hierarchical path planning framework called SG-RL (subgoal graphs-reinforcement learning), to plan rational paths for agents maneuvering in continuous and uncertain environments. By "rational", we mean (1) efficient path planning to eliminate first-move lags; (2) collision-free and smooth for agents with kinematic constraints satisfied. SG-RL works in a two-level manner. At the first level, SG-RL uses a geometric path-planning method, i.e., Simple Subgoal Graphs (SSG), to efficiently find optimal abstract paths, also called subgoal sequences. At the second level, SG-RL uses an RL method, i.e., Least-Squares Policy Iteration (LSPI), to learn near-optimal motion-planning policies which can generate kinematically feasible and collision-free trajectories between adjacent subgoals. The first advantage of the proposed method is that SSG can solve the limitations of sparse reward and local minima trap for RL agents; thus, LSPI can be used to generate paths in complex environments. The second advantage is that, when the environment changes slightly (i.e., unexpected obstacles appearing), SG-RL does not need to reconstruct subgoal graphs and replan subgoal sequences using SSG, since LSPI can deal with uncertainties by exploiting its generalization ability to handle changes in environments. Simulation experiments in representative scenarios demonstrate that, compared with existing methods, SG-RL can work well on large-scale maps with relatively low action-switching frequencies and shorter path lengths, and SG-RL can deal with small changes in environments. We further demonstrate that the design of reward functions and the types of training environments are important factors for learning feasible policies.Comment: 20 page

Robot motion planning using real-time heuristic search

Autonomous mobile robots must be able to plan quickly and stay reactive to the world around them. Currently, navigating in the presence of dynamic obstacles is a problem that modern techniques struggle to handle in a real-time manner, even when the environment is known. The solutions range from using: 1) sampling-based algorithms which cut down on the shear size of these state spaces, 2) algorithms which quickly try to plan complete paths to the goal (to avoid local minima) and 3) using real-time search techniques designed for static worlds. Each of these methods have fundamental flaws that prevent it from being used in practice. In this thesis I offer three proposed techniques to help improve planning among dynamic obstacles. First, I present a new partitioned learning technique for splitting the costs estimates used by heuristic search techniques into those caused by the static environment and those caused by the dynamic obstacles in the world. This allows for much more accurate learning. Second, I introduce a novel decaying heuristic technique for generalizing cost-to-go over states of the same pose (x. y.theta.v) in the world. Third, I show a garbage collection mechanism for removing useless states from our search to cut down on the overall memory usage. Finally, I present a new algorithm called Partitioned Learning Real-time A*. PLRTA* uses all three of these new enhancements to navigate through worlds with dynamic obstacles in a real-time manner while handling the complex situations in which other algorithms fail. I empirically compare our algorithm to other competing algorithms in a number of random instances as well as hand crafted scenarios designed to highlight desirable behavior in specific situations. I show that PLRTA* outperforms the current state-of-the-art algorithms in terms of minimizing cost over a large number of robot motion planning problems, even when planning in fairly confined environments with up to ten dynamic obstacles

Solving the potential field local minimum problem using internal agent states

We propose a new, extended artificial potential field method, which uses dynamic internal agent states. The internal states are modelled as a dynamical system of coupled first order differential equations that manipulate the potential field in which the agent is situated. The internal state dynamics are forced by the interaction of the agent with the external environment. Local equilibria in the potential field are then manipulated by the internal states and transformed from stable equilibria to unstable equilibria, allowiong escape from local minima in the potential field. This new methodology successfully solves reactive path planning problems, such as a complex maze with multiple local minima, which cannot be solved using conventional static potential fields

PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-based Planning

We present PRM-RL, a hierarchical method for long-range navigation task completion that combines sampling based path planning with reinforcement learning (RL). The RL agents learn short-range, point-to-point navigation policies that capture robot dynamics and task constraints without knowledge of the large-scale topology. Next, the sampling-based planners provide roadmaps which connect robot configurations that can be successfully navigated by the RL agent. The same RL agents are used to control the robot under the direction of the planning, enabling long-range navigation. We use the Probabilistic Roadmaps (PRMs) for the sampling-based planner. The RL agents are constructed using feature-based and deep neural net policies in continuous state and action spaces. We evaluate PRM-RL, both in simulation and on-robot, on two navigation tasks with non-trivial robot dynamics: end-to-end differential drive indoor navigation in office environments, and aerial cargo delivery in urban environments with load displacement constraints. Our results show improvement in task completion over both RL agents on their own and traditional sampling-based planners. In the indoor navigation task, PRM-RL successfully completes up to 215 m long trajectories under noisy sensor conditions, and the aerial cargo delivery completes flights over 1000 m without violating the task constraints in an environment 63 million times larger than used in training.Comment: 9 pages, 7 figure