Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning

Robots that navigate among pedestrians use collision avoidance algorithms to enable safe and efficient operation. Recent works present deep reinforcement learning as a framework to model the complex interactions and cooperation. However, they are implemented using key assumptions about other agents' behavior that deviate from reality as the number of agents in the environment increases. This work extends our previous approach to develop an algorithm that learns collision avoidance among a variety of types of dynamic agents without assuming they follow any particular behavior rules. This work also introduces a strategy using LSTM that enables the algorithm to use observations of an arbitrary number of other agents, instead of previous methods that have a fixed observation size. The proposed algorithm outperforms our previous approach in simulation as the number of agents increases, and the algorithm is demonstrated on a fully autonomous robotic vehicle traveling at human walking speed, without the use of a 3D Lidar

Modeling rationality to control self-organization of crowds: An environmental approach

In this paper we propose a classification of crowd models in built environments based on the assumed pedestrian ability to foresee the movements of other walkers. At the same time, we introduce a new family of macroscopic models, which make it possible to tune the degree of predictiveness (i.e., rationality) of the individuals. By means of these models we describe both the natural behavior of pedestrians, i.e., their expected behavior according to their real limited predictive ability, and a target behavior, i.e., a particularly efficient behavior one would like them to assume (for, e.g., logistic or safety reasons). Then we tackle a challenging shape optimization problem, which consists in controlling the environment in such a way that the natural behavior is as close as possible to the target one, thereby inducing pedestrians to behave more rationally than what they would naturally do. We present numerical tests which elucidate the role of rational/predictive abilities and show some promising results about the shape optimization problem

Intention-Aware Motion Planning

As robots venture into new application domains as autonomous vehicles on the road or as domestic helpers at home, they must recognize human intentions and behaviors in order to operate effectively. This paper investigates a new class of motion planning problems with uncertainty in human intention. We propose a method for constructing a practical model by assuming a finite set of unknown intentions. We first construct a motion model for each intention in the set and then combine these models together into a single Mixed Observability Markov Decision Process (MOMDP), which is a structured variant of the more common Partially Observable Markov Decision Process (POMDP). By leveraging the latest advances in POMDP/MOMDP approximation algorithms, we can construct and solve moderately complex models for interesting robotic tasks. Experiments in simulation and with an autonomous vehicle show that the proposed method outperforms common alternatives because of its ability in recognizing intentions and using the information effectively for decision making.Singapore-MIT Alliance for Research and Technology (SMART) (grant R-252- 000-447-592)Singapore-MIT GAMBIT Game Lab (grant R-252-000-398-490)Singapore. Ministry of Education (AcRF grant 2010-T2-2-071