Fine-grained acceleration control for autonomous intersection management using deep reinforcement learning

Recent advances in combining deep learning and Reinforcement Learning have shown a promising path for designing new control agents that can learn optimal policies for challenging control tasks. These new methods address the main limitations of conventional Reinforcement Learning methods such as customized feature engineering and small action/state space dimension requirements. In this paper, we leverage one of the state-of-the-art Reinforcement Learning methods, known as Trust Region Policy Optimization, to tackle intersection management for autonomous vehicles. We show that using this method, we can perform fine-grained acceleration control of autonomous vehicles in a grid street plan to achieve a global design objective.Comment: Accepted in IEEE Smart World Congress 201

Analysis of Model Predictive Intersection Control for Autonomous Vehicles

Autonomous vehicles are in the main focus for automotive companies and urban traffic engineers as well. As their penetration rate in traffic becomes more and more pronounced due to improvement in sensor technologies and the corresponding infrastructure, new methods for autonomous vehicle controls become a necessity. For instance, autonomous vehicles can improve the performance of urban traffic and prevent the formation of congestions with the usage of Vehicle-to-Vehicle (V2V) and Vehicle-to-Infrastructure (V2I) communication based control methods. One of the key area for improvement is centralized intersection control for autonomous vehicles, by which traveling times can be reduced and efficiency of traffic flow can be improved, while safety of passengers can be guaranteed through constraints built in the centralized design. The paper presents the analysis of a Model Predictive Control (MPC) method for the coordination of autonomous vehicles at intersections by comparing it with an offline constraint optimization considering time and energy optimal intervention of vehicles. The analysis has been evaluated in high-fidelity simulation environment CarSim, where the speed trajectories, traveling times and energy consumptions have been compared for the different methods. The simulations show that the proposed time-optimal MPC intersection control method results in similar traveling times of that given by the time-optimal offline constraint optimization, while the energy optimal optimization re-quires significantly more time for the autonomous vehicle to achieve. Due to the possibility of a congestion forming in the latter case, the proposed centralized MPC method is more applicable in real traffic scenarios