Traffic Light Control Using Deep Policy-Gradient and Value-Function Based Reinforcement Learning

Recent advances in combining deep neural network architectures with reinforcement learning techniques have shown promising potential results in solving complex control problems with high dimensional state and action spaces. Inspired by these successes, in this paper, we build two kinds of reinforcement learning algorithms: deep policy-gradient and value-function based agents which can predict the best possible traffic signal for a traffic intersection. At each time step, these adaptive traffic light control agents receive a snapshot of the current state of a graphical traffic simulator and produce control signals. The policy-gradient based agent maps its observation directly to the control signal, however the value-function based agent first estimates values for all legal control signals. The agent then selects the optimal control action with the highest value. Our methods show promising results in a traffic network simulated in the SUMO traffic simulator, without suffering from instability issues during the training process

Fine-grained acceleration control for autonomous intersection management using deep reinforcement learning

Recent advances in combining deep learning and Reinforcement Learning have shown a promising path for designing new control agents that can learn optimal policies for challenging control tasks. These new methods address the main limitations of conventional Reinforcement Learning methods such as customized feature engineering and small action/state space dimension requirements. In this paper, we leverage one of the state-of-the-art Reinforcement Learning methods, known as Trust Region Policy Optimization, to tackle intersection management for autonomous vehicles. We show that using this method, we can perform fine-grained acceleration control of autonomous vehicles in a grid street plan to achieve a global design objective.Comment: Accepted in IEEE Smart World Congress 201

A multi-agent Framework for dynamic traffic management Considering Priority Link

To favor emergency vehicles, promote collective modes of transport in Moroccan cities, we propose in this paper a control system to manage traffic at signalized intersections with priority links in urban settings. This system combines multi-agent technology and fuzzy logic to regulate traffic flows. The traffic system flow is divided into two types of vehicles; priority and regular vehicles. The regular vehicles can use only the regular links, while the priority vehicles may use both priority and the regular links. This approach aims to favor emergency vehicles and promote collective modes of transport, it acts on the traffic light phases length and order to control all traffic flows. We proposed a decentralized system of regulation based on real-time monitoring to develop a local inter-section state, and intelligent coordination between neighboring intersections to build an overview of the traffic state. The regulation and prioritization decisions are made through cooperation, communication, and coordination between different agents. The performance of the proposed system is investigated and instantiated in ANYLOGIC simulator, using a section of the Marrakesh road network that contains priority links. The results indicate that the designed system can significantly develop the efficiency of the traffic regulation system

Multi-Agent Chance-Constrained Stochastic Shortest Path with Application to Risk-Aware Intelligent Intersection

In transportation networks, where traffic lights have traditionally been used for vehicle coordination, intersections act as natural bottlenecks. A formidable challenge for existing automated intersections lies in detecting and reasoning about uncertainty from the operating environment and human-driven vehicles. In this paper, we propose a risk-aware intelligent intersection system for autonomous vehicles (AVs) as well as human-driven vehicles (HVs). We cast the problem as a novel class of Multi-agent Chance-Constrained Stochastic Shortest Path (MCC-SSP) problems and devise an exact Integer Linear Programming (ILP) formulation that is scalable in the number of agents' interaction points (e.g., potential collision points at the intersection). In particular, when the number of agents within an interaction point is small, which is often the case in intersections, the ILP has a polynomial number of variables and constraints. To further improve the running time performance, we show that the collision risk computation can be performed offline. Additionally, a trajectory optimization workflow is provided to generate risk-aware trajectories for any given intersection. The proposed framework is implemented in CARLA simulator and evaluated under a fully autonomous intersection with AVs only as well as in a hybrid setup with a signalized intersection for HVs and an intelligent scheme for AVs. As verified via simulations, the featured approach improves intersection's efficiency by up to $200\%$ while also conforming to the specified tunable risk threshold