EMVLight: a Multi-agent Reinforcement Learning Framework for an Emergency Vehicle Decentralized Routing and Traffic Signal Control System

Emergency vehicles (EMVs) play a crucial role in responding to time-critical calls such as medical emergencies and fire outbreaks in urban areas. Existing methods for EMV dispatch typically optimize routes based on historical traffic-flow data and design traffic signal pre-emption accordingly; however, we still lack a systematic methodology to address the coupling between EMV routing and traffic signal control. In this paper, we propose EMVLight, a decentralized reinforcement learning (RL) framework for joint dynamic EMV routing and traffic signal pre-emption. We adopt the multi-agent advantage actor-critic method with policy sharing and spatial discounted factor. This framework addresses the coupling between EMV navigation and traffic signal control via an innovative design of multi-class RL agents and a novel pressure-based reward function. The proposed methodology enables EMVLight to learn network-level cooperative traffic signal phasing strategies that not only reduce EMV travel time but also shortens the travel time of non-EMVs. Simulation-based experiments indicate that EMVLight enables up to a $42.6\%$ reduction in EMV travel time as well as an $23.5\%$ shorter average travel time compared with existing approaches.Comment: 19 figures, 10 tables. Manuscript extended on previous work arXiv:2109.05429, arXiv:2111.0027

A Micro-Simulation Study of the Generalized Proportional Allocation Traffic Signal Control

In this paper, we study the problem of determining phase activations for signalized junctions by utilizing feedback, more specifically, by measure the queue-lengths on the incoming lanes to each junction. The controller we are investigating is the Generalized Proportional Allocation (GPA) controller, which has previously been shown to have desired stability and throughput properties in a continuous averaged dynamical model for queueing networks. In this paper, we provide and implement two discretized versions of the GPA controller in the SUMO micro simulator. We also compare the GPA controllers with the MaxPressure controller, a controller that requires more information than the GPA, in an artificial Manhattan-like grid. To show that the GPA controller is easy to implement in a real scenario, we also implement it in a previously published realistic traffic scenario for the city of Luxembourg and compare its performance with the static controller provided with the scenario. The simulations show that the GPA performs better than a static controller for the Luxembourg scenario, and better than the MaxPressure pressure controller in the Manhattan-grid when the demands are low