Back-pressure traffic signal control with unknown routing rates

The control of a network of signalized intersections is considered. Previous works proposed a feedback control belonging to the family of the so-called back-pressure controls that ensures provably maximum stability given pre-specified routing probabilities. However, this optimal back-pressure controller (BP*) requires routing rates and a measure of the number of vehicles queuing at a node for each possible routing decision. It is an idealistic assumption for our application since vehicles (going straight, turning left/right) are all gathered in the same lane apart from the proximity of the intersection and cameras can only give estimations of the aggregated queue length. In this paper, we present a back-pressure traffic signal controller (BP) that does not require routing rates, it requires only aggregated queue lengths estimation (without direction information) and loop detectors at the stop line for each possible direction. A theoretical result on the Lyapunov drift in heavy load conditions under BP control is provided and tends to indicate that BP should have good stability properties. Simulations confirm this and show that BP stabilizes the queuing network in a significant part of the capacity region.Comment: accepted for presentation at IFAC 2014, 6 pages. arXiv admin note: text overlap with arXiv:1309.648

EMVLight: a Multi-agent Reinforcement Learning Framework for an Emergency Vehicle Decentralized Routing and Traffic Signal Control System

Emergency vehicles (EMVs) play a crucial role in responding to time-critical calls such as medical emergencies and fire outbreaks in urban areas. Existing methods for EMV dispatch typically optimize routes based on historical traffic-flow data and design traffic signal pre-emption accordingly; however, we still lack a systematic methodology to address the coupling between EMV routing and traffic signal control. In this paper, we propose EMVLight, a decentralized reinforcement learning (RL) framework for joint dynamic EMV routing and traffic signal pre-emption. We adopt the multi-agent advantage actor-critic method with policy sharing and spatial discounted factor. This framework addresses the coupling between EMV navigation and traffic signal control via an innovative design of multi-class RL agents and a novel pressure-based reward function. The proposed methodology enables EMVLight to learn network-level cooperative traffic signal phasing strategies that not only reduce EMV travel time but also shortens the travel time of non-EMVs. Simulation-based experiments indicate that EMVLight enables up to a $42.6\%$ reduction in EMV travel time as well as an $23.5\%$ shorter average travel time compared with existing approaches.Comment: 19 figures, 10 tables. Manuscript extended on previous work arXiv:2109.05429, arXiv:2111.0027

Two-layer adaptive signal control framework for large-scale dynamically-congested networks: Combining efficient Max Pressure with Perimeter Control

Traffic-responsive signal control is a cost-effective and easy-to-implement network management strategy with high potential in improving performance in congested networks with dynamic characteristics. Max Pressure (MP) distributed controller gained significant popularity due to its theoretically proven ability of queue stabilization and throughput maximization under specific assumptions. However, its effectiveness under saturated conditions is questionable, while network-wide application is limited due to high instrumentation cost. Perimeter control (PC) based on the concept of the Macroscopic Fundamental Diagram (MFD) is a state-of-the-art aggregated strategy that regulates exchange flows between regions, in order to maintain maximum regional travel production and prevent over-saturation. Yet, homogeneity assumption is hardly realistic in congested states, thus compromising PC efficiency. In this paper, the effectiveness of network-wide, parallel application of PC and MP embedded in a two-layer control framework is assessed with mesoscopic simulation. Aiming at reducing implementation cost of MP without significant performance loss, we propose a method to identify critical nodes for partial MP deployment. A modified version of Store-and-forward paradigm incorporating finite queue and spill-back consideration is used to test different configurations of the proposed framework, for a real large-scale network, in moderately and highly congested scenarios. Results show that: (i) combined control of MP and PC outperforms separate MP and PC applications in both demand scenarios; (ii) MP control in reduced critical node sets leads to similar or even better performance compared to full-network implementation, thus allowing for significant cost reduction; iii) the proposed control schemes improve system performance even under demand fluctuations of up to 20% of mean.Comment: Submitted to Transportation Research Part C: Emerging Technologie