Adaptive traffic signal control using approximate dynamic programming

This paper presents a study on an adaptive traffic signal controller for real-time operation. The controller aims for three operational objectives: dynamic allocation of green time, automatic adjustment to control parameters, and fast revision of signal plans. The control algorithm is built on approximate dynamic programming (ADP). This approach substantially reduces computational burden by using an approximation to the value function of the dynamic programming and reinforcement learning to update the approximation. We investigate temporal-difference learning and perturbation learning as specific learning techniques for the ADP approach. We find in computer simulation that the ADP controllers achieve substantial reduction in vehicle delays in comparison with optimised fixed-time plans. Our results show that substantial benefits can be gained by increasing the frequency at which the signal plans are revised, which can be achieved conveniently using the ADP approach

Objectives, stimulus and feedback in signal control of road traffic

This article identifies the prospective role of a range of intelligent transport systems technologies for the signal control of road traffic. We discuss signal control within the context of traffic management and control in urban road networks and then present a control-theoretic formulation for it that distinguishes the various roles of detector data, objectives of optimization, and control feedback. By reference to this, we discuss the importance of different kinds of variability in traffic flows and review the state of knowledge in respect of control in the presence of different combinations of them. In light of this formulation and review, we identify a range of important possibilities for contributions to traffic management and control through traffic measurement and detection technology, and contemporary flexible optimization techniques that use various kinds of automated learning

Adaptive performance optimization for large-scale traffic control systems

In this paper, we study the problem of optimizing (fine-tuning) the design parameters of large-scale traffic control systems that are composed of distinct and mutually interacting modules. This problem usually requires a considerable amount of human effort and time to devote to the successful deployment and operation of traffic control systems due to the lack of an automated well-established systematic approach. We investigate the adaptive fine-tuning algorithm for determining the set of design parameters of two distinct mutually interacting modules of the traffic-responsive urban control (TUC) strategy, i.e., split and cycle, for the large-scale urban road network of the city of Chania, Greece. Simulation results are presented, demonstrating that the network performance in terms of the daily mean speed, which is attained by the proposed adaptive optimization methodology, is significantly better than the original TUC system in the case in which the aforementioned design parameters are manually fine-tuned to virtual perfection by the system operators

Data-driven linear decision rule approach for distributionally robust optimization of on-line signal control

We propose a two-stage, on-line signal control strategy for dynamic networks using a linear decision rule (LDR) approach and a distributionally robust optimization (DRO) technique. The first (off-line) stage formulates a LDR that maps real-time traffic data to optimal signal control policies. A DRO problem is solved to optimize the on-line performance of the LDR in the presence of uncertainties associated with the observed traffic states and ambiguity in their underlying distribution functions. We employ a data-driven calibration of the uncertainty set, which takes into account historical traffic data. The second (on-line) stage implements a very efficient linear decision rule whose performance is guaranteed by the off-line computation. We test the proposed signal control procedure in a simulation environment that is informed by actual traffic data obtained in Glasgow, and demonstrate its full potential in on-line operation and deployability on realistic networks, as well as its effectiveness in improving traffic

Integrated Approach for Diversion Route Performance Management during Incidents

Non-recurrent congestion is one of the critical sources of congestion on the highway. In particular, traffic incidents create congestion in unexpected times and places that travelers do not prepare for. During incidents on freeways, route diversion has been proven to be a useful tactic to mitigate non-recurrent congestion. However, the capacity constraints created by the signals on the alternative routes put limits on the diversion process since the typical time-of-day signal control cannot handle the sudden increase in the traffic on the arterials due to diversion. Thus, there is a need for proactive strategies for the management of the diversion routes performance and for coordinated freeway and arterial (CFA) operation during incidents on the freeway. Proactive strategies provide better opportunities for both the agency and the traveler to make and implement decisions to improve performance. This dissertation develops a methodology for the performance management of diversion routes through integrating freeway and arterials operation during incidents on the freeway. The methodology includes the identification of potential diversion routes for freeway incidents and the generation and implementation of special signal plans under different incident and traffic conditions. The study utilizes machine learning, data analytics, multi-resolution modeling, and multi-objective optimization for this purpose. A data analytic approach based on the long short term memory (LSTM) deep neural network method is used to predict the utilized alternative routes dynamically using incident attributes and traffic status on the freeway and travel time on both the freeway and alternative routes during the incident. Then, a combination of clustering analysis, multi- resolution modeling (MRM), and multi-objective optimization techniques are used to develop and activate special signal plans on the identified alternative routes. The developed methods use data from different sources, including connected vehicle (CV) data and high- resolution controller (HRC) data for congestion patterns identification at the critical intersections on the alternative routes and signal plans generation. The results indicate that implementing signal timing plans to better accommodate the diverted traffic can improve the performance of the diverted traffic without significantly deteriorating other movements\u27 performance at the intersection. The findings show the importance of using data from emerging sources in developing plans to improve the performance of the diversion routes and ensure CFA operation with higher effectiveness

Development and evaluation of cooperative intersection management algorithm under connected vehicles environment

Recent technological advancements in the automotive and transportation industry established a firm foundation for development and implementation of various automated and connected vehicle (C/AV) solutions around the globe. Wireless communication technologies such as the dedicated short-range communication (DSRC) protocol are enabling instantaneous information exchange between vehicles and infrastructure. Such information exchange produces tremendous benefits with the possibility to automate conventional traffic streams and enhance existing signal control strategies. While many promising studies in the area of signal control under connected vehicle (CV) environment have been introduced, they mainly offer solutions designed to operate a single isolated intersection or they require high technology penetration rates to operate in a safe and efficient manner. Applications designed to operate on a signalized corridor with imperfect market penetration rates of connected vehicle technology represent a bridge between conventional traffic control paradigm and fully automated corridors of the future. Assuming utilization of the connected vehicle environment and vehicle to infrastructure (V2I) technology, all vehicular and signal-related parameters are known and can be shared with the control agent to control automated vehicles while improving the mobility of the signalized corridor. This dissertation research introduces an intersection management strategy for a corridor with automated vehicles utilizing vehicular trajectory-driven optimization method. The Trajectory-driven Optimization for Automated Driving (TOAD) provides an optimal trajectory for automated vehicles while maintaining safe and uninterrupted movement of general traffic, consisting of regular unequipped vehicles. Signal status parameters such as cycle length and splits are continuously captured. At the same time, vehicles share their position information with the control agent. Both inputs are then used by the control algorithm to provide optimal trajectories for automated vehicles, resulting in the reduction of vehicle delay along the signalized corridor with fixed-time signal control. To determine the most efficient trajectory for automated vehicles, an evolutionary-based optimization is utilized. Influence of the prevailing traffic conditions is incorporated into a control algorithm using conventional data collection methods such as loop detectors, Bluetooth or Wi-Fi sensors to collect vehicle counts, travel time on corridor segments, and spot speed. Moreover, a short-term, artificial intelligence prediction model is developed to achieve reasonable deployment of data collection devices and provide accurate vehicle delay predictions producing realistic and highly-efficient longitudinal vehicle trajectories. The concept evaluation through microsimulation reveals significant mobility improvements compared to contemporary corridor management approach. The results for selected test-bed locations on signalized arterials in New Jersey reveals up to 19.5 % reduction in overall corridor travel time depending on different market penetration and lane configuration scenario. It is also discovered that operational scenarios with a possibility of utilizing reserved lanes for movement of automated vehicles further increases the effectiveness of the proposed algorithm. In addition, the proposed control algorithm is feasible under imperfect C/AV market penetrations showing mobility improvements even with low market penetration rates

Adaptive traffic signal control using approximate dynamic programming

This thesis presents a study on an adaptive traffic signal controller for real-time operation. An approximate dynamic programming (ADP) algorithm is developed for controlling traffic signals at isolated intersection and in distributed traffic networks. This approach is derived from the premise that classic dynamic programming is computationally difficult to solve, and approximation is the second-best option for establishing sequential decision-making for complex process. The proposed ADP algorithm substantially reduces computational burden by using a linear approximation function to replace the exact value function of dynamic programming solution. Machine-learning techniques are used to improve the approximation progressively. Not knowing the ideal response for the approximation to learn from, we use the paradigm of unsupervised learning, and reinforcement learning in particular. Temporal-difference learning and perturbation learning are investigated as appropriate candidates in the family of unsupervised learning. We find in computer simulation that the proposed method achieves substantial reduction in vehicle delays in comparison with optimised fixed-time plans, and is competitive against other adaptive methods in computational efficiency and effectiveness in managing varying traffic. Our results show that substantial benefits can be gained by increasing the frequency at which the signal plans are revised. The proposed ADP algorithm is in compliance with a range of discrete systems of resolution from 0.5 to 5 seconds per temporal step. This study demonstrates the readiness of the proposed approach for real-time operations at isolated intersections and the potentials for distributed network control

Determination of traffic responsive plan selection factors and thresholds using artificial neural networks

Traffic congestion has become a menace to civilized society. It degrades air quality, jeopardizes safety and causes delay. Traffic congestion can be alleviated by providing an effective traffic control signal system. Closed-loop traffic control systems are an example of such a system. Closed-loop traffic control systems can be operated primarily in either of two modes: Time of Day Mode (TOD) or Traffic Responsive Plan Selection Mode (TRPS). TRPS mode, if properly configured, can easily handle time independent variation in traffic volumes. It can also reduce the effect of timing plan aging. Despite these advantages, TRPS mode is not used as frequently as TOD mode. The reason being a lack of methodologies and formal guidelines for predicting the factors and thresholds associated with TRPS mode. In this research, a new methodology is developed for determining the thresholds and factors associated with the TRPS mode. This methodology, when tested on a closed-loop system in Odem, Texas, produced a classification accuracy of 94%. The classification accuracy can be increased to 98% with a proposed TRPS architecture