A Novel Ramp Metering Approach Based on Machine Learning and Historical Data

The random nature of traffic conditions on freeways can cause excessive congestions and irregularities in the traffic flow. Ramp metering is a proven effective method to maintain freeway efficiency under various traffic conditions. Creating a reliable and practical ramp metering algorithm that considers both critical traffic measures and historical data is still a challenging problem. In this study we use machine learning approaches to develop a novel real-time prediction model for ramp metering. We evaluate the potentials of our approach in providing promising results by comparing it with a baseline traffic-responsive ramp metering algorithm.Comment: 5 pages, 11 figures, 2 table

Development of traffic micro-simulation model for motorway merges with ramp metering

This thesis focuses on the development of a micro-simulation model for motorway merge sections. The aim is to study the effectiveness of applying some traffic management controls and particularly focuses on applying ramp metering (RM) systems.The new model has been developed based on car-following, lane changing and gap acceptance rules. The model considered the multi-decisions undertaken by merging traffic when a driver, for example, accepts the lead gap and rejects the lag gap. The cooperative nature of drivers is also considered where motorway drivers allow others to merge in front of them either by decelerating or shifting to other lanes (yielding) in the vicinity of motorway merge sections. Video recordings, as well as data from the Motorway Incident Detection and Automatic Signalling (MIDAS) were obtained from a selection of sites. The data was used in the verification, calibration and validation processes of the developed model. Other main sources of information include more than 4 million cases of successive vehicles taken from UK motorway sites. These cases were analysed to study the effect of vehicle types on the following behaviour for drivers. The main finding is that there is no evidence that the average spacing between successive vehicles is significantly affected by the type of leading vehicle.Different RM algorithms have been integrated within the developed model. The results of testing the effectiveness of RM controls using the developed model reveal the benefits of RM in reducing time spent by motorway traffic (TTSM) but it significantly increases the time spent by the merging traffic (TTSM). The overall benefits of implementing RM in reducing total time spent (TTS) is limited to situations where the sum of motorway and merge flows exceeds the capacity of the downstream section. Other issues related to RM design and effectiveness have been tested such as the effects of having different durations for peak periods, finding the optimum parameters for each algorithm, the effect of ramp length (storage area) and the effect of RM signals position. The results suggest that RM is very efficient when implemented for short peak periods (e.g. less than 30 minutes). The effectiveness of RM in decreasing the travel time for motorway traffic is increased with an increasing ramp length but with a significant increase in ramp traffic delay. No significant effect is obtained from altering the ramp signals' position.Other tests include the use of other types of traffic management controls (e.g. applying different speed limits and lane changing restrictions (LCR) at the approach to merge sections). No significant improvements were obtained from testing different speed limit values. The results suggest that LCR could reduce travel time for motorway traffic. However, there are other practical considerations which need to be addressed before this could be recommended

Reinforcement Learning with Model Predictive Control for Highway Ramp Metering

In the backdrop of an increasingly pressing need for effective urban and highway transportation systems, this work explores the synergy between model-based and learning-based strategies to enhance traffic flow management by use of an innovative approach to the problem of highway ramp metering control that embeds Reinforcement Learning techniques within the Model Predictive Control framework. The control problem is formulated as an RL task by crafting a suitable stage cost function that is representative of the traffic conditions, variability in the control action, and violations of a safety-critical constraint on the maximum number of vehicles in queue. An MPC-based RL approach, which merges the advantages of the two paradigms in order to overcome the shortcomings of each framework, is proposed to learn to efficiently control an on-ramp and to satisfy its constraints despite uncertainties in the system model and variable demands. Finally, simulations are performed on a benchmark from the literature consisting of a small-scale highway network. Results show that, starting from an MPC controller that has an imprecise model and is poorly tuned, the proposed methodology is able to effectively learn to improve the control policy such that congestion in the network is reduced and constraints are satisfied, yielding an improved performance compared to the initial controller.Comment: 14 pages, 10 figures, 3 tables, submitted to IEEE Transactions on Intelligent Transportation System

A Q-LEARNING BASED INTEGRATED VARIABLE SPEED LIMIT AND HARD SHOULDER RUNNING CONTROL TO REDUCE TRAVEL TIME AT FREEWAY BOTTLENECK

To increase traffic mobility and safety, several types of active traffic management (ATM) strategies, such as variable speed limit (VSL) and hard shoulder running (HSR), are implemented in many countries. While all kinds of ATM strategies show promise in releasing traffic congestion, many studies indicate that stand-alone strategies have very limited capability. This paper proposes an integrated VSL and HSR control strategy based on a reinforcement learning (RL) technique, Q-learning (QL). The proposed strategy bridges a direct connection between the traffic flow data and the ATM control strategies via intensive self-learning processes thus reduces the need for human knowledge. A typical congested interstate highway, I-270 in Maryland, U.S. is simulated using a dynamic traffic assignment (DTA) model to evaluate the proposed strategy. Simulation results indicated that the integrated strategy outperforms the stand-alone strategies and traditional feedback-based VSL strategy in mitigating congestions and reducing travel time on the freeway corridor

Deep Reinforcement Learning Approach for Lagrangian Control: Improving Freeway Bottleneck Throughput Via Variable Speed Limit

Connected vehicles (CVs) will enable new applications to improve traffic flow. The focus of this dissertation is to investigate how reinforcement learning (RL) control for the variable speed limit (VSL) through CVs can be generalized to improve traffic flow at different freeway bottlenecks. Three different bottlenecks are investigated: A sag curve, where the gradient changes from negative to positive values causes a reduction in the roadway capacity and congestion; a lane reduction, where three lanes merge to two lanes and cause congestion, and finally, an on-ramp, where increase in demand on a multilane freeway causes capacity drop. An RL algorithm is developed and implemented in a simulation environment for controlling a VSL in the upstream to manipulate the inflow of vehicles to the bottleneck on a freeway to minimize delays and increase the throughput. CVs are assumed to receive VSL messages through Infrastructure-to-Vehicle (I2V) communications technologies. Asynchronous Advantage Actor-Critic (A3C) algorithms are developed for each bottleneck to determine optimal VSL policies. Through these RL control algorithms, the speed of CVs are manipulated in the upstream of the bottleneck to avoid or minimize congestion. Various market penetration rates for CVs are considered in the simulations. It is demonstrated that the RL algorithm is able to adapt to stochastic arrivals of CVs and achieve significant improvements even at low market penetration rates of CVs, and the RL algorithm is able to find solution for all three bottlenecks. The results also show that the RL-based solutions outperform feedback-control-based solutions