A self-learning motorway traffic control system for ramp metering

Self-learning systems have attracted increasing attention in the ramp metering domain in recent years. These systems are based on reinforcement learning (RL) and can learn to control motorway traffic adaptively. However, RL-based ramp metering systems are still in their early stages and have shown limitations regarding their design and evaluation. This research aims to develop a new RL-based system (known as RAS) for ramp metering to overcome these limitations. A general framework for designing a RL-based system is proposed in this research. It contains the definition of three RL elements in a ramp metering scenario and a system structure which brings together all modules to accomplish the reinforcement learning process. Under this framework, two control algorithms for both single- and multi-objective problems are developed. In addition, to evaluate the proposed system, a software platform combining the new system and a traffic flow model is developed in the research. Based on the platform developed, a systematic evaluation is carried out through a series of simulation-based experiments. By comparing with a widely used control strategy, ALINEA, the proposed system, RAS, has shown its effectiveness in learning the optimal control actions for different control objectives in both hypothetical and real motorway networks. It is found that RAS outperforms ALINEA on improving traffic efficiency in the situation with severe congestion and on maintaining user equity when multiple on-ramps are included in the motorway network. Moreover, this research has been extended to use indirect learning technology to deal with incident-induced congestion. Tests for this extension to the work are carried out based on the platform developed and a commercial software package, AIMSUN, which have shown the potential of the extended system in tackling incident-induced congestion

Network Maintenance and Capacity Management with Applications in Transportation

abstract: This research develops heuristics to manage both mandatory and optional network capacity reductions to better serve the network flows. The main application discussed relates to transportation networks, and flow cost relates to travel cost of users of the network. Temporary mandatory capacity reductions are required by maintenance activities. The objective of managing maintenance activities and the attendant temporary network capacity reductions is to schedule the required segment closures so that all maintenance work can be completed on time, and the total flow cost over the maintenance period is minimized for different types of flows. The goal of optional network capacity reduction is to selectively reduce the capacity of some links to improve the overall efficiency of user-optimized flows, where each traveler takes the route that minimizes the traveler’s trip cost. In this dissertation, both managing mandatory and optional network capacity reductions are addressed with the consideration of network-wide flow diversions due to changed link capacities. This research first investigates the maintenance scheduling in transportation networks with service vehicles (e.g., truck fleets and passenger transport fleets), where these vehicles are assumed to take the system-optimized routes that minimize the total travel cost of the fleet. This problem is solved with the randomized fixed-and-optimize heuristic developed. This research also investigates the maintenance scheduling in networks with multi-modal traffic that consists of (1) regular human-driven cars with user-optimized routing and (2) self-driving vehicles with system-optimized routing. An iterative mixed flow assignment algorithm is developed to obtain the multi-modal traffic assignment resulting from a maintenance schedule. The genetic algorithm with multi-point crossover is applied to obtain a good schedule. Based on the Braess’ paradox that removing some links may alleviate the congestion of user-optimized flows, this research generalizes the Braess’ paradox to reduce the capacity of selected links to improve the efficiency of the resultant user-optimized flows. A heuristic is developed to identify links to reduce capacity, and the corresponding capacity reduction amounts, to get more efficient total flows. Experiments on real networks demonstrate the generalized Braess’ paradox exists in reality, and the heuristic developed solves real-world test cases even when commercial solvers fail.Dissertation/ThesisDoctoral Dissertation Industrial Engineering 201