Deep learning for real-time traffic signal control on urban networks

Real-time traffic signal controls are frequently challenged by (1) uncertain knowledge about the traffic states; (2) need for efficient computation to allow timely decisions; (3) multiple objectives such as traffic delays and vehicle emissions that are difficult to optimize; and (4) idealized assumptions about data completeness and quality that are often made in developing many theoretical signal control models. This thesis addresses these challenges by proposing two real-time signal control frameworks based on deep learning techniques, followed by extensive simulation tests that verifies their effectiveness in view of the aforementioned challenges. The first method, called the Nonlinear Decision Rule (NDR), defines a nonlinear mapping between network states and signal control parameters to network performances based on prevailing traffic conditions, and such a mapping is optimized via off-line simulation. The NDR is instantiated with two neural networks: feedforward neural network (FFNN) and recurrent neural network (RNN), which have different ways of processing traffic information in the near past. The NDR is implemented and tested within microscopic traffic simulation (S-Paramics) for a real-world network in West Glasgow, where the off-line training of the NDR amounts to a simulation-based optimization procedure aiming to reduce delay, CO2 and black carbon emissions. Extensive tests are performed to assess the NDR framework, not only in terms of its effectiveness in optimizing different traffic and environmental objectives, but also in relation to local vs. global benefits, trade-off between delay and emissions, impact of sensor locations, and different levels of network saturation. The second method, called the Advanced Reinforcement Learning (ARL), employs the potential-based reward shaping function using Q-learning and 3rd party advisor to enhance its performance over conventional reinforcement learning. The potential-based reward shaping in this thesis obtains an opinion from the 3rd party advisor when calculating reward. This technique can resolve the problem of sparse reward and slow learning speed. The ARL is tested with a range of existing reinforcement learning methods. The results clearly show that ARL outperforms the other models in almost all the scenarios. Lastly, this thesis evaluates the impact of information availability and quality on different real-time signal control methods, including the two proposed ones. This is driven by the observation that most responsive signal control models in the literature tend to make idealized assumptions on the quality and availability of data. This research shows the varying levels of performance deterioration of different signal controllers in the presence of missing data, data noise, and different data types. Such knowledge and insights are crucial for real-world implementation of these signal control methods.Open Acces

Road network equilibrium approaches to environmental sustainability

Environmental sustainability is closely related to transportation, especially to the road network, because vehicle emissions and noise damage the environment and have adverse effects on human health. It is, therefore, important to take their effect into account when designing and managing road networks. Road network equilibrium approaches have been used to estimate this impact and to design and manage road networks accordingly. However, no comprehensive review has summarized the applications of these approaches to the design and management of road networks that explicitly address environmental concerns. More importantly, it is necessary to identify this gap in the literature so that future research can improve the existing methodologies. Hence, this paper summarizes these applications and identifies potential future research directions in terms of theories, modelling approaches, algorithms, analyses, and applications.postprin

Demand response within the energy-for-water-nexus - A review. ESRI WP637, October 2019

A promising tool to achieve more flexibility within power systems is demand re-sponse (DR). End-users in many strands of industry have been subject to research up to now regarding the opportunities for implementing DR programmes. One sector that has received little attention from the literature so far, is wastewater treatment. However, case studies indicate that the potential for wastewater treatment plants to provide DR services might be significant. This review presents and categorises recent modelling approaches for industrial demand response as well as for the wastewater treatment plant operation. Furthermore, the main sources of flexibility from wastewater treatment plants are presented: a potential for variable electricity use in aeration, the time-shifting operation of pumps, the exploitation of built-in redundan-cy in the system and flexibility in the sludge processing. Although case studies con-note the potential for DR from individual WWTPs, no study acknowledges the en-dogeneity of energy prices which arises from a large-scale utilisation of DR. There-fore, an integrated energy systems approach is required to quantify system and market effects effectively