Towards Robust Deep Reinforcement Learning for Traffic Signal Control: Demand Surges, Incidents and Sensor Failures

Reinforcement learning (RL) constitutes a promising solution for alleviating the problem of traffic congestion. In particular, deep RL algorithms have been shown to produce adaptive traffic signal controllers that outperform conventional systems. However, in order to be reliable in highly dynamic urban areas, such controllers need to be robust with the respect to a series of exogenous sources of uncertainty. In this paper, we develop an open-source callback-based framework for promoting the flexible evaluation of different deep RL configurations under a traffic simulation environment. With this framework, we investigate how deep RL-based adaptive traffic controllers perform under different scenarios, namely under demand surges caused by special events, capacity reductions from incidents and sensor failures. We extract several key insights for the development of robust deep RL algorithms for traffic control and propose concrete designs to mitigate the impact of the considered exogenous uncertainties.Comment: 8 page

From Social Simulation to Integrative System Design

As the recent financial crisis showed, today there is a strong need to gain "ecological perspective" of all relevant interactions in socio-economic-techno-environmental systems. For this, we suggested to set-up a network of Centers for integrative systems design, which shall be able to run all potentially relevant scenarios, identify causality chains, explore feedback and cascading effects for a number of model variants, and determine the reliability of their implications (given the validity of the underlying models). They will be able to detect possible negative side effect of policy decisions, before they occur. The Centers belonging to this network of Integrative Systems Design Centers would be focused on a particular field, but they would be part of an attempt to eventually cover all relevant areas of society and economy and integrate them within a "Living Earth Simulator". The results of all research activities of such Centers would be turned into informative input for political Decision Arenas. For example, Crisis Observatories (for financial instabilities, shortages of resources, environmental change, conflict, spreading of diseases, etc.) would be connected with such Decision Arenas for the purpose of visualization, in order to make complex interdependencies understandable to scientists, decision-makers, and the general public.Comment: 34 pages, Visioneer White Paper, see http://www.visioneer.ethz.c

Modeling Location Choice of Secondary Activities with a Social Network of Cooperative Agents

Activity-based models in transportation science focus on the description of human trips and activities. Modeling the spatial decision for so-called secondary activities is addressed in this paper. Given both home and work locations, where do individuals perform activities such as shopping and leisure? Simulation of these decisions using random utility models requires a full enumeration of possible outcomes. For large data sets, it becomes computationally unfeasible because of the combinatorial complexity. To overcome that limitation, a model is proposed in which agents have limited, accurate information about a small subset of the overall spatial environment. Agents are interconnected by a social network through which they can exchange information. This approach has several advantages compared with the explicit simulation of a standard random utility model: (a) it computes plausible choice sets in reasonable computing times, (b) it can be extended easily to integrate further empirical evidence about travel behavior, and (c) it provides a useful framework to study the propagation of any newly available information. This paper emphasizes the computational efficiency of the approach for real-world examples

The MATSim Network Flow Model for Traffic Simulation Adapted to Large-Scale Emergency Egress and an Application to the Evacuation of the Indonesian City of Padang in Case of a Tsunami Warning

The evacuation of whole cities or even regions is an important problem, as demonstrated by recent events such as evacuation of Houston in the case of Hurricane Rita or the evacuation of coastal cities in the case of Tsunamis. This paper describes a complex evacuation simulation framework for the city of Pandang, with approximately 1,000,000 inhabitants. Padang faces a high risk of being inundated by a tsunami wave. The evacuation simulation is based on the MATSim framework for large-scale transport simulations. Different optimization parameters like evacuation distance, evacuation time, or the variation of the advance warning time are investigated. The results are given as overall evacuation times, evacuation curves, an detailed GIS analysis of the evacuation directions. All these results are discussed with regard to their usability for evacuation recommendations.BMBF, 03G0666E, Verbundprojekt FW: Last-mile Evacuation; Vorhaben: Evakuierungsanalyse und Verkehrsoptimierung, Evakuierungsplan einer Stadt - Sonderprogramm GEOTECHNOLOGIENBMBF, 03NAPAI4, Transport und Verkehr: Verbundprojekt ADVEST: Adaptive Verkehrssteuerung; Teilprojekt Verkehrsplanung und Verkehrssteuerung in Megacitie

Risk Minimizing Evacuation Strategies under Uncertainty

This paper presents results on the simulation of the evacuation of the city of Padang with approximately 1,000,000 inhabitants. The model used is MATSim (www.matsim.org). Three different strategies were applied: shortest path solution, user optimum, system optimum, together with a constraint that moves should reduce risk whenever possible. The introduction of the risk minimization increases the overall required safe egress time (RSET). The differences between the RSET for the three risk minimizing strategies are small. Further quantities used for the assessment of the evacuation are the formation of congestion and the individual RSETs (in comparison with the available SET).BMBF, 03G0666E, Verbundprojekt FW: Last-mile Evacuation; Vorhaben: Evakuierungsanalyse und Verkehrsoptimierung, Evakuierungsplan einer Stadt - Sonderprogramm GEOTECHNOLOGIENBMBF, 03NAPAI4, Transport und Verkehr: Verbundprojekt ADVEST: Adaptive Verkehrssteuerung; Teilprojekt Verkehrsplanung und Verkehrssteuerung in Megacitie

Adaptive traffic signal control using approximate dynamic programming

This paper presents a study on an adaptive traffic signal controller for real-time operation. The controller aims for three operational objectives: dynamic allocation of green time, automatic adjustment to control parameters, and fast revision of signal plans. The control algorithm is built on approximate dynamic programming (ADP). This approach substantially reduces computational burden by using an approximation to the value function of the dynamic programming and reinforcement learning to update the approximation. We investigate temporal-difference learning and perturbation learning as specific learning techniques for the ADP approach. We find in computer simulation that the ADP controllers achieve substantial reduction in vehicle delays in comparison with optimised fixed-time plans. Our results show that substantial benefits can be gained by increasing the frequency at which the signal plans are revised, which can be achieved conveniently using the ADP approach