CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario

Traffic signal control is an emerging application scenario for reinforcement learning. Besides being as an important problem that affects people's daily life in commuting, traffic signal control poses its unique challenges for reinforcement learning in terms of adapting to dynamic traffic environment and coordinating thousands of agents including vehicles and pedestrians. A key factor in the success of modern reinforcement learning relies on a good simulator to generate a large number of data samples for learning. The most commonly used open-source traffic simulator SUMO is, however, not scalable to large road network and large traffic flow, which hinders the study of reinforcement learning on traffic scenarios. This motivates us to create a new traffic simulator CityFlow with fundamentally optimized data structures and efficient algorithms. CityFlow can support flexible definitions for road network and traffic flow based on synthetic and real-world data. It also provides user-friendly interface for reinforcement learning. Most importantly, CityFlow is more than twenty times faster than SUMO and is capable of supporting city-wide traffic simulation with an interactive render for monitoring. Besides traffic signal control, CityFlow could serve as the base for other transportation studies and can create new possibilities to test machine learning methods in the intelligent transportation domain.Comment: WWW 2019 Demo Pape

A State-of-the-art Integrated Transportation Simulation Platform

Nowadays, universities and companies have a huge need for simulation and modelling methodologies. In the particular case of traffic and transportation, making physical modifications to the real traffic networks could be highly expensive, dependent on political decisions and could be highly disruptive to the environment. However, while studying a specific domain or problem, analysing a problem through simulation may not be trivial and may need several simulation tools, hence raising interoperability issues. To overcome these problems, we propose an agent-directed transportation simulation platform, through the cloud, by means of services. We intend to use the IEEE standard HLA (High Level Architecture) for simulators interoperability and agents for controlling and coordination. Our motivations are to allow multiresolution analysis of complex domains, to allow experts to collaborate on the analysis of a common problem and to allow co-simulation and synergy of different application domains. This paper will start by presenting some preliminary background concepts to help better understand the scope of this work. After that, the results of a literature review is shown. Finally, the general architecture of a transportation simulation platform is proposed

Jamming transition in air transportation networks

In this work we present a model of an air transportation traffic system from the complex network modelling viewpoint. In the network, every node corresponds to a given airport, and two nodes are connected by means of flight routes. Each node is weighted according to its load capacity, and links are weighted according to the Euclidean distance that separates each pair of nodes. Local rules describing the behavior of individual nodes in terms of the surrounding flow have been also modelled, and a random network topology has been chosen in a baseline approach. Numerical simulations describing the diffusion of a given number of agents (aircraft) in this network show the onset of a jamming transition that distinguishes an efficient regime with null amount of airport queues and high diffusivity (free phase) and a regime where bottlenecks suddenly take place, leading to a poor aircraft diffusion (congested phase). Fluctuations are maximal around the congestion threshold, suggesting that the transition is critical. We then proceed by exploring the robustness of our results in neutral random topologies by embedding the model in heterogeneous networks. Specifically, we make use of the European air transportation network formed by 858 airports and 11170 flight routes connecting them, which we show to be scale-free. The jamming transition is also observed in this case. These results and methodologies may introduce relevant decision making procedures in order to optimize the air transportation traffic