CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario

Traffic signal control is an emerging application scenario for reinforcement learning. Besides being as an important problem that affects people's daily life in commuting, traffic signal control poses its unique challenges for reinforcement learning in terms of adapting to dynamic traffic environment and coordinating thousands of agents including vehicles and pedestrians. A key factor in the success of modern reinforcement learning relies on a good simulator to generate a large number of data samples for learning. The most commonly used open-source traffic simulator SUMO is, however, not scalable to large road network and large traffic flow, which hinders the study of reinforcement learning on traffic scenarios. This motivates us to create a new traffic simulator CityFlow with fundamentally optimized data structures and efficient algorithms. CityFlow can support flexible definitions for road network and traffic flow based on synthetic and real-world data. It also provides user-friendly interface for reinforcement learning. Most importantly, CityFlow is more than twenty times faster than SUMO and is capable of supporting city-wide traffic simulation with an interactive render for monitoring. Besides traffic signal control, CityFlow could serve as the base for other transportation studies and can create new possibilities to test machine learning methods in the intelligent transportation domain.Comment: WWW 2019 Demo Pape

The urban real-time traffic control (URTC) system : a study of designing the controller and its simulation

The growth of the number of automobiles on the roads in China has put higher demands on the traffic control system that needs to efficiently reduce the level of congestion occurrence, which increases travel delay, fuel consumption, and air pollution. The traffic control system, urban real-time traffic control system based on multi-agent (MA-URTC) is presented in this thesis. According to the present situation and the traffic's future development in China, the researches on intelligent traffic control strategy and simulation based on agent lays a foundation for the realization of the system. The thesis is organized as follows: The first part focuses on the intersection' real-time signal control strategy. It contains the limitations of current traffic control systems, application of artificial intelligence in the research, how to bring the dynamic traffic flow forecast into effect by combining the neural network with the genetic arithmetic, and traffic signal real-time control strategy based on fuzzy control. The author uses sorne simple simulation results to testify its superiority. We adopt the latest agent technology in designing the logical structure of the MA-URTC system. By exchanging traffic flows information among the relative agents, MA-URTC provides a new concept in urban traffic control. With a global coordination and cooperation on autonomy-based view of the traffic in cities, MA-URTC anticipates the congestion and control traffic flows. It is designed to support the real-time dynamic selection of intelligent traffic control strategy and the real-time communication requirements, together with a sufficient level of fault-tolerance. Due to the complexity and levity of urban traffic, none strategy can be universally applicable. The agent can independently choose the best scheme according to the real-time situation. To develop an advanced traffic simulation system it can be helpful for us to find the best scheme and the best switch-point of different schemes. Thus we can better deal with the different real-time traffic situations. The second part discusses the architecture and function of the intelligent traffic control simulation based on agent. Meanwhile the author discusses the design model of the vehicle-agent, road agent in traffic network and the intersection-agent so that we can better simulate the real-time environment. The vehicle-agent carries out the intelligent simulation based on the characteristics of the drivers in the actual traffic condition to avoid the disadvantage of the traditional traffic simulation system, simple-functioned algorithm of the vehicles model and unfeasible forecasting hypothesis. It improves the practicability of the whole simulation system greatly. The road agent's significance lies in its guidance of the traffic participants. It avoids the urban traffic control that depends on only the traffic signal control at intersection. It gives the traffic participants the most comfortable and direct guidance in traveling. It can also make a real-time and dynamic adjustment on the urban traffic flow, thus greatly lighten the pressure of signal control in intersection area. To sorne extent, the road agent is equal to the pre-caution mechanism. In the future, the construction of urban roads tends to be more intelligent. Therefore, the research on road agent is very important. All kinds of agents in MA-URTC are interconnected through a computer network. In the end, the author discusses the direction of future research. As the whole system is a multi-agent system, the intersection, the road and the vehicle belongs to multi-agent system respectively. So the emphasis should be put on the structure design and communication of all kinds of traffic agents in the system. Meanwhile, as an open and flexible real-time traffic control system, it is also concerned with how to collaborate with other related systems effectively, how to conform the resources and how to make the traffic participants anywhere throughout the city be in the best traffic guidance at all times and places. To actualize the genuine ITS will be our final goal. \ud ______________________________________________________________________________ \ud MOTS-CLÉS DE L’AUTEUR : Artificial Intelligence, Computer simulation, Fuzzy control, Genetic Algorithm, Intelligent traffic control, ITS, Multi-agent, Neural Network, Real-time

A State-of-the-art Integrated Transportation Simulation Platform

Nowadays, universities and companies have a huge need for simulation and modelling methodologies. In the particular case of traffic and transportation, making physical modifications to the real traffic networks could be highly expensive, dependent on political decisions and could be highly disruptive to the environment. However, while studying a specific domain or problem, analysing a problem through simulation may not be trivial and may need several simulation tools, hence raising interoperability issues. To overcome these problems, we propose an agent-directed transportation simulation platform, through the cloud, by means of services. We intend to use the IEEE standard HLA (High Level Architecture) for simulators interoperability and agents for controlling and coordination. Our motivations are to allow multiresolution analysis of complex domains, to allow experts to collaborate on the analysis of a common problem and to allow co-simulation and synergy of different application domains. This paper will start by presenting some preliminary background concepts to help better understand the scope of this work. After that, the results of a literature review is shown. Finally, the general architecture of a transportation simulation platform is proposed

An agent-based approach to assess drivers’ interaction with pre-trip information systems.

This article reports on the practical use of a multi-agent microsimulation framework to address the issue of assessing drivers’ responses to pretrip information systems. The population of drivers is represented as a community of autonomous agents, and travel demand results from the decision-making deliberation performed by each individual of the population as regards route and departure time. A simple simulation scenario was devised, where pretrip information was made available to users on an individual basis so that its effects at the aggregate level could be observed. The simulation results show that the overall performance of the system is very likely affected by exogenous information, and these results are ascribed to demand formation and network topology. The expressiveness offered by cognitive approaches based on predicate logics, such as the one used in this research, appears to be a promising approximation to fostering more complex behavior modelling, allowing us to represent many of the mental aspects involved in the deliberation process