CoLight: Learning Network-level Cooperation for Traffic Signal Control

Cooperation among the traffic signals enables vehicles to move through intersections more quickly. Conventional transportation approaches implement cooperation by pre-calculating the offsets between two intersections. Such pre-calculated offsets are not suitable for dynamic traffic environments. To enable cooperation of traffic signals, in this paper, we propose a model, CoLight, which uses graph attentional networks to facilitate communication. Specifically, for a target intersection in a network, CoLight can not only incorporate the temporal and spatial influences of neighboring intersections to the target intersection, but also build up index-free modeling of neighboring intersections. To the best of our knowledge, we are the first to use graph attentional networks in the setting of reinforcement learning for traffic signal control and to conduct experiments on the large-scale road network with hundreds of traffic signals. In experiments, we demonstrate that by learning the communication, the proposed model can achieve superior performance against the state-of-the-art methods.Comment: 10 pages. Proceedings of the 28th ACM International on Conference on Information and Knowledge Management. ACM, 201

A self-learning intersection control system for connected and automated vehicles

This study proposes a Decentralized Sparse Coordination Learning System (DSCLS) based on Deep Reinforcement Learning (DRL) to control intersections under the Connected and Automated Vehicles (CAVs) environment. In this approach, roadway sections are divided into small areas; vehicles try to reserve their desired area ahead of time, based on having a common desired area with other CAVs; the vehicles would be in an independent or coordinated state. Individual CAVs are set accountable for decision-making at each step in both coordinated and independent states. In the training process, CAVs learn to minimize the overall delay at the intersection. Due to the chain impact of taking random actions in the training course, the trained model can deal with unprecedented volume circumstances, the main challenge in intersection management. Application of the model to a single-lane intersection with no turning movement as a proof-of-concept test reveals noticeable improvements in traffic measures compared to three other intersection control systems. A Spring Mass Damper (SMD) model is developed to control platooning behavior of CAVs. In the SMD model, each vehicle is assumed as a mass, coupled with its preceding vehicle with a spring and a damper. The spring constant and damper coefficient control the interaction between vehicles. Limitations on communication range and the number of vehicles in each platoon are applied in this model, and the SMD model controls intra-platoon and inter-platoon interactions. The simulation result for a regular highway section reveals that the proposed platooning algorithm increases the maximum throughput by 29% and 63% under 50% and 100% market penetration rate of CAVs. A merging section with different volume combinations on the main section and merging section and different market penetration rates of CAVs is also modeled to test inter-platoon spacing performance in accommodating merging vehicles. Noticeable travel time reduction is observed in both mainline and merging lanes and under all volume combinations in 80% and higher MPR of CAVs. For a more reliable assessment of the DSCLS, the model is applied to a more realistic intersection, including three approaching lanes in each direction and turning movements. The proposed algorithm decreases delay by 58%, 19%, and 13% in moderate, high, and extreme volume regimes, improving travel time accordingly. Comparison of safety measures reveals 28% improvement in Post Encroachment Time (PET) in the extreme volume regime and minor improvements in high and moderate volume regimes. Due to the limited acceleration and deceleration rates, the proposed model does not show a better performance in environmental measures, including fuel consumption and CO2 emission, compared to the conventional control systems. However, the DSCLS noticeably outperforms the other pixel-reservation counterpart control system, with limited acceleration and deceleration rates. The application of the model to a corridor of four interactions shows the same trends in traffic, safety, and environmental measures as the single intersection experiment. An automated intersection control system for platooning CAVs is developed by combining the two proposed models, which remarkably improves traffic and safety measures, specifically in extreme volume regimes compared to the regular DSCLS model

The urban real-time traffic control (URTC) system : a study of designing the controller and its simulation

The growth of the number of automobiles on the roads in China has put higher demands on the traffic control system that needs to efficiently reduce the level of congestion occurrence, which increases travel delay, fuel consumption, and air pollution. The traffic control system, urban real-time traffic control system based on multi-agent (MA-URTC) is presented in this thesis. According to the present situation and the traffic's future development in China, the researches on intelligent traffic control strategy and simulation based on agent lays a foundation for the realization of the system. The thesis is organized as follows: The first part focuses on the intersection' real-time signal control strategy. It contains the limitations of current traffic control systems, application of artificial intelligence in the research, how to bring the dynamic traffic flow forecast into effect by combining the neural network with the genetic arithmetic, and traffic signal real-time control strategy based on fuzzy control. The author uses sorne simple simulation results to testify its superiority. We adopt the latest agent technology in designing the logical structure of the MA-URTC system. By exchanging traffic flows information among the relative agents, MA-URTC provides a new concept in urban traffic control. With a global coordination and cooperation on autonomy-based view of the traffic in cities, MA-URTC anticipates the congestion and control traffic flows. It is designed to support the real-time dynamic selection of intelligent traffic control strategy and the real-time communication requirements, together with a sufficient level of fault-tolerance. Due to the complexity and levity of urban traffic, none strategy can be universally applicable. The agent can independently choose the best scheme according to the real-time situation. To develop an advanced traffic simulation system it can be helpful for us to find the best scheme and the best switch-point of different schemes. Thus we can better deal with the different real-time traffic situations. The second part discusses the architecture and function of the intelligent traffic control simulation based on agent. Meanwhile the author discusses the design model of the vehicle-agent, road agent in traffic network and the intersection-agent so that we can better simulate the real-time environment. The vehicle-agent carries out the intelligent simulation based on the characteristics of the drivers in the actual traffic condition to avoid the disadvantage of the traditional traffic simulation system, simple-functioned algorithm of the vehicles model and unfeasible forecasting hypothesis. It improves the practicability of the whole simulation system greatly. The road agent's significance lies in its guidance of the traffic participants. It avoids the urban traffic control that depends on only the traffic signal control at intersection. It gives the traffic participants the most comfortable and direct guidance in traveling. It can also make a real-time and dynamic adjustment on the urban traffic flow, thus greatly lighten the pressure of signal control in intersection area. To sorne extent, the road agent is equal to the pre-caution mechanism. In the future, the construction of urban roads tends to be more intelligent. Therefore, the research on road agent is very important. All kinds of agents in MA-URTC are interconnected through a computer network. In the end, the author discusses the direction of future research. As the whole system is a multi-agent system, the intersection, the road and the vehicle belongs to multi-agent system respectively. So the emphasis should be put on the structure design and communication of all kinds of traffic agents in the system. Meanwhile, as an open and flexible real-time traffic control system, it is also concerned with how to collaborate with other related systems effectively, how to conform the resources and how to make the traffic participants anywhere throughout the city be in the best traffic guidance at all times and places. To actualize the genuine ITS will be our final goal. \ud ______________________________________________________________________________ \ud MOTS-CLÉS DE L’AUTEUR : Artificial Intelligence, Computer simulation, Fuzzy control, Genetic Algorithm, Intelligent traffic control, ITS, Multi-agent, Neural Network, Real-time

Multi-Agent Reinforcement Learning for Connected and Automated Vehicles Control: Recent Advancements and Future Prospects

Connected and automated vehicles (CAVs) have emerged as a potential solution to the future challenges of developing safe, efficient, and eco-friendly transportation systems. However, CAV control presents significant challenges, given the complexity of interconnectivity and coordination required among the vehicles. To address this, multi-agent reinforcement learning (MARL), with its notable advancements in addressing complex problems in autonomous driving, robotics, and human-vehicle interaction, has emerged as a promising tool for enhancing the capabilities of CAVs. However, there is a notable absence of current reviews on the state-of-the-art MARL algorithms in the context of CAVs. Therefore, this paper delivers a comprehensive review of the application of MARL techniques within the field of CAV control. The paper begins by introducing MARL, followed by a detailed explanation of its unique advantages in addressing complex mobility and traffic scenarios that involve multiple agents. It then presents a comprehensive survey of MARL applications on the extent of control dimensions for CAVs, covering critical and typical scenarios such as platooning control, lane-changing, and unsignalized intersections. In addition, the paper provides a comprehensive review of the prominent simulation platforms used to create reliable environments for training in MARL. Lastly, the paper examines the current challenges associated with deploying MARL within CAV control and outlines potential solutions that can effectively overcome these issues. Through this review, the study highlights the tremendous potential of MARL to enhance the performance and collaboration of CAV control in terms of safety, travel efficiency, and economy

Human Motion Trajectory Prediction: A Survey

With growing numbers of intelligent autonomous systems in human environments, the ability of such systems to perceive, understand and anticipate human behavior becomes increasingly important. Specifically, predicting future positions of dynamic agents and planning considering such predictions are key tasks for self-driving vehicles, service robots and advanced surveillance systems. This paper provides a survey of human motion trajectory prediction. We review, analyze and structure a large selection of work from different communities and propose a taxonomy that categorizes existing methods based on the motion modeling approach and level of contextual information used. We provide an overview of the existing datasets and performance metrics. We discuss limitations of the state of the art and outline directions for further research.Comment: Submitted to the International Journal of Robotics Research (IJRR), 37 page