HARL: A Novel Hierachical Adversary Reinforcement Learning for Automoumous Intersection Management

As an emerging technology, Connected Autonomous Vehicles (CAVs) are believed to have the ability to move through intersections in a faster and safer manner, through effective Vehicle-to-Everything (V2X) communication and global observation. Autonomous intersection management is a key path to efficient crossing at intersections, which reduces unnecessary slowdowns and stops through adaptive decision process of each CAV, enabling fuller utilization of the intersection space. Distributed reinforcement learning (DRL) offers a flexible, end-to-end model for AIM, adapting for many intersection scenarios. While DRL is prone to collisions as the actions of multiple sides in the complicated interactions are sampled from a generic policy, restricting the application of DRL in realistic scenario. To address this, we propose a hierarchical RL framework where models at different levels vary in receptive scope, action step length, and feedback period of reward. The upper layer model accelerate CAVs to prevent them from being clashed, while the lower layer model adjust the trends from upper layer model to avoid the change of mobile state causing new conflicts. And the real action of CAV at each step is co-determined by the trends from both levels, forming a real-time balance in the adversarial process. The proposed model is proven effective in the experiment undertaken in a complicated intersection with 4 branches and 4 lanes each branch, and show better performance compared with baselines

Simulation Research on Driving Behaviour of Autonomous Vehicles on Expressway Ramp Under the Background of Vehicle-Road Coordination

Constructing a risk model with the subject of autonomous vehicles to screen out the vehicles of potential conflicts and analyze their choices under different strategies. Based on the co-simulation of Python and SUMO, establishing a model of on-ramp merge driving behaviour of autonomous vehicles based on non-cooperative static game. Under this model, the experiment results that the average speed in the merging area is increased by 12.7%, the standard deviation of the average speed is reduced by 35.46%, and the number of the vehicles successfully merged before the end of the merging area is 4.86 times that of traditional method, indicate that the model can effectively help the vehicles be merged and improve the traffic efficiency to a certain extent

LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving

Existing learning-based autonomous driving (AD) systems face challenges in comprehending high-level information, generalizing to rare events, and providing interpretability. To address these problems, this work employs Large Language Models (LLMs) as a decision-making component for complex AD scenarios that require human commonsense understanding. We devise cognitive pathways to enable comprehensive reasoning with LLMs, and develop algorithms for translating LLM decisions into actionable driving commands. Through this approach, LLM decisions are seamlessly integrated with low-level controllers by guided parameter matrix adaptation. Extensive experiments demonstrate that our proposed method not only consistently surpasses baseline approaches in single-vehicle tasks, but also helps handle complex driving behaviors even multi-vehicle coordination, thanks to the commonsense reasoning capabilities of LLMs. This paper presents an initial step toward leveraging LLMs as effective decision-makers for intricate AD scenarios in terms of safety, efficiency, generalizability, and interoperability. We aspire for it to serve as inspiration for future research in this field. Project page: https://sites.google.com/view/llm-mp

A Systematic Survey of Control Techniques and Applications: From Autonomous Vehicles to Connected and Automated Vehicles

Vehicle control is one of the most critical challenges in autonomous vehicles (AVs) and connected and automated vehicles (CAVs), and it is paramount in vehicle safety, passenger comfort, transportation efficiency, and energy saving. This survey attempts to provide a comprehensive and thorough overview of the current state of vehicle control technology, focusing on the evolution from vehicle state estimation and trajectory tracking control in AVs at the microscopic level to collaborative control in CAVs at the macroscopic level. First, this review starts with vehicle key state estimation, specifically vehicle sideslip angle, which is the most pivotal state for vehicle trajectory control, to discuss representative approaches. Then, we present symbolic vehicle trajectory tracking control approaches for AVs. On top of that, we further review the collaborative control frameworks for CAVs and corresponding applications. Finally, this survey concludes with a discussion of future research directions and the challenges. This survey aims to provide a contextualized and in-depth look at state of the art in vehicle control for AVs and CAVs, identifying critical areas of focus and pointing out the potential areas for further exploration

Improved Consensus ADMM for Cooperative Motion Planning of Large-Scale Connected Autonomous Vehicles with Limited Communication

This paper investigates a cooperative motion planning problem for large-scale connected autonomous vehicles (CAVs) under limited communications, which addresses the challenges of high communication and computing resource requirements. Our proposed methodology incorporates a parallel optimization algorithm with improved consensus ADMM considering a more realistic locally connected topology network, and time complexity of O(N) is achieved by exploiting the sparsity in the dual update process. To further enhance the computational efficiency, we employ a lightweight evolution strategy for the dynamic connectivity graph of CAVs, and each sub-problem split from the consensus ADMM only requires managing a small group of CAVs. The proposed method implemented with the receding horizon scheme is validated thoroughly, and comparisons with existing numerical solvers and approaches demonstrate the efficiency of our proposed algorithm. Also, simulations on large-scale cooperative driving tasks involving 80 vehicles are performed in the high-fidelity CARLA simulator, which highlights the remarkable computational efficiency, scalability, and effectiveness of our proposed development. Demonstration videos are available at https://henryhcliu.github.io/icadmm_cmp_carla.Comment: 15 pages, 10 figure

Learn to Bet: Using Reinforcement Learning to Improve Vehicle Bids in Auction-Based Smart Intersections

With the advent of IoT, cities will soon be populated by autonomous vehicles and managed by intelligent systems capable of actively interacting with city infrastructures and vehicles. In this work, we propose a model based on reinforcement learning that teaches to autonomous connected vehicles how to save resources while navigating in such an environment. In particular, we focus on budget savings in the context of auction-based intersection management systems. We trained several models with Deep Q-learning by varying traffic conditions to find the most performance-effective variant in terms of the trade-off between saved currency and trip times. Afterward, we compared the performance of our model with previously proposed and random strategies, even under adverse traffic conditions. Our model appears to be robust and manages to save a considerable amount of currency without significantly increasing the waiting time in traffic. For example, the learner bidder saves at least 20% of its budget with heavy traffic conditions and up to 74% in lighter traffic with respect to a standard bidder, and around three times the saving of a random bidder. The results and discussion suggest practical adoption of the proposal in a foreseen future real-life scenario

The Hierarchical Control Method for Coordinating a Group of Connected Vehicles on Urban Roads

Safety, mobility and environmental impact are the three major challenges in today\u27s transportation system. As the advances in wireless communication and vehicle automation technologies, they have rapidly led to the emergence and development of connected and automated vehicles (CAVs). We can expect fully CAVs by 2030. The CAV technologies offer another solution for the issues we are dealing with in the current transportation system. In the meanwhile, urban roads are one of the most important part in the transportation network. Urban roads are characterized by multiple interconnected intersections. They are more complicated than highway traffic, because the vehicles on the urban roads are moving in multiple directions with higher relative velocity. Most of the traffic accidents happened at intersections and the intersections are the major contribution to the traffic congestions. Our urban road infrastructures are also becoming more intelligent. Sensor-embedded roadways are continuously gathering traffic data from passing vehicles. Our smart vehicles are meeting intelligent roads. However, we have not taken the fully advantages of the data rich traffic environment provided by the connected vehicle technologies and intelligent road infrastructures. The objective of this research is to develop a coordination control strategy for a group of connected vehicles under intelligent traffic environment, which can guide the vehicles passing through the intersections and make smart lane change decisions with the objective of improving overall fuel economy and traffic mobility. The coordination control strategy should also be robust to imperfect connectivity conditions with various connected vehicle penetration rate. This dissertation proposes a hierarchical control method to coordinate a group of connected vehicles travelling on urban roads with intersections. The dissertation includes four parts of the application of our proposed method: First, we focus on the coordination of the connected vehicles on the multiple interconnected unsignalized intersection roads, where the traffic signals are removed and the collision avoidance at the intersection area relays on the communication and cooperation of the connected vehicles and intersection controllers. Second, a fuel efficient hierarchical control method is proposed to control the connected vehicles travel on the signalized intersection roads. With the signal phase and timing (SPAT) information, our proposed approach is able to help the connected vehicles minimize red light idling and improve the fuel economy at the same time. Third, the research is extended form single lane to multiple lane, where the connected vehicle discretionary and cooperative mandatory lane change have been explored. Finally, we have analysis the real-world implementation potential of our proposed algorithm including the communication delay and real-time implementation analysis

A comprehensive survey on cooperative intersection management for heterogeneous connected vehicles

Nowadays, with the advancement of technology, world is trending toward high mobility and dynamics. In this context, intersection management (IM) as one of the most crucial elements of the transportation sector demands high attention. Today, road entities including infrastructures, vulnerable road users (VRUs) such as motorcycles, moped, scooters, pedestrians, bicycles, and other types of vehicles such as trucks, buses, cars, emergency vehicles, and railway vehicles like trains or trams are able to communicate cooperatively using vehicle-to-everything (V2X) communications and provide traffic safety, efficiency, infotainment and ecological improvements. In this paper, we take into account different types of intersections in terms of signalized, semi-autonomous (hybrid) and autonomous intersections and conduct a comprehensive survey on various intersection management methods for heterogeneous connected vehicles (CVs). We consider heterogeneous classes of vehicles such as road and rail vehicles as well as VRUs including bicycles, scooters and motorcycles. All kinds of intersection goals, modeling, coordination architectures, scheduling policies are thoroughly discussed. Signalized and semi-autonomous intersections are assessed with respect to these parameters. We especially focus on autonomous intersection management (AIM) and categorize this section based on four major goals involving safety, efficiency, infotainment and environment. Each intersection goal provides an in-depth investigation on the corresponding literature from the aforementioned perspectives. Moreover, robustness and resiliency of IM are explored from diverse points of view encompassing sensors, information management and sharing, planning universal scheme, heterogeneous collaboration, vehicle classification, quality measurement, external factors, intersection types, localization faults, communication anomalies and channel optimization, synchronization, vehicle dynamics and model mismatch, model uncertainties, recovery, security and privacy