Decentralized Cooperative Planning for Automated Vehicles with Continuous Monte Carlo Tree Search

Urban traffic scenarios often require a high degree of cooperation between traffic participants to ensure safety and efficiency. Observing the behavior of others, humans infer whether or not others are cooperating. This work aims to extend the capabilities of automated vehicles, enabling them to cooperate implicitly in heterogeneous environments. Continuous actions allow for arbitrary trajectories and hence are applicable to a much wider class of problems than existing cooperative approaches with discrete action spaces. Based on cooperative modeling of other agents, Monte Carlo Tree Search (MCTS) in conjunction with Decoupled-UCT evaluates the action-values of each agent in a cooperative and decentralized way, respecting the interdependence of actions among traffic participants. The extension to continuous action spaces is addressed by incorporating novel MCTS-specific enhancements for efficient search space exploration. The proposed algorithm is evaluated under different scenarios, showing that the algorithm is able to achieve effective cooperative planning and generate solutions egocentric planning fails to identify

Decentralized Cooperative Planning for Automated Vehicles with Hierarchical Monte Carlo Tree Search

Today's automated vehicles lack the ability to cooperate implicitly with others. This work presents a Monte Carlo Tree Search (MCTS) based approach for decentralized cooperative planning using macro-actions for automated vehicles in heterogeneous environments. Based on cooperative modeling of other agents and Decoupled-UCT (a variant of MCTS), the algorithm evaluates the state-action-values of each agent in a cooperative and decentralized manner, explicitly modeling the interdependence of actions between traffic participants. Macro-actions allow for temporal extension over multiple time steps and increase the effective search depth requiring fewer iterations to plan over longer horizons. Without predefined policies for macro-actions, the algorithm simultaneously learns policies over and within macro-actions. The proposed method is evaluated under several conflict scenarios, showing that the algorithm can achieve effective cooperative planning with learned macro-actions in heterogeneous environments

Bringing Diversity to Autonomous Vehicles: An Interpretable Multi-vehicle Decision-making and Planning Framework

With the development of autonomous driving, it is becoming increasingly common for autonomous vehicles (AVs) and human-driven vehicles (HVs) to travel on the same roads. Existing single-vehicle planning algorithms on board struggle to handle sophisticated social interactions in the real world. Decisions made by these methods are difficult to understand for humans, raising the risk of crashes and making them unlikely to be applied in practice. Moreover, vehicle flows produced by open-source traffic simulators suffer from being overly conservative and lacking behavioral diversity. We propose a hierarchical multi-vehicle decision-making and planning framework with several advantages. The framework jointly makes decisions for all vehicles within the flow and reacts promptly to the dynamic environment through a high-frequency planning module. The decision module produces interpretable action sequences that can explicitly communicate self-intent to the surrounding HVs. We also present the cooperation factor and trajectory weight set, bringing diversity to autonomous vehicles in traffic at both the social and individual levels. The superiority of our proposed framework is validated through experiments with multiple scenarios, and the diverse behaviors in the generated vehicle trajectories are demonstrated through closed-loop simulations

Deep Reinforcement Learning and Game Theoretic Monte Carlo Decision Process for Safe and Efficient Lane Change Maneuver and Speed Management

Predicting the states of the surrounding traffic is one of the major problems in automated driving. Maneuvers such as lane change, merge, and exit management could pose challenges in the absence of intervehicular communication and can benefit from driver behavior prediction. Predicting the motion of surrounding vehicles and trajectory planning need to be computationally efficient for real-time implementation. This dissertation presents a decision process model for real-time automated lane change and speed management in highway and urban traffic. In lane change and merge maneuvers, it is important to know how neighboring vehicles will act in the imminent future. Human driver models, probabilistic approaches, rule-base techniques, and machine learning approach have addressed this problem only partially as they do not focus on the behavioral features of the vehicles. The main goal of this research is to develop a fast algorithm that predicts the future states of the neighboring vehicles, runs a fast decision process, and learns the regretfulness and rewardfulness of the executed decisions. The presented algorithm is developed based on level-K game theory to model and predict the interaction between the vehicles. Using deep reinforcement learning, this algorithm encodes and memorizes the past experiences that are recurrently used to reduce the computations and speed up motion planning. Also, we use Monte Carlo Tree Search (MCTS) as an effective tool that is employed nowadays for fast planning in complex and dynamic game environments. This development leverages the computation power efficiently and showcases promising outcomes for maneuver planning and predicting the environment’s dynamics. In the absence of traffic connectivity that may be due to either passenger’s choice of privacy or the vehicle’s lack of technology, this development can be extended and employed in automated vehicles for real-world and practical applications

Behavior planning for automated highway driving

This work deals with certain components of an automated driving system for highways, focusing on lane change behavior planning. It presents a variety of algorithms of a modular system aiming at safe and comfortable driving. A major contribution of this work is a method for analyzing traffic scenes in a spatio-temporal, curvilinear coordinate system. The results of this analysis are used in a further step to generate lane change trajectories. A total of three approaches with increasing levels of complexity and capabilities are compared. The most advanced approach formulates the problem as a linear-quadratic cooperative game and accounts for the inherently uncertain and multimodal nature of trajectory predictions for surrounding road users. Evaluations on real data show that the developed algorithms can be integrated into current generation automated driving software systems fulfilling runtime constraints

A Computationally Efficient Bi-level Coordination Framework for CAVs at Unsignalized Intersections

In this paper, we investigate cooperative vehicle coordination for connected and automated vehicles (CAVs) at unsignalized intersections. To support high traffic throughput while reducing computational complexity, we present a novel collision region model and decompose the optimal coordination problem into two sub-problems: \textit{centralized} priority scheduling and \textit{distributed} trajectory planning. Then, we propose a bi-level coordination framework which includes: (i) a Monte Carlo Tree Search (MCTS)-based high-level priority scheduler aims to find high-quality passing orders to maximize traffic throughput, and (ii) a priority-based low-level trajectory planner that generates optimal collision-free control inputs. Simulation results demonstrate that our bi-level strategy achieves near-optimal coordination performance, comparable to state-of-the-art centralized strategies, and significantly outperform the traffic signal control systems in terms of traffic throughput. Moreover, our approach exhibits good scalability, with computational complexity scaling linearly with the number of vehicles. Video demonstrations can be found online at \url{https://youtu.be/WYAKFMNnQfs}

Automated driving and autonomous functions on road vehicles

In recent years, road vehicle automation has become an important and popular topic for research and development in both academic and industrial spheres. New developments received extensive coverage in the popular press, and it may be said that the topic has captured the public imagination. Indeed, the topic has generated interest across a wide range of academic, industry and governmental communities, well beyond vehicle engineering; these include computer science, transportation, urban planning, legal, social science and psychology. While this follows a similar surge of interest – and subsequent hiatus – of Automated Highway Systems in the 1990’s, the current level of interest is substantially greater, and current expectations are high. It is common to frame the new technologies under the banner of “self-driving cars” – robotic systems potentially taking over the entire role of the human driver, a capability that does not fully exist at present. However, this single vision leads one to ignore the existing range of automated systems that are both feasible and useful. Recent developments are underpinned by substantial and long-term trends in “computerisation” of the automobile, with developments in sensors, actuators and control technologies to spur the new developments in both industry and academia. In this paper we review the evolution of the intelligent vehicle and the supporting technologies with a focus on the progress and key challenges for vehicle system dynamics. A number of relevant themes around driving automation are explored in this article, with special focus on those most relevant to the underlying vehicle system dynamics. One conclusion is that increased precision is needed in sensing and controlling vehicle motions, a trend that can mimic that of the aerospace industry, and similarly benefit from increased use of redundant by-wire actuators

TrafficMCTS: A Closed-Loop Traffic Flow Generation Framework with Group-Based Monte Carlo Tree Search

Digital twins for intelligent transportation systems are currently attracting great interests, in which generating realistic, diverse, and human-like traffic flow in simulations is a formidable challenge. Current approaches often hinge on predefined driver models, objective optimization, or reliance on pre-recorded driving datasets, imposing limitations on their scalability, versatility, and adaptability. In this paper, we introduce TrafficMCTS, an innovative framework that harnesses the synergy of groupbased Monte Carlo tree search (MCTS) and Social Value Orientation (SVO) to engender a multifaceted traffic flow replete with varying driving styles and cooperative tendencies. Anchored by a closed-loop architecture, our framework enables vehicles to dynamically adapt to their environment in real time, and ensure feasible collision-free trajectories. Through comprehensive comparisons with state-of-the-art methods, we illuminate the advantages of our approach in terms of computational efficiency, planning success rate, intent completion time, and diversity metrics. Besides, we simulate highway and roundabout scenarios to illustrate the effectiveness of the proposed framework and highlight its ability to induce diverse social behaviors within the traffic flow. Finally, we validate the scalability of TrafficMCTS by showcasing its prowess in simultaneously mass vehicles within a sprawling road network, cultivating a landscape of traffic flow that mirrors the intricacies of human behavior