Decision-Making for Automated Vehicles Using a Hierarchical Behavior-Based Arbitration Scheme

Behavior planning and decision-making are some of the biggest challenges for highly automated systems. A fully automated vehicle (AV) is confronted with numerous tactical and strategical choices. Most state-of-the-art AV platforms implement tactical and strategical behavior generation using finite state machines. However, these usually result in poor explainability, maintainability and scalability. Research in robotics has raised many architectures to mitigate these problems, most interestingly behavior-based systems and hybrid derivatives. Inspired by these approaches, we propose a hierarchical behavior-based architecture for tactical and strategical behavior generation in automated driving. It is a generalizing and scalable decision-making framework, utilizing modular behavior blocks to compose more complex behaviors in a bottom-up approach. The system is capable of combining a variety of scenario- and methodology-specific solutions, like POMDPs, RRT* or learning-based behavior, into one understandable and traceable architecture. We extend the hierarchical behavior-based arbitration concept to address scenarios where multiple behavior options are applicable but have no clear priority against each other. Then, we formulate the behavior generation stack for automated driving in urban and highway environments, incorporating parking and emergency behaviors as well. Finally, we illustrate our design in an explanatory evaluation

A self-learning intersection control system for connected and automated vehicles

This study proposes a Decentralized Sparse Coordination Learning System (DSCLS) based on Deep Reinforcement Learning (DRL) to control intersections under the Connected and Automated Vehicles (CAVs) environment. In this approach, roadway sections are divided into small areas; vehicles try to reserve their desired area ahead of time, based on having a common desired area with other CAVs; the vehicles would be in an independent or coordinated state. Individual CAVs are set accountable for decision-making at each step in both coordinated and independent states. In the training process, CAVs learn to minimize the overall delay at the intersection. Due to the chain impact of taking random actions in the training course, the trained model can deal with unprecedented volume circumstances, the main challenge in intersection management. Application of the model to a single-lane intersection with no turning movement as a proof-of-concept test reveals noticeable improvements in traffic measures compared to three other intersection control systems. A Spring Mass Damper (SMD) model is developed to control platooning behavior of CAVs. In the SMD model, each vehicle is assumed as a mass, coupled with its preceding vehicle with a spring and a damper. The spring constant and damper coefficient control the interaction between vehicles. Limitations on communication range and the number of vehicles in each platoon are applied in this model, and the SMD model controls intra-platoon and inter-platoon interactions. The simulation result for a regular highway section reveals that the proposed platooning algorithm increases the maximum throughput by 29% and 63% under 50% and 100% market penetration rate of CAVs. A merging section with different volume combinations on the main section and merging section and different market penetration rates of CAVs is also modeled to test inter-platoon spacing performance in accommodating merging vehicles. Noticeable travel time reduction is observed in both mainline and merging lanes and under all volume combinations in 80% and higher MPR of CAVs. For a more reliable assessment of the DSCLS, the model is applied to a more realistic intersection, including three approaching lanes in each direction and turning movements. The proposed algorithm decreases delay by 58%, 19%, and 13% in moderate, high, and extreme volume regimes, improving travel time accordingly. Comparison of safety measures reveals 28% improvement in Post Encroachment Time (PET) in the extreme volume regime and minor improvements in high and moderate volume regimes. Due to the limited acceleration and deceleration rates, the proposed model does not show a better performance in environmental measures, including fuel consumption and CO2 emission, compared to the conventional control systems. However, the DSCLS noticeably outperforms the other pixel-reservation counterpart control system, with limited acceleration and deceleration rates. The application of the model to a corridor of four interactions shows the same trends in traffic, safety, and environmental measures as the single intersection experiment. An automated intersection control system for platooning CAVs is developed by combining the two proposed models, which remarkably improves traffic and safety measures, specifically in extreme volume regimes compared to the regular DSCLS model

Validation of trajectory planning strategies for automated driving under cooperative, urban, and interurban scenarios.

149 p.En esta Tesis se estudia, diseña e implementa una arquitectura de control para vehículos automatizados de forma dual, que permite realizar pruebas en simulación y en vehículos reales con los mínimos cambios posibles. La arquitectura descansa sobre seis módulos: adquisición de información de sensores, percepción del entorno, comunicaciones e interacción con otros agentes, decisión de maniobras, control y actuación, además de la generación de mapas en el módulo de decisión, que utiliza puntos simples para la descripción de las estructuras de la ruta (rotondas, intersecciones, tramos rectos y cambios de carril)Tecnali

A comprehensive survey on cooperative intersection management for heterogeneous connected vehicles

Nowadays, with the advancement of technology, world is trending toward high mobility and dynamics. In this context, intersection management (IM) as one of the most crucial elements of the transportation sector demands high attention. Today, road entities including infrastructures, vulnerable road users (VRUs) such as motorcycles, moped, scooters, pedestrians, bicycles, and other types of vehicles such as trucks, buses, cars, emergency vehicles, and railway vehicles like trains or trams are able to communicate cooperatively using vehicle-to-everything (V2X) communications and provide traffic safety, efficiency, infotainment and ecological improvements. In this paper, we take into account different types of intersections in terms of signalized, semi-autonomous (hybrid) and autonomous intersections and conduct a comprehensive survey on various intersection management methods for heterogeneous connected vehicles (CVs). We consider heterogeneous classes of vehicles such as road and rail vehicles as well as VRUs including bicycles, scooters and motorcycles. All kinds of intersection goals, modeling, coordination architectures, scheduling policies are thoroughly discussed. Signalized and semi-autonomous intersections are assessed with respect to these parameters. We especially focus on autonomous intersection management (AIM) and categorize this section based on four major goals involving safety, efficiency, infotainment and environment. Each intersection goal provides an in-depth investigation on the corresponding literature from the aforementioned perspectives. Moreover, robustness and resiliency of IM are explored from diverse points of view encompassing sensors, information management and sharing, planning universal scheme, heterogeneous collaboration, vehicle classification, quality measurement, external factors, intersection types, localization faults, communication anomalies and channel optimization, synchronization, vehicle dynamics and model mismatch, model uncertainties, recovery, security and privacy

A convex optimal control framework for autonomous vehicle intersection crossing

Cooperative vehicle management emerges as a promising solution to improve road traffic safety and efficiency. This paper addresses the speed planning problem for connected and autonomous vehicles (CAVs) at an unsignalized intersection with consideration of turning maneuvers. The problem is approached by a hierarchical centralized coordination scheme that successively optimizes the crossing order and velocity trajectories of a group of vehicles so as to minimize their total energy consumption and travel time required to pass the intersection. For an accurate estimate of the energy consumption of each CAV, the vehicle modeling framework in this paper captures 1) friction losses that affect longitudinal vehicle dynamics, and 2) the powertrain of each CAV in line with a battery-electric architecture. It is shown that the underlying optimization problem subject to safety constraints for powertrain operation, cornering and collision avoidance, after convexification and relaxation in some aspects can be formulated as two second-order cone programs, which ensures a rapid solution search and a unique global optimum. Simulation case studies are provided showing the tightness of the convex relaxation bounds, the overall effectiveness of the proposed approach, and its advantages over a benchmark solution invoking the widely used first-in-first-out policy. The investigation of Pareto optimal solutions for the two objectives (travel time and energy consumption) highlights the importance of optimizing their trade-off, as small compromises in travel time could produce significant energy savings

A Modular, Adaptive, and Autonomous Transit System (MAATS): A In-motion Transfer Strategy and Performance Evaluation in Urban Grid Transit Networks

Dynamic traffic demand has been a longstanding challenge for the conventional transit system design and operation. The recent development of autonomous vehicles (AVs) makes it increasingly realistic to develop the next generation of transportation systems with the potential to improve operational performance and flexibility. In this study, we propose an innovative transit system with autonomous modular buses (AMBs) that is adaptive to dynamic traffic demands and not restricted to fixed routes and timetables. A unique transfer operation, termed as “in-motion transfer”, is introduced in this paper to transfer passengers between coupled modular buses in motion. A two-stage model is developed to facilitate in-motion transfer operations in optimally designing passenger transfer plans and AMB trajectories at intersections. In the proposed AMB system, all passengers can travel in the shortest path smoothly without having to actually alight and transfer between different bus lines. Numerical experiments demonstrate that the proposed transit system results in shorter travel time and a significantly reduced average number of transfers. While enjoying the above-mentioned benefits, the modular, adaptive, and autonomous transit system (MAATS) does not impose substantially higher energy consumption in comparison to the conventional bus syste

Graph Reinforcement Learning Application to Co-operative Decision-Making in Mixed Autonomy Traffic: Framework, Survey, and Challenges

Proper functioning of connected and automated vehicles (CAVs) is crucial for the safety and efficiency of future intelligent transport systems. Meanwhile, transitioning to fully autonomous driving requires a long period of mixed autonomy traffic, including both CAVs and human-driven vehicles. Thus, collaboration decision-making for CAVs is essential to generate appropriate driving behaviors to enhance the safety and efficiency of mixed autonomy traffic. In recent years, deep reinforcement learning (DRL) has been widely used in solving decision-making problems. However, the existing DRL-based methods have been mainly focused on solving the decision-making of a single CAV. Using the existing DRL-based methods in mixed autonomy traffic cannot accurately represent the mutual effects of vehicles and model dynamic traffic environments. To address these shortcomings, this article proposes a graph reinforcement learning (GRL) approach for multi-agent decision-making of CAVs in mixed autonomy traffic. First, a generic and modular GRL framework is designed. Then, a systematic review of DRL and GRL methods is presented, focusing on the problems addressed in recent research. Moreover, a comparative study on different GRL methods is further proposed based on the designed framework to verify the effectiveness of GRL methods. Results show that the GRL methods can well optimize the performance of multi-agent decision-making for CAVs in mixed autonomy traffic compared to the DRL methods. Finally, challenges and future research directions are summarized. This study can provide a valuable research reference for solving the multi-agent decision-making problems of CAVs in mixed autonomy traffic and can promote the implementation of GRL-based methods into intelligent transportation systems. The source code of our work can be found at https://github.com/Jacklinkk/Graph_CAVs.Comment: 22 pages, 7 figures, 10 tables. Currently under review at IEEE Transactions on Intelligent Transportation System