Towards Learning Feasible Hierarchical Decision-Making Policies in Urban Autonomous Driving

Modern learning-based algorithms, powered by advanced deep structured neural nets, have multifacetedly facilitated automated driving platforms, spanning from scene characterization and perception to low-level control and state estimation schemes. Nonetheless, urban autonomous driving is regarded as a challenging application for machine learning (ML) and artificial intelligence (AI) since the learnt driving policies must handle complex multi-agent driving scenarios with indeterministic intentions of road participants. In the case of unsignalized intersections, automating the decision-making process at these safety-critical environments entails comprehending numerous layers of abstractions associated with learning robust driving behaviors to allow the vehicle to drive safely and efficiently. Based on our in-depth investigation, we discern that an efficient, yet safe, decision-making scheme for navigating real-world unsignalized intersections does not exist yet. The state-of-the-art schemes lacked practicality to handle real-life complex scenarios as they utilize Low-fidelity vehicle dynamic models which makes them incapable of simulating the real dynamic motion in real-life driving applications. In addition, the conservative behavior of autonomous vehicles, which often overreact to threats which have low likelihood, degrades the overall driving quality and jeopardizes safety. Hence, enhancing driving behavior is essential to attain agile, yet safe, traversing maneuvers in such multi-agent environments. Therefore, the main goal of conducting this PhD research is to develop high-fidelity learning-based frameworks to enhance the autonomous decision-making process at these safety-critical environments. We focus this PhD dissertation on three correlated and complementary research challenges. In our first research challenge, we conduct an in-depth and comprehensive survey on the state-of-the-art learning-based decision-making schemes with the objective of identifying the main shortcomings and potential research avenues. Based on the research directions concluded, we propose, in Problem II and Problem III, novel learning-based frameworks with the objective of enhancing safety and efficiency at different decision-making levels. In Problem II, we develop a novel sensor-independent state estimation for a safety-critical system in urban driving using deep learning techniques. A neural inference model is developed and trained via deep-learning training techniques to obtain accurate state estimates using indirect measurements of vehicle dynamic states and powertrain states. In Problem III, we propose a novel hierarchical reinforcement learning-based decision-making architecture for learning left-turn policies at four-way unsignalized intersections with feasibility guarantees. The proposed technique involves an integration of two main decision-making layers; a high-level learning-based behavioral planning layer which adopts soft actor-critic principles to learn high-level, non-conservative yet safe, driving behaviors, and a motion planning layer that uses low-level Model Predictive Control (MPC) principles to ensure feasibility of the two-dimensional left-turn maneuver. The high-level layer generates reference signals of velocity and yaw angle for the ego vehicle taking into account safety and collision avoidance with the intersection vehicles, whereas the low-level planning layer solves an optimization problem to track these reference commands considering several vehicle dynamic constraints and ride comfort

Intention-Aware Decision-Making for Mixed Intersection Scenarios

This paper presents a white-box intention-aware decision-making for the handling of interactions between a pedestrian and an automated vehicle (AV) in an unsignalized street crossing scenario. Moreover, a design framework has been developed, which enables automated parameterization of the decision-making. This decision-making is designed in such a manner that it can understand pedestrians in urban traffic and can react accordingly to their intentions. That way, a human-like response to the actions of the pedestrian is ensured, leading to a higher acceptance of AVs. The core notion of this paper is that the intention prediction of the pedestrian to cross the street and decision-making are divided into two subsystems. On the one hand, the intention detection is a data-driven, black-box model. Thus, it can model the complex behavior of the pedestrians. On the other hand, the decision-making is a white-box model to ensure traceability and to enable a rapid verification and validation of AVs. This white-box decision-making provides human-like behavior and a guaranteed prevention of deadlocks. An additional benefit is that the proposed decision-making requires low computational resources only enabling real world usage. The automated parameterization uses a particle swarm optimization and compares two different models of the pedestrian: The social force model and the Markov decision process model. Consequently, a rapid design of the decision-making is possible and different pedestrian behaviors can be taken into account. The results reinforce the applicability of the proposed intention-aware decision-making

The Design of Performance Guaranteed Autonomous Vehicle Control for Optimal Motion in Unsignalized Intersections

The design of the motion of autonomous vehicles in non-signalized intersections with the consideration of multiple criteria and safety constraints is a challenging problem with several tasks. In this paper, a learning-based control solution with guarantees for collision avoidance is proposed. The design problem is formed in a novel way through the division of the control problem, which leads to reduced complexity for achieving real-time computation. First, an environment model for the intersection was created based on a constrained quadratic optimization, with which guarantees on collision avoidance can be provided. A robust cruise controller for the autonomous vehicle was also designed. Second, the environment model was used in the training process, which was based on a reinforcement learning method. The goal of the training was to improve the economy of autonomous vehicles, while guaranteeing collision avoidance. The effectiveness of the method is presented through simulation examples in non-signalized intersection scenarios with varying numbers of vehicles