A behavior driven approach for sampling rare event situations for autonomous vehicles

Performance evaluation of urban autonomous vehicles requires a realistic model of the behavior of other road users in the environment. Learning such models from data involves collecting naturalistic data of real-world human behavior. In many cases, acquisition of this data can be prohibitively expensive or intrusive. Additionally, the available data often contain only typical behaviors and exclude behaviors that are classified as rare events. To evaluate the performance of AV in such situations, we develop a model of traffic behavior based on the theory of bounded rationality. Based on the experiments performed on a large naturalistic driving data, we show that the developed model can be applied to estimate probability of rare events, as well as to generate new traffic situations

Dynamic-Occlusion-Aware Risk Identification for Autonomous Vehicles Using Hypergames

A particular challenge for both autonomous vehicles (AV) and human drivers is dealing with risk associated with dynamic occlusion, i.e., occlusion caused by other vehicles in traffic. In order to overcome this challenge, we use the theory of hypergames to develop a novel dynamic-occlusion risk measure (DOR). We use DOR to evaluate the safety of strategic planners, a type of AV behaviour planner that reasons over the assumptions other road users have of each other. We also present a method for augmenting naturalistic driving data to artificially generate occlusion situations. Combining our risk identification and occlusion generation methods, we are able to discover occlusion-caused collisions (OCC), which rarely occur in naturalistic driving data. Using our method we are able to increase the number of dynamic-occlusion situations in naturalistic data by a factor of 70, which allows us to increase the number of OCCs we can discover in naturalistic data by a factor of 40. We show that the generated OCCs are realistic and cover a diverse range of configurations. We then characterize the nature of OCCs at intersections by presenting an OCC taxonomy, which categorizes OCCs based on if they are left-turning or right-turning situations, and if they are reveal or tagging-on situations. Finally, in order to analyze the impact of collisions, we perform a severity analysis, where we find that the majority of OCCs result in high-impact collisions, demonstrating the need to evaluate AVs under occlusion situations before they can be released for commercial use

WiseBench: A Motion Planning Benchmarking Framework for Autonomous Vehicles

Rapid advances in every sphere of autonomous driving technology have intensified the need to be able to benchmark and compare different approaches. While many benchmarking tools tailored to different sub-systems of an autonomous vehicle, such as perception, already exist, certain aspects of autonomous driving still lack the necessary depth and diversity of coverage in suitable benchmarking approaches - autonomous vehicle motion planning is one such aspect. While motion planning benchmarking tools are abundant in the robotics community in general, they largely tend to lack the specificity and scope required to rigorously compare algorithms that are tailored to the autonomous vehicle domain. Furthermore, approaches that are targeted at autonomous vehicle motion planning are generally either not sensitive enough to distinguish subtle differences between different approaches, or not able to scale across problems and operational design domains of varying complexity. This work aims to address these issues by proposing WiseBench, an autonomous vehicle motion planning benchmark framework aimed at comprehensively uncovering fine and coarse-grained differences in motion planners across a wide range of operational design domains. WiseBench outlines a robust set of requirements for a suitable autonomous vehicle motion planner. These include simulation requirements that determine the environmental representation and physics models used by the simulator, scenario-suite requirements that govern the type and complexity of interactions with the environment and other traffic agents, and comparison metrics requirements that are geared towards distinguishing the behavioral capabilities and decision making processes of different motion planners. WiseBench is implemented using a carefully crafted set of scenarios and robust comparison metrics that operate within an in-house simulation environment, all of which satisfy these requirements. The benchmark proved to be successful in comparing and contrasting two different autonomous vehicle motion planners, and was shown to be an effective measure of passenger comfort and safety in a real-life experiment. The main contributions of our work on WiseBench thus include: a scenario creation methodology for the representative scenario suite, a comparison methodology to evaluate different motion planning algorithms, and a proof-of-concept implementation of the WiseBench framework as a whole