Characterization and Mitigation of Insufficiencies In Automated Driving Systems

Automated Driving (AD) systems have the potential to increase safety, comfort and energy efficiency. Recently, major automotive companies have started testing and validating AD systems (ADS) on public roads. Nevertheless, the commercial deployment and wide adoption of ADS have been moderate, partially due to system functional insufficiencies (FI) that undermine passenger safety and lead to hazardous situations on the road. In contrast to system faults that are analyzed by the automotive functional safety standard ISO 26262, FIs are defined in ISO 21448 Safety Of The Intended Functionality (SOTIF). FIs are insufficiencies in sensors, actuators and algorithm implementations, including neural networks and probabilistic calculations. Examples of FIs in ADS include inaccurate ego-vehicle localization on the road, incorrect prediction of a cyclist maneuver, unreliable detection of a pedestrian in rainy weather using cameras and image processing algorithms, etc. The main goal of the study is to formulate a generic architectural design pattern, which is compatible with existing methods and ADS, to improve FI mitigation and enable faster commercial deployment of ADS. First, the authors studied the 2021 autonomous vehicles disengagement reports published by the California Department of Motor Vehicles (DMV). The data clearly show that disengagements are five times more often caused by FIs rather than by system faults. They then made a comprehensive list of insufficiencies and their characteristics by analyzing over 10 hours of publicly available road test videos. In particular, the authors identified insufficiency types in four major categories: world model, motion plan, traffic rule, and operational design domain. The insufficiency characterization helps making the SOTIF analyses of triggering conditions more systematic and comprehensive. To handle faults, modern ADS already integrate multiple AD channels, where each channel is composed of sensors and processors running AD software. The characterization study triggered a hypothesis that these heterogeneous channels can also complement each other’s capabilities to mitigate insufficiencies in vehicle operation. To verify the hypothesis, the authors built an open-loop automated driving simulation environment based on the LG SVL simulator. Three realistic AD channels (Baidu Apollo, Autoware.Auto, and comma.ai openpilot) were tested in the same driving scenario. The experiments suggest that even advanced AD channels have insufficiencies that can be mitigated by switching control to another (possibly less advanced) AD channel at the right moment. Based on the FI characterization, simulation experiments and literature survey, the authors define a novel generic architectural design pattern Daruma to dynamically select the channel that is least likely to have a FI at the moment. The key component of the pattern does cross-channel analysis, in which planned trajectories and world models from different AD channels are mutually evaluated. The output of the cross-channel analysis is combined with more traditional fault detections in a safety fusion component. The safety fusion then feeds an aggregated per-channel safety score to the high-level arbiter, which eventually selects the AD channel to control the vehicle. The formulated architectural pattern can help manufactures of autonomous vehicles in mitigating FIs. Limitations of the study suggest interesting future work, including algorithmic research on cross-channel analysis and safety fusion, as well as evaluation of the cross-channel analysis in simulations and road tests