Scenario description language for automated driving systems : a two level abstraction approach

The complexities associated with Automated Driving Systems (ADSs) and their interaction with the environment pose a challenge for their safety evaluation. Number of miles driven has been suggested as one of the metrics to demonstrate technological maturity. However, the experiences or the scenarios encountered by the ADSs is a more meaningful metric, and has led to a shift to scenario-based testing approach in the automotive industry and research community. Variety of scenario generation techniques have been advocated, including real-world data analysis, accident data analysis and via systems hazard analysis. While scenario generation can be done via these methods, there is a need for a scenario description language format which enables the exchange of scenarios between diverse stakeholders (as part of the systems engineering lifecycle) with varied usage requirements. In this paper, we propose a two-level abstraction approach to scenario description language (SDL) - SDL level 1 and SDL level 2. SDL level 1 is a textual description of the scenario at a higher abstraction level to be used by regulators or system engineers. SDL level 2 is a formal machine-readable language which is ingested by testing platform e.g. simulation or test track. One can transform a scenario in SDL level 1 into SDL level 2 by adding more details or from SDL level 2 to SDL level 1 by abstracting

Identifying accident causes of driver-vehicle interactions using system theoretic process analysis (STPA)

Latest generations of automobiles are gradually being equipped with technologies that have increasing automation, a trend which had led to increase in the system complexity as well as increased human-automation interactions. Failures in such complex human-automation interactions increasingly occur due to the mismatch between what operators know about the system and what the designers expect operators to know. Causes of road accidents also change due to role shift of drivers from controlling the vehicle to monitoring the in-vehicle controllers. Failures in such complex systems involving human-automation interactions increasingly occur due to the emergent behaviours from the interactions, and are less likely due to reliability of individual components. Traditional safety analysis methods fall short in identifying such emergent failures. This paper focuses on using a systems thinking inspired safety analysis method called System Theoretic Process Analysis (STPA) to identify potential failures. The analysis focuses on a SAE Level-4 Vehicle that is in the development phase, and is controlled partially by a safety driver and its built-in Autonomous Driving System (ADS). The analysis yields that while increase in complexity does increase system functionality, it also brings a challenge to evaluate the safety of the system and potentially causes incorrect human-automation interactions, leading to an accident. After the possible inadequate driver-vehicle interactions are identified by STPA, corresponding requirements were then proposed in order to avoid the unsafe behaviour and thus preventing the hazards

Calibrating trust through knowledge : introducing the concept of informed safety for automation in vehicles

There has been an increasing focus on the development of automation in vehicles due its many potential benefits like safety, improved traffic efficiency, reduced emissions etc. One of the key factors influencing public acceptance of automated vehicle technologies is their level of trust. Development of trust is a dynamic process and needs to be calibrated to the correct levels for safe deployment to ensure appropriate use of such systems. One of the factors influencing trust is the knowledge provided to the driver about the system’s true capabilities and limitations. After a 56 participants driving simulator study, the authors found that with the introduction of knowledge about the true capabilities and limitations of the automated system, trust in the automated system increased as compared to when no knowledge was provided about the system. Participants experienced two different types of automated systems: low capability automated system and high capability automated system. Interestingly, with the introduction of knowledge, the average trust levels for both low and high capability automated systems were similar. Based on the experimental results, the authors introduce the concept of informed safety, i.e., informing the drivers about the safety limits of the automated system to enable them to calibrate their trust in the system to an appropriate level

Towards increased reliability by objectification of Hazard Analysis and Risk Assessment (HARA) of automated automotive systems

Hazard Analysis and Risk Assessment (HARA) in various domains like automotive, aviation, process industry etc. suffer from the issues of validity and reliability. While there has been an increasing appreciation of this subject, there have been limited approaches to overcome these issues. In the automotive domain, HARA is influenced by the ISO 26262 international standard which details functional safety of road vehicles. While ISO 26262 was a major step towards analysing hazards and risks, like other domains, it is also plagued by the issues of reliability. In this paper, the authors discuss the automotive HARA process. While exposing the reliability challenges of the HARA process detailed by the standard, the authors present an approach to overcome the reliability issues. The approach is obtained by creating a rule-set for automotive HARA to determine the Automotive Safety Integrity Level (ASIL) by parametrizing the individual components of an automotive HARA, i.e., severity, exposure and controllability. The initial rule-set was put to test by conducting a workshop involving international functional safety experts as participants in an experiment where rules were provided for severity and controllability ratings. Based on the qualitative results of the experiments, the rule-set was re-calibrated. The proposed HARA approach by the creation of a rule- set demonstrated reduction in variation. However, the caveat lies in the fact that the rule-set needs to be exhaustive or sufficiently explained in order to avoid any degree of subjective interpretation which is a source of variation and unreliability

Identification of test cases for Automated Driving Systems using Bayesian optimization

With advancements in technology, the automotive industry is experiencing a paradigm shift from assisted driving to highly automated driving. However, autonomous driving systems are highly safety critical in nature and need to be thoroughly tested for a diverse set of conditions before being commercially deployed. Due to the huge complexities involved with Advanced Driver Assistance Systems (ADAS) and Automated Driving Systems (ADS), traditional software testing methods have well-known limitations. They also fail to cover the infinite number of adverse conditions that can occur due to a slight change in the interactions between the environment and the system. Hence, it is important to identify test conditions that push the vehicle under test to breach its safe boundaries. Hazard Based Testing (HBT) methods, inspired by Systems-Theoretic Process Analysis (STPA), identify such parameterized test conditions that can lead to system failure. However, these techniques fall short of discovering the exact parameter values that lead to the failure condition. The presented paper proposes a test case identification technique using Bayesian Optimization. For a given test scenario, the proposed method learns parameter values by observing the system's output. The identified values create test cases that drive the system to violate its safe boundaries. STPA inspired outputs (parameters and pass/fail criteria) are used as inputs to the Bayesian Optimization model. The proposed method was applied to an SAE Level-4 Low Speed Automated Driving (LSAD) system which was modelled in a driving simulator

Domain Knowledge Distillation from Large Language Model: An Empirical Study in the Autonomous Driving Domain

Engineering knowledge-based (or expert) systems require extensive manual effort and domain knowledge. As Large Language Models (LLMs) are trained using an enormous amount of cross-domain knowledge, it becomes possible to automate such engineering processes. This paper presents an empirical automation and semi-automation framework for domain knowledge distillation using prompt engineering and the LLM ChatGPT. We assess the framework empirically in the autonomous driving domain and present our key observations. In our implementation, we construct the domain knowledge ontology by "chatting" with ChatGPT. The key finding is that while fully automated domain ontology construction is possible, human supervision and early intervention typically improve efficiency and output quality as they lessen the effects of response randomness and the butterfly effect. We, therefore, also develop a web-based distillation assistant enabling supervision and flexible intervention at runtime. We hope our findings and tools could inspire future research toward revolutionizing the engineering of knowledge-based systems across application domains.Comment: Accepted by ITSC 202

Analyzing real-world accidents for test scenario generation for automated vehicles

Identification of test scenarios for Automated Driving Systems (ADSs) remains a key challenge for the Verification & Validation of ADSs. Various approaches including data based approaches and knowledge based approaches have been proposed for scenario generation. Identifying the conditions that lead to high severity traffic accidents can help us not only identify test scenarios for ADSs, but also implement measures to save lives and infrastructure resources. Taking a data based approach, in this paper, we introduce a novel accident data analysis method for generating test scenarios where we analyze UK’s Stats19 accident data to identify trends in high severity accidents for test scenario generation. This paper first focuses on the severity of the accidents with the goal of relating it to static and time-dependent internal and external factors in a comprehensive way taking into account Operational Design Domain (ODD) properties, e.g. road, environmental conditions, and vehicle properties and driver characteristics. For this purpose, the paper utilizes a data grouping strategy (coarse-graining) and builds a logistic regression approach, derived from conventional regression models, in which emerging features become more pronounced, while uninteresting features and noise weaken. The approach makes the relationship between the factors and outcome variable more visible and hence well suited for the severity analysis. The method shows superior performance as compared to ordinary logistic models measured by goodness of fit and accounting for model variance (R2=0.05 for the ordinary model, R2=0.85 for the current model). The model is then used to solve the inverse problem of constructing high-risk pre-crash conditions as test scenarios for simulation based testing of ADSs

Introducing ASIL inspired dynamic tactical safety decision framework for automated vehicles

Existing automotive Hazard Analysis and Risk Assessment (HARA) process as discussed by the international standard ISO 26262 is static in nature. While the standard describes a systematic process to incorporate functional safety in the development process of Electrical & Electronic (E/E) systems, it fails to address the needs of Advanced Driver Assistance Systems (ADAS) and Automated Driving (AD) systems. In order to ensure the safety of ADAS and AD systems, it is important to incorporate the changing nature of interactions between the system and the environment, in the safety analysis process for ADAS and AD systems. In this paper, the authors argue the need for a dynamic approach for automotive safety analysis by adapting the tactical safety for ADAS and AD systems depending on the real-time operational capability and real-time ASIL (Automotive Safety Integrity Level) rating of a situation, and discuss a framework for this process. The novelty and therefore contribution of this paper lies in the proposed ASIL inspired dynamic tactical safety framework, which evaluates the severity, controllability and exposure ratings in real-time based on the real time values of the various vehicle and environment parameters. These ratings are used to assign a real-time ASIL value which is used to determine the tactical decisions in order to lower the ASIL value in real-time by altering the functional (operational) capability of the system. Furthermore, the framework is explained with the help of a case study based on a combined Adaptive Cruise Control (ACC) and Autonomous Emergency Braking (AEB) system

Systems approach to creating test scenarios for automated driving systems

Increased safety has been advocated as one of the major benefits of the introduction of Automated Driving Systems (ADSs). Incorporation of ADSs in vehicles mean that associated software has safety critical application, thus requiring exhaustive testing. To prove ADSs are safer than human drivers, some work has suggested that they will need to be driven for over 11 billion miles. The number of test miles driven is not, by itself, a meaningful metric for judging the safety of ADSs. Rather, the types of scenarios encountered by the ADSs during testing are critically important. With a Hazard Based Testing approach, this paper proposes that the extent to which testing miles are ‘smart miles’ that reflect hazard-based scenarios relevant to the way in which an ADS fails or handles hazards is a fundamental, if not pivotal, consideration for safety-assurance of ADSs. Using Systems Theoretic Process Analysis (STPA) method as a foundation, an extension to the STPA method has been developed to identify test scenarios. The approach has been applied to a real-world case study of a SAE Level 4 Low-Speed Automated Driving system (a.k.a. a shuttle). This paper, discusses the STPA analysis and a newly-developed test scenarios creation method derived from STPA

OmniCAV : a simulation and modelling system that enables “CAVs for All”

OmniCAV is laying the foundations for the development of a comprehensive, robust and secure simulator, aimed at providing a certification tool for Connected Autonomous Vehicles (CAVs) that can be used by regulatory and accreditation bodies, insurers and manufacturers to accelerate the safe development of CAVs. To achieve this, OmniCAV is using highly detailed road maps, together with a powerful combination of traffic management, accident and CCTV data, to create a high-fidelity traffic and driving simulation environment to interact with the AV under test. Scenarios for testing are developed and randomised in a holistic way to avoid CAVs training to specific conditions. Critically, the simulator offers coverage of a representative element of the U.K. road network, through encompassing rural roads, peri-urban and urban roads to enable autonomy for all. The validity of the synthetic test environment compared to the real-world is of particular importance, and OmniCAV will be tested and refined through an iterative approach involving real-world comparisons and working in conjunction with a CAV testbed