A Survey on Modelling of Automotive Radar Sensors for Virtual Test and Validation of Automated Driving

Radar sensors were among the first perceptual sensors used for automated driving. Although several other technologies such as lidar, camera, and ultrasonic sensors are available, radar sensors have maintained and will continue to maintain their importance due to their reliability in adverse weather conditions. Virtual methods are being developed for verification and validation of automated driving functions to reduce the time and cost of testing. Due to the complexity of modelling high-frequency wave propagation and signal processing and perception algorithms, sensor models that seek a high degree of accuracy are challenging to simulate. Therefore, a variety of different modelling approaches have been presented in the last two decades. This paper comprehensively summarises the heterogeneous state of the art in radar sensor modelling. Instead of a technology-oriented classification as introduced in previous review articles, we present a classification of how these models can be used in vehicle development by using the V-model originating from software development. Sensor models are divided into operational, functional, technical, and individual models. The application and usability of these models along the development process are summarised in a comprehensive tabular overview, which is intended to support future research and development at the vehicle level and will be continuously updated

Sensormodelle zur Simulation der Umfelderfassung für Systeme des automatisierten Fahrens

The use of sensor models allows the simulation of environmental perception in automated driving systems, aiding in development and testing efforts. This work systematically discusses the different types of sensor models and introduces an architecture for statistics based as well as for physically motivated sensor models. Each approach is grounded in real world observations of sensor measurements and is designed for portability and the ease of further extensions.Die Nutzung von Sensormodellen für die Umfelderfassung ebnet den Weg für die simulationsgestützte Entwicklung von Systemen des automatisierten Fahrens. In dieser Arbeit wird eine Systematik für verschiedene Arten von Sensormodellen eingeführt und eine Umsetzung von statistischen sowie von physikalisch motivierten Modellen vorgestellt. Beide Ansätze basieren auf realen Sensormessdaten und zielen auf eine leichte Übertragbarkeit sowie die Möglichkeit der Erweiterung der Modelle für verschiedene Anwendungsbereiche

Sensormodelle zur Simulation der Umfelderfassung für Systeme des automatisierten Fahrens

The use of sensor models allows the simulation of environmental perception in automated driving systems, aiding in development and testing efforts. This work systematically discusses the different types of sensor models and introduces an architecture for statistics based as well as for physically motivated sensor models. Each approach is grounded in real world observations of sensor measurements and is designed for portability and the ease of further extensions.Die Nutzung von Sensormodellen für die Umfelderfassung ebnet den Weg für die simulationsgestützte Entwicklung von Systemen des automatisierten Fahrens. In dieser Arbeit wird eine Systematik für verschiedene Arten von Sensormodellen eingeführt und eine Umsetzung von statistischen sowie von physikalisch motivierten Modellen vorgestellt. Beide Ansätze basieren auf realen Sensormessdaten und zielen auf eine leichte Übertragbarkeit sowie die Möglichkeit der Erweiterung der Modelle für verschiedene Anwendungsbereiche

Metrics for Specification, Validation, and Uncertainty Prediction for Credibility in Simulation of Active Perception Sensor Systems

The immense effort required for the safety validation of an automated driving system of SAE level 3 or higher is known not to be feasible by real test drives alone. Therefore, simulation is key even for limited operational design domains for homologation of automated driving functions. Consequently, all simulation models used as tools for this purpose must be qualified beforehand. For this, in addition to their verification and validation, uncertainty quantification (VV&UQ) and prediction for the application domain are required for the credibility of the simulation model. To enable such VV&UQ, a particularly developed lidar sensor system simulation is utilized to present new metrics that can be used holistically to demonstrate the model credibility and -maturity for simulation models of active perception sensor systems. The holistic process towards model credibility starts with the formulation of the requirements for the models. In this context, the threshold values of the metrics as acceptance criteria are quantifiable by the relevance analysis of the cause-effect chains prevailing in different scenarios, and should intuitively be in the same unit as the simulated metric for this purpose. These relationships can be inferred via the presented aligned methods “Perception Sensor Collaborative Effect and Cause Tree” (PerCollECT) and “Cause, Effect, and Phenomenon Relevance Analysis” (CEPRA). For sample validation, each experiment must be accompanied by reference measurements, as these then serve as simulation input. Since the reference data collection is subject to epistemic as well as aleatory uncertainty, which are both propagated through the simulation in the form of input data variation, this leads to several slightly different simulation results. In the simulation of measured signals and data over time considered here, this combination of uncertainties is best expressed as superimposed cumulative distribution functions. The metric must therefore be able to handle such so-called p-boxes as a result of the large set of simulations. In the present work, the area validation metric (AVM) is selected by a detailed analysis as the best of the metrics already used and extended to be able to fulfill all the requirements. This results in the corrected AVM (CAVM), which quantifies the model scattering error with respect to the real scatter. Finally, the double validation metric (DVM) is elaborated as a double-vector of the former metric with the estimate for the model bias. The novel metric is exemplarily applied to the empirical cumulative distribution functions of lidar measurements and the p-boxes from their re-simulations. In this regard, aleatory and epistemic uncertainties are taken into account for the first time and the novel metrics are successfully established. The quantification of the uncertainties and error prediction of a sensor model based on the sample validation is also demonstrated for the first time

Statistical modelling of algorithms for signal processing in systems based on environment perception

One cornerstone for realising automated driving systems is an appropriate handling of uncertainties in the environment perception and situation interpretation. Uncertainties arise due to noisy sensor measurements or the unknown future evolution of a traffic situation. This work contributes to the understanding of these uncertainties by modelling and propagating them with parametric probability distributions

Advances in Automated Driving Systems

Electrification, automation of vehicle control, digitalization and new mobility are the mega-trends in automotive engineering, and they are strongly connected. While many demonstrations for highly automated vehicles have been made worldwide, many challenges remain in bringing automated vehicles to the market for private and commercial use. The main challenges are as follows: reliable machine perception; accepted standards for vehicle-type approval and homologation; verification and validation of the functional safety, especially at SAE level 3+ systems; legal and ethical implications; acceptance of vehicle automation by occupants and society; interaction between automated and human-controlled vehicles in mixed traffic; human–machine interaction and usability; manipulation, misuse and cyber-security; the system costs of hard- and software and development efforts. This Special Issue was prepared in the years 2021 and 2022 and includes 15 papers with original research related to recent advances in the aforementioned challenges. The topics of this Special Issue cover: Machine perception for SAE L3+ driving automation; Trajectory planning and decision-making in complex traffic situations; X-by-Wire system components; Verification and validation of SAE L3+ systems; Misuse, manipulation and cybersecurity; Human–machine interactions, driver monitoring and driver-intention recognition; Road infrastructure measures for the introduction of SAE L3+ systems; Solutions for interactions between human- and machine-controlled vehicles in mixed traffic

How Stochastic can Help to Introduce Automated Driving

Status Quo: Automated systems will replace the human operator at different tasks in everyday life. From today’s perspective, these new technologies offer predicted but also unknown benefits. However, as every other new technology, also automated systems will have drawbacks for some stakeholders in our society. As long as new technologies are within readiness levels of research, their impact is mostly negligible. The technology readiness level of automated driving in road traffic is pushed forward strongly by many researchers and developers all over the world. Consequently, the demand for safety assurance gets urgent. From today’s perspective, a concept that evaluates the safety of automated driving in an affordable and meaningful way is missing. However, this concept is necessary to enable the introduction of automated driving to public road traffic. Objectives: The objective of this thesis is to improve the understanding of the challenge for safety assurance on automated vehicles. Therefore a concept is aimed for, that estimates the safety impact for the stakeholders of automated driving. Estimations are always based on assumptions and suffer from uncertainty. For that reason the concept needs to consider and express the underlying assumptions and uncertainties. Methodology: The methodology for reaching the objectives is formed around the core assumption of the concept: The safety of an Object under Test can be described by the parameter of a probability distribution. This parameter connects the number of events that result from driving a distance with the safety performance of the OuT. Based on this core assumption a model for safety evaluation is developed iteratively. First of all the relevant stakeholders that are influenced by the technology are identified and analyzed. The second step identifies measurable requirements for the safety of automated vehicles from the stakeholder’s perspectives. Based on this preliminary work on the one hand a usage strategy is defined that controls the introduction of automated vehicles. On the other hand an examination strategy is developed to evaluate whether this strategy enables the automation to meet the requirements. In step four the usage strategy is examined for the Autobahn automation being one representative use case. The results, meaning testing effort and introduction possibilities, are compared and discussed. A refinement of stakeholders as well as requirements is performed. Such a refinement is necessary as only a more precise and subtle analysis will lead to a share between efforts and benefits of the introduction of automated vehicles that forms a basis for the discussion on the safety assurance challenge. Results: The results of the thesis can be grouped into four mayor insights. Firstly, the number of rare events like accidents can be handled as being a product of a random experiment that depends on a safety performance of a traffic participant and the number of driven kilometers. From today’s perspective a falsification of this approach was not found and thus builds a simple first approach. Secondly, the statistical proof of safety based on real-world driving is not economically feasible before mass application of the automated vehicle. Thirdly, refinement of the requirements is necessary and justifiable to reduce the safety requirements. Splitting up the requirements of society and vehicle users leads to reduced testing efforts and an uncertainty-based usage strategy. This uncertainty most likely will reduce during usage, thus also enabling a statistical statement on safety at one point in future. Lastly, a method consisting of evaluation criteria as well as an introduction simulation is developed to examine proposed usage strategies. Thereby the possible safety impacts of the usage are studied. Conclusion: As the safety of automated driving cannot be proven statistically before introduction, the introduction needs to be performed despite and under consideration of an estimated uncertainty. This does not mean that the introduced vehicles are less safe compared to their benchmark; however during introduction it will be uncertain. As long as the uncertainty stays above a threshold a usage strategy that is included into the safety assurance concept is necessary. Such a usage strategy would be cautious and based on regular observation of the events encountered by introduced vehicles. Several challenges have been identified for the developed introduction concept of automated vehicles. Based on these challenges further work should mainly address two topics: 1. The identification and collection of data that is necessary for concept application. 2. The answer of an unavoidable question: How much harm, caused by a human built machine, is acceptable for the exposed humans

Towards testing of automated driving functions in virtual driving environments

In dieser Arbeit wird ein Beitrag für den methodischen Test von automatisierten Fahrfunktionen mit Hilfe von virtuellen Umgebungen geleistet. Im ersten Teil wird die Notwendigkeit eines systematischen Testkonzepts begründet und die These aufgestellt, dass ein szenariobasiertes Testkonzept eine mögliche Lösung für das Testdilemma darstellen könnte. Dazu werden sechs Forschungsfragen aufgestellt, die für die Entwicklung eines szenariobasierten Ansatzes beantwortet werden müssen. Im zweiten Teil werden die Grundlagen und Voraussetzungen der Arbeit dargestellt. Hierfür werden Begriffe und Definitionen vorgestellt. Weiterhin wird der Begriff der Komplexität von Szenarien untersucht. Die Automatisierungsgrade und eine funktionale Systemarchitektur für automatisierte Fahrfunktionen werden vorgestellt. Der Teil schließt mit einer Klassifikation von verschiedenen X-in-the-Loop-Verfahren ab. Im dritten Teil wird das Testkonzept des modularen virtuellen Testbaukastens vorgestellt. Es werden Anforderungen definiert sowie der Aufbau und die Schnittstellen zwischen den Modulen des Testbaukastens präsentiert. Für die Auswahl und Analyse der Einflussparameter, die Testfallerstellung und die Testdurchführung mittels X-in-the-Loop-Verfahren werden Anforderungen definiert und der relevante Stand der Technik vorgestellt. Daraus wird der Forschungsbedarf abgeleitet. Für die Auswahl und Analyse der Einflussparameter wird ein Schema zur Beschreibung der Einflussparameter hergeleitet und Informationsquellen für die Auswahl und Analyse von Einflussparametern werden bewertet. Für die Testfallerstellung wird ein generisches Modell zur Beschreibung von Szenarien vorgestellt und eine kombinatorische Testfallableitung präsentiert. Für die Testdurchführung wird eine Zuordnungsmethode für Testfälle auf verschiedene X-in-the-Loop-Verfahren beschrieben. Zusätzlich werden Testtreiber für die Module einer funktionalen Systemarchitektur analysiert und die Testtreiber des modularen virtuellen Testbaukastens vorgestellt. Für die Testfallauswertung werden Anforderungen definiert und Methoden aus dem Stand der Technik zur Bewertung und zur Analyse der Testergebnissen präsentiert. Der Teil schließt mit einer Beschreibung der Limitationen des Testbaukastens ab. Der vierte Teil beschreibt die Anwendung des Testbaukastens im Fallbeispiel des Engstellenassistenten. Das Projekt wird vorgestellt und die verschiedenen Module des Testbaukastens werden angewendet.This dissertation contributes to the systematical test of driving functions with virtual environments. The first part establishes the necessity of a systematic test concept for automated driving functions. The challenge of testing automated driving functions is presented and the assumption that scenario-based test concept can be a possible solution. Six research questions will be asked in this section, which have to be answered for the development of a scenario-based test concept. The second part defines important terms and analyses prior art as a foundation for this dissertation. Furthermore the levels of automated driving functions are presented and a functional system architecture is suggested. Finally, methods of software testing, traffic simulations, and classification methods for x-in-the-loop techniques are discussed. The third part purposes a concept for a modular virtual testing toolbox. The structure and interfaces between modules of the toolbox are described. Furthermore, requirements are stated for the following modules: selection and analysis of influence parameters, test case generation, test execution with x-in-the-loop techniques, and test case evaluation. For each of these modules selected state of the art methods are presented. Hence, the need for research is deduced. For the selection and analysis of influence parameters, a schema for describing influence parameters is introduced. Furthermore, resources for the selection and analysis of influence parameters are evaluated. For the test case generation, a unified model for the description of scenarios is presented. Additionally, a combinatorial test case deduction is described. For the test case execution, a method for assigning test cases to x-in-the-loop techniques is suggested. For the test case evaluation, a method for the evaluation and analysis of the test result is presented. A review of the limitation of the modular virtual testing toolbox closes this part. The fourth part presents the application of the modular virtual test toolbox to the constriction assistance system. The project is described and the single modules of the toolbox are applied to the assistance system