Search-based Test Generation for Automated Driving Systems: From Perception to Control Logic

abstract: Automated driving systems are in an intensive research and development stage, and the companies developing these systems are targeting to deploy them on public roads in a very near future. Guaranteeing safe operation of these systems is crucial as they are planned to carry passengers and share the road with other vehicles and pedestrians. Yet, there is no agreed-upon approach on how and in what detail those systems should be tested. Different organizations have different testing approaches, and one common approach is to combine simulation-based testing with real-world driving. One of the expectations from fully-automated vehicles is never to cause an accident. However, an automated vehicle may not be able to avoid all collisions, e.g., the collisions caused by other road occupants. Hence, it is important for the system designers to understand the boundary case scenarios where an autonomous vehicle can no longer avoid a collision. Besides safety, there are other expectations from automated vehicles such as comfortable driving and minimal fuel consumption. All safety and functional expectations from an automated driving system should be captured with a set of system requirements. It is challenging to create requirements that are unambiguous and usable for the design, testing, and evaluation of automated driving systems. Another challenge is to define useful metrics for assessing the testing quality because in general, it is impossible to test every possible scenario. The goal of this dissertation is to formalize the theory for testing automated vehicles. Various methods for automatic test generation for automated-driving systems in simulation environments are presented and compared. The contributions presented in this dissertation include (i) new metrics that can be used to discover the boundary cases between safe and unsafe driving conditions, (ii) a new approach that combines combinatorial testing and optimization-guided test generation methods, (iii) approaches that utilize global optimization methods and random exploration to generate critical vehicle and pedestrian trajectories for testing purposes, (iv) a publicly-available simulation-based automated vehicle testing framework that enables application of the existing testing approaches in the literature, including the new approaches presented in this dissertation.Dissertation/ThesisDoctoral Dissertation Computer Engineering 201

On Optimization-Based Falsification of Cyber-Physical Systems

In what is commonly referred to as cyber-physical systems (CPSs), computational and physical resources are closely interconnected. An example is the closed-loop behavior of perception, planning, and control algorithms, executing on a computer and interacting with a physical environment. Many CPSs are safety-critical, and it is thus important to guarantee that they behave according to given specifications that define the correct behavior. CPS models typically include differential equations, state machines, and code written in general-purpose programming languages. This heterogeneity makes it generally not feasible to use analytical methods to evaluate the system’s correctness. Instead, model-based testing of a simulation of the system is more viable. Optimization-based falsification is an approach to, using a simulation model, automatically check for the existence of input signals that make the CPS violate given specifications. Quantitative semantics estimate how far the specification is from being violated for a given scenario. The decision variables in the optimization problems are parameters that determine the type and shape of generated input signals. This thesis contributes to the increased efficiency of optimization-based falsification in four ways. (i) A method for using multiple quantitative semantics during optimization-based falsification. (ii) A direct search approach, called line-search falsification that prioritizes extreme values, which are known to often falsify specifications, and has a good balance between exploration and exploitation of the parameter space. (iii) An adaptation of Bayesian optimization that allows for injecting prior knowledge and uses a special acquisition function for finding falsifying points rather than the global minima. (iv) An investigation of different input signal parameterizations and their coverability of the space and time and frequency domains. The proposed methods have been implemented and evaluated on standard falsification benchmark problems. Based on these empirical studies, we show the efficiency of the proposed methods. Taken together, the proposed methods are important contributions to the falsification of CPSs and in enabling a more efficient falsification process

Falsification of Signal-Based Specifications for Cyber-Physical Systems

In the development of software for modern Cyber-Physical Systems, testing is an integral part that is rightfully given a lot of attention. Testing is done on many different abstraction levels, and especially for large-scale industrial systems, it can be difficult to know when the testing should conclude and the software can be considered correct enough for making its way into production. This thesis proposes new methods for analyzing and generating test cases as a means of being more certain that proper testing has been performed for the system under test. For analysis, the proposed approach includes automatically finding how much a given test suite has executed the physical properties of the simulated system. For test case generation, an up-and-coming approach to find errors in Cyber-Physical Systems is simulation-based falsification. While falsification is suitable also for some large-scale industrial systems, sometimes there is a gap between what has been researched and what problems need to be solved to make the approach tractable in the industry. This thesis attempts to close this gap by applying falsification techniques to real-world models from Volvo Car Corporation, and adapting the falsification procedure where it has shortcomings for certain classes of systems. Specifically, the thesis includes a method for automatically transforming a signal-based specification into a formal specification in temporal logic, as well as a modification to the underlying optimization problem that makes falsification more viable in an industrial setting. The proposed methods have been evaluated for both academic benchmark examples and real-world industrial models. One of the main conclusions is that the proposed additions and changes to analysis and generation of tests can be useful, given that one has enough information about the system under test. It is difficult to provide a general solution that will always work best -- instead, the challenge lies in identifying which properties of the given system should be taken into account when trying to find potential errors in the system

On Falsification of Large-Scale Cyber-Physical Systems

In the development of modern Cyber-Physical Systems, Model-Based Testingof the closed-loop system is an approach for finding potential faults andincreasing quality of developed products. Testing is done on many differentabstraction levels, and for large-scale industrial systems, there are severalchallenges. Executing tests on the systems can be time-consuming and largenumbers of complex specifications need to be thoroughly tested, while manyof the popular academic benchmarks do not necessarily reflect on this complexity.This thesis proposes new methods for analyzing and generating test casesas a means for being more certain that proper testing has been performed onthe system under test. For analysis, the proposed approach can automaticallyfind out how much of the physical parts of the system that the test suite hasexecuted.For test case generation, an approach to find errors is optimization-basedfalsification. This thesis attempts to close the gap between academia and industryby applying falsification techniques to real-world models from VolvoCar Corporation and adapting the falsification procedure where it has shortcomingsfor certain classes of systems. Specifically, the main contributionsof this thesis are (i) a method for automatically transforming a signal-basedspecification into a formal specification allowing an optimization-based falsificationapproach, (ii) a new collection of specifications inspired by large-scalespecifications from industry, (iii) an algorithm to perform optimization-basedfalsification for such a large set of specifications, and (iv) a new type of coveragecriterion for Cyber-Physical Systems that can help to assess when testingcan be concluded.The proposed methods have been evaluated for both academic benchmarkexamples and real-world industrial models. One of the main conclusions isthat the proposed additions and changes to the analysis and generation oftests can be useful, given that one has enough information about the systemunder test. The methods presented in this thesis have been applied to realworldmodels in a way that allows for higher-quality products by finding morefaults in early phases of development

From Formal Requirement Analysis to Testing and Monitoring of Cyber-Physical Systems

abstract: Cyber-Physical Systems (CPS) are being used in many safety-critical applications. Due to the important role in virtually every aspect of human life, it is crucial to make sure that a CPS works properly before its deployment. However, formal verification of CPS is a computationally hard problem. Therefore, lightweight verification methods such as testing and monitoring of the CPS are considered in the industry. The formal representation of the CPS requirements is a challenging task. In addition, checking the system outputs with respect to requirements is a computationally complex problem. In this dissertation, these problems for the verification of CPS are addressed. The first method provides a formal requirement analysis framework which can find logical issues in the requirements and help engineers to correct the requirements. Also, a method is provided to detect tests which vacuously satisfy the requirement because of the requirement structure. This method is used to improve the test generation framework for CPS. Finally, two runtime verification algorithms are developed for off-line/on-line monitoring with respect to real-time requirements. These monitoring algorithms are computationally efficient, and they can be used in practical applications for monitoring CPS with low runtime overhead.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Reachability Analysis of Cyber-Physical Systems Using Symbolic-Numeric Techniques

In this thesis, we address the problem of reachability analysis in cyber-physical systems. These are systems engineered by interfacing computational components with the physical world. They provide partially or fully automated safety-critical services in the form of medical devices, autonomous vehicles, avionics and power systems. We propose techniques to reason about the reachability of such systems, and provide methods for falsifying their safety properties. We model the cyber component as a software program and the physical component as a hybrid dynamical system. Unlike model based analysis, which uses either a purely symbolic or a numerical approach, we argue in favor of using a combination of the two. We justify this by noting that the software program running on a computer is completely specified and has precise semantics. In contrast, the model of the physical system is only an approximation. Hence, we treat the former as a white box, but treat the latter as a black box. Using symbolic methods for the cyber components and numerical methods for hybrid systems, we carefully capture the complex behaviors of software programs and circumvent the difficulty in analyzing complex models developed through first principles. To combine the two techniques, we use a Counterexample Guided Abstraction Refinement (CEGAR) framework. Furthermore, we explore learning techniques like regression and piecewise affine modeling to estimate and represent black box hybrid dynamical systems for the purpose of falsification. We use prototype implementations to demonstrate the effectiveness of presented ideas. Using non-trivial benchmarks, we compare their performance against the state of the art. We also comment on their applicability and discuss ideas for further improvement

Impact Assessment, Detection, And Mitigation Of False Data Attacks In Electrical Power Systems

The global energy market has seen a massive increase in investment and capital flow in the last few decades. This has completely transformed the way power grids operate - legacy systems are now being replaced by advanced smart grid infrastructures that attest to better connectivity and increased reliability. One popular example is the extensive deployment of phasor measurement units, which is referred to PMUs, that constantly provide time-synchronized phasor measurements at a high resolution compared to conventional meters. This enables system operators to monitor in real-time the vast electrical network spanning thousands of miles. However, a targeted cyber attack on PMUs can prompt operators to take wrong actions that can eventually jeopardize the power system reliability. Such threats originating from the cyber-space continue to increase as power grids become more dependent on PMU communication networks. Additionally, these threats are becoming increasingly efficient in remaining undetected for longer periods while gaining deep access into the power networks. An attack on the energy sector immediately impacts national defense, emergency services, and all aspects of human life. Cyber attacks against the electric grid may soon become a tactic of high-intensity warfare between nations in near future and lead to social disorder. Within this context, this dissertation investigates the cyber security of PMUs that affects critical decision-making for a reliable operation of the power grid. In particular, this dissertation focuses on false data attacks, a key vulnerability in the PMU architecture, that inject, alter, block, or delete data in devices or in communication network channels. This dissertation addresses three important cyber security aspects - (1) impact assessment, (2) detection, and (3) mitigation of false data attacks. A comprehensive background of false data attack models targeting various steady-state control blocks is first presented. By investigating inter-dependencies between the cyber and the physical layers, this dissertation then identifies possible points of ingress and categorizes risk at different levels of threats. In particular, the likelihood of cyber attacks against the steady-state power system control block causing the worst-case impacts such as cascading failures is investigated. The case study results indicate that false data attacks do not often lead to widespread blackouts, but do result in subsequent line overloads and load shedding. The impacts are magnified when attacks are coordinated with physical failures of generators, transformers, or heavily loaded lines. Further, this dissertation develops a data-driven false data attack detection method that is independent of existing in-built security mechanisms in the state estimator. It is observed that a convolutional neural network classifier can quickly detect and isolate false measurements compared to other deep learning and traditional classifiers. Finally, this dissertation develops a recovery plan that minimizes the consequence of threats when sophisticated attacks remain undetected and have already caused multiple failures. Two new controlled islanding methods are developed that minimize the impact of attacks under the lack of, or partial information on the threats. The results indicate that the system operators can successfully contain the negative impacts of cyber attacks while creating stable and observable islands. Overall, this dissertation presents a comprehensive plan for fast and effective detection and mitigation of false data attacks, improving cyber security preparedness, and enabling continuity of operations

Impact Assessment, Detection, and Mitigation of False Data Attacks in Electrical Power Systems

The global energy market has seen a massive increase in investment and capital flow in the last few decades. This has completely transformed the way power grids operate - legacy systems are now being replaced by advanced smart grid infrastructures that attest to better connectivity and increased reliability. One popular example is the extensive deployment of phasor measurement units, which is referred to PMUs, that constantly provide time-synchronized phasor measurements at a high resolution compared to conventional meters. This enables system operators to monitor in real-time the vast electrical network spanning thousands of miles. However, a targeted cyber attack on PMUs can prompt operators to take wrong actions that can eventually jeopardize the power system reliability. Such threats originating from the cyber-space continue to increase as power grids become more dependent on PMU communication networks. Additionally, these threats are becoming increasingly efficient in remaining undetected for longer periods while gaining deep access into the power networks. An attack on the energy sector immediately impacts national defense, emergency services, and all aspects of human life. Cyber attacks against the electric grid may soon become a tactic of high-intensity warfare between nations in near future and lead to social disorder. Within this context, this dissertation investigates the cyber security of PMUs that affects critical decision-making for a reliable operation of the power grid. In particular, this dissertation focuses on false data attacks, a key vulnerability in the PMU architecture, that inject, alter, block, or delete data in devices or in communication network channels. This dissertation addresses three important cyber security aspects - (1) impact assessment, (2) detection, and (3) mitigation of false data attacks. A comprehensive background of false data attack models targeting various steady-state control blocks is first presented. By investigating inter-dependencies between the cyber and the physical layers, this dissertation then identifies possible points of ingress and categorizes risk at different levels of threats. In particular, the likelihood of cyber attacks against the steady-state power system control block causing the worst-case impacts such as cascading failures is investigated. The case study results indicate that false data attacks do not often lead to widespread blackouts, but do result in subsequent line overloads and load shedding. The impacts are magnified when attacks are coordinated with physical failures of generators, transformers, or heavily loaded lines. Further, this dissertation develops a data-driven false data attack detection method that is independent of existing in-built security mechanisms in the state estimator. It is observed that a convolutional neural network classifier can quickly detect and isolate false measurements compared to other deep learning and traditional classifiers. Finally, this dissertation develops a recovery plan that minimizes the consequence of threats when sophisticated attacks remain undetected and have already caused multiple failures. Two new controlled islanding methods are developed that minimize the impact of attacks under the lack of, or partial information on the threats. The results indicate that the system operators can successfully contain the negative impacts of cyber attacks while creating stable and observable islands. Overall, this dissertation presents a comprehensive plan for fast and effective detection and mitigation of false data attacks, improving cyber security preparedness, and enabling continuity of operations