Falsification of Signal-Based Specifications for Cyber-Physical Systems

In the development of software for modern Cyber-Physical Systems, testing is an integral part that is rightfully given a lot of attention. Testing is done on many different abstraction levels, and especially for large-scale industrial systems, it can be difficult to know when the testing should conclude and the software can be considered correct enough for making its way into production. This thesis proposes new methods for analyzing and generating test cases as a means of being more certain that proper testing has been performed for the system under test. For analysis, the proposed approach includes automatically finding how much a given test suite has executed the physical properties of the simulated system. For test case generation, an up-and-coming approach to find errors in Cyber-Physical Systems is simulation-based falsification. While falsification is suitable also for some large-scale industrial systems, sometimes there is a gap between what has been researched and what problems need to be solved to make the approach tractable in the industry. This thesis attempts to close this gap by applying falsification techniques to real-world models from Volvo Car Corporation, and adapting the falsification procedure where it has shortcomings for certain classes of systems. Specifically, the thesis includes a method for automatically transforming a signal-based specification into a formal specification in temporal logic, as well as a modification to the underlying optimization problem that makes falsification more viable in an industrial setting. The proposed methods have been evaluated for both academic benchmark examples and real-world industrial models. One of the main conclusions is that the proposed additions and changes to analysis and generation of tests can be useful, given that one has enough information about the system under test. It is difficult to provide a general solution that will always work best -- instead, the challenge lies in identifying which properties of the given system should be taken into account when trying to find potential errors in the system

RuSTL: Runtime Verification using Signal Temporal Logic

A system is classified to be a safety-critical system if its failure and/or malfunction of these devices may result in severe injuries or in extreme cases loss of human life. Such systems are all around us, examples of which include pacemakers, respiratory equipment, electrical locks, fire sprinklers and cars among many others. Runtime Verification (RV) is used to monitor the execution of such systems either while running or after execution to ensure that the system under observation does not violate any safety constraints. RV employs formal specification languages to evaluate a real-world systems. Pnueli introduced the formal specification for Linear Temporal Logic (LTL) in 1977 for specifying propositional time properties of reactive and concurrent systems. Signal Temporal Logic (STL) is a popular extension of LTL, which analyzes dense-time real-valued signal properties with quantitative timing constraints. In this thesis, we introduce Runtime Verification using Signal Temporal Logic (RuSTL), an offline qualitative semantic tool for monitoring STL properties. RuSTL is designed to parse any valid STL formula ’ and create a stand-alone executable monitor program, which checks the property against a given trace σ. RuSTL also take in as input structured English text and convert it into an equivalent STL formula. The application also has the capability to automatically generate diagnostic plots that help the user visually inspect the results of the monitor against a given trace. We prove that the monitor program generated by RuSTL is sound and it terminates for any given valid STL property. Furthermore, we prove that the parsing algorithm used to create the monitor program is complete. We evaluated RuSTL’s performance over traces collected from an autonomous self-driving vehicle. The experimental results for our RV monitor show that the execution time of the monitor grows linearly with respect to the length of the signal trace provided

From Formal Requirement Analysis to Testing and Monitoring of Cyber-Physical Systems

abstract: Cyber-Physical Systems (CPS) are being used in many safety-critical applications. Due to the important role in virtually every aspect of human life, it is crucial to make sure that a CPS works properly before its deployment. However, formal verification of CPS is a computationally hard problem. Therefore, lightweight verification methods such as testing and monitoring of the CPS are considered in the industry. The formal representation of the CPS requirements is a challenging task. In addition, checking the system outputs with respect to requirements is a computationally complex problem. In this dissertation, these problems for the verification of CPS are addressed. The first method provides a formal requirement analysis framework which can find logical issues in the requirements and help engineers to correct the requirements. Also, a method is provided to detect tests which vacuously satisfy the requirement because of the requirement structure. This method is used to improve the test generation framework for CPS. Finally, two runtime verification algorithms are developed for off-line/on-line monitoring with respect to real-time requirements. These monitoring algorithms are computationally efficient, and they can be used in practical applications for monitoring CPS with low runtime overhead.Dissertation/ThesisDoctoral Dissertation Computer Science 201