Multilevel Runtime Verification for Safety and Security Critical Cyber Physical Systems from a Model Based Engineering Perspective

Advanced embedded system technology is one of the key driving forces behind the rapid growth of Cyber-Physical System (CPS) applications. CPS consists of multiple coordinating and cooperating components, which are often software-intensive and interact with each other to achieve unprecedented tasks. Such highly integrated CPSs have complex interaction failures, attack surfaces, and attack vectors that we have to protect and secure against. This dissertation advances the state-of-the-art by developing a multilevel runtime monitoring approach for safety and security critical CPSs where there are monitors at each level of processing and integration. Given that computation and data processing vulnerabilities may exist at multiple levels in an embedded CPS, it follows that solutions present at the levels where the faults or vulnerabilities originate are beneficial in timely detection of anomalies. Further, increasing functional and architectural complexity of critical CPSs have significant safety and security operational implications. These challenges are leading to a need for new methods where there is a continuum between design time assurance and runtime or operational assurance. Towards this end, this dissertation explores Model Based Engineering methods by which design assurance can be carried forward to the runtime domain, creating a shared responsibility for reducing the overall risk associated with the system at operation. Therefore, a synergistic combination of Verification & Validation at design time and runtime monitoring at multiple levels is beneficial in assuring safety and security of critical CPS. Furthermore, we realize our multilevel runtime monitor framework on hardware using a stream-based runtime verification language

Trajectory planning based on adaptive model predictive control: Study of the performance of an autonomous vehicle in critical highway scenarios

Increasing automation in automotive industry is an important contribution to overcome many of the major societal challenges. However, testing and validating a highly autonomous vehicle is one of the biggest obstacles to the deployment of such vehicles, since they rely on data-driven and real-time sensors, actuators, complex algorithms, machine learning systems, and powerful processors to execute software, and they must be proven to be reliable and safe. For this reason, the verification, validation and testing (VVT) of autonomous vehicles is gaining interest and attention among the scientific community and there has been a number of significant efforts in this field. VVT helps developers and testers to determine any hidden faults, increasing systems confidence in safety, security, functional analysis, and in the ability to integrate autonomous prototypes into existing road networks. Other stakeholders like higher-management, public authorities and the public are also crucial to complete the VTT process. As autonomous vehicles require hundreds of millions of kilometers of testing driven on public roads before vehicle certification, simulations are playing a key role as they allow the simulation tools to virtually test millions of real-life scenarios, increasing safety and reducing costs, time and the need for physical road tests. In this study, a literature review is conducted to classify approaches for the VVT and an existing simulation tool is used to implement an autonomous driving system. The system will be characterized from the point of view of its performance in some critical highway scenarios.O aumento da automação na indústria automotiva é uma importante contribuição para superar muitos dos principais desafios da sociedade. No entanto, testar e validar um veículo altamente autónomo é um dos maiores obstáculos para a implantação de tais veículos, uma vez que eles contam com sensores, atuadores, algoritmos complexos, sistemas de aprendizagem de máquina e processadores potentes para executar softwares em tempo real, e devem ser comprovadamente confiáveis e seguros. Por esta razão, a verificação, validação e teste (VVT) de veículos autónomos está a ganhar interesse e atenção entre a comunidade científica e tem havido uma série de esforços significativos neste campo. A VVT ajuda os desenvolvedores e testadores a determinar quaisquer falhas ocultas, aumentando a confiança dos sistemas na segurança, proteção, análise funcional e na capacidade de integrar protótipos autónomos em redes rodoviárias existentes. Outras partes interessadas, como a alta administração, autoridades públicas e o público também são cruciais para concluir o processo de VTT. Como os veículos autónomos exigem centenas de milhões de quilómetros de testes conduzidos em vias públicas antes da certificação do veículo, as simulações estão a desempenhar cada vez mais um papel fundamental, pois permitem que as ferramentas de simulação testem virtualmente milhões de cenários da vida real, aumentando a segurança e reduzindo custos, tempo e necessidade de testes físicos em estrada. Neste estudo, é realizada uma revisão da literatura para classificar abordagens para a VVT e uma ferramenta de simulação existente é usada para implementar um sistema de direção autónoma. O sistema é caracterizado do ponto de vista do seu desempenho em alguns cenários críticos de autoestrad

Compositional Falsification of Cyber-Physical Systems with Machine Learning Components

Cyber-physical systems (CPS), such as automotive systems, are starting to include sophisticated machine learning (ML) components. Their correctness, therefore, depends on properties of the inner ML modules. While learning algorithms aim to generalize from examples, they are only as good as the examples provided, and recent efforts have shown that they can produce inconsistent output under small adversarial perturbations. This raises the question: can the output from learning components can lead to a failure of the entire CPS? In this work, we address this question by formulating it as a problem of falsifying signal temporal logic (STL) specifications for CPS with ML components. We propose a compositional falsification framework where a temporal logic falsifier and a machine learning analyzer cooperate with the aim of finding falsifying executions of the considered model. The efficacy of the proposed technique is shown on an automatic emergency braking system model with a perception component based on deep neural networks

From Verified Models to Verified Code for Safe Medical Devices

Medical devices play an essential role in the care of patients around the world, and can have a life-saving effect. An emerging category of autonomous medical devices like implantable pacemakers and implantable cardioverter defibrillators (ICD) diagnose conditions of the patient and autonomously deliver therapies. Without trained professionals in the loop, the software component of autonomous medical devices is responsible for making critical therapeutic decisions, which pose a new set of challenges to guarantee patient safety. As regulation effort to guarantee patient safety, device manufacturers are required to submit evidence for the safety and efficacy of the medical devices before they can be released to the market. Due to the closed-loop interaction between the device and the patient, the safety and efficacy of autonomous medical devices must ultimately be evaluated within their physiological context. Currently the primary closed-loop validation of medical devices is in form of clinical trials, in which the devices are evaluated on real patients. Clinical trials are expensive and expose the patients to risks associated with untested devices. Clinical trials are also conducted after device development, therefore issues found during clinical trials are expensive to fix. There is urgent need for closed-loop validation of autonomous medical devices before the devices are used in clinical trials. In this thesis, I used implantable cardiac devices to demonstrate the applications of model-based approaches during and after device development to provide confidence towards the safety and efficacy of the devices. A heart model structure is developed to mimic the electrical behaviors of the heart in various heart conditions. The heart models created with the model structure are capable of interacting with implantable cardiac devices in closed-loop and can provide physiological interpretations for a large variety of heart conditions. With the heart models, I demonstrated that closed-loop model checking is capable of identifying known and unknown safety violations within the pacemaker design. More importantly, I developed a framework to choose the most appropriate heart models to cover physiological conditions that the pacemaker may encounter, and provide physiological context to counter-examples returned by the model checker. A model translation tool UPP2SF is then developed to translate the pacemaker design in UPPAAL to Stateflow, and automatically generated to C code. The automated and rigorous translation ensures that the properties verified during model checking still hold in the implementation, which justifies the model checking effort. Finally, the devices are evaluated with a virtual patient cohort consists of a large number of heart models before evaluated in clinical trials. These in-silico pre-clinical trials provide useful insights which can be used to increase the success rate of a clinical trial. The work in this dissertation demonstrated the importance and challenges to represent physiological behaviors during closed-loop validation of autonomous medical devices, and demonstrated the capability of model-based approaches to provide safety and efficacy evidence during and after device development

Developing a distributed electronic health-record store for India

The DIGHT project is addressing the problem of building a scalable and highly available information store for the Electronic Health Records (EHRs) of the over one billion citizens of India

High-Confidence Medical Device Software Development

The design of bug-free and safe medical device software is challenging, especially in complex implantable devices. This is due to the device\u27s closed-loop interaction with the patient\u27s organs, which are stochastic physical environments. The life-critical nature and the lack of existing industry standards to enforce software validation make this an ideal domain for exploring design automation challenges for integrated functional and formal modeling with closed-loop analysis. The primary goal of high-confidence medical device software is to guarantee the device will never drive the patient into an unsafe condition even though we do not have complete understanding of the physiological plant. There are two major differences between modeling physiology and modeling man-made systems: first, physiology is much more complex and less well-understood than man-made systems like cars and airplanes, and spans several scales from the molecular to the entire human body. Secondly, the variability between humans is orders of magnitude larger than that between two cars coming off the assembly line. Using the implantable cardiac pacemaker as an example of closed-loop device, and the heart as the organ to be modeled, we present several of the challenges and early results in model-based device validation. We begin with detailed timed automata model of the pacemaker, based on the specifications and algorithm descriptions from Boston Scientific. For closed-loop evaluation, a real-time Virtual Heart Model (VHM) has been developed to model the electrophysiological operation of the functioning and malfunctioning (i.e., during arrhythmia) hearts. By extracting the timing properties of the heart and pacemaker device, we present a methodology to construct timed-automata models for formal model checking and functional testing of the closed-loop system. The VHM\u27s capability of generating clinically-relevant response has been validated for a variety of common arrhythmias. Based on a set of requirements, we describe a framework of Abstraction Trees that allows for interactive and physiologically relevant closed-loop model checking and testing for basic pacemaker device operations such as maintaining the heart rate, atrial-ventricle synchrony and complex conditions such as avoiding pacemaker-mediated tachycardia. Through automatic model translation of abstract models to simulation-based testing and code generation for platform-level testing, this model-based design approach ensures the closed-loop safety properties are retained through the design toolchain and facilitates the development of verified software from verified models. This system is a step toward a validation and testing approach for medical cyber-physical systems with the patient-in-the-loop

Testing Strategies for Model-Based Development

This report presents an approach for testing artifacts generated in a model-based development process. This approach divides the traditional testing process into two parts: requirements-based testing (validation testing) which determines whether the model implements the high-level requirements and model-based testing (conformance testing) which determines whether the code generated from a model is behaviorally equivalent to the model. The goals of the two processes differ significantly and this report explores suitable testing metrics and automation strategies for each. To support requirements-based testing, we define novel objective requirements coverage metrics similar to existing specification and code coverage metrics. For model-based testing, we briefly describe automation strategies and examine the fault-finding capability of different structural coverage metrics using tests automatically generated from the model

Performance-Driven Metamorphic Testing of Cyber-Physical Systems

Cyber-physical systems (CPSs) are a new generation of systems, which integrate software with physical processes. The increasing complexity of these systems, combined with the un certainty in their interactions with the physical world, makes the definition of effective test oracles especially challenging, facing the well-known test oracle problem. Metamorphic testing has shown great potential to alleviate the test oracle problem by exploiting the relations among the inputs and outputs of different executions of the system, so-called metamorphic relations (MRs). In this article, we propose an MR pattern called PV for the identification of performance-driven MRs, and we show its applicability in two CPSs from different domains, which are automated navigation systems and elevator control systems. For the evaluation, we as sessed the effectiveness of this approach for detecting failures in an open-source simulation-based autonomous navigation system, as well as in an industrial case study from the elevation domain. We derive concrete MRs based on the PV pattern for both case studies, and we evaluate their effectiveness with seeded faults. Results show that the approach is effective at detecting over 88% of the seeded faults, while keeping the ratio of FPs at 4% or lower.European Union's Horizon 2020 Research and Innovation Programme (Grant Number: 871319)Junta de Andalucía US-1264651 (APOLO)Junta de Andalucía P18-FR-2895 (EKIPMENT-PLUS)Ministerio de Ciencia e Innovación RTI2018-101204-B-C21 (HORATIO)Mondragon Unibertsitatea IT1519-2