High-Confidence Medical Device Software Development

The design of bug-free and safe medical device software is challenging, especially in complex implantable devices. This is due to the device\u27s closed-loop interaction with the patient\u27s organs, which are stochastic physical environments. The life-critical nature and the lack of existing industry standards to enforce software validation make this an ideal domain for exploring design automation challenges for integrated functional and formal modeling with closed-loop analysis. The primary goal of high-confidence medical device software is to guarantee the device will never drive the patient into an unsafe condition even though we do not have complete understanding of the physiological plant. There are two major differences between modeling physiology and modeling man-made systems: first, physiology is much more complex and less well-understood than man-made systems like cars and airplanes, and spans several scales from the molecular to the entire human body. Secondly, the variability between humans is orders of magnitude larger than that between two cars coming off the assembly line. Using the implantable cardiac pacemaker as an example of closed-loop device, and the heart as the organ to be modeled, we present several of the challenges and early results in model-based device validation. We begin with detailed timed automata model of the pacemaker, based on the specifications and algorithm descriptions from Boston Scientific. For closed-loop evaluation, a real-time Virtual Heart Model (VHM) has been developed to model the electrophysiological operation of the functioning and malfunctioning (i.e., during arrhythmia) hearts. By extracting the timing properties of the heart and pacemaker device, we present a methodology to construct timed-automata models for formal model checking and functional testing of the closed-loop system. The VHM\u27s capability of generating clinically-relevant response has been validated for a variety of common arrhythmias. Based on a set of requirements, we describe a framework of Abstraction Trees that allows for interactive and physiologically relevant closed-loop model checking and testing for basic pacemaker device operations such as maintaining the heart rate, atrial-ventricle synchrony and complex conditions such as avoiding pacemaker-mediated tachycardia. Through automatic model translation of abstract models to simulation-based testing and code generation for platform-level testing, this model-based design approach ensures the closed-loop safety properties are retained through the design toolchain and facilitates the development of verified software from verified models. This system is a step toward a validation and testing approach for medical cyber-physical systems with the patient-in-the-loop

Model-Based Closed-Loop Testing of Implantable Pacemakers

The increasing complexity of software in implantable medical devices such as cardiac pacemakers and defibrillators accounts for over 40% of device recalls. Testing remains the principal means of verification in the medical device certification regime. Traditional software test generation techniques, where the tests are generated independently of the operational environment, are not effective as the device must be tested within the context of the patient\u27s condition and the current state of the heart. It is necessary for the testing system to observe the system state and conditionally generate the next input to advance the purpose of the test. To this effect, a set of general and patient condition-specific temporal requirements is specified for the closed-loop heart and pacemaker system. Based on these requirements, we describe a closed-loop testing environment between a timed automata-based heart model and a pacemaker. This allows for interactive and physiologically relevant model-based test generation for basic pacemaker device operations such as maintaining the heart rate and atrial-ventricle synchrony. We also demonstrate the flexibility and efficacy of the testing environment for more complex common timing anomalies such as reentry circuits, pacemaker mode switch operation and pacemaker-mediated tachycardia. This system is a step toward a testing approach for medical cyber-physical systems with the patient-in-the-loop

Cyber-Physical Modeling of Implantable Cardiac Medical Devices

The design of bug-free and safe medical device software is challenging, especially in complex implantable devices that control and actuate organs in unanticipated contexts. Safety recalls of pacemakers and implantable cardioverter defibrillators between 1990 and 2000 affected over 600,000 devices. Of these, 200,000 or 41%, were due to firmware issues and their effect continues to increase in frequency. There is currently no formal methodology or open experimental platform to test and verify the correct operation of medical device software within the closed-loop context of the patient. To this effect, a real-time Virtual Heart Model (VHM) has been developed to model the electrophysiological operation of the functioning and malfunctioning (i.e., during arrhythmia) heart. By extracting the timing properties of the heart and pacemaker device, we present a methodology to construct a timed-automata model for functional and formal testing and verification of the closed-loop system. The VHM\u27s capability of generating clinically-relevant response has been validated for a variety of common arrhythmias. Based on a set of requirements, we describe a closed-loop testing environment that allows for interactive and physiologically relevant model-based test generation for basic pacemaker device operations such as maintaining the heart rate, atrial-ventricle synchrony and complex conditions such as pacemaker-mediated tachycardia. This system is a step toward a testing and verification approach for medical cyber-physical systems with the patient-in-the-loop

On Falsification of Large-Scale Cyber-Physical Systems

In the development of modern Cyber-Physical Systems, Model-Based Testingof the closed-loop system is an approach for finding potential faults andincreasing quality of developed products. Testing is done on many differentabstraction levels, and for large-scale industrial systems, there are severalchallenges. Executing tests on the systems can be time-consuming and largenumbers of complex specifications need to be thoroughly tested, while manyof the popular academic benchmarks do not necessarily reflect on this complexity.This thesis proposes new methods for analyzing and generating test casesas a means for being more certain that proper testing has been performed onthe system under test. For analysis, the proposed approach can automaticallyfind out how much of the physical parts of the system that the test suite hasexecuted.For test case generation, an approach to find errors is optimization-basedfalsification. This thesis attempts to close the gap between academia and industryby applying falsification techniques to real-world models from VolvoCar Corporation and adapting the falsification procedure where it has shortcomingsfor certain classes of systems. Specifically, the main contributionsof this thesis are (i) a method for automatically transforming a signal-basedspecification into a formal specification allowing an optimization-based falsificationapproach, (ii) a new collection of specifications inspired by large-scalespecifications from industry, (iii) an algorithm to perform optimization-basedfalsification for such a large set of specifications, and (iv) a new type of coveragecriterion for Cyber-Physical Systems that can help to assess when testingcan be concluded.The proposed methods have been evaluated for both academic benchmarkexamples and real-world industrial models. One of the main conclusions isthat the proposed additions and changes to the analysis and generation oftests can be useful, given that one has enough information about the systemunder test. The methods presented in this thesis have been applied to realworldmodels in a way that allows for higher-quality products by finding morefaults in early phases of development

Cyber-Virtual Systems: Simulation, Validation & Visualization

We describe our ongoing work and view on simulation, validation and visualization of cyber-physical systems in industrial automation during development, operation and maintenance. System models may represent an existing physical part - for example an existing robot installation - and a software simulated part - for example a possible future extension. We call such systems cyber-virtual systems. In this paper, we present the existing VITELab infrastructure for visualization tasks in industrial automation. The new methodology for simulation and validation motivated in this paper integrates this infrastructure. We are targeting scenarios, where industrial sites which may be in remote locations are modeled and visualized from different sites anywhere in the world. Complementing the visualization work, here, we are also concentrating on software modeling challenges related to cyber-virtual systems and simulation, testing, validation and verification techniques for them. Software models of industrial sites require behavioural models of the components of the industrial sites such as models for tools, robots, workpieces and other machinery as well as communication and sensor facilities. Furthermore, collaboration between sites is an important goal of our work.Comment: Preprint, 9th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE 2014

Advanced Testing Chain Supporting the Validation of Smart Grid Systems and Technologies

New testing and development procedures and methods are needed to address topics like power system stability, operation and control in the context of grid integration of rapidly developing smart grid technologies. In this context, individual testing of units and components has to be reconsidered and appropriate testing procedures and methods need to be described and implemented. This paper addresses these needs by proposing a holistic and enhanced testing methodology that integrates simulation/software- and hardware-based testing infrastructure. This approach presents the advantage of a testing environment, which is very close to f i eld testing, includes the grid dynamic behavior feedback and is risks-free for the power system, for the equipment under test and for the personnel executing the tests. Furthermore, this paper gives an overview of successful implementation of the proposed testing approach within different testing infrastructure available at the premises of different research institutes in Europe.Comment: 2018 IEEE Workshop on Complexity in Engineering (COMPENG

ERIGrid Holistic Test Description for Validating Cyber-Physical Energy Systems

Smart energy solutions aim to modify and optimise the operation of existing energy infrastructure. Such cyber-physical technology must be mature before deployment to the actual infrastructure, and competitive solutions will have to be compliant to standards still under development. Achieving this technology readiness and harmonisation requires reproducible experiments and appropriately realistic testing environments. Such testbeds for multi-domain cyber-physical experiments are complex in and of themselves. This work addresses a method for the scoping and design of experiments where both testbed and solution each require detailed expertise. This empirical work first revisited present test description approaches, developed a newdescription method for cyber-physical energy systems testing, and matured it by means of user involvement. The new Holistic Test Description (HTD) method facilitates the conception, deconstruction and reproduction of complex experimental designs in the domains of cyber-physical energy systems. This work develops the background and motivation, offers a guideline and examples to the proposed approach, and summarises experience from three years of its application.This work received funding in the European Community’s Horizon 2020 Program (H2020/2014–2020) under project “ERIGrid” (Grant Agreement No. 654113)

Evaluating Model Testing and Model Checking for Finding Requirements Violations in Simulink Models

Matlab/Simulink is a development and simulation language that is widely used by the Cyber-Physical System (CPS) industry to model dynamical systems. There are two mainstream approaches to verify CPS Simulink models: model testing that attempts to identify failures in models by executing them for a number of sampled test inputs, and model checking that attempts to exhaustively check the correctness of models against some given formal properties. In this paper, we present an industrial Simulink model benchmark, provide a categorization of different model types in the benchmark, describe the recurring logical patterns in the model requirements, and discuss the results of applying model checking and model testing approaches to identify requirements violations in the benchmarked models. Based on the results, we discuss the strengths and weaknesses of model testing and model checking. Our results further suggest that model checking and model testing are complementary and by combining them, we can significantly enhance the capabilities of each of these approaches individually. We conclude by providing guidelines as to how the two approaches can be best applied together.Comment: 10 pages + 2 page reference