Using Formal Methods for Autonomous Systems: Five Recipes for Formal Verification

Formal Methods are mathematically-based techniques for software design and engineering, which enable the unambiguous description of and reasoning about a system's behaviour. Autonomous systems use software to make decisions without human control, are often embedded in a robotic system, are often safety-critical, and are increasingly being introduced into everyday settings. Autonomous systems need robust development and verification methods, but formal methods practitioners are often asked: Why use Formal Methods for Autonomous Systems? To answer this question, this position paper describes five recipes for formally verifying aspects of an autonomous system, collected from the literature. The recipes are examples of how Formal Methods can be an effective tool for the development and verification of autonomous systems. During design, they enable unambiguous description of requirements; in development, formal specifications can be verified against requirements; software components may be synthesised from verified specifications; and behaviour can be monitored at runtime and compared to its original specification. Modern Formal Methods often include highly automated tool support, which enables exhaustive checking of a system's state space. This paper argues that Formal Methods are a powerful tool for the repertoire of development techniques for safe autonomous systems, alongside other robust software engineering techniques.Comment: Accepted at Journal of Risk and Reliabilit

Formal Specifications and Analysis of the Computer Assisted Resuscitation Algorithm (CARA) Infusion Pump Control System

Reliability of medical devices such as the CARA Infusion Pump Control System is of extreme importance given that these devices are being used on patients in critical condition. The Infusion Pump Control System includes embedded processors and accompanying embedded software for monitoring as well as controlling sensors and actuators that allow the embedded systems to interact with their environments. This nature of the Infusion Pump Control System adds to the complexity of assuring the reliability of the total system. The traditional methods of developing embedded systems are inadequate for such safety-critical devices. In this paper, we study the application of formal methods to the requirements capture and analysis for the Infusion Pump Control System. Our approach consists of two phases. The first phase is to convert the informal design requirements into a set of reference specifications using a formal system, in this case EFSMs (Extended Finite State Machines). The second phase is to translate the reference specifications to the tools supporting formal analysis, such as SCR and Hermes. This allows us to conclude properties of the reference specifications. Our research goal is to develop a framework and methodology for the integrated use of formal methods in the development of embedded medical systems that require high assurance and confidence

Formal Guarantees for Safety Critical Code Generation: the Case of Highly Variable Languages

Control and command softwares play a key role in safety-critical embedded systems used for human related activities such as transportation, healthcare or energy. Their impact on safety makes the assessment of their correctness the central point in their development activities. Such systems verification activities are usually conducted according to normative certification guidelines providing objectives to be reached in order to ensure development process reliability and thus prevent flaws. Verification activities usually relies on tests and proof reading of the software but recent versions of certification guidelines are taking into account the deployment of new development paradigms such as model-based development, and formal methods; or the use of tools in assistance of the development processes. Automatic code generators are used in most safety-critical embedded systems development in order to avoid human related software production errors and to ensure the respect of development quality standards. As these tools are supposed to replace humans in the software code production activities, errors in these tools may result in embedded software flaws. It is thus in turn mandatory to ensure the same level of correctness for the tool itself than for the expected produced code. Tools verification shall be done according to qualification guidelines. We advocate in our work the use of model-based development and formal methods for the development of these tools in order to reach a higher quality level. Critical control and command software are mostly designed using graphical dataflow languages. These languages are used to express complex systems relying on atomic operations embedded in blocks that are gathered in block libraries. Blocks may be sophisticated pieces of software with highly variable structure and semantics. This variability is dependent on the values of the block parameters and of the block's context of use. In our work, we focus on the formal specification and verification of such block based languages. We experimented various techniques in order to ensure a formal, sound, verifiable and usable specification for blocks. We developed a domain specific formal model-based language specifically tailored for the specification of structure and semantics of blocks. This specification language is inspired from software product line concepts in order to ensure a correct and scalable management of the blocks variability. We have applied this specification and verification approach on chosen block examples from common industrial use cases and we have validated it on tool prototypes. Blocks are the core elements of the input language of automatic code generators used for control and command systems development. We show how our blocks formal specification can be translated as code annotations in order to ease and automate the generated code verification. Code annotations are verified using specialised static code analysis tools. Relying on synchronous observers to express high level requirements at the input model level, we show how formal block specification can also be used for the translation of high level requirements as verifiable code annotations discharged using the same specialised tooling. We finally target the assistance of code generation tools qualification activities by arguing on the ability to automatically generate qualification data such as requirements, tests or simulation results for the verification and development of automatic code generators from the formal block specification

A framework for the requirements analysis of safety-critical computing systems

PhD ThesisDigital computers are increasingly being used in safety-critical applications (e.g., avionics, chemical plant and railway systems). The main motivations for introducing computers into such environments are to increase performance, flexibility and efficiency. However, the cost to safety in achieving these benefits using computing systems is unclear. The general class of systems considered in this thesis are process control systems. More specifically the thesis examines the class of safety-critical computing systems which are a component of a process control system that could cause or allow the overall system to enter into a hazardous state. This thesis investigates the role oiformal methods in safety-critical computing systems. The phase of system development considered is requirements analysis. Experience in safety-critical systems has shown that errors in the identified requirements are one of the major causes of mishap. It is argued that to gain a complete understanding of such computing systems, the requirements of the overall system and the properties of the environment must be analyzed in a common formal framework. A system development model based on the separation of safety and mission issues is discussed, which highlights the essential specifications that must be produced during requirements analysis. A formal model for the representation of these essential specifications is presented. The semantics of this formal model are based on the notion of a system history. To structure the specifications expressed by this formal model the concept of a mode is introduced. This thesis suggests that for a formal model to be useful during requirements analysis a related systematic methodology, which provides comprehensive guidelines for the analysts who use the model must be made available. An appropriate methodology, based upon the system development model, which incorporates some traditional system safety techniques is described. Overall, the thesis presents a framework for requirements analysis by providing a system development model, formal model and related development methodology. An example of how this framework can support requirements analysis is presented in the appendices Band C.UK Science and Engineering Research Council: Alvey Software Reliability Project Grant

Formal Verification and Validation of ERTMS Industrial Railway Train Spacing System

Abstract. Formal verification and validation is a fundamental step for the certifi-cation of railways critical systems. Many railways safety standards (e.g. the CEN-ELEC EN-50126, EN-50128 and EN-50129 standards implement the mandatory safety requirements of IEC-61508-7 standard for Functional and Safety) currently mandate the use of formal methods in the design to certify correctness. In this paper we describe an industrial application of formal methods for the ver-ification and validation of “Logica di Sicurezza ” (LDS), the safety logic of a railways ERTMS Level 2 system developed by Ansaldo-STS. LDS is a generic control software that needs to be instantiated on a railways network configuration. We developed a methodology for the verification and validation of a critical sub-set of LDS deployed on typical realistic railways network configurations. To show feasibility, effectiveness and scalability, we have experimented with several state of the art symbolic software model checking techniques and tools on different network configurations. From the experiments, we have successfully identified an effective strategy for the verification and validation of our case study. More-over, the results of experiments show that formal verification and validation is feasible and effective, and also scales reasonably well with the size of the config-uration. Given the results, Ansaldo-STS is currently integrating the methodology in its internal Development and Verification & Validation Flow.

Analysing MTL Properties using NuSMV model checker

Reliability and safety property of any hardware is an important parameter. To achieve this and to improve the performance bounds of the designed system it is important to enhance the efficiency by proper verification techniques. To overcome the problems arising due to the software crisis, formal methods are used. The use of formal methods in aerospace domain is the latest research that is being carried out. Formal verification, a part of formal methods is a mathematical modelling technique used to verify the hardware systems. Technique such as model checking is used to efficiently bridge the gap between design and developed stage of the system with less errors and more efficiency. In this paper, we propose to use NuSMV for verifying the vertical mode functionality of the Mode Transition Logic (MTL). MTL is a very critical functionality in aircraft. It assists the control of trajectories, weather and systems. The NuSMV model checker is used to analyse the functional behaviour of the model. The model is initially designed and developed using Mat-lab/Simulink tool suite. The semantic translation of the MTL model to NuSMV is done by means of specification languages such as CTL and LTL. Test cases generated at the Simulink model level are used as a reference to test the linear and non-linear properties of the MTL vertical model in NuSMV. These test cases are compared with the results obtained using NuSMV analysis. The efficiency is defined by earlier fault detection and improving the software development life cycle of the system

Implementing Software Safety in the NASA Environment

Until recently, NASA did not consider allowing computers total control of flight systems. Human operators, via hardware, have constituted the ultimate safety control. In an attempt to reduce costs, NASA has come to rely more and more heavily on computers and software to control space missions. (For example. software is now planned to control most of the operational functions of the International Space Station.) Thus the need for systematic software safety programs has become crucial for mission success. Concurrent engineering principles dictate that safety should be designed into software up front, not tested into the software after the fact. 'Cost of Quality' studies have statistics and metrics to prove the value of building quality and safety into the development cycle. Unfortunately, most software engineers are not familiar with designing for safety, and most safety engineers are not software experts. Software written to specifications which have not been safety analyzed is a major source of computer related accidents. Safer software is achieved step by step throughout the system and software life cycle. It is a process that includes requirements definition, hazard analyses, formal software inspections, safety analyses, testing, and maintenance. The greatest emphasis is placed on clearly and completely defining system and software requirements, including safety and reliability requirements. Unfortunately, development and review of requirements are the weakest link in the process. While some of the more academic methods, e.g. mathematical models, may help bring about safer software, this paper proposes the use of currently approved software methodologies, and sound software and assurance practices to show how, to a large degree, safety can be designed into software from the start. NASA's approach today is to first conduct a preliminary system hazard analysis (PHA) during the concept and planning phase of a project. This determines the overall hazard potential of the system to be built. Shortly thereafter, as the system requirements are being defined, the second iteration of hazard analyses takes place, the systems hazard analysis (SHA). During the systems requirements phase, decisions are made as to what functions of the system will be the responsibility of software. This is the most critical time to affect the safety of the software. From this point, software safety analyses as well as software engineering practices are the main focus for assuring safe software. While many of the steps proposed in this paper seem like just sound engineering practices, they are the best technical and most cost effective means to assure safe software within a safe system

Support for collaborative component-based software engineering

Collaborative system composition during design has been poorly supported by traditional CASE tools (which have usually concentrated on supporting individual projects) and almost exclusively focused on static composition. Little support for maintaining large distributed collections of heterogeneous software components across a number of projects has been developed. The CoDEEDS project addresses the collaborative determination, elaboration, and evolution of design spaces that describe both static and dynamic compositions of software components from sources such as component libraries, software service directories, and reuse repositories. The GENESIS project has focussed, in the development of OSCAR, on the creation and maintenance of large software artefact repositories. The most recent extensions are explicitly addressing the provision of cross-project global views of large software collections and historical views of individual artefacts within a collection. The long-term benefits of such support can only be realised if OSCAR and CoDEEDS are widely adopted and steps to facilitate this are described. This book continues to provide a forum, which a recent book, Software Evolution with UML and XML, started, where expert insights are presented on the subject. In that book, initial efforts were made to link together three current phenomena: software evolution, UML, and XML. In this book, focus will be on the practical side of linking them, that is, how UML and XML and their related methods/tools can assist software evolution in practice. Considering that nowadays software starts evolving before it is delivered, an apparent feature for software evolution is that it happens over all stages and over all aspects. Therefore, all possible techniques should be explored. This book explores techniques based on UML/XML and a combination of them with other techniques (i.e., over all techniques from theory to tools). Software evolution happens at all stages. Chapters in this book describe that software evolution issues present at stages of software architecturing, modeling/specifying, assessing, coding, validating, design recovering, program understanding, and reusing. Software evolution happens in all aspects. Chapters in this book illustrate that software evolution issues are involved in Web application, embedded system, software repository, component-based development, object model, development environment, software metrics, UML use case diagram, system model, Legacy system, safety critical system, user interface, software reuse, evolution management, and variability modeling. Software evolution needs to be facilitated with all possible techniques. Chapters in this book demonstrate techniques, such as formal methods, program transformation, empirical study, tool development, standardisation, visualisation, to control system changes to meet organisational and business objectives in a cost-effective way. On the journey of the grand challenge posed by software evolution, the journey that we have to make, the contributory authors of this book have already made further advances

Augmenting a Hazard Analysis Method with Error Propagation Information for Safety-Critical Systems

Safety-critical systems need specific activities in the software development life cycle to ensure that the system will operate safely. The objective of this dissertation is to develop a new safety analysis method to identify hazards. The method uses error propagation information and the internal structure rather than the interfaces of a system. We propose development procedures to augment STPA (System-Theoretic Process Analysis) with error propagation information derived from the architecture description of a system represented in the AADL (Architecture Analysis Design Language). We will focus on how the AADL error ontology can be used to assist in identifying errors, how those errors propagate among components, and whether the errors lead to hazards in the system. Our research shows that tracing error propagation leads to the discovery of hazards and additional information that other methods miss. The new safety analysis method, Architecture Safety Analysis Method (ASAM), by augmenting STPA with early design information, is able to find more hazards, unsafe control actions, safety constraints and causes of the unsafe control actions than by using STPA alone. Our method leaves more false positives than STPA, but in safety analysis having false positive is preferred over missing actual hazards. We use the AADL error ontology to rigorously describe system component errors and how they propagate among components. We illustrate this rigorous description through several examples and we demonstrate that it yields hazards that an STPA analysis of the example did not find. In addition, we provide a mathematical notation and expressions so that formal analysis and verification of the hazards can be done to ensure that all causes of the hazards have been identified and that any developed safety constraints fully mitigate the hazards, through the use of compositional reasoning

SPECIFICATION FORMALIZATION OF STATE CHARTS FOR COMPLEX SYSTEM MANAGEMENT

This article presents a formalization approach for the requirements of object-oriented programs with state machines, using a spacecraft control system as a case study. It proposes a state pattern implementation, where each state is represented as a class with clearly defined responsibilities, and the transitions between states are controlled by the state objects themselves. Additionally, the application of model checking, theorem proving, and code generation techniques are discussed. The effectiveness of the proposed approach in ensuring compliance with the specified requirements is demonstrated, while also identifying potential drawbacks and limitations of the approach. The implementation is validated using a range of formal verification techniques, including model checking and theorem proving. The article also discusses how the approach can be extended and applied to other complex systems. Overall, the valuable insights into the formalization of requirements for object-oriented programs with state machines are provided, offering a practical and effective approach for verifying the correctness and completeness of such implementations. The results of this work have important implications for the development of safety-critical systems and can potentially improve the quality and reliability of software systems in various domains. By using mathematical models and rigorous formal methods, it is possible to detect and eliminate errors early in the development process, leading to higher confidence in the correctness of the final product. Future research in this area could explore the use of more advanced techniques, such as model-driven development and automatic code synthesis, to further streamline the software development process. Additionally, the development of more efficient and user-friendly tools could make these techniques more accessible to a wider range of developers and organizations. Altogether, the combination of formal methods and software engineering has the potential to revolutionize the way software systems are designed, developed, and verified, leading to safer and more reliable software for critical applications