Formal mutation testing for Circus

International audienceContext: The demand from industry for more dependable and scalable test-development mechanisms has fostered the use of formal models to guide the generation of tests. Despite many advancements having been obtained with state-based models, such as Finite State Machines (FSMs) and Input/Output Transition Systems (IOTSs), more advanced formalisms are required to specify large, state-rich, concurrent systems. Circus, a state-rich process algebra combining Z, CSP and a refinement calculus, is suitable for this; however, deriving tests from such models is accordingly more challenging. Recently, a testing theory has been stated for Circus, allowing the verification of process refinement based on exhaustive test sets. Objective: We investigate fault-based testing for refinement from Circus specifications using mutation. We seek the benefits of such techniques in test-set quality assertion and fault-based test-case selection. We target results relevant not only for Circus, but to any process algebra for refinement that combines CSP with a data language. Method: We present a formal definition for fault-based test sets, extending the Circus testing theory, and an extensive study of mutation operators for Circus. Using these results, we propose an approach to generate tests to kill mutants. Finally, we explain how prototype tool support can be obtained with the implementation of a mutant generator, a translator from Circus to CSP, and a refinement checker for CSP, and with

Testing robots using CSP

This paper presents a technique for automatic generation of tests for robotic systems based on a domain-specific notation called RoboChart. This is a UML-like diagrammatic notation that embeds a component model suitable for robotic systems, and supports the definition of behavioural models using enriched state machines that can feature time properties. The formal semantics of RoboChart is given using tockCSP, a discrete-time variant of the process algebra CSP. In this paper, we use the example of a simple drone to illustrate an approach to generate tests from RoboChart models using a mutation tool called Wodel. From mutated models, tests are generated using the CSP model checker FDR. The testing theory of CSP justifies the soundness of the tests

Towards a UTP semantics for modelica

We describe our work on a UTP semantics for the dynamic systems modelling language Modelica. This is a language for modelling a system’s continuous behaviour using a combination of differential algebraic equations and an event-handling system. We develop a novel UTP theory of hybrid relations, inspired by Hybrid CSP and Duration Calculus, that is purely relational and provides uniform handling of continuous and discrete variables. This theory is mechanised in our Isabelle implementation of the UTP, Isabelle/UTP, with which we verify some algebraic properties. Finally, we show how a subset of Modelica models can be given semantics using our theory. When combined with the wealth of existing UTP theories for discrete system modelling, our work enables a sound approach to heterogeneous semantics for Cyber-Physical systems by leveraging the theory linking facilities of the UTP

The 14th Overture Workshop: Towards Analytical Tool Chains

This report contains the proceedings from the 14th Overture workshop organized in connection with the Formal Methods 2016 symposium. This includes nine papers describing different technological progress in relation to the Overture/VDM tool support and its connection with other tools such as Crescendo, Symphony, INTO-CPS, TASTE and ViennaTalk

Safety assurance of an industrial robotic control system using hardware/software co-verification

As a general trend in industrial robotics, an increasing number of safety functions are being developed or re-engineered to be handled in software rather than by physical hardware such as safety relays or interlock circuits. This trend reinforces the importance of supplementing traditional, input-based testing and quality procedures which are widely used in industry today, with formal verification and model-checking methods. To this end, this paper focuses on a representative safety-critical system in an ABB industrial paint robot, namely the High-Voltage electrostatic Control system (HVC). The practical convergence of the high-voltage produced by the HVC, essential for safe operation, is formally verified using a novel and general co-verification framework where hardware and software models are related via platform mappings. This approach enables the pragmatic combination of highly diverse and specialised tools. The paper's main contribution includes details on how hardware abstraction and verification results can be transferred between tools in order to verify system-level safety properties. It is noteworthy that the HVC application considered in this paper has a rather generic form of a feedback controller. Hence, the co-verification framework and experiences reported here are also highly relevant for any cyber-physical system tracking a setpoint reference

An Empirical Investigation of Using Models During Requirements Engineering in the Automotive Industry

Context:The automotive industry is undergoing a major transformation from a manufacturing industry towards an industry that relies heavily on software. As one of the main factors for project success, requirements engineering (RE) plays a major role in this transition. Similar to other areas of automotive engineering, the use of models during RE has been suggested to increase productivity and tackle increasing complexity by means of abstraction. Existing modelling frameworks often prescribe a variety of different, formal models for RE, trying to maximise the benefit obtained from model-based engineering (MBE). However, these frameworks are typically based on assumptions from anecdotal evidence and experience, without empirical data supporting these assumptions.Objective:The overall aim of our research is to investigate the potential benefits and drawbacks of using model-based RE in an automotive environment based on empirical evidence. To do so, we present an investigation of the current industrial practice of MBE in the automotive industry, existing challenges in automotive RE, and potential use cases for model-based RE. Furthermore, we explore two use cases for model-based RE, namely the creation of behavioural requirements models for validation and verification purposes and the use of existing trace models to support communication.Method:We address the aims of this thesis using three empirical strategies: case study, design science and survey. We collected quantitative and qualitative data using interviews as well as questionnaires.Results:Our results show that using models during automotive RE can be beneficial, if restricted to certain aspects of RE. In particular, models supporting communication and stakeholder interaction are promising. We show that the use of abstract models of behavioural requirements are considered beneficial for system testing purposes, even though they abstract from the detailed functional requirements. Furthermore, we demonstrate that existing data can be understood as a model to uncover dependencies between stakeholders. Conclusions:Our results question the feasibility to construct and maintain large amounts of formal models for RE. Instead, models during RE should be used for a few, important use cases. Additionally, MBE can be used as a means to understand existing problems in software engineering

Integration of Formal Proof into Unified Assurance Cases with Isabelle/SACM

Assurance cases are often required to certify critical systems. The use of formal methods in assurance can improve automation, increase confidence, and overcome errant reasoning. However, assurance cases can never be fully formalised, as the use of formal methods is contingent on models that are validated by informal processes. Consequently, assurance techniques should support both formal and informal artifacts, with explicated inferential links between them. In this paper, we contribute a formal machine-checked interactive language, called Isabelle/SACM, supporting the computer-assisted construction of assurance cases compliant with the OMG Structured Assurance Case Meta-Model. The use of Isabelle/SACM guarantees well-formedness, consistency, and traceability of assurance cases, and allows a tight integration of formal and informal evidence of various provenance. In particular, Isabelle brings a diverse range of automated verification techniques that can provide evidence. To validate our approach, we present a substantial case study based on the Tokeneer secure entry system benchmark. We embed its functional specification into Isabelle, verify its security requirements, and form a modular security case in Isabelle/SACM that combines the heterogeneous artifacts. We thus show that Isabelle is a suitable platform for critical systems assurance

A model-based approach to System of Systems risk management

The failure of many System of Systems (SoS) enterprises can be attributed to the inappropriate application of traditional Systems Engineering (SE) processes within the SoS domain, because of the mistaken belief that a SoS can be regarded as a single large, or complex, system. SoS Engineering (SoSE) is a sub-discipline of SE; Risk Management and Modelling and Simulation (M&S) are key areas within SoSE, both of which also lie within the traditional SE domain. Risk Management of SoS requires a different approach to that currently taken for individual systems; if risk is managed for each component system then it cannot be assumed that the aggregated affect will be to mitigate risk at the SoS level. A literature review was undertaken examining three themes: (1) SoS Engineering (SoSE), (2) M&S and (3) Risk. Theme 1 of the literature provided insight into the activities comprising SoSE and its difference from traditional SE with risk management identified as a key activity. The second theme discussed the application of M&S to SoS, providing an output, which supported the identification of appropriate techniques and concluding that, the inherent complexity of a SoS required the use of M&S in order to support SoSE activities. Current risk management approaches were reviewed in theme 3 as well as the management of SoS risk. Although some specific examples of the management of SoS risk were found, no mature, general approach was identified, indicating a gap in current knowledge. However, it was noted most of these examples were underpinned by M&S approaches. It was therefore concluded a general approach SoS risk management utilising M&S methods would be of benefit. In order to fill the gap identified in current knowledge, this research proposed a new model based approach to Risk Management where risk identification was supported by a framework, which combined SoS system of interest dimensions with holistic risk types, where the resulting risks and contributing factors are captured in a causal network. Analysis of the causal network using a model technique selection tool, developed as part of this research, allowed the causal network to be simplified through the replacement of groups of elements within the network by appropriate supporting models. The Bayesian Belief Network (BBN) was identified as a suitable method to represent SoS risk. Supporting models run in Monte Carlo Simulations allowed data to be generated from which the risk BBNs could learn, thereby providing a more quantitative approach to SoS risk management. A method was developed which provided context to the BBN risk output through comparison with worst and best-case risk probabilities. The model based approach to Risk Management was applied to two very different case studies: Close Air Support mission planning and the Wheat Supply Chain, UK National Food Security risks, demonstrating its effectiveness and adaptability. The research established that the SoS SoI is essential for effective SoS risk identification and analysis of risk transfer, effective SoS modelling requires a range of techniques where suitability is determined by the problem context, the responsibility for SoS Risk Management is related to the overall SoS classification and the model based approach to SoS risk management was effective for both application case studies