Enablers and Impediments for Collaborative Research in Software Testing: An Empirical Exploration

When it comes to industrial organizations, current collaboration efforts in software engineering research are very often kept in-house, depriving these organizations off the skills necessary to build independent collaborative research. The current trend, towards empirical software engineering research, requires certain standards to be established which would guide these collaborative efforts in creating a strong partnership that promotes independent, evidence-based, software engineering research. This paper examines key enabling factors for an efficient and effective industry-academia collaboration in the software testing domain. A major finding of the research was that while technology is a strong enabler to better collaboration, it must be complemented with industrial openness to disclose research results and the use of a dedicated tooling platform. We use as an example an automated test generation approach that has been developed in the last two years collaboratively with Bombardier Transportation AB in Sweden

A Case Study in Formal System Engineering with SysML

International audienceIn the development of complex critical systems, an important source of errors is the misinterpretation of system requirements allocated to the software, due to inadequate communication between system engineering teams and software teams. In response, organizations that develop such systems are searching for solutions allowing formal system engineering and system to software bridging, based on standard languages like SysML. As part of this effort, we have defined a formal profile for SysML (OMEGA SysML) and we have built a simulation and verification toolbox for this profile (IFx). This paper reports on the experience of modelling and validating an industry-grade system, the Solar Generation System (SGS) of the Automated Transfer Vehicle (ATV) built by Astrium, using IFx-OMEGA. The experience reveals what can currently be expected from such an approach and what are the weak points that should be addressed by future research and development

Model-Based Verification for SIMULINK Design

Testing a Model-Based design is the only way to determine the correctness of the designed model but not enough to conclude that the design is error free. Verification exposes all the design errors and describes the functionality of the system. Assertion based verification helps to determine whether the model obey the actual design requirements. This thesis work is mainly based on verification of a Water Tank control system modeling using SIMULINK model

METAMOC: Modular Execution Time Analysis using Model Checking

Safe and tight worst-case execution times (WCETs) are important when scheduling hard real-time systems. This paper presents METAMOC, a path-based, modular method, based on model checking and static analysis, that determines safe and tight WCETs for programs running on platforms fea-turing caching and pipelining. The method works by constructing a UPPAAL model of the program being analysed and annotating the model with information from an inter-procedural value analysis. The program model is then combined with a model of the hardware platform, and model checked for the WCET. Through support for the platforms ARM7, ARM9 and ATMEL AVR 8-bit the modularity and retargetability of the method is demonstrated, as only the pipeline needs to be remodelled. Mod-elling the hardware is performed in a state-of-the-art graphical modeling environment. Experiments on the Mälardalen WCET benchmark programs show that taking caching into account yields much tighter WCETs, and that METAMOC is a fast and versatile approach for WCET analysis. 1

Developing a distributed electronic health-record store for India

The DIGHT project is addressing the problem of building a scalable and highly available information store for the Electronic Health Records (EHRs) of the over one billion citizens of India

Simulation and Formal Verification for Improving Safety of PLC Programs

The use of analysis techniques for improving quality of software for industrial controllers is widely used. Mainly Simulation and Formal Verification can be used as complementary techniques improving dependability of mechatronic systems behavior. In this paper there are used Simulation and Formal Verification for guaranteeing safe software for Programmable Logic Controllers, mainly related with using Function blocks of IEC 61131-3 standard. For studying, simulating and verifying behavior of those blocks are used timed automata, as modeling formalism, and UPPAAL, as tool for simulation and Formal Verification purposes

Proceedings of the Second NASA Formal Methods Symposium

This publication contains the proceedings of the Second NASA Formal Methods Symposium sponsored by the National Aeronautics and Space Administration and held in Washington D.C. April 13-15, 2010. Topics covered include: Decision Engines for Software Analysis using Satisfiability Modulo Theories Solvers; Verification and Validation of Flight-Critical Systems; Formal Methods at Intel -- An Overview; Automatic Review of Abstract State Machines by Meta Property Verification; Hardware-independent Proofs of Numerical Programs; Slice-based Formal Specification Measures -- Mapping Coupling and Cohesion Measures to Formal Z; How Formal Methods Impels Discovery: A Short History of an Air Traffic Management Project; A Machine-Checked Proof of A State-Space Construction Algorithm; Automated Assume-Guarantee Reasoning for Omega-Regular Systems and Specifications; Modeling Regular Replacement for String Constraint Solving; Using Integer Clocks to Verify the Timing-Sync Sensor Network Protocol; Can Regulatory Bodies Expect Efficient Help from Formal Methods?; Synthesis of Greedy Algorithms Using Dominance Relations; A New Method for Incremental Testing of Finite State Machines; Verification of Faulty Message Passing Systems with Continuous State Space in PVS; Phase Two Feasibility Study for Software Safety Requirements Analysis Using Model Checking; A Prototype Embedding of Bluespec System Verilog in the PVS Theorem Prover; SimCheck: An Expressive Type System for Simulink; Coverage Metrics for Requirements-Based Testing: Evaluation of Effectiveness; Software Model Checking of ARINC-653 Flight Code with MCP; Evaluation of a Guideline by Formal Modelling of Cruise Control System in Event-B; Formal Verification of Large Software Systems; Symbolic Computation of Strongly Connected Components Using Saturation; Towards the Formal Verification of a Distributed Real-Time Automotive System; Slicing AADL Specifications for Model Checking; Model Checking with Edge-valued Decision Diagrams; and Data-flow based Model Analysis

Automated requirements-driven testing of embedded systems based on use case specifications and timed automata

The complexity of embedded software in safety-critical domains, such as automotive and avionics, has significantly increased over the years. For most embedded systems, standards require system testing to explicitly demonstrate that the software meets its functional and safety requirements. In these domains, system test cases are often manually derived from functional requirements in natural language plus other design artefacts, like UML statecharts. The definition of system test cases is therefore time-consuming and error-prone, especially given the quickly rising complexity of embedded systems. The benefits of automatic test generation are widely acknowledged today but existing approaches often require behavioural models that tend to be complex and expensive to produce, and are thus often not part of development practice. The work proposed in this dissertation focusses on the automated generation of test cases for testing the compliance between software and its functional and timing requirements. This dissertation is inspired by contexts where functional and timing requirements are expressed by means of use case specifications and timing automata, respectively. This is the development context of our industrial partner, IEE, an automotive company located in Luxembourg, who provided the case study used to validate the approach and tool described in this dissertation. This dissertation presents five main contributions: (1) A set of guidelines for the definition of functional and timing requirements to enable the automated generation of system test cases. (2) A technique for the automated generation of functional test cases from requirements elicited in the form of use case specifications following a prescribed template and natural-language restrictions. (3) A technique that reuses the automatically generated functional test cases to generate timeliness test cases from minimal models of the timing requirements of the system. (4) A technique for the automated generation of oracles for non-deterministic systems whose specifications are expressed by means of timed automata. In the context of this dissertation, automated oracles for non-deterministic systems are necessary to evaluate the results of the generated timeliness test cases. (5) The evaluation of the applicability and effectiveness of the proposed guidelines and techniques on an industrial case study, a representative automotive embedded system developed by IEE