Complete Model-Based Testing Applied to the Railway Domain

Testing is the most important verification technique to assert the correctness of an embedded system. Model-based testing (MBT) is a popular approach that generates test cases from models automatically. For the verification of safety-critical systems, complete MBT strategies are most promising. Complete testing strategies can guarantee that all errors of a certain kind are revealed by the generated test suite, given that the system-under-test fulfils several hypotheses. This work presents a complete testing strategy which is based on equivalence class abstraction. Using this approach, reactive systems, with a potentially infinite input domain but finitely many internal states, can be abstracted to finite-state machines. This allows for the generation of finite test suites providing completeness. However, for a system-under-test, it is hard to prove the validity of the hypotheses which justify the completeness of the applied testing strategy. Therefore, we experimentally evaluate the fault-detection capabilities of our equivalence class testing strategy in this work. We use a novel mutation-analysis strategy which introduces artificial errors to a SystemC model to mimic typical HW/SW integration errors. We provide experimental results that show the adequacy of our approach considering case studies from the railway domain (i.e., a speed-monitoring function and an interlocking-system controller) and from the automotive domain (i.e., an airbag controller). Furthermore, we present extensions to the equivalence class testing strategy. We show that a combination with randomisation and boundary-value selection is able to significantly increase the probability to detect HW/SW integration errors

IEEE/NASA Workshop on Leveraging Applications of Formal Methods, Verification, and Validation

This volume contains the Preliminary Proceedings of the 2005 IEEE ISoLA Workshop on Leveraging Applications of Formal Methods, Verification, and Validation, with a special track on the theme of Formal Methods in Human and Robotic Space Exploration. The workshop was held on 23-24 September 2005 at the Loyola College Graduate Center, Columbia, MD, USA. The idea behind the Workshop arose from the experience and feedback of ISoLA 2004, the 1st International Symposium on Leveraging Applications of Formal Methods held in Paphos (Cyprus) last October-November. ISoLA 2004 served the need of providing a forum for developers, users, and researchers to discuss issues related to the adoption and use of rigorous tools and methods for the specification, analysis, verification, certification, construction, test, and maintenance of systems from the point of view of their different application domains

Integrating Abstraction Techniques for Formal Verification of Analog Designs

The verification of analog designs is a challenging and exhaustive task that requires deep understanding of physical behaviours. In this paper, we propose a qualitative based predicate abstraction method for the verification of a class of non-linear analog circuits. In the proposed method, system equations are automatically extracted from a circuit diagram by means of a bond graph. Verification is applied based on combining techniques from constraint solving and computer algebra along with symbolic model checking. Our methodology has the advantage of avoiding exhaustive simulation normally encountered in the verification of analog designs. To this end, we have used Dymola, Hsolver, SMV and Mathematica to implement the verification flow. We illustrate the methodology on several analog examples including Colpitts and tunnel diode oscillators

Finding False Assurance in Formal Verification of Software Systems

Formal verification plays a crucial role in enhancing the reliability of computing systems by mathematically checking the correctness of a program. Although recent years have witnessed lots of research and applications that optimize the formal verification process, the issue of false assurance persists in certain stages of the formal verification pipeline. The false assurance problem is critical as it can easily undermine months if not years of verification efforts. In this thesis, we first generalized the formal verification process. We then identified and analyzed specific stages susceptible to false assurance. Subsequently, a systematization of knowledge pertaining to the false assurance issues observed at these stages is provided, accompanied by a discussion on the existing defense mechanisms that are currently available. Specifically, we focused on the problem of formal specification incompleteness. We presented FAST in this thesis, which is short for underlineFuzzing-underlineAssisted underlineSpecification underlineTesting. FAST examines the spec for incompleteness issues in an automated way: it first locates spec gaps via mutation testing, i.e., by checking whether a code variant conforms to the original spec. If so, FAST further leverages the test suites to infer whether the gap is introduced by intention or by mistake. Depending on the codebase size, FAST may choose to generate code variants in either an enumerative or evolutionary way. FAST is applied to two open-source codebases that feature formal verification and helps to confirm 13 and 21 blind spots in their spec respectively. This highlights the prevalence of spec incompleteness in real-world applications

A model-based approach for the specification and refinement of streaming applications

Embedded systems can be found in a wide range of applications. Depending on the application, embedded systems must meet a wide range of constraints. Thus, designing and programming embedded systems is a challenging task. Here, model-based design flows can be a solution. This thesis proposes novel approaches for the specification and refinement of streaming applications. To this end, it focuses on dataflow models. As key result, the proposed dataflow model provides for a seamless model-based design flow from system level to the instruction/logic level for a wide range of streaming applications

Yippelia: Triggering Deep Property Violations in Hardware Designs through Symbolic Execution

We in Yippelia attempt to automatically identify deep bugs in hardware designs by symbolically exploring hardware designs for one clock cycle and then stitching the generated simple paths to form a multi-cycle path from the reset state to the buggy state. Compared to a state-of-the-art symbolic execution engine, Yippelia has an average speedup of at least four orders of magnitude on finding deep bugs on the up-down counter hardware design.Bachelor of Scienc

Automated Validation of State-Based Client-Centric Isolation with TLA ⁺

Clear consistency guarantees on data are paramount for the design and implementation of distributed systems. When implementing distributed applications, developers require approaches to verify the data consistency guarantees of an implementation choice. Crooks et al. define a state-based and client-centric model of database isolation. This paper formalizes this state-based model in, reproduces their examples and shows how to model check runtime traces and algorithms with this formalization. The formalized model in enables semi-automatic model checking for different implementation alternatives for transactional operations and allows checking of conformance to isolation levels. We reproduce examples of the original paper and confirm the isolation guarantees of the combination of the well-known 2-phase locking and 2-phase commit algorithms. Using model checking this formalization can also help finding bugs in incorrect specifications. This improves feasibility of automated checking of isolation guarantees in synthesized synchronization implementations and it provides an environment for experimenting with new designs.</p