Bug Hunting with False Negatives Revisited

Safe data abstractions are widely used for verification purposes. Positive verification results can be transferred from the abstract to the concrete system. When a property is violated in the abstract system, one still has to check whether a concrete violation scenario exists. However, even when the violation scenario is not reproducible in the concrete system (a false negative), it may still contain information on possible sources of bugs. Here, we propose a bug hunting framework based on abstract violation scenarios. We first extract a violation pattern from one abstract violation scenario. The violation pattern represents multiple abstract violation scenarios, increasing the chance that a corresponding concrete violation exists. Then, we look for a concrete violation that corresponds to the violation pattern by using constraint solving techniques. Finally, we define the class of counterexamples that we can handle and argue correctness of the proposed framework. Our method combines two formal techniques, model checking and constraint solving. Through an analysis of contracting and precise abstractions, we are able to integrate overapproximation by abstraction with concrete counterexample generation

Using fairness to make abstractions work

Abstractions often introduce infinite traces which have no corresponding traces at the concrete level and can lead to the failure of the verification. Refinement does not always help to eliminate those traces. In this paper, we consider a timer abstraction that introduces a cyclic behaviour on abstract timers and we show how one can exclude cycles by imposing a strong fairness constraint on the abstract model. By employing the fact that the loop on the abstract timer is a self-loop, we render the strong fairness constraint into a weak fairness constraint and embed it into the verification algorithm. We implemented the algorithm in the DTSpin model checker and showed its efficiency on case studies. The same approach can be used for other data abstractions that introduce self-loops

Abstraction and flow analysis for model checking open asynchronous systems

Formal methods, especially model checking, are an indispensable part of the software engineering process. With large software systems currently beyond the range of fully automatic verification, however, a combination of decomposition and abstraction techniques is needed. To model check components of a system, a standard approach is to close the component with an abstraction of its environment. To make it useful in practice, the closing of the component should be automatic, both for data and for control abstraction. Specifically for model checking asynchronous open systems, external input queues should be removed, as they are a potential source of a combinatorial state explosion. In this paper, we close a component synchronously by embedding the external environment directly into the system to avoid the external queues, while for the data, we use a two-valued abstraction, namely data influenced from the outside or not. This gives a more precise analysis than the one investigated in [7]. To further combat the state explosion problem, we combine this data abstraction with a static analysis to remove superfluous code fragments. The static analysis we use is reminiscent to the one presented in [7], but we use a combination of a may and a must-analysis instead of a may-analysis

Bug Hunting with False Negatives Revisited

Safe data abstractions are widely used for verification purposes. Positive verification results can be transferred from the abstract to the concrete system. When a property is violated in the abstract system, one still has to check whether a concrete violation scenario exists. However, even when the violation scenario is not reproducible in the concrete system (a false negative), it may still contain information on possible sources of bugs. Here, we propose a bug hunting framework based on abstract violation scenarios. We first extract a violation pattern from one abstract violation scenario. The violation pattern represents multiple abstract violation scenarios, increasing the chance that a corresponding concrete violation exists. Then, we look for a concrete violation that corresponds to the violation pattern by using constraint solving techniques. Finally, we define the class of counterexamples that we can handle and argue correctness of the proposed framework. Our method combines two formal techniques, model checking and constraint solving. Through an analysis of contracting and precise abstractions, we are able to integrate overapproximation by abstraction with concrete counterexample generation

Simulated time for testing railway interlockings with TTCN-3

In this report, we first give an overview of software systems based on Vital Processor Interlocking (VPI). Interlockings guarantee safety of railway control systems, so testing these software systems is a key issue. We show why testing such systems with real time and scaled time is inefficient. We also provide a time semantics for simulated time that is more suitable for testing VPI's software. We provide a solution that allows simulated time for TTCN-3 test systems. TTCN-3 is a standard language for specifying and executing test suites. In the context of the TT-MEDAL project, TTCN-3 is applied to various domains, in particular to testing railway and automotive systems. TTCN-3 supports real-time and scaled-time testing but not simulated-time testing. The solution is based on a distributed termination detection algorithm that we extend to provide the main ingredients of simulated time: idleness detection and correct time progress. We implemented our solution as a TTCN-3 module and several Java classes that can be reused for testing other systems that have characteristics similar to those of VPI

Towards automatic generation of parameterized test cases from abstractions

Model-based tools for automatic test generation usually can handle systems of a rather limited size. Therefore, they cannot be applied directly to systems of real industrial size. Here, we propose an approach to test generation combining enumerative data abstraction, test generation methods and constraint solving. The approach allows applying enumerative test generation tools like TGV to large and infinite systems. Given such a system, abstractions allow to derive a finite abstract system suitable for automatic test generation with enumerative tools. Abstract test cases need to be parameterized with actual test data, in order to execute them. For data selection, we make use of constraint solving techniques. Test case execution will later be done by TTCN-

Bug hunting with false negatives

Safe data abstractions are widely used for verification purposes. Positive verification results can be transferred to the concrete system. When a property is violated in the abstract system, one still has to check whether a concrete violation exists. However, even when the violation scenario is not reproducible in the concrete system (a false negative), it may still contain information on possible sources of bugs. Here we propose a bug hunting framework based on abstract violation scenarios. We first extract a violation pattern from an abstract violation scenario. The violation pattern represents multiple violation scenarios, increasing the chance that a corresponding concrete violation exists. Then we look for a concrete violation that corresponds to the violation pattern by using constraint solving techniques

Synchronous Closing and Flow Analysis for Model Checking Timed Systems

Abstract. Formal methods, in particular model checking, are increas-ingly accepted as integral part of system development. With large soft-ware systems beyond the range of fully automatic verification, however, a combination of decomposition and abstraction techniques is needed. To model check components of a system, a standard approach is to close the component with an abstraction of its environment, as standard model checkers often do not handle open reactive systems directly. To make it useful in practice, the closing of the component should be automatic, both for data and for control abstraction. Specifically for model checking asynchronous open systems, external input queues should be removed, as they are a potential source of a combinatorial state explosion. In this paper we investigate a class of environmental processes for which the asynchronous communication scheme can safely be replaced by a synchronous one. Such a replacement is possible only if the environment is constructed under rather a severe restriction on the behavior, which can be partially softened via the use of a discrete-time semantics. We employ data-flow analysis to detect instances of variables and timers influenced by the data passing between the system and the environment

A transformation of SDL specifications : a step towards the verification

Industrial-size specifications/models (whose state space is often infinite) can not be model checked in a direct way— a verification model of a system is model checked instead. Program transformation is a way to build a finite-state verification model that can be submitted to a model checker. Abstraction is another technique that can be used for the same purpose. This paper presents a transformation of SDL timers aimed at the reduction of the infinite domain of timer values to a finite one with preserving the behaviour of a system. A timer abstraction is proposed to further reduce the state space. We discuss the ideas behind these transformations and argue their correctness