Generating natural language descriptions of Z test cases

Critical software most often requires an independent validation and verification (IVV). IVV is usually performed by domain experts, who are not familiar with specific, many times formal, development technologies. In addition, model-based testing (MBT) is a promising testing technique for the verification of critical software. Test cases generated by MBT tools are logical descriptions. The problem is, then, to provide natural language (NL) descriptions of these test cases, making them accessible to domain experts. In this paper, we present ongoing research aimed at finding a suitable method for generating NL descriptions from test cases in a formal specification language. A first prototype has been developed and applied to a real-world project in the aerospace sector

Applying SMT Solvers to the Test Template Framework

The Test Template Framework (TTF) is a model-based testing method for the Z notation. In the TTF, test cases are generated from test specifications, which are predicates written in Z. In turn, the Z notation is based on first-order logic with equality and Zermelo-Fraenkel set theory. In this way, a test case is a witness satisfying a formula in that theory. Satisfiability Modulo Theory (SMT) solvers are software tools that decide the satisfiability of arbitrary formulas in a large number of built-in logical theories and their combination. In this paper, we present the first results of applying two SMT solvers, Yices and CVC3, as the engines to find test cases from TTF's test specifications. In doing so, shallow embeddings of a significant portion of the Z notation into the input languages of Yices and CVC3 are provided, given that they do not directly support Zermelo-Fraenkel set theory as defined in Z. Finally, the results of applying these embeddings to a number of test specifications of eight cases studies are analysed.Comment: In Proceedings MBT 2012, arXiv:1202.582

An Automatically Verified Prototype of the Tokeneer ID Station Specification

The Tokeneer project was an initiative set forth by the National Security Agency (NSA, USA) to be used as a demonstration that developing highly secure systems can be made by applying rigorous methods in a cost effective manner. Altran Praxis (UK) was selected by NSA to carry out the development of the Tokeneer ID Station. The company wrote a Z specification later implemented in the SPARK Ada programming language, which was verified using the SPARK Examiner toolset. In this paper, we show that the Z specification can be easily and naturally encoded in the {log} set constraint language, thus generating a functional prototype. Furthermore, we show that {log}'s automated proving capabilities can discharge all the proof obligations concerning state invariants as well as important security properties. As a consequence, the prototype can be regarded as correct with respect to the verified properties. This provides empirical evidence that Z users can use {log} to generate correct prototypes from their Z specifications. In turn, these prototypes enable or simplify some verificatio activities discussed in the paper

Combining Type Checking and Set Constraint Solving to Improve Automated Software Verification

In this paper we show how prescritive type checking and constraint solving can be combined to increase automation during software verification. We do so by defining a type system and implementing a typechecker for {log} (read `setlog'), a Constraint Logic Programming (CLP) language and satisfiability solver based on set theory. Hence, we proceed as follows: a) a type system for {log} is defined; b) the constraint solver is proved to be safe w.r.t. the type system; c) the implementation of a concrete typechecker is presented; d) the integration of type checking and set constraint solving to increase automation during software verification is discussed; and f) two industrial-strength case studies are presented where this combination is used with very good results

Comparing EvenB, {log}\{log\} and Why3 Models of Sparse Sets

Many representations for sets are available in programming languages libraries. The paper focuses on sparse sets used, e.g., in some constraint solvers for representing integer variable domains which are finite sets of values, as an alternative to range sequence. We propose in this paper verified implementations of sparse sets, in three deductive formal verification tools, namely EventB, $\{log\}$ and Why3. Furthermore, we draw some comparisons regarding specifications and proofs

Soporte herramental para el Test Template Framework

Este proyecto trata sobre la automatización y extensión del Test Template Framework (TTF). El TTF es un método de testing basado en modelos (MBT) especialmente orientado a testing de unidad a partir de especificaciones Z. Aunque el TTF es un método sólido y fue ampliamente estudiado desde su primera publicación, la comunidad de MBT fue perdiendo interés en él. Nosotros creemos que esto se debió, al menos en parte, a la falta o dificultad aparente en dotarlo de un apoyo herramental. De hecho, algunos han sugerido que la generación de casos de prueba abstractos siguiendo el TTF es una actividad manual que requiere que los usuarios manipulen predicados complejos. La intención de este proyecto es mostrar que estas conclusiones son al menos dudosas, implementando una herramienta, llamada Fastest. Fastest no solo es capaz de producir automáticamente casos de prueba abstractos sino que además podrá cubrir las necesidades de la comunidad Z en relación a herramientas de MBT.Eje: Ingeniería de SoftwareRed de Universidades con Carreras en Informática (RedUNCI

Coverage Criteria for Set-Based Specifications

Model-based testing (MBT) studies how test cases are generated from a model of the system under test (SUT). Many MBT methods rely on building an automaton from the model and then they generate test cases by covering the automaton with different path coverage criteria. However, if a model of the SUT is a logical formula over some complex mathematical theories (such as set theory) it may be more natural or intuitive to apply coverage criteria directly over the formula. On the other hand, domain partition, i.e. the partition of the input domain of model operations, is one of the main techniques in MBT. Partitioning is conducted by applying different rules or heuristics. Engineers may find it difficult to decide what, where and how these rules should be applied. In this paper we propose a set of coverage criteria based on domain partition for set-based specifications. We call them testing strategies. Testing strategies play a similar role to path- or data-based coverage criteria in structural testing. Furthermore, we show a partial order of testing strategies as is done in structural testing. We also describe an implementation of testing strategies for the Test Template Framework, which is a MBT method for the Z notation; and a scripting language that allows users to implement testing strategies

Declarative Programming with Intensional Sets in Java Using JSetL

Intensional sets are sets given by a property rather than by enumerating their elements. In previous work, we have proposed a decision procedure for a first-order logic language which provides Restricted Intensional Sets (RIS), i.e., a sub-class of intensional sets that are guaranteed to denote finite---though unbounded---sets. In this paper we show how RIS can be exploited as a convenient programming tool also in a conventional setting, namely, the imperative O-O language Java. We do this by considering a Java library, called JSetL, that integrates the notions of logical variable, (set) unification and constraints that are typical of constraint logic programming languages into the Java language. We show how JSetL is naturally extended to accommodate for RIS and RIS constraints, and how this extension can be exploited, on the one hand, to support a more declarative style of programming and, on the other hand, to effectively enhance the expressive power of the constraint language provided by the library

Automated Reasoning with Restricted Intensional Sets

Intensional sets, i.e., sets given by a property rather than by enumerating elements, are widely recognized as a key feature to describe complex problems (see, e.g., specification languages such as B and Z). Notwithstanding, very few tools exist supporting high-level automated reasoning on general formulas involving intensional sets. In this paper we present a decision procedure for a first-order logic language offering both extensional and (a restricted form of) intensional sets (RIS). RIS are introduced as first-class citizens of the language and set-theoretical operators on RIS are dealt with as constraints. Syntactic restrictions on RIS guarantee that the denoted sets are finite, though unbounded. The language of RIS, called L_RIS , is parametric with respect to any first-order theory X providing at least equality and a decision procedure for X-formulas. In particular, we consider the instance of L_RIS when X is the theory of hereditarily finite sets and binary relations. We also present a working implementation of this instance as part of the {log} tool and we show through a number of examples and two case studies that, although RIS are a subclass of general intensional sets, they are still sufficiently expressive as to encode and solve many interesting problems. Finally, an extensive empirical evaluation provides evidence that the tool can be used in practice

Proof Automation in the Theory of Finite Sets and Finite Set Relation Algebra

{log} ('setlog') is a satisfiability solver for formulas of the theory of finite sets and finite set relation algebra (FSTRA). As such, it can be used as an automated theorem prover (ATP) for this theory. {log} is able to automatically prove a number of FSTRA theorems, but not all of them. Nevertheless, we have observed that many theorems that {log} cannot automatically prove can be divided into a few subgoals automatically dischargeable by {log}. The purpose of this work is to present a prototype interactive theorem prover (ITP), called {log}-ITP, providing evidence that a proper integration of {log} into world-class ITP's can deliver a great deal of proof automation concerning FSTRA. An empirical evaluation based on 210 theorems from the TPTP and Coq's SSReflect libraries shows a noticeable reduction in the size and complexity of the proofs with respect to Coq