Exact and approximate strategies for symmetry reduction in model checking

Symmetry reduction techniques can help to combat the state space explosion problem for model checking, but are restricted by the hard problem of determining equivalence of states during search. Consequently, existing symmetry reduction packages can only exploit full symmetry between system components, as checking the equivalence of states is straightforward in this special case. We present a framework for symmetry reduction with an arbitrary group of structural symmetries. By generalising existing techniques for efficiently exploiting symmetry, and introducing an approximate strategy for use with groups for which fast, exact strategies are not available, our approach allows for significant state-space reduction with minimal time overhead. We show how computational group theoretic techniques can be used to analyse the structure of a symmetry group so that an appropriate symmetry reduction strategy can be chosen, and we describe a symmetry reduction package for the Spin model checker which interfaces with the computational algebra system Gap. Experimental results on a variety of Promela models illustrate the effectiveness of our methods

Teaching Model-Based Testing with {L}eirios {T}est {G}enerator

International audienceThis paper proposes a technique to encourage the interest of students in learning formal methods. Our course is focused on the B method, involving basic knowledge of set theory, invariance proofs, refinement techniques and so on. While lectures and tutorials cover a large range of such concepts, the practical work is focused on applying the principles of a model-based approach in the context of test generation. This paper explains the pratical outcome of the course, through the Leirios Test Generator tool, that gives an interesting and playful use of the B method, by simulating the execution of the model through animation, and by generating tests –based on the B model– that can be run on an implementation. In order to make sure that students will be interested in applying these techniques, we challenge them to play a game consisting in detecting mutants of a program with their model-based tests. The feedback from the students is very positive here, and suggests that formal methods are more likely to be understood if their interest is shown through a concrete application

Grey-box Testing and Verification of Java/JML

International audienceWe present in this paper the application of constraint solving techniques to the validation and automated test cases generation for Java programs, annotated with JML specifications. The Java/JML code is translated into a constraint representation based on a subset of the set-theory, which is well-suited for modelling object-oriented programs. Symbolic code execution techniques can then be applied to produce test cases, using classical structural test selection criteria, or to detect possible runtime errors, and non-conformances between the Java code and its embedded JML model

Génération automatique de tests à partir de patrons de propriétés

National audienceCet article propose une technique originale de génération de tests, à partir d'un modèle formel d'une application, écrit sous la forme d'une machine abstraite B, et d'une propriété définie suivant certains patrons. Les patrons sont des structures paramétrées génériques permettant d'exprimer une propriété formelle se basant sur des prédicats d'états et des événements du système. La technique présentée se base sur la production automatique de scénarios de tests qui sont issus de la propriété en elle-même et d'un besoin de test, ce dernier décrivant informellement une intention de test d'un ingénieur validation. En fonction du patron de propriété considérée et de son instanciation concrète, un ou plusieurs besoins de tests peuvent s'appliquer. Les scénarios de tests produits sont exprimés sous la forme d'expressions régulières décrivant des enchaînements d'opérations amenant à des états pertinents du système. Un mécanisme d'animation symbolique du modèle est utilisé pour déplier les scénarios et instancier les tests, notamment les paramètres des opérations, jusqu'ici abstraits. Ceci permet de produire des cas de tests abstraits prêts à être concrétisés pour le système sous test. Nous présentons l'application de ces principes à travers une étude de cas issue du milieu industriel

Praspel: Contract-Driven Testing for PHP using Realistic Domains

We present an integrated contract-based testing framework for PHP. It relies on a behavioral interface specification language called Praspel, for "PHP Realistic Annotation and Specification Language". Using Praspel developers can easily annotate their PHP scripts with formal contracts, namely class invariants, and method pre- and postconditions. These contracts describe assertions either by predicates or by assigning realistic domains to data. Realistic domains introduce types in PHP and describe complex structures frequently encountered in applications, such as email addresses or SQL queries. Realistic domains display two properties: predicability, which allows to check if a data belongs to a given realistic domain, and samplability, which allows to generate valid data. This paper introduces coverage criteria dedicated to contracts, designed to exhibit relevant behaviors of the annotated methods. Test data are then computed to satisfy these coverage criteria, by using dedicated data generators for complex realistic domains, such as arrays or strings. This framework has been implemented and disseminated within the PHP community, which gave us feedback on their usage of the tool and the relevance of this integrated process with respect to their practice of manual testing

Safety Property Driven Test Generation from {JML} Specifications

International audienceThis paper describes the automated generation of test sequences derived from a JML specification and a safety property written in an ad hoc language, named JTPL. The functional JML model is animated to build the test sequences w.r.t. the safety properties, which represent the test targets. From these properties, we derive strategies that are used to guide the symbolic animation. Moreover, additional JML annotations reinforce the oracle in order to guarantee that the safety properties are not violated during the execution of the test suite. Finally, we illustrate this approach on an industrial JavaCard case study

Actes des 14e journées sur les Approches Formelles dans l'Assistance au Développement de Logiciels

National audienceCet ouvrage présente les actes des 14èmes journées sur les Approches Formelles dans l'Assistance au Développement de Logiciels (AFADL'2015) qui se sont tenues à Bordeaux les 9 et 10 juin 2015

Automated UML models merging for web services testing

International audienceThis paper presents a method for merging UML models which takes place in a quality evaluation framework for Web Services (WS). This framework, called iTac-QoS, is an extended UDDI server (a yellow pages system dedicated to WS), using model based testing to assess quality. WS vendors have to create UML model of their product and our framework extracts tests from it. Depending on the results of the test execution, a mark is given to WS. This mark gives to the cus- tomers an idea about the quality of WS they find on our UDDI server. Up today, our framework was limited to WS which did not use other WS. This was justified by the fact that it is impossible for vendors to cre- ate a good model of a foreign product. Our method for model merging solves this problem: each vendor produces models of its own product, and we automatically merge the different models. The resulting model from this merging represents the composition of the different WS. For each type of diagram present in the models (class, instance or state- chart diagram), a method is proposed in order to produce a unique model. In addition to this, a solution is proposed to merge all OCL code in the class modeling the WS under test. Unfortunately, this pro- cess introduces inconsistencies in the resulting model, that falsify the results of the subsequent test generation phase. We thus propose to detect such inconsistencies in order to distinguish inconsistent and un- reachable test targets

Praspel: Contract-Driven Testing for PHP using Realistic Domains

We present an integrated contract-based testing framework for PHP. It relies on a behavioral interface specification language called Praspel, for "PHP Realistic Annotation and Specification Language". Using Praspel developers can easily annotate their PHP scripts with formal contracts, namely class invariants, and method pre- and postconditions. These contracts describe assertions either by predicates or by assigning realistic domains to data. Realistic domains introduce types in PHP and describe complex structures frequently encountered in applications, such as email addresses or SQL queries. Realistic domains display two properties: predicability, which allows to check if a data belongs to a given realistic domain, and samplability, which allows to generate valid data. This paper introduces coverage criteria dedicated to contracts, designed to exhibit relevant behaviors of the annotated methods. Test data are then computed to satisfy these coverage criteria, by using dedicated data generators for complex realistic domains, such as arrays or strings. This framework has been implemented and disseminated within the PHP community, which gave us feedback on their usage of the tool and the relevance of this integrated process with respect to their practice of manual testing

Test Generation and Evaluation from High-Level Properties for Common Criteria Evaluations - The TASCCC Testing Tool

International audienceIn this paper, we present a model-based testing tool resulting from a research project, named TASCCC. This tool is a complete tool chain dedicated to property-based testing in UML/OCL, that integrates various technologies inside a dedicated Eclipse plug-in. The test properties are expressed in a dedicated language based on property patterns. These properties are then used for two purposes. First, they can be employed to evaluate the relevance of a test suite according to specific coverage criteria. Second, it is possible to generate test scenarios that will illustrate or exercise the property. These test scenarios are then unfolded and animated on the Smartesting's CertifyIt model animator, that is used to filter out infeasible sequences. This tool has been used in industrial partnership, aiming at providing an assistance for Common Criteria evaluations, especially by providing test generation reports used to show the link between the test cases and the Common Criteria artefacts