Formal mutation testing for Circus

International audienceContext: The demand from industry for more dependable and scalable test-development mechanisms has fostered the use of formal models to guide the generation of tests. Despite many advancements having been obtained with state-based models, such as Finite State Machines (FSMs) and Input/Output Transition Systems (IOTSs), more advanced formalisms are required to specify large, state-rich, concurrent systems. Circus, a state-rich process algebra combining Z, CSP and a refinement calculus, is suitable for this; however, deriving tests from such models is accordingly more challenging. Recently, a testing theory has been stated for Circus, allowing the verification of process refinement based on exhaustive test sets. Objective: We investigate fault-based testing for refinement from Circus specifications using mutation. We seek the benefits of such techniques in test-set quality assertion and fault-based test-case selection. We target results relevant not only for Circus, but to any process algebra for refinement that combines CSP with a data language. Method: We present a formal definition for fault-based test sets, extending the Circus testing theory, and an extensive study of mutation operators for Circus. Using these results, we propose an approach to generate tests to kill mutants. Finally, we explain how prototype tool support can be obtained with the implementation of a mutant generator, a translator from Circus to CSP, and a refinement checker for CSP, and with

Formal Methods and Testing: Hypotheses, and Correctness Approximations

mcg at lri.fr Abstract. It has been recognised for a while that formal specifications can bring much to software testing. Numerous methods have been proposed for the derivation of test cases from various kinds of formal specifications, their submission, and verdict. All these methods rely upon some hypotheses on the system under test that formalise the gap between the success of a test campaign and the correctness of the system under test.

Testing Processes from Formal Specifications with Inputs, Outputs and Data Types

Deriving test cases from formal specifications of communicating processes has been studied for a while. Several methods have been proposed for specifications based on FSM (Finite State Machines), LTS (Labelled Transition Systems), IOTS (Input Output Transition Systems), etc. However, most approaches are limited to a finite set of actions, excluding the possibility of communicating typed values between processes. This article presents a test derivation and selection method based on a model of communicating processes with inputs, outputs and data types, which is closer to actual implementations of communication protocols

Contributions au test de logiciel basé sur des spécifications formelles

Dans cette thèse, nous nous intéressons au test de conformité qui vise à vérifier que l'implémentation d'un système satisfait à sa spécification selon une relation de conformité entre le modèle de la spécification et celui de l'implémentation. Nous avons défini le modèle RIOLTS signifiant Restrictive Input/Output Labeled Transition System et la relation de conformité rioco pour Restrictive Input/Output COnformance dont la spécificité est de permettre la description de systèmes dont certaines entrées sont interdites dans certains états. Dans la seconde partie de cette thèse nous nous intéressons a la génération de tests à partir de modèles infinis manipulant des types de données complexes, encore appelés modèles symboliques. La présence de types de données complexes rend plus difficile le problème de la sélection des tests: en effet, à la potentielle infinité des comportements d'un système s'ajoute celle des valeurs que peuvent prendre les symboles apparaissant dans les actions symboliques. De plus, des gardes peuvent conditionner les transitions ce qui nous confronte au fait que certaines traces symboliques ne sont pas faisables. Nous proposons ici une stratégie de sélection applicable sur tout modèle basé sur les systèmes de transitions symboliques. Nous utilisons un solveur de contraintes pour déterminer les chemins de l'automate de la spécification qui sont faisables. Les difficultés rencontrées pour obtenir de tels chemins de manière rapide nous ont conduit à optimiser l'utilisation du solveur. Ainsi plusieurs méthodes ont été proposées et expérimentées afin d'améliorer les temps de résolution et d'en assurer la terminaison.In this thesis, we get interested in conformance testing whose goal is to check that the implementation of a system conforms to its specification w.r.t. a conformance relation between the model of the specification and the one of the implementation. We have defined the RIOLTS model standing for Restrictive Input/Output Labeled Transition System and the conformance relation rioco standing for Restrictive Input/Output COnformance. The particularity of this model is that it makes it possible to describe systems in which some inputs are forbidden in some states. In the second part of this thesis, we have worked on test generation and selection from infinite models using complex data types. These models are called symbolic models. Complex data types makes it harder the test selection problem: we must not only deal with the possible unlimited number of behaviors of systems but also with the unlimited number of values possible for the symbols appearing in symbolic actions. Moreover as guards may condition transitions, some symbolic traces are unfeasible. We propose a selection strategy that can be applied on any model based on symbolic transition systems. We use a constraint solver to determine feasible paths of the specification automata. Such paths are difficult to calculate, and to get them faster, we have had to optimize the use of the solver. Though, we have proposed and carried out experiments on many methods to reduce solving time and ensure the termination of calculations.ORSAY-PARIS 11-BU Sciences (914712101) / SudocSudocFranceF

Unfolding-based Test Selection for Concurrent Conformance

Abstract. Model-based testing has mainly focused on models where currency is interpreted as interleaving (like the ioco theory for labeled transition systems), which may be too coarse when one wants concurrency to be preserved in the implementation. In order to test such concurrent systems, we choose to use Petri nets as specifications and define a concurrent conformance relation named coioco. We propose a test generation algorithm based on Petri net unfolding able to build a complete test suite w.r.t our co-ioco conformance relation. In addition we propose a coverage criterion based on a dedicated notion of complete prefixes that selects a manageable test suite. Model-based Testing. The aim of testing is to execute a software system, the implementation, on a set of input data selected so as to find discrepancies between actual behavior and intended behavior described by the specification. The testing process is usually decomposed into three phases: selection of relevant input data, called a test suite, among the possible inputs of the system; submission of this test suite to the implementation, its execution; and decision of the success or the failure of the test suite submission, know

Using formal specifications to support testing

Formal methods and testing are two important approaches that assist in the development of high-quality software. While traditionally these approaches have been seen as rivals, in recent years a new consensus has developed in which they are seen as complementary. This article reviews the state of the art regarding ways in which the presence of a formal specification can be used to assist testing