664 research outputs found

    Model-based Testing

    Get PDF
    This paper provides a comprehensive introduction to a framework for formal testing using labelled transition systems, based on an extension and reformulation of the ioco theory introduced by Tretmans. We introduce the underlying models needed to specify the requirements, and formalise the notion of test cases. We discuss conformance, and in particular the conformance relation ioco. For this relation we prove several interesting properties, and we provide algorithms to derive test cases (either in batches, or on the fly)

    Talking quiescence: a rigorous theory that supports parallel composition, action hiding and determinisation

    Get PDF
    The notion of quiescence - the absence of outputs - is vital in both behavioural modelling and testing theory. Although the need for quiescence was already recognised in the 90s, it has only been treated as a second-class citizen thus far. This paper moves quiescence into the foreground and introduces the notion of quiescent transition systems (QTSs): an extension of regular input-output transition systems (IOTSs) in which quiescence is represented explicitly, via quiescent transitions. Four carefully crafted rules on the use of quiescent transitions ensure that our QTSs naturally capture quiescent behaviour. We present the building blocks for a comprehensive theory on QTSs supporting parallel composition, action hiding and determinisation. In particular, we prove that these operations preserve all the aforementioned rules. Additionally, we provide a way to transform existing IOTSs into QTSs, allowing even IOTSs as input that already contain some quiescent transitions. As an important application, we show how our QTS framework simplifies the fundamental model-based testing theory formalised around ioco.Comment: In Proceedings MBT 2012, arXiv:1202.582

    Testing Divergent Transition Systems

    Get PDF

    Tester versus Bug: A Generic Framework for Model-Based Testing via Games

    Get PDF
    We propose a generic game-based approach for test case generation. We set up a game between the tester and the System Under Test, in such a way that test cases correspond to game strategies, and the conformance relation ioco corresponds to alternating refinement. We show that different test assumptions from the literature can be easily incorporated, by slightly varying the moves in the games and their outcomes. In this way, our framework allows a wide plethora of game-theoretic techniques to be deployed for model based testing.Comment: In Proceedings GandALF 2018, arXiv:1809.0241

    Divergent Quiescent Transition Systems (extended version)

    Get PDF
    Quiescence is a fundamental concept in modelling system behaviour, as it explicitly represents the fact that no output is produced in certain states. The notion of quiescence is also essential to model-based testing: if a particular implementation under test does not provide any output, then the test evaluation algorithm must decide whether or not to allow this behaviour. To explicitly model quiescence in all its glory, we introduce Divergent Quiescent Transition Systems (DQTSs). DQTSs model quiescence using explicit delta-labelled transitions, analogous to Suspension Automata (SAs) in the well-known ioco framework. Whereas SAs have only been defined implicitly, DQTSs for the first time provide a fully-formalised framework for quiescence. Also, while SAs are restricted to convergent systems (i.e., without tau-cycles), we show how quiescence can be treated naturally using a notion of fairness, allowing systems exhibiting divergence to be modelled as well. We study compositionality under the familiar automata-theoretical operations of determinisation, parallel composition and action hiding. We provide a non-trivial algorithm for detecting divergent states, and discuss its complexity. Finally, we show how to use DQTSs in the context of model-based testing, for the first time presenting a full-fledged theory that allows ioco to be applied to divergent systems

    Using schedulers to test probabilistic distributed systems

    Get PDF
    This is the author's accepted manuscript. The final publication is available at Springer via http://dx.doi.org/10.1007/s00165-012-0244-5. Copyright © 2012, British Computer Society.Formal methods are one of the most important approaches to increasing the confidence in the correctness of software systems. A formal specification can be used as an oracle in testing since one can determine whether an observed behaviour is allowed by the specification. This is an important feature of formal testing: behaviours of the system observed in testing are compared with the specification and ideally this comparison is automated. In this paper we study a formal testing framework to deal with systems that interact with their environment at physically distributed interfaces, called ports, and where choices between different possibilities are probabilistically quantified. Building on previous work, we introduce two families of schedulers to resolve nondeterministic choices among different actions of the system. The first type of schedulers, which we call global schedulers, resolves nondeterministic choices by representing the environment as a single global scheduler. The second type, which we call localised schedulers, models the environment as a set of schedulers with there being one scheduler for each port. We formally define the application of schedulers to systems and provide and study different implementation relations in this setting

    Test Generation with Inputs, Outputs, and Quiescence

    Get PDF

    Software doping – Theory and detection

    Get PDF
    Software is doped if it contains a hidden functionality that is intentionally included by the manufacturer and is not in the interest of the user or society. This thesis complements this informal definition by a set of formal cleanness definitions that characterise the absence of software doping. These definitions reflect common expectations on clean software behaviour and are applicable to many types of software, from printers to cars to discriminatory AI systems. We use these definitions to propose white-box and black-box analysis techniques to detect software doping. In particular, we present a provably correct, model-based testing algorithm that is intertwined with a probabilistic-falsification-based test input selection technique. We identify and explain how to overcome the challenges that are specific to real-world software doping tests and analyses. The most prominent example of software doping in recent years is the Diesel Emissions Scandal. We demonstrate the strength of our cleanness definitions and analysis techniques by applying them to emission cleaning systems of diesel cars. All our car related research is unified in a Car Data Platform. The mobile app LolaDrives is one building block of this platform; it supports conducting real-driving emissions tests and provides feedback to the user in how far a trip satisfies driving conditions that are defined by official regulations.Software ist gedopt wenn sie eine versteckte Funktionalität enthält, die vom Hersteller beabsichtigt ist und deren Existenz nicht im Interesse des Benutzers oder der Gesellschaft ist. Die vorliegende Arbeit ergänzt diese nicht formale Definition um eine Menge von Cleanness-Definitionen, die die Abwesenheit von Software Doping charakterisieren. Diese Definitionen spiegeln allgemeine Erwartungen an "sauberes" Softwareverhalten wider und sie sind auf viele Arten von Software anwendbar, vom Drucker über Autos bis hin zu diskriminierenden KI-Systemen. Wir verwenden diese Definitionen um sowohl white-box, als auch black-box Analyseverfahren zur Verfügung zu stellen, die in der Lage sind Software Doping zu erkennen. Insbesondere stellen wir einen korrekt bewiesenen Algorithmus für modellbasierte Tests vor, der eng verflochten ist mit einer Test-Input-Generierung basierend auf einer Probabilistic-Falsification-Technik. Wir identifizieren Hürden hinsichtlich Software-Doping-Tests in der echten Welt und erklären, wie diese bewältigt werden können. Das bekannteste Beispiel für Software Doping in den letzten Jahren ist der Diesel-Abgasskandal. Wir demonstrieren die Fähigkeiten unserer Cleanness-Definitionen und Analyseverfahren, indem wir diese auf Abgasreinigungssystem von Dieselfahrzeugen anwenden. Unsere gesamte auto-basierte Forschung kommt in der Car Data Platform zusammen. Die mobile App LolaDrives ist eine Kernkomponente dieser Plattform; sie unterstützt bei der Durchführung von Abgasmessungen auf der Straße und gibt dem Fahrer Feedback inwiefern eine Fahrt den offiziellen Anforderungen der EU-Norm der Real-Driving Emissions entspricht
    corecore