Using formal methods to support testing

Formal methods and testing are two important approaches that assist in the development of high quality software. While traditionally these approaches have been seen as rivals, in recent years a new consensus has developed in which they are seen as complementary. This article reviews the state of the art regarding ways in which the presence of a formal specification can be used to assist testing

Validating specifications of dynamic systems using automated reasoning techniques

In this paper, we propose a new approach to validating formal specifications of observable behavior of discrete dynamic systems. By observable behavior we mean system behavior as observed by users or other systems in the environment of the system. Validation of a formal specification of an informal domain tries to answer the question whether the specification actually describes the intended domain. This differs from the verification problem, which deals with the correspondence between formal objects, e.g. between a formal specification of a system and an implementation of it. We consider formal specifications of object-oriented dynamic systems that are subject to static and dynamic integrity constraints. To validate that such a specification expresses the intended behavior, we propose to use a tool that can answer reachability queries. In a reachability query we ask whether the system can evolve from one state into another without violating the integrity constraints. If the query is answered positively, the system should exhibit an example path between the states; if the answer is negative, the system should explain why this is so. An example path produced by the tool can be used to produce scenarios for presentations of system behavior, but can also be used as a basis for acceptance testing. In this paper, we discuss the use of planning and theoremproving techniques to answer such queries, and illustrate the use of reachability queries in the context of information system development

Testing a deterministic implementation against a non-controllable non-deterministic stream X-machine

A stream X-machine is a type of extended finite state machine with an associated development approach that consists of building a system from a set of trusted components. One of the great benefits of using stream X-machines for the purpose of specification is the existence of test generation techniques that produce test suites that are guaranteed to determine correctness as long as certain well-defined conditions hold. One of the conditions that is traditionally assumed to hold is controllability: this insists that all paths through the stream X-machine are feasible. This restrictive condition has recently been weakened for testing from a deterministic stream X-machine. This paper shows how controllability can be replaced by a weaker condition when testing a deterministic system against a non-deterministic stream X-machine. This paper therefore develops a new, more general, test generation algorithm for testing from a non-deterministic stream X-machine

Development of a framework for automated systematic testing of safety-critical embedded systems

“This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder." “Copyright IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.”In this paper we introduce the development of a framework for testing safety-critical embedded systems based on the concepts of model-based testing. In model-based testing the test cases are derived from a model of the system under test. In our approach the model is an automaton model that is automatically extracted from the C-source code of the system under test. Beside random test data generation the test case generation uses formal methods, in detail model checking techniques. To find appropriate test cases we use the requirements defined in the system specification. To cover further execution paths we developed an additional, to our best knowledge, novel method based on special structural coverage criteria. We present preliminary results on the model extraction using a concrete industrial case study from the automotive domain

Leveraging Semantic Web Service Descriptions for Validation by Automated Functional Testing

Recent years have seen the utilisation of Semantic Web Service descriptions for automating a wide range of service-related activities, with a primary focus on service discovery, composition, execution and mediation. An important area which so far has received less attention is service validation, whereby advertised services are proven to conform to required behavioural specifications. This paper proposes a method for validation of service-oriented systems through automated functional testing. The method leverages ontology-based and rule-based descriptions of service inputs, outputs, preconditions and effects (IOPE) for constructing a stateful EFSM specification. The specification is subsequently utilised for functional testing and validation using the proven Stream X-machine (SXM) testing methodology. Complete functional test sets are generated automatically at an abstract level and are then applied to concrete Web services, using test drivers created from the Web service descriptions. The testing method comes with completeness guarantees and provides a strong method for validating the behaviour of Web services