Generating feasible transition paths for testing from an extended finite state machine (EFSM)

The problem of testing from an extended finite state machine (EFSM) can be expressed in terms of finding suitable paths through the EFSM and then deriving test data to follow the paths. A chosen path may be infeasible and so it is desirable to have methods that can direct the search for appropriate paths through the EFSM towards those that are likely to be feasible. However, generating feasible transition paths (FTPs) for model based testing is a challenging task and is an open research problem. This paper introduces a novel fitness metric that analyzes data flow dependence among the actions and conditions of the transitions in order to estimate the feasibility of a transition path. The proposed fitness metric is evaluated by being used in a genetic algorithm to guide the search for FTPs

Automatic test selection based on CEFSM specifications

Mutation analysis is a fault based testing method used initially for code based software testing. In this paper, this method is applied to formal specifications and used for automatic conformance test selection. This paper defines formally a set of mutation operators for CEFSM (Communicating Extended Finite State Machine) systems to enable the automated creation of mutant specifications. Mutants of a specification are used as selection criteria to pick out adequate test cases. Two different algorithms are proposed for the generation and selection of efficient test suites. Additionally, the operators and algorithms provide the basis of an automatic tool developed at the Budapest University of Technology and Economics. We present the results of an empirical study on the well-known INRES protocol acquired using the tool

Automatic generation of test sequences form EFSM models using evolutionary algorithms

Automated test data generation through evolutionary testing (ET) is a topic of interest to the software engineering community. While there are many ET-based techniques for automatically generating test data from code, the problem of generating test data from an extended finite state machine (EFSMs) is more complex and has received little attention. In this paper, we introduce a novel approach that addresses the problem of generating input test sequences that trigger given feasible paths in an EFSM model by employing an ET-based technique. The proposed approach expresses the problem as a search for input parameters to be applied to a set of functions to be called sequentially. In order to apply ET-based technique, a new fitness function is introduced to cope with the case when a test target involves calls to a set of transitions sequentially. We evaluate our approach empirically using five sets of randomly generated paths through two EFSM case studies: INRES and class 2 transport protocols. In the experiments, we apply two search techniques: a random and an ET-based which utilizes our new fitness function. Experimental results show that the proposed approach produces input test sequences that trigger all the feasible paths used with a success rate of 100%, however, the random technique failed in most cases with a success rate of 20.8%

An integrated search-based approach for automatic testing from extended finite state machine (EFSM) models

This is the post-print version of the Article - Copyright @ 2011 ElsevierThe extended finite state machine (EFSM) is a modelling approach that has been used to represent a wide range of systems. When testing from an EFSM, it is normal to use a test criterion such as transition coverage. Such test criteria are often expressed in terms of transition paths (TPs) through an EFSM. Despite the popularity of EFSMs, testing from an EFSM is difficult for two main reasons: path feasibility and path input sequence generation. The path feasibility problem concerns generating paths that are feasible whereas the path input sequence generation problem is to find an input sequence that can traverse a feasible path. While search-based approaches have been used in test automation, there has been relatively little work that uses them when testing from an EFSM. In this paper, we propose an integrated search-based approach to automate testing from an EFSM. The approach has two phases, the aim of the first phase being to produce a feasible TP (FTP) while the second phase searches for an input sequence to trigger this TP. The first phase uses a Genetic Algorithm whose fitness function is a TP feasibility metric based on dataflow dependence. The second phase uses a Genetic Algorithm whose fitness function is based on a combination of a branch distance function and approach level. Experimental results using five EFSMs found the first phase to be effective in generating FTPs with a success rate of approximately 96.6%. Furthermore, the proposed input sequence generator could trigger all the generated feasible TPs (success rate = 100%). The results derived from the experiment demonstrate that the proposed approach is effective in automating testing from an EFSM

A search-based technique for testing from extended finite state machine model

Extended finite state machines (EFSMs), and languages such as state-charts that are similar to EFSMs, are widely used to model state-based systems. When testing from an EFSM M it is common to aim to produce a set of test sequences (input sequences) that satisfies a test criterion that relates to the transition paths (TPs) of M that are executed by the test sequences. For example, we might require that the set of TPs triggered includes all of the transitions of M. One approach to generating such a set of test sequences is to split the problem into two stages: choosing a set of TPs that achieves the test criterion and then producing test sequences to trigger these TPs. However, the EFSM may contain infeasible TPs and the problem of generating a test sequence to trigger a given feasible TP (FTP) is generally uncomputable. In this paper we present a search-based approach that uses two techniques: (1) A TP fitness metric based on our previous work that estimates the feasibility of a given transition path; and (2) A fitness function to guide the search for a test sequence to trigger a given FTP. We evaluated our approach on five EFSMs: A simple in-flight safety system; a class II transport protocol; a lift system; an ATM; and the Inres initiator. In the experiments the proposed approach successfully tested approximately 96.75 % of the transitions and the proposed test sequence generation technique triggered all of the generated FTPs

Mutation Testing Applied to Validate SDL Specifications

Abstract. Mutation Testing is an error-based criterion that provides mechanisms to evaluate the quality of a test set and/or to generate test sets. This criterion, originally proposed to program testing, has also been applied to specification testing. In this paper, we propose the application of Mutation Testing for testing SDL specifications. We define a mutant operator set for SDL that intends to model errors related to the behavioral aspect of the processes, the communication among processes, the structure of the specification and some intrinsic characteristics of SDL. A testing strategy to apply the mutant operators to test SDL specifications is proposed. We illustrate our approach using the Alternating-Bit protocol

Verification of Modular Systems with Unknown Components Combining Testing and Inference

26 pagesVerification of a modular system composed of communicating components is a difficult problem, especially when the formal specifications, i.e., models of the components are not available. Conventional testing techniques are not efficient in detecting erroneous interactions of components because interleavings of internal events are difficult to reproduce in a modular system. The problem of detecting intermittent errors and other compositional problems in the absence of components' models is addressed in this paper. A method to infer a controllable approximation of communicating components through testing is elaborated. The inferred finite state models of components are used to detect compositional problems in the system through reachability analysis. To confirm a flaw in a particular component, a witness trace is used to construct a test applied to the component in isolation. The models are refined at each analysis step thus making the approach iterative

Comparison of two approaches for test case generations from EFSMs.

Testing is one of the vital steps in software development process. To convey testing, test cases need to be generated to check whether an implementation conforms to the design specification. Design specifications are usually expressed as Extended Finite State Machines (EFSMs) and test cases are actually a path from the initial state to a specific state on that EFSM. One of the most difficult issues of test case generation for EFSMs comes from the fact that infeasible paths exist on EFSMs. Two approaches have been developed in earlier 90s\u27 to generate feasible paths from EFSMs: one is to develop algorithm to search EFSMs directly to generate feasible paths, and the other is to expand EFSMs into Finite State Machines (FSMs), followed by applying FSM techniques to generate feasible paths. Model checking method was proposed recently as a new approach for test case generation. It has some advantages over previous methods such as efficiency on number of states explored. However, by nature, it also has some disadvantages such as time inefficiency. Here we present a comparison between the model checking method and the previous expansion method from pragmatic aspect by running experiments. To carry on this comparison, we implemented a classical expansion algorithm, defined the translation from EFSMs to Promela models, and used SPIN model checker in the model checking approach. We have run sufficient number of test case generation experiments, compared the two approaches on their time consumptions, numbers of states explored, performance changes when EFSMs\u27 sizes increase etc. By this comparison, we can see the tradeoff between time consumptions and the number of states explored in the two approaches and observe their performance changes while EFSMs change. Finally, we show the existence of the trade-off between state efficiency and time efficiency of the two approaches, the impact of domain size of variable value, the native drawbacks of the expansion algorithm and the performance improvement by tuning Premela models.Dept. of Computer Science. Paper copy at Leddy Library: Theses & Major Papers - Basement, West Bldg. / Call Number: Thesis2005 .T36. Source: Masters Abstracts International, Volume: 44-03, page: 1415. Thesis (M.Sc.)--University of Windsor (Canada), 2005

Improving fault coverage and minimising the cost of fault identification when testing from finite state machines

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Software needs to be adequately tested in order to increase the confidence that the system being developed is reliable. However, testing is a complicated and expensive process. Formal specification based models such as finite state machines have been widely used in system modelling and testing. In this PhD thesis, we primarily investigate fault detection and identification when testing from finite state machines. The research in this thesis is mainly comprised of three topics - construction of multiple Unique Input/Output (UIO) sequences using Metaheuristic Optimisation Techniques (MOTs), the improved fault coverage by using robust Unique Input/Output Circuit (UIOC) sequences, and fault diagnosis when testing from finite state machines. In the studies of the construction of UIOs, a model is proposed where a fitness function is defined to guide the search for input sequences that are potentially UIOs. In the studies of the improved fault coverage, a new type of UIOCs is defined. Based upon the Rural Chinese Postman Algorithm (RCPA), a new approach is proposed for the construction of more robust test sequences. In the studies of fault diagnosis, heuristics are defined that attempt to lead to failures being observed in some shorter test sequences, which helps to reduce the cost of fault isolation and identification. The proposed approaches and techniques were evaluated with regard to a set of case studies, which provides experimental evidence for their efficacy.Brunel Research Initiative and Enterprise Fund (BRIEF) Award from Brunel University and Departmental bursary from Department of Information Systems and Computing, Brunel Universit