Data and analysis code for GP EFSM inference

This artifact captures the workflow that we adopted for our experimental evaluation in our ICSME paper on inferring state transition functions during EFSM inference. To summarise, the paper uses Genetic Programming to infer data transformations, to enable the inference of fully 'computational' extended finite state machine models. This submission shows how we generated, transformed, analysed, and visualised our raw data. It includes everything needed to generate raw results and provides the relevant R code in the form of a re-usable Jupyter Notebook (accompanied by a descriptive narrative)

Inferring Computational State Machine Models from Program Executions

The challenge of inferring state machines from log data or execution traces is well-established, and has led to the development of several powerful techniques. Current approaches tend to focus on the inference of conventional finite state machines or, in few cases, state machines with guards. However, these machines are ultimately only partial, because they fail to model how any underlying variables are computed during the course of an execution, they are not computational. In this paper we introduce a technique based upon Genetic Programming to infer these data transformation functions, which in turn render inferred automata fully computational. Instead of merely determining whether or not a sequence is possible, they can be simulated, and be used to compute the variable values throughout the course of an execution. We demonstrate the approach by using a Cross-Validation study to reverse-engineer complete (computational) EFSMs from traces of established implementations

Inference of Extended Finite State Machines

This is the final version. Available from AFP via the link in this recordIn this AFP entry, we provide a formal implementation of a state-merging technique to infer extended finite state machines (EFSMs), complete with output and update functions, from black-box traces. In particular, we define the subsumption in context relation as a means of determining whether one transition is able to account for the behaviour of another. Building on this, we define the direct subsumption relation, which lifts the subsumption in context relation to EFSM level such that we can use it to determine whether it is safe to merge a given pair of transitions. Key proofs include the conditions necessary for subsumption to occur and that subsumption and direct subsumption are preorder relations. We also provide a number of different heuristics which can be used to abstract away concrete values into registers so that more states and transitions can be merged and provide proofs of the various conditions which must hold for these abstractions to subsume their ungeneralised counterparts. A Code Generator setup to create executable Scala code is also defined

Reverse Engineering Systems to Identify Flaws and Understand Behaviour

Accurate system models are applicable to many software engineering tasks. Despite their utility, models are often neglected during development. It is therefore desirable to reverse engineer them from existing systems. One way to do this is to record traces of the system and infer a model by generalising from this behaviour. Unfortunately, the models inferred by current techniques often cannot represent how the data values associated with each action affect system behaviour. This raises the following questions. What kind of model do we need in order to show the interplay between behaviour and data? How can we infer such models from system traces? How can we infer functions to relate input data with subsequent outputs? How can we use our models once they have been inferred? To answer these questions, the first contribution of this thesis is a new model definition designed to show the relationship between data and behaviour. Secondly, I present a technique to infer such models from system traces, and define a preprocessing step to infer functions that relate system inputs and outputs. I then empirically evaluate the models produced by my technique and compare them to those produced by a state-of-the-art tool. Finally, I show how the inferred models can be used to analyse properties of the systems they represent. The results show that my technique infers models which are more accurate and intuitive than the current state of the art. My tool can also handle circumstances where the output of a system depends on data values not present in the traces, and can identify situations where the result of particular actions depends on specific data values. The models inferred by my tool can be used by existing verification tools to prove and refute properties of the underlying systems

Reverse-engineering EFSMs with data dependencies

EFSMs provide a way to model systems with internal data variables. In situations where they do not already exist, we need to infer them from system behaviour. A key challenge here is inferring the functions which relate inputs, outputs, and internal variables. Existing approaches either work with white-box traces, which expose variable values, or rely upon the user to provide heuristics to recognise and generalise particular data-usage patterns. This paper presents a preprocessing technique for the inference process which generalises the concrete values from the traces into symbolic functions which calculate output from input, even when this depends on values not present in the original traces. Our results show that our technique leads to more accurate models than are produced by the current state-of-the-art and that somewhat accurate models can still be inferred even when the output of particular transitions depends on values not present in the original traces

Automated test sequence generation for finite state machines using genetic algorithms

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Testing software implementations, formally specified using finite state automata (FSA) has been of interest. Such systems include communication protocols and control sections of safety critical systems. There is extensive literature regarding how to formally validate an FSM based specification, but testing that an implementation conforms to the specification is still an open problem. Two aspects of FSA based testing, both NP-hard problems, are discussed in this thesis and then combined. These are the generation of state verification sequences (UIOs) and the generation of sequences of conditional transitions that are easy to trigger. In order to facilitate test sequence generation a novel representation of the transition conditions and a number of fitness function algorithms are defined. An empirical study of the effectiveness on real FSA based systems and example FSAs provides some interesting positive results. The use of genetic algorithms (GAs) makes these problems scalable for large FSAs. The experiments used a software tool that was developed in Java

Comparison of two approaches for test case generations from EFSMs.

Testing is one of the vital steps in software development process. To convey testing, test cases need to be generated to check whether an implementation conforms to the design specification. Design specifications are usually expressed as Extended Finite State Machines (EFSMs) and test cases are actually a path from the initial state to a specific state on that EFSM. One of the most difficult issues of test case generation for EFSMs comes from the fact that infeasible paths exist on EFSMs. Two approaches have been developed in earlier 90s\u27 to generate feasible paths from EFSMs: one is to develop algorithm to search EFSMs directly to generate feasible paths, and the other is to expand EFSMs into Finite State Machines (FSMs), followed by applying FSM techniques to generate feasible paths. Model checking method was proposed recently as a new approach for test case generation. It has some advantages over previous methods such as efficiency on number of states explored. However, by nature, it also has some disadvantages such as time inefficiency. Here we present a comparison between the model checking method and the previous expansion method from pragmatic aspect by running experiments. To carry on this comparison, we implemented a classical expansion algorithm, defined the translation from EFSMs to Promela models, and used SPIN model checker in the model checking approach. We have run sufficient number of test case generation experiments, compared the two approaches on their time consumptions, numbers of states explored, performance changes when EFSMs\u27 sizes increase etc. By this comparison, we can see the tradeoff between time consumptions and the number of states explored in the two approaches and observe their performance changes while EFSMs change. Finally, we show the existence of the trade-off between state efficiency and time efficiency of the two approaches, the impact of domain size of variable value, the native drawbacks of the expansion algorithm and the performance improvement by tuning Premela models.Dept. of Computer Science. Paper copy at Leddy Library: Theses & Major Papers - Basement, West Bldg. / Call Number: Thesis2005 .T36. Source: Masters Abstracts International, Volume: 44-03, page: 1415. Thesis (M.Sc.)--University of Windsor (Canada), 2005

Search-based software engineering: A search-based approach for testing from extended finite state machine (EFSM) models

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.The extended finite state machine (EFSM) is a powerful modelling approach that has been applied to represent a wide range of systems. Despite its popularity, testing from an EFSM is a substantial problem for two main reasons: path feasibility and path test case generation. The path feasibility problem concerns generating transition paths through an EFSM that are feasible and satisfy a given test criterion. In an EFSM, guards and assignments in a path‟s transitions may cause some selected paths to be infeasible. The problem of path test case generation is to find a sequence of inputs that can exercise the transitions in a given feasible path. However, the transitions‟ guards and assignments in a given path can impose difficulties when producing such data making the range of acceptable inputs narrowed down to a possibly tiny range. While search-based approaches have proven efficient in automating aspects of testing, these have received little attention when testing from EFSMs. This thesis proposes an integrated search-based approach to automatically test from an EFSM. The proposed approach generates paths through an EFSM that are potentially feasible and satisfy a test criterion. Then, it generates test cases that can exercise the generated feasible paths. The approach is evaluated by being used to test from five EFSM cases studies. The achieved experimental results demonstrate the value of the proposed approach.Aleppo University, Syri

Automated test sequence generation for finite state machines using genetic algorithms

Testing software implementations, formally specified using finite state automata (FSA) has been of interest. Such systems include communication protocols and control sections of safety critical systems. There is extensive literature regarding how to formally validate an FSM based specification, but testing that an implementation conforms to the specification is still an open problem. Two aspects of FSA based testing, both NP-hard problems, are discussed in this thesis and then combined. These are the generation of state verification sequences (UIOs) and the generation of sequences of conditional transitions that are easy to trigger. In order to facilitate test sequence generation a novel representation of the transition conditions and a number of fitness function algorithms are defined. An empirical study of the effectiveness on real FSA based systems and example FSAs provides some interesting positive results. The use of genetic algorithms (GAs) makes these problems scalable for large FSAs. The experiments used a software tool that was developed in Java.EThOS - Electronic Theses Online ServiceGBUnited Kingdo