Using distinguishing and UIO sequences together in a checking sequence

If a finite state machine M does not have a distinguishing sequence, but has UIO sequences for its states, there are methods to produce a checking sequence for M. However, if M has a distinguishing sequence D, then there are methods that make use of D to construct checking sequences that are much shorter than the ones that would be constructed by using only the UIO sequences for M. The methods to applied when a distinguishing sequence exists, only make use of the distinguishing sequences. In this paper we show that, even if M has a distinguishing sequence D, the UIO sequences can still be used together with D to construct shorter checking sequences

Testing from a finite state machine: Extending invertibility to sequences

When testing a system modelled as a finite state machine it is desirable to minimize the effort required. It has been demonstrated that it is possible to utilize test sequence overlap in order to reduce the test effort and this overlap has been represented by using invertible transitions. In this paper invertibility will be extended to sequences in order to reduce the test effort further and encapsulate a more general type of test sequence overlap. It will also be shown that certain properties of invertible sequences can be used in the generation of state identification sequences

Constructing multiple unique input/output sequences using metaheuristic optimisation techniques

Multiple unique input/output sequences (UIOs) are often used to generate robust and compact test sequences in finite state machine (FSM) based testing. However, computing UIOs is NP-hard. Metaheuristic optimisation techniques (MOTs) such as genetic algorithms (GAs) and simulated annealing (SA) are effective in providing good solutions for some NP-hard problems. In the paper, the authors investigate the construction of UIOs by using MOTs. They define a fitness function to guide the search for potential UIOs and use sharing techniques to encourage MOTs to locate UIOs that are calculated as local optima in a search domain. They also compare the performance of GA and SA for UIO construction. Experimental results suggest that, after using a sharing technique, both GA and SA can find a majority of UIOs from the models under test

Testing in context: Efficiency and executability

Testing each software component in isolation is not always feasible. We consider testing a deterministic Implementation Under Test (IUT) together with some other correctly implemented components as its context. One of the essential issues of testing in context is test executability problem, i.e., tests generated solely from the specification of the IUT may not be executable due to the uncontrollable interaction between the IUT and its context. On the other hand, generating a test sequence from the abstract specifications of a stateful IUT and its context often suffers from the well-known state explosion problem. In this dissertation, we solve the problem of generating a minimal-length test sequence from a given specification of a stateful IUT and its embedded context. By adopting model checking techniques, we avoid the state explosion problem during test generation and avoid the test executability problem during testing in context

Using formal methods to support testing

Formal methods and testing are two important approaches that assist in the development of high quality software. While traditionally these approaches have been seen as rivals, in recent years a new consensus has developed in which they are seen as complementary. This article reviews the state of the art regarding ways in which the presence of a formal specification can be used to assist testing

Using a SAT solver to generate checking sequences

Methods for software testing based on Finite State Machines (FSMs) have been researched since the early 60’s. Many of these methods are about generating a checking sequence from a given FSM which is an input sequence that determines whether an implementation of the FSM is faulty or correct. In this paper, we consider one of these methods, which constructs a checking sequence by reducing the problem of generating a checking sequence to finding a Chinese rural postman tour on a graph induced by the FSM; we re-formulate the constraints used in this method as a set of Boolean formulas; and use a SAT solver to generate a checking sequence of minimal length

K-branching UIO sequences for partially specified observable non-deterministic FSMs

In black-box testing, test sequences may be constructed from systems modelled as deterministic finite-state machines (DFSMs) or, more generally, observable non-deterministic finite state machines (ONFSMs). Test sequences usually contain state identification sequences, with unique input output sequences (UIOs) often being used with DFSMs. This paper extends the notion of UIOs to ONFSMs. One challenge is that, as a result of non-determinism, the application of an input sequence can lead to exponentially many expected output sequences. To address this scalability problem, we introduce K-UIOs: K-UIOs that lead to at most K output sequences from states of M. We show that checking K-UIO existence is PSPACE-Complete if the problem is suitably bounded; otherwise it is in EXPSPACE and PSPACE-Hard. We provide a massively parallel algorithm for constructing K-UIOs and the results of experiments on randomly generated and real FSM specifications. The proposed algorithm was able to construct UIOs in cases where the existing UIO generation algorithm could not and was able to construct UIOs from FSMs with 38K states and 400K transitions

Improving fault coverage and minimising the cost of fault identification when testing from finite state machines

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Software needs to be adequately tested in order to increase the confidence that the system being developed is reliable. However, testing is a complicated and expensive process. Formal specification based models such as finite state machines have been widely used in system modelling and testing. In this PhD thesis, we primarily investigate fault detection and identification when testing from finite state machines. The research in this thesis is mainly comprised of three topics - construction of multiple Unique Input/Output (UIO) sequences using Metaheuristic Optimisation Techniques (MOTs), the improved fault coverage by using robust Unique Input/Output Circuit (UIOC) sequences, and fault diagnosis when testing from finite state machines. In the studies of the construction of UIOs, a model is proposed where a fitness function is defined to guide the search for input sequences that are potentially UIOs. In the studies of the improved fault coverage, a new type of UIOCs is defined. Based upon the Rural Chinese Postman Algorithm (RCPA), a new approach is proposed for the construction of more robust test sequences. In the studies of fault diagnosis, heuristics are defined that attempt to lead to failures being observed in some shorter test sequences, which helps to reduce the cost of fault isolation and identification. The proposed approaches and techniques were evaluated with regard to a set of case studies, which provides experimental evidence for their efficacy.Brunel Research Initiative and Enterprise Fund (BRIEF) Award from Brunel University and Departmental bursary from Department of Information Systems and Computing, Brunel Universit

Distinguishing sequence based checking sequence generation implementation and improvements

With advances in computer technology and software engineering, systems are constantly becoming larger and more complex. Straightforward testing methods are insufficient to cope with the complexity and maintaining quality of service demands the use of more structured testing methods. Checking sequences are testing mechanisms based on finite state behavior models that can offer guarantees about a system under test, under certain assumptions. However, their complexities are high, and to make their implementation feasible methods of their construction need to be refined. We have studied several methods of checking sequence construction in the presence of distinguishing sequences, developed fully formed algorithms from loose specifications, then implemented and compared their performances. We have also proposed several improvements that will allow generation of shorter checking sequences. We are confident that these developments will be instrumental in making the use of checking sequences feasible in a larger scope