Distinguishing sequences for partially specified FSMs

Distinguishing Sequences (DSs) are used inmany Finite State Machine (FSM) based test techniques. Although Partially Specified FSMs (PSFSMs) generalise FSMs, the computational complexity of constructing Adaptive and Preset DSs (ADSs/PDSs) for PSFSMs has not been addressed. This paper shows that it is possible to check the existence of an ADS in polynomial time but the corresponding problem for PDSs is PSPACE-complete. We also report on the results of experiments with benchmarks and over 8 * 106 PSFSMs. © 2014 Springer International Publishing

Using a SAT solver to generate checking sequences

Methods for software testing based on Finite State Machines (FSMs) have been researched since the early 60’s. Many of these methods are about generating a checking sequence from a given FSM which is an input sequence that determines whether an implementation of the FSM is faulty or correct. In this paper, we consider one of these methods, which constructs a checking sequence by reducing the problem of generating a checking sequence to finding a Chinese rural postman tour on a graph induced by the FSM; we re-formulate the constraints used in this method as a set of Boolean formulas; and use a SAT solver to generate a checking sequence of minimal length

The complexity of asynchronous model based testing

This is the post-print version of the final paper published in Theoretical Computer Science. The published article is available from the link below. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. Copyright @ 2012 Elsevier B.V.In model based testing (MBT), testing is based on a model MM that typically is expressed using a state-based language such as an input output transition system (IOTS). Most approaches to MBT assume that communications between the system under test (SUT) and its environment are synchronous. However, many systems interact with their environment through asynchronous channels and the presence of such channels changes the nature of testing. In this paper we investigate the situation in which the SUT interacts with its environment through asynchronous channels and the problems of producing test cases to reach a state, execute a transition, or to distinguish two states. In addition, we investigate the Oracle Problem. All four problems are explored for both FIFO and non-FIFO channels. It is known that the Oracle Problem can be solved in polynomial time for FIFO channels but we also show that the three test case generation problems can also be solved in polynomial time in the case where the IOTS is observable but the general test generation problems are EXPTIME-hard. For non-FIFO channels we prove that all of the test case generation problems are EXPTIME-hard and the Oracle Problem in NP-hard, even if we restrict attention to deterministic IOTSs

Generating minimum height ADSs for partially specified finite state machines

In earlier work, the problem of generating a preset distinguishing sequence from a finite state machine (FSM) was converted into a Boolean formulae to be fed into a SAT solver, with experiments suggesting that such approaches are required as the size of input alphabet grows. In this paper we extend the approach to the minimum height adaptive distinguishing sequence construction problem for partially specified FSMs (PSFMSs), which is known to be an NP- Hard problem. The results of experimentswith randomly generated PSFSMs and case studies from the literature show that SAT solvers can perform better than a previously proposed brute-force algorithm.The Scientific and Technological Research Council of Turkey under the grant reference no B.14.2.TBT.0.06.01-219-115543

Distinguishing Sequences for Distributed Testing: Preset Distinguishing Sequences

There has been long-standing interest in automatically generating test sequences from a finite state machine (FSM) and more recently this has been extended to the case where there are multiple physically distributed testers and so we are testing from a multi-port FSM. This paper explores the problem of generating a controllable preset distinguishing sequence (PDS) from a multi-port FSM, motivated by the fact that many FSM-based test generation algorithms use PDSs. We prove that it is generally undecidable whether a multi-port FSM has a controllable PDS but provide a class of multi-port FSMs for which the problem is decidable. We also consider the important case where there is an upper bound ` on the length of PDSs of interest, proving that controllable PDS existence is PSPACE-hard and in EXPSPACE. In practice the upper bound ` is likely to be a polynomial in terms of the size of the multi-port FSM and in this case controllable PDS existence is NP- Complete

Parallel algorithms for generating distinguishing sequences for observable non-deterministic FSMs

A distinguishing sequence (DS) for a finite state machine (FSM) is an input sequence that distinguishes every pair of states of the FSM. There are techniques that generate a test sequence with guaranteed fault detection power and it has been found that shorter test sequence can be produced if DSs are used. Despite these benefits, however, until recently the only published DS generation algorithms have been for deterministic FSMs. This paper develops a massively parallel algorithm, which can be used in GPU Computing, to generate DSs from partial observable non-deterministic FSMs. We also present the results of experiments using randomly generated FSMs and some benchmark FSMs. The results are promising and indicate that the proposed algorithm can derive DSs from partial observable non-deterministic FSMs with 32,000 states in an acceptable amount of time.This work is supported by the Scientific and Technological Research Council of Turkey (TUBITAK) under Grant #1059B191400424 and by the NVIDIA corporation

K-branching UIO sequences for partially specified observable non-deterministic FSMs

In black-box testing, test sequences may be constructed from systems modelled as deterministic finite-state machines (DFSMs) or, more generally, observable non-deterministic finite state machines (ONFSMs). Test sequences usually contain state identification sequences, with unique input output sequences (UIOs) often being used with DFSMs. This paper extends the notion of UIOs to ONFSMs. One challenge is that, as a result of non-determinism, the application of an input sequence can lead to exponentially many expected output sequences. To address this scalability problem, we introduce K-UIOs: K-UIOs that lead to at most K output sequences from states of M. We show that checking K-UIO existence is PSPACE-Complete if the problem is suitably bounded; otherwise it is in EXPSPACE and PSPACE-Hard. We provide a massively parallel algorithm for constructing K-UIOs and the results of experiments on randomly generated and real FSM specifications. The proposed algorithm was able to construct UIOs in cases where the existing UIO generation algorithm could not and was able to construct UIOs from FSMs with 38K states and 400K transitions

Using formal methods to support testing

Formal methods and testing are two important approaches that assist in the development of high quality software. While traditionally these approaches have been seen as rivals, in recent years a new consensus has developed in which they are seen as complementary. This article reviews the state of the art regarding ways in which the presence of a formal specification can be used to assist testing

Incomplete distinguishing sequences for finite state machines

Given a Finite State Machine (FSM) M, a Distinguishing Sequence (DS) is a test that identifies the state of M. While there are two types of DSs, preset DSs (PDSs) and adaptive DSs (ADSs), not all FSMs possess a DS. In this paper, we examine the problem of finding incomplete PDSs and ADSs, exploring associated optimisation problems: finding a largest set of states that has a DS and finding a smallest set of DSs that, between them, distinguish all of the states. We also propose a greedy algorithm to produce a small set of incomplete ADSs and use experiments to compare this with two previously published algorithms for generating state identifiers. We show that the optimisation problems related to incomplete ADSs and PDSs are PSPACE-Complete as are corresponding approximation problems. In the experiments we found that incomplete ADSs produced by the proposed greedy algorithm led to relatively compact state identifiers