Adaptive Homing is in P

Homing preset and adaptive experiments with Finite State Machines (FSMs) are widely used when a non-initialized discrete event system is given for testing and thus, has to be set to the known state at the first step. The length of a shortest homing sequence is known to be exponential with respect to the number of states for a complete observable nondeterministic FSM while the problem of checking the existence of such sequence (Homing problem) is PSPACE-complete. In order to decrease the complexity of related problems, one can consider adaptive experiments when a next input to be applied to a system under experiment depends on the output responses to the previous inputs. In this paper, we study the problem of the existence of an adaptive homing experiment for complete observable nondeterministic machines. We show that if such experiment exists then it can be constructed with the use of a polynomial-time algorithm with respect to the number of FSM states.Comment: In Proceedings MBT 2015, arXiv:1504.0192

Using formal methods to support testing

Formal methods and testing are two important approaches that assist in the development of high quality software. While traditionally these approaches have been seen as rivals, in recent years a new consensus has developed in which they are seen as complementary. This article reviews the state of the art regarding ways in which the presence of a formal specification can be used to assist testing

Improvements in finite state machines

Finite State Machine (FSM) based testing methods have a history of over half a century, starting in 1956 with the works on machine identi cation. This was then followed by works checking the conformance of a given implementation to a given speci cation. When it is possible to identify the states of an FSM using an appropriate input sequence, it's been long known that it is possible to generate a Fault Detection Experiment with fault coverage with respect to a certain fault model in polynomial time. In this thesis, we investigate two notions of fault detection sequences; Checking Sequence (CS), Checking Experiment (CE). Since a fault detection sequence (either a CS or a CE) is constructed once but used many times, the importance of having short fault detection sequences is obvious and hence recent works in this eld aim to generate shorter fault detection sequences. In this thesis, we rst investigate a strategy and related problems to reduce the length of a CS. A CS consists several components such as Reset Sequences and State Identi - cation Sequences. All works assume that for a given FSM, a reset sequence and a state identi cation sequence are also given together with the speci cation FSM M. Using the given reset and state identi cation sequences, a CS is formed that gives full fault coverage under certain assumptions. In other words, any faulty implementation N can be identi ed by using this test sequence. In the literature, di erent methods for CS construction take di erent approaches to put these components together, with the aim of coming up with a shorter CS incorporating all of these components. One obvious way of keeping the CS short is to keep components short. As the reset sequence and the state identi cation sequence are the biggest components, having short reset and state identi cation sequences is very important as well. It was shown in 1991 that for a given FSM M, shortest reset sequence cannot be computed in polynomial time if P 6≠NP. Recently it was shown that when the FSM has particular type (\monotonic") of transition structure, constructing one of the shortest reset word is polynomial time solvable. However there has been no work on constructing one of the shortest reset word for a monotonic partially speci ed machines. In this thesis, we showed that this problem is NP-hard. On the other hand, in 1994 it was shown that one can check if M has special type of state identi cation sequence (known as an adaptive distinguishing sequence) in polynomial time. The same work also suggests a polynomial time algorithm to construct a state identi cation sequence when one exists. However, this algorithm generates a state identi cation sequence without any particular emphasis on generating a short one. There has been no work on the generation of state identi cation sequences for complete or partial machines after this work. In this thesis, we showed that construction of short state identi cation sequences is NP-complete and NP-hard to approximate. We propose methods of generating short state identi cation sequences and experimentally validate that such state identi cation sequences can reduce the length of fault detection sequences by 29:2% on the average. Another line of research, in this thesis, devoted for reducing the cost of checking experiments. A checking experiment consist of a set of input sequences each of which aim to test di erent properties of the implementation. As in the case of CSs, a large portion of these input sequences contain state identi cation sequences. There are several kinds of state identi cation sequences that are applicable in CEs. In this work, we propose a new kind of state identi cation sequence and show that construction of such sequences are PSPACE-complete. We propose a heuristic and we perform experiments on benchmark FSMs and experimentally show that the proposed notion of state identi cation sequence can reduce the cost of CEs by 65% in the extreme case. Testing distributed architectures is another interesting eld for FSM based fault detection sequence generation. The additional challenge when such distributed architectures are considered is to generate a fault detection sequence which does not pose controllability or observability problem. Although the existing methods again assume that a state identi cation sequence is given using which a fault detection sequence is constructed, there is no work on how to generate a state identi cation sequence which do not have controllability/observability problem itself. In this thesis we investigate the computational complexities to generate such state identi cation sequences and show that no polynomial time algorithm can construct a state identi cation sequence for a given distributed FSM

Синтез установочных последовательностей для автоматов с временными ограничениями

State identification is the well-known problem in the theory of Finite State Machines (FSM) where homing sequences (HS) are used for the identification of a current FSM state, and this fact is widely used in the area of software and hardware testing and verification. For various kinds of FSMs, such as partial, complete, deterministic, non-deterministic, there exist sufficient and necessary conditions for the existence ofpreset and adaptive HS and algorithms for their derivation. Nowadays timed aspects become very important for hardware and software systems and for this reason classical FSMs are extended by clock variables. In this work, we address the problem of checking the existence and derivation of homing sequences for FSMs with timed guards and show that the length estimation for timed homing sequence coincides with that for untimed FSM. The investigation is based on the FSM abstraction of a Timed FSM, i.e. on a classical FSM which describes behavior of corresponding TFSM and inherits some of its properties. When solving state identification problems for timed FSMs, the existing FSM abstraction is properly optimized.Идентификация состояний является хорошо известной задачей теории конечных автоматов, и установочные последовательности, которые позволяют идентифицировать текущее состояние конечного автомата, широко используются в областях тестирования и верификации программного и аппаратного обеспечения. Для автоматов различных классов, полностью определенных и частичных, детерминированных и недетерминированных, установлены необходимые и достаточные условия существования безусловных и адаптивных установочных последовательностей и предложены алгоритмы их синтеза, если такая последовательность существует. В настоящее время при верификации и тестировании программного и аппаратного обеспечения необходимо учитывать временные аспекты, что приводит к расширению автоматных моделей временными переменными. В настоящей работе мы исследуем задачи проверки существования и синтеза безусловных и адаптивных установочных последовательностей для автоматов с временными ограничениями и показываем, что оценки на длину таких последовательностей совпадают с оценками для классических конечных автоматов. Предлагаемый подход основан на использовании конечно-автоматной абстракции временного автомата, то есть описании временного автомата соответствующим конечным автоматом, который сохраняет свойства временного автомата относительно установочных последовательностей

К синтезу синхронизирующих и установочных последовательностей для входо-выходных полуавтоматов

In this paper, we study the problem of existence check and derivation of synchronizing and homing sequences for finite input/output automata. Corresponding sequences can be effectively used for the current state identification of a system under test / verification, after the input sequence is applied. In the model considered in the paper, the alphabet of actions is divided into disjoint sets of inputs and outputs; however, no sets of possible initial or final states are defined. We introduce the notions of homing and synchronizing sequences for a specific class of such machines for which at each state the transitions only under inputs or under outputs are defined, and the machine transition diagram does not contain cycles labeled by outputs, i.e. the language of the machine does not contain traces with infinite postfix of outputs. For such a class of input/output automata, we establish necessary and sufficient conditions for the existence of synchronizing and homing sequences and discuss the length of such sequences. We also define some subclasses of automata for which the worst-case upper bounds (normally, exponential) are not reachable. В работе рассматриваются задачи проверки существования и синтеза синхронизирующих и установочных последовательностей для конечных входо-выходных полуавтоматов. Соответствующие последовательности могут быть использованы при идентификации состояния проверяемой системы после подачи подходящей входной последовательности. В модели, исследуемой в работе, действия разделены на входные и выходные, однако отсутствуют выделенные явно семейства начальных и финальных состояний. В статье определяются понятия синхронизирующей и установочной последовательностей и предлагаются методы их синтеза для специального класса входо-выходных полуавтоматов, у которых в каждом состоянии определены переходы или только по входным, или только по выходным действиям; кроме того, в соответствующем графе переходов отсутствуют циклы по выходным символам. Для описанного класса входо-выходных полуавтоматов устанавливаются необходимые и достаточные условия существования синхронизирующих и установочных последовательностей и оценивается длина таких последовательностей. Выделяются подклассы полуавтоматов, для которых худшие (в основном экспоненциальные) оценки сложности не являются достижимыми.

Accelerating finite state machine-based testing using reinforcement learning

Testing is a crucial phase in the development of complex systems, and this has led to interest in automated test generation techniques based on state-based models. Many approaches use models that are types of finite state machine (FSM). Corresponding test generation algorithms typically require that certain test components, such as reset sequences (RSs) and preset distinguishing sequences (PDSs), have been produced for the FSM specification. Unfortunately, the generation of RSs and PDSs is computationally expensive, and this affects the scalability of such FSM-based test generation algorithms. This paper addresses this scalability problem by introducing a reinforcement learning framework: the Q -Graph framework for MBT. We show how this framework can be used in the generation of RSs and PDSs and consider both (potentially partial) timed and untimed models. The proposed approach was evaluated using three types of FSMs: randomly generated FSMs, FSMs from a benchmark, and an FSM of an Engine Status Manager for a printer. In experiments, the proposed approach was much faster and used much less memory than the state-of-the-art methods in computing PDSs and RSs

Improving fault coverage and minimising the cost of fault identification when testing from finite state machines

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Software needs to be adequately tested in order to increase the confidence that the system being developed is reliable. However, testing is a complicated and expensive process. Formal specification based models such as finite state machines have been widely used in system modelling and testing. In this PhD thesis, we primarily investigate fault detection and identification when testing from finite state machines. The research in this thesis is mainly comprised of three topics - construction of multiple Unique Input/Output (UIO) sequences using Metaheuristic Optimisation Techniques (MOTs), the improved fault coverage by using robust Unique Input/Output Circuit (UIOC) sequences, and fault diagnosis when testing from finite state machines. In the studies of the construction of UIOs, a model is proposed where a fitness function is defined to guide the search for input sequences that are potentially UIOs. In the studies of the improved fault coverage, a new type of UIOCs is defined. Based upon the Rural Chinese Postman Algorithm (RCPA), a new approach is proposed for the construction of more robust test sequences. In the studies of fault diagnosis, heuristics are defined that attempt to lead to failures being observed in some shorter test sequences, which helps to reduce the cost of fault isolation and identification. The proposed approaches and techniques were evaluated with regard to a set of case studies, which provides experimental evidence for their efficacy.Brunel Research Initiative and Enterprise Fund (BRIEF) Award from Brunel University and Departmental bursary from Department of Information Systems and Computing, Brunel Universit

Generating synchronizable test sequences with overlaps

Finite-state-machine-based conformance testing has been comprehensively studied in the literature in the context of centralized test architecture. As distributed test architecture involves multiple remote testers, applying a test sequence generated from a given n-port finite state machine to an implementation under test (IUT) may result in controllability problems. A possible way to resolve this problem is to select an appropriate test sequence, whose application to the IUT will not involve controllability problems. Thus generating such efficient test sequences is an interesting issue. There are several possibilities for such test sequence generation and we provide empirical study to compare the efficiency of two typical solutions proposed in the literature in terms of the length of the generated test sequences. While both of the two techniques rely on solutions to the Rural Postman Problem (RPP), a well-used RPP solution has been adopted and further improved in this thesis work

Improving fault coverage and minimising the cost of fault identification when testing from finite state machines

Software needs to be adequately tested in order to increase the confidence that the system being developed is reliable. However, testing is a complicated and expensive process. Formal specification based models such as finite state machines have been widely used in system modelling and testing. In this PhD thesis, we primarily investigate fault detection and identification when testing from finite state machines. The research in this thesis is mainly comprised of three topics - construction of multiple Unique Input/Output (UIO) sequences using Metaheuristic Optimisation Techniques (MOTs), the improved fault coverage by using robust Unique Input/Output Circuit (UIOC) sequences, and fault diagnosis when testing from finite state machines. In the studies of the construction of UIOs, a model is proposed where a fitness function is defined to guide the search for input sequences that are potentially UIOs. In the studies of the improved fault coverage, a new type of UIOCs is defined. Based upon the Rural Chinese Postman Algorithm (RCPA), a new approach is proposed for the construction of more robust test sequences. In the studies of fault diagnosis, heuristics are defined that attempt to lead to failures being observed in some shorter test sequences, which helps to reduce the cost of fault isolation and identification. The proposed approaches and techniques were evaluated with regard to a set of case studies, which provides experimental evidence for their efficacy.EThOS - Electronic Theses Online ServiceGBUnited Kingdo