1,801 research outputs found
Recommended from our members
Applying adaptive test cases to nondeterministic implementations
The testing of a state-based system involves the application of sequences of
inputs and the observation of the resultant input/output sequences (traces).
These traces can result from preset input sequences or adaptive test cases in
which the choice of the next input depends on the trace that has observed
up to that input. Adaptive test cases are used in a number of areas including
protocol conformance testing and adaptivity forms
the basis of the standardised test language TTCN.
Suppose that we apply adaptive test case ° to the system under test (SUT)
and observe the trace ¯¾. If the SUT is deterministic and we apply ° again, after
resetting the SUT, then we will observe ¯¾ again. Further, if we have another
adaptive test case °0 where a prefix ¯¾0 of ¯¾ is a possible response to °0 then we
know that the application of °0 must lead to ¯¾0. Thus, for a deterministic SUT
the response of the SUT to an adaptive test case °0 might be deduced from
the response of the SUT to another adaptive test case. This observation
can be used to reduce the cost of testing: we only apply adaptive test case °0
if we cannot deduce the response to °0 from the set of observations.
While many systems are deterministic, nondeterminism is becoming increasingly
common. Nondeterminism in the SUT is typically a consequence of limits
in the ability to observe the SUT. For example, it could be a result of information
hiding, real time properties, or of different possible interleavings in a
concurrent system (see, for example. This paper investigates the case
where the SUT is nondeterministic. We consider the situation in which a set
O of traces has been observed in testing and we are considering applying an adaptive test case °. In general we cannot expect to be able to deduce the
response of a nondeterministic SUT to an adaptive test case ° since there may
be more than one possible response. Instead we consider the question of how
we can decide whether the application of ° could lead to a trace that has not
been observed. A solution to this would allow us to reduce the cost of testing:
if all possible responses of the SUT to ° have already been observed then we
do not have to apply ° in testing and thus reduce the cost of test execution.
This paper considers three cases. Section 3 considers the case where we can
apply a fairness assumption. Section 4 weakens this assumption to us having
a lower bound p on the probability of observing alternative responses of the
SUT to any input and in any state. Section 5 then considers the general case
Adaptive Homing is in P
Homing preset and adaptive experiments with Finite State Machines (FSMs) are
widely used when a non-initialized discrete event system is given for testing
and thus, has to be set to the known state at the first step. The length of a
shortest homing sequence is known to be exponential with respect to the number
of states for a complete observable nondeterministic FSM while the problem of
checking the existence of such sequence (Homing problem) is PSPACE-complete. In
order to decrease the complexity of related problems, one can consider adaptive
experiments when a next input to be applied to a system under experiment
depends on the output responses to the previous inputs. In this paper, we study
the problem of the existence of an adaptive homing experiment for complete
observable nondeterministic machines. We show that if such experiment exists
then it can be constructed with the use of a polynomial-time algorithm with
respect to the number of FSM states.Comment: In Proceedings MBT 2015, arXiv:1504.0192
Recommended from our members
Testing from a stochastic timed system with a fault model
In this paper we present a method for testing a system against a non-deterministic stochastic finite state machine. As usual, we assume that the functional behaviour of the system under test
(SUT) is deterministic but we allow the timing to be non-deterministic. We extend the state counting method of deriving tests, adapting it to the presence of temporal requirements represented by means of random variables. The notion of conformance is introduced using an implementation relation considering temporal aspects and the limitations imposed by a black-box framework. We propose an algorithm for generating a test suite that determines the conformance of a deterministic SUT with respect to a non-deterministic specification. We show how previous work on testing from stochastic systems can be encoded into the framework presented in this paper as an instantiation of our parameterized implementation relation. In this setting, we use a notion of conformance up to a given confidence level
Checking experiments for stream X-machines
This article is a post-print version of the published article which may be accessed at the link below. Copyright © 2010 Elsevier B.V. All rights reserved.Stream X-machines are a state based formalism that has associated with it a particular development process in which a system is built from trusted components. Testing thus essentially checks that these components have been combined in a correct manner and that the orders in which they can occur are consistent with the specification. Importantly, there are test generation methods that return a checking experiment: a test that is guaranteed to determine correctness as long as the implementation under test (IUT) is functionally equivalent to an unknown element of a given fault domain Ψ. Previous work has show how three methods for generating checking experiments from a finite state machine (FSM) can be adapted to testing from a stream X-machine. However, there are many other methods for generating checking experiments from an
FSM and these have a variety of benefits that correspond to different testing scenarios. This paper shows how any method for generating a checking experiment from an FSM can be adapted to generate a checking experiment for testing an implementation against a stream X-machine. This is the case whether we are testing to check that the IUT is functionally equivalent to a specification or we are testing to check that every trace (input/output sequence) of the IUT is also a trace of a nondeterministic specification. Interestingly, this holds even if the fault domain Ψ used is not that traditionally associated with testing from a stream
X-machine. The results also apply for both deterministic and nondeterministic implementations
Adaptive testing of a deterministic implementation against a nondeterministic finite state machine
A number of authors have looked at the problem of deriving a checking experiment from a nondeterministic finite state machine that models the required behaviour of a system. We show that these methods can be extended if it is known that the implementation is equivalent to some (unknown) deterministic finite state machine. When testing a deterministic implementation, the test output provides information about the implementation under test and can thus guide future testing. The use of an adaptive test process is thus proposed
The complexity of asynchronous model based testing
This is the post-print version of the final paper published in Theoretical Computer Science. The published article is available from the link below. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. Copyright @ 2012 Elsevier B.V.In model based testing (MBT), testing is based on a model MM that typically is expressed using a state-based language such as an input output transition system (IOTS). Most approaches to MBT assume that communications between the system under test (SUT) and its environment are synchronous. However, many systems interact with their environment through asynchronous channels and the presence of such channels changes the nature of testing. In this paper we investigate the situation in which the SUT interacts with its environment through asynchronous channels and the problems of producing test cases to reach a state, execute a transition, or to distinguish two states. In addition, we investigate the Oracle Problem. All four problems are explored for both FIFO and non-FIFO channels. It is known that the Oracle Problem can be solved in polynomial time for FIFO channels but we also show that the three test case generation problems can also be solved in polynomial time in the case where the IOTS is observable but the general test generation problems are EXPTIME-hard. For non-FIFO channels we prove that all of the test case generation problems are EXPTIME-hard and the Oracle Problem in NP-hard, even if we restrict attention to deterministic IOTSs
- …