Testing a deterministic implementation against a non-controllable non-deterministic stream X-machine

A stream X-machine is a type of extended finite state machine with an associated development approach that consists of building a system from a set of trusted components. One of the great benefits of using stream X-machines for the purpose of specification is the existence of test generation techniques that produce test suites that are guaranteed to determine correctness as long as certain well-defined conditions hold. One of the conditions that is traditionally assumed to hold is controllability: this insists that all paths through the stream X-machine are feasible. This restrictive condition has recently been weakened for testing from a deterministic stream X-machine. This paper shows how controllability can be replaced by a weaker condition when testing a deterministic system against a non-deterministic stream X-machine. This paper therefore develops a new, more general, test generation algorithm for testing from a non-deterministic stream X-machine

Testing conformance of a deterministic implementation against a non-deterministic stream X-machine

Stream X-machines are a formalisation of extended finite state machines that have been used to specify systems. One of the great benefits of using stream X-machines, for the purpose of specification, is the associated test generation technique which produces a test that is guaranteed to determine correctness under certain design for test conditions. This test generation algorithm has recently been extended to the case where the specification is non-deterministic. However, the algorithms for testing from a non-deterministic stream X-machine currently have limitations: either they test for equivalence, rather than conformance or they restrict the source of non-determinism allowed in the specification. This paper introduces a new test generation algorithm that overcomes both of these limitations, for situations where the implementation is known to be deterministic

Using formal methods to support testing

Formal methods and testing are two important approaches that assist in the development of high quality software. While traditionally these approaches have been seen as rivals, in recent years a new consensus has developed in which they are seen as complementary. This article reviews the state of the art regarding ways in which the presence of a formal specification can be used to assist testing

Formal Methods in Conformance Testing: A Probabilistic Refinement

This paper refines the framework of ‘Formal Methods in Conformance Testing’ by introducing probabilities for concepts which have a stochastic nature. Test execution is refined into test runs, where each test run is considered as a stochastic process that returns a possible observa- tion with a certain probability. This implies that not every possible observation that could be made, will actually be made. The development process of an implementation from a specifica- tion is also viewed as a stochastic process that may result in a specific implementation with a certain probability. Together with a weight assignment on implementations this introduces a valuation measure on implementations. The test run probabilities and the valuation measures are integrated in generalized definitions of soundness and exhaustiveness, which can be used to compare test suites with respect to their ability to accept correct, and to reject erroneous implementations

Test of preemptive real-time systems

Time Petri nets with stopwatches not only model system/environment interactions and time constraints. They further enable modeling of suspend/resume operations in real-time systems. Assuming the modelled systems are non deterministic and partially observable, the paper proposes a test generation approach which implements an online testing policy and outputs test results that are valid for the (part of the) selected environment. A relativized conformance relation named rswtioco is defined and a test generation algorithm is presented. The proposed approach is illustrated on an example

The complexity of asynchronous model based testing

This is the post-print version of the final paper published in Theoretical Computer Science. The published article is available from the link below. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. Copyright @ 2012 Elsevier B.V.In model based testing (MBT), testing is based on a model MM that typically is expressed using a state-based language such as an input output transition system (IOTS). Most approaches to MBT assume that communications between the system under test (SUT) and its environment are synchronous. However, many systems interact with their environment through asynchronous channels and the presence of such channels changes the nature of testing. In this paper we investigate the situation in which the SUT interacts with its environment through asynchronous channels and the problems of producing test cases to reach a state, execute a transition, or to distinguish two states. In addition, we investigate the Oracle Problem. All four problems are explored for both FIFO and non-FIFO channels. It is known that the Oracle Problem can be solved in polynomial time for FIFO channels but we also show that the three test case generation problems can also be solved in polynomial time in the case where the IOTS is observable but the general test generation problems are EXPTIME-hard. For non-FIFO channels we prove that all of the test case generation problems are EXPTIME-hard and the Oracle Problem in NP-hard, even if we restrict attention to deterministic IOTSs

Adaptive Homing is in P

Homing preset and adaptive experiments with Finite State Machines (FSMs) are widely used when a non-initialized discrete event system is given for testing and thus, has to be set to the known state at the first step. The length of a shortest homing sequence is known to be exponential with respect to the number of states for a complete observable nondeterministic FSM while the problem of checking the existence of such sequence (Homing problem) is PSPACE-complete. In order to decrease the complexity of related problems, one can consider adaptive experiments when a next input to be applied to a system under experiment depends on the output responses to the previous inputs. In this paper, we study the problem of the existence of an adaptive homing experiment for complete observable nondeterministic machines. We show that if such experiment exists then it can be constructed with the use of a polynomial-time algorithm with respect to the number of FSM states.Comment: In Proceedings MBT 2015, arXiv:1504.0192

A Semiparametric Test of Agent's Information Sets for Games of Incomplete Information

We propose semiparametric tests of misspecification of agent's information for games of incomplete information. The tests use the intuition that the opponent's choices should not predict a player's choice conditional on the proposed information available to the player. The tests are designed to check against some commonly used null hypotheses (Bajari et al. (2010), Aradillas-Lopez (2010)). We show that our tests have power to discriminate between common alternatives even in small samples. We apply our tests to data on entry in the US airline industry. Both the assumptions of independent and correlated private shocks are not supported by the data

On the Use of Formative Measurement Specifications in Structural Equation Modeling: A Monte Carlo Simulation Study to Compare Covariance-Based and Partial Least Squares Model Estimation Methodologies

The broader goal of this paper is to provide social researchers with some analytical guidelines when investigating structural equation models (SEM) with predominantly a formative specification. This research is the first to investigate the robustness and precision of parameter estimates of a formative SEM specification. Two distinctive scenarios (normal and non-normal data scenarios) are compared with the aid of a Monte Carlo simulation study for various covariance-based structural equation modeling (CBSEM) estimators and various partial least squares path modeling (PLS-PM) weighting schemes. Thus, this research is also one of the first to compare CBSEM and PLS-PM within the same simulation study. We establish that the maximum likelihood (ML) covariance-based discrepancy function provides accurate and robust parameter estimates for the formative SEM model under investigation when the methodological assumptions are met (e.g., adequate sample size, distributional assumptions, etc.). Under these conditions, ML-CBSEM outperforms PLS-PM. We also demonstrate that the accuracy and robustness of CBSEM decreases considerably when methodological requirements are violated, whereas PLS-PM results remain comparatively robust, e.g. irrespective of the data distribution. These findings are important for researchers and practitioners when having to choose between CBSEM and PLS-PM methodologies to estimate formative SEM in their particular research situation.PLS, path modeling, covariance structure analysis, structural equation modeling, formative measurement, simulation study