Formal Methods in Conformance Testing: A Probabilistic Refinement

This paper refines the framework of ‘Formal Methods in Conformance Testing’ by introducing probabilities for concepts which have a stochastic nature. Test execution is refined into test runs, where each test run is considered as a stochastic process that returns a possible observa- tion with a certain probability. This implies that not every possible observation that could be made, will actually be made. The development process of an implementation from a specifica- tion is also viewed as a stochastic process that may result in a specific implementation with a certain probability. Together with a weight assignment on implementations this introduces a valuation measure on implementations. The test run probabilities and the valuation measures are integrated in generalized definitions of soundness and exhaustiveness, which can be used to compare test suites with respect to their ability to accept correct, and to reject erroneous implementations

Conformance Testing with Labelled Transition Systems: Implementation Relations and Test Generation

This paper studies testing based on labelled transition systems, presenting two test generation algorithms with their corresponding implementation relations. The first algorithm assumes that implementations communicate with their environment via symmetric, synchronous interactions. It is based on the theory of testing equivalence and preorder, as is most of the testing theory for labelled transition systems, and it is found in the literature in some slightly different variations. The second algorithm is based on the assumption that implementations communicate with their environment via inputs and outputs. Such implementations are formalized by restricting the class of labelled transition systems to those systems that can always accept input actions. For these implementations a testing theory is developed, analogous to the theory of testing equivalence and preorder. It consists of implementation relations formalizing the notion of conformance of these implementations with respect to labelled transition system specifications, test cases and test suites, test execution, the notion of passing a test suite, and the test generation algorithm, which is proved to produce sound test suites for one of the implementation relations

From FPGA to ASIC: A RISC-V processor experience

This work document a correct design flow using these tools in the Lagarto RISC- V Processor and the RTL design considerations that must be taken into account, to move from a design for FPGA to design for ASIC

Testing from Partial Finite State Machines without Harmonised Traces

This paper concerns the problem of testing from a partial, possibly non-deterministic, finite state machine (FSM) S. Two notions of correctness (quasi-reduction and quasi-equivalence) have previously been defined for partial FSMs but these, and the corresponding test generation techniques, only apply to FSMs that have harmonised traces. We show how quasi-reduction and quasi equivalence can be generalised to all partial FSMs. We also consider the problem of generating an m-complete test suite from a partial FSM S: a test suite that is guaranteed to determine correctness as long as the system under test has no more than m states. We prove that we can complete S to form a completely-specified non-deterministic FSM S0 such that any m-complete test suite generated from S0 can be converted into an m-complete test suite for S. We also show that there is a correspondence between test suites that are reduced for S and S0 and also that are minimal for S and S0

Testing in context: Efficiency and executability

Testing each software component in isolation is not always feasible. We consider testing a deterministic Implementation Under Test (IUT) together with some other correctly implemented components as its context. One of the essential issues of testing in context is test executability problem, i.e., tests generated solely from the specification of the IUT may not be executable due to the uncontrollable interaction between the IUT and its context. On the other hand, generating a test sequence from the abstract specifications of a stateful IUT and its context often suffers from the well-known state explosion problem. In this dissertation, we solve the problem of generating a minimal-length test sequence from a given specification of a stateful IUT and its embedded context. By adopting model checking techniques, we avoid the state explosion problem during test generation and avoid the test executability problem during testing in context

Test generation algorithm for the All-Transition-State criteria of Finite State Machines

In the current article a novel test generation algorithm is presented for deterministic finite state machine specifications based on the recently introduced All-Transition-State criteria. The size of the resulting test suite and the time required for test suite generation are investigated through analytical and practical analyses and are also compared to the Transition Tour, Harmonized State Identifiers and random walk test generation methods. The fault detection capabilities of the different approaches are also investigated with simulations applying randomly injected transfer faults