8,357 research outputs found
Avoiding coincidental correctness in boundary value analysis
In partition analysis we divide the input domain to form subdomains on which the system's behaviour should be uniform. Boundary value analysis produces test inputs near each subdomain's boundaries to find failures caused by incorrect implementation of the boundaries. However, boundary value analysis can be adversely affected by coincidental correctness---the system produces the expected output, but for the wrong reason. This article shows how boundary value analysis can be adapted in order to reduce the likelihood of coincidental correctness. The main contribution is to cases of automated test data generation in which we cannot rely on the expertise of a tester
Formal Verification of Real-Time Function Blocks Using PVS
A critical step towards certifying safety-critical systems is to check their
conformance to hard real-time requirements. A promising way to achieve this is
by building the systems from pre-verified components and verifying their
correctness in a compositional manner. We previously reported a formal approach
to verifying function blocks (FBs) using tabular expressions and the PVS proof
assistant. By applying our approach to the IEC 61131-3 standard of Programmable
Logic Controllers (PLCs), we constructed a repository of precise specification
and reusable (proven) theorems of feasibility and correctness for FBs. However,
we previously did not apply our approach to verify FBs against timing
requirements, since IEC 61131-3 does not define composite FBs built from
timers. In this paper, based on our experience in the nuclear domain, we
conduct two realistic case studies, consisting of the software requirements and
the proposed FB implementations for two subsystems of an industrial control
system. The implementations are built from IEC 61131-3 FBs, including the
on-delay timer. We find issues during the verification process and suggest
solutions.Comment: In Proceedings ESSS 2015, arXiv:1506.0325
Generating Complete and Finite Test Suite for ioco: Is It Possible?
Testing from Input/Output Transition Systems has been intensely investigated.
The conformance between the implementation and the specification is often
determined by the so-called ioco-relation. However, generating tests for ioco
is usually hindered by the problem of conflicts between inputs and outputs.
Moreover, the generation is mainly based on nondeterministic methods, which may
deliver complete test suites but require an unbounded number of executions. In
this paper, we investigate whether it is possible to construct a finite test
suite which is complete in a predefined fault domain for the classical ioco
relation even in the presence of input/output conflicts. We demonstrate that it
is possible under certain assumptions about the specification and
implementation, by proposing a method for complete test generation, based on a
traditional method developed for FSM.Comment: In Proceedings MBT 2014, arXiv:1403.704
Recommended from our members
Testing from a stochastic timed system with a fault model
In this paper we present a method for testing a system against a non-deterministic stochastic finite state machine. As usual, we assume that the functional behaviour of the system under test
(SUT) is deterministic but we allow the timing to be non-deterministic. We extend the state counting method of deriving tests, adapting it to the presence of temporal requirements represented by means of random variables. The notion of conformance is introduced using an implementation relation considering temporal aspects and the limitations imposed by a black-box framework. We propose an algorithm for generating a test suite that determines the conformance of a deterministic SUT with respect to a non-deterministic specification. We show how previous work on testing from stochastic systems can be encoded into the framework presented in this paper as an instantiation of our parameterized implementation relation. In this setting, we use a notion of conformance up to a given confidence level
Recommended from our members
Using formal methods to support testing
Formal methods and testing are two important approaches that assist in the development of high quality software. While traditionally these approaches have been seen as rivals, in recent
years a new consensus has developed in which they are seen as complementary. This article reviews the state of the art regarding ways in which the presence of a formal specification can be used to assist testing
Recommended from our members
Comparing test sets and criteria in the presence of test hypotheses and fault domains
A number of authors have considered the problem of comparing test sets and criteria. Ideally
test sets are compared using a preorder with the property that test set T1 is at least as strong
as T2 if whenever T2 determines that an implementation p is faulty, T1 will also determine that
p is faulty. This notion can be extended to test criteria. However, it has been noted that very
few test sets and criteria are comparable under such an ordering; instead orderings are based
on weaker properties such as subsumes. This paper explores an alternative approach, in which
comparisons are made in the presence of a test hypothesis or fault domain. This approach allows
strong statements about fault detecting ability to be made and yet for a number of test sets and
criteria to be comparable. It may also drive incremental test generation
JWalk: a tool for lazy, systematic testing of java classes by design introspection and user interaction
Popular software testing tools, such as JUnit, allow frequent retesting of modified code; yet the manually created test scripts are often seriously incomplete. A unit-testing tool called JWalk has therefore been developed to address the need for systematic unit testing within the context of agile methods. The tool operates directly on the compiled code for Java classes and uses a new lazy method for inducing the changing design of a class on the fly. This is achieved partly through introspection, using Javaās reflection capability, and partly through interaction with the user, constructing and saving test oracles on the fly. Predictive rules reduce the number of oracle values that must be confirmed by the tester. Without human intervention, JWalk performs bounded exhaustive exploration of the classās method protocols and may be directed to explore the space of algebraic constructions, or the intended design state-space of the tested class. With some human interaction, JWalk performs up to the equivalent of fully automated state-based testing, from a specification that was acquired incrementally
Low-Effort Specification Debugging and Analysis
Reactive synthesis deals with the automated construction of implementations
of reactive systems from their specifications. To make the approach feasible in
practice, systems engineers need effective and efficient means of debugging
these specifications.
In this paper, we provide techniques for report-based specification
debugging, wherein salient properties of a specification are analyzed, and the
result presented to the user in the form of a report. This provides a
low-effort way to debug specifications, complementing high-effort techniques
including the simulation of synthesized implementations.
We demonstrate the usefulness of our report-based specification debugging
toolkit by providing examples in the context of generalized reactivity(1)
synthesis.Comment: In Proceedings SYNT 2014, arXiv:1407.493
Towards Symbolic Model-Based Mutation Testing: Combining Reachability and Refinement Checking
Model-based mutation testing uses altered test models to derive test cases
that are able to reveal whether a modelled fault has been implemented. This
requires conformance checking between the original and the mutated model. This
paper presents an approach for symbolic conformance checking of action systems,
which are well-suited to specify reactive systems. We also consider
nondeterminism in our models. Hence, we do not check for equivalence, but for
refinement. We encode the transition relation as well as the conformance
relation as a constraint satisfaction problem and use a constraint solver in
our reachability and refinement checking algorithms. Explicit conformance
checking techniques often face state space explosion. First experimental
evaluations show that our approach has potential to outperform explicit
conformance checkers.Comment: In Proceedings MBT 2012, arXiv:1202.582
- ā¦