Equivalence Partitioning as a Basis for Dynamic Conditional Invariant Detection

Program invariants are statements asserting properties of programs at certain points. They can assist developers and testers in understanding the program, and can be used for automated formal verification of the program. However, despite their usefulness they are often omitted from code. Dynamic invariant detection is a technique that discovers program invariants by observing execution of the program. One type of invariants that presents challenge to this technique is conditional invariants, which are considered to be computationally infeasible to be computed exhaustively. We present a new approach to assist conditional invariants detection, by analysing test suites used to drive the execution of the programs for their use of equivalence partitioning – a very common testing technique – and inferring conditional invariants from this information. A prototype implementation, named Yacon, is developed to work in conjunction with a mature dynamic invariant detection tool Daikon. Given a set of splitting conditions, Daikon can use them to infer conditional invariants. Yacon attempts to recover partitioning information from a given test suite, producing splitting conditions as a result. We introduced two strategies to recover partitioning information, one based on the presence of boundary value analysis testing technique; the other based on invariants within the test suite itself. We evaluated the effectiveness of each recovery strategy and the approach as a whole, and found that our approach can help make Daikon perform significantly better. However, the two recovery strategies only work well in limited circumstances, suggesting possible improvement in finding more effective recovery strategies

Adopting A Particle Swarm-Based Test Generator Strategy For Variable-Strength And T-Way Testing

Recently, researchers have started to explore the use of Artificial Intelligence (AI)-based algorithms as t-way (where t indicates the interaction strength) and variable-strength testing strategies. Many AI-based strategies have been developed, such as Ant Colony, Simulated Annealing, Genetic Algorithm, and Tabu Search. Although useful, most existing AI-based strategies adopt complex search processes and require heavy computations. For this reason, existing AI-based strategies have been confined to small interaction strengths (i.e., t≤3) and small test configurations. Recent studies demonstrate the need to go up to t=6 in order to capture most faults. This thesis presents the design and implementation of a new interaction test generation strategy, known as the Particle Swarm-based Test Generator (PSTG), for generating t-way and variable-strength test suites. Unlike other existing AI-based strategies, the lightweight computation of the particle swarm search process enables PSTG to support high interaction strengths of up to t=6. The performance of PSTG is evaluated using several sets of benchmark experiments. Comparatively, PSTG consistently outperforms its AI counterparts and other existing strategies as far as the size of the test suite is concerned. Furthermore, the case study demonstrates the usefulness of PSTG for detecting faulty interactions of the input components

Boundary Value Exploration for Software Analysis

For software to be reliable and resilient, it is widely accepted that tests must be created and maintained alongside the software itself. One safeguard from vulnerabilities and failures in code is to ensure correct behavior on the boundaries between sub-domains of the input space. So-called boundary value analysis (BVA) and boundary value testing (BVT) techniques aim to exercise those boundaries and increase test effectiveness. However, the concepts of BVA and BVT themselves are not clearly defined and it is not clear how to identify relevant sub-domains, and thus the boundaries delineating them, given a specification. This has limited adoption and hindered automation. We clarify BVA and BVT and introduce Boundary Value Exploration (BVE) to describe techniques that support them by helping to detect and identify boundary inputs. Additionally, we propose two concrete BVE techniques based on information-theoretic distance functions: (i) an algorithm for boundary detection and (ii) the usage of software visualization to explore the behavior of the software under test and identify its boundary behavior. As an initial evaluation, we apply these techniques on a much used and well-tested date handling library. Our results reveal questionable behavior at boundaries highlighted by our techniques. In conclusion, we argue that the boundary value exploration that our techniques enable is a step towards automated boundary value analysis and testing which can foster their wider use and improve test effectiveness and efficiency