154 research outputs found
Towards Statistical Prioritization for Software Product Lines Testing
Software Product Lines (SPL) are inherently difficult to test due to the
combinatorial explosion of the number of products to consider. To reduce the
number of products to test, sampling techniques such as combinatorial
interaction testing have been proposed. They usually start from a feature model
and apply a coverage criterion (e.g. pairwise feature interaction or
dissimilarity) to generate tractable, fault-finding, lists of configurations to
be tested. Prioritization can also be used to sort/generate such lists,
optimizing coverage criteria or weights assigned to features. However, current
sampling/prioritization techniques barely take product behavior into account.
We explore how ideas of statistical testing, based on a usage model (a Markov
chain), can be used to extract configurations of interest according to the
likelihood of their executions. These executions are gathered in featured
transition systems, compact representation of SPL behavior. We discuss possible
scenarios and give a prioritization procedure illustrated on an example.Comment: Extended version published at VaMoS '14
(http://dx.doi.org/10.1145/2556624.2556635
Towards Sampling and Simulation-Based Analysis of Featured Weighted Automata
International audienceWe consider the problem of model checking Variability-Intensive Systems (VIS) against non-functional requirements. These requirements are typically expressed as an optimization problem over quality attributes of interest, whose value is determined by the executions of the system. Identifying the optimal variant can be hard for two reasons. First, the state-explosion problem inherent to model checking makes it increasingly complex to find the optimal executions within a given variant. Second, the number of variants can grow exponentially with respect to the number of variation points in the VIS. In this paper, we lay the foundations for the application of smart sampling and statistical model checking to solve this problem faster. We design a simple method that samples variants and executions in a uniform manner from a featured weighted automaton and that assesses which of the sampled variants/executions are optimal. We implemented our approach on top of ProVeLines, a tool suite for model-checking VIS and carried out a preliminary evaluation on an industrial embedded system design case study. Our results tend to show that sampling-based approaches indeed holds the potential to improve scalability but should be supported by better sampling heuristics to be competitive
The "Baggaging" of Theory-Based Evaluation
We introduce SeMu, a Dynamic Symbolic Execution technique that generates test
inputs capable of killing stubborn mutants (killable mutants that remain
undetected after a reasonable amount of testing). SeMu aims at mutant
propagation (triggering erroneous states to the program output) by
incrementally searching for divergent program behaviours between the original
and the mutant versions. We model the mutant killing problem as a symbolic
execution search within a specific area in the programs' symbolic tree. In this
framework, the search area is defined and controlled by parameters that allow
scalable and cost-effective mutant killing. We integrate SeMu in KLEE and
experimented with Coreutils (a benchmark frequently used in symbolic execution
studies). Our results show that our modelling plays an important role in mutant
killing. Perhaps more importantly, our results also show that, within a
two-hour time limit, SeMu kills 37% of the stubborn mutants, where KLEE kills
none and where the mutant infection strategy (strategy suggested by previous
research) kills 17%
- …