14,979 research outputs found
A Large-scale Study of Call Graph-based Impact Prediction using Mutation Testing
International audienceIn software engineering, impact analysis consists in predicting the software elements (e.g. modules, classes, methods) potentially impacted by a change in the source code. Impact analysis is required to optimize the testing effort. In this paper, we propose a framework to predict error propagation. Based on 10 open-source Java projects and 5 classical mutation operators, we create 17000 mutants and study how the error they introduce propagates. This framework enables us to analyze impact prediction based on four types of call graph. Our results show that the sophistication indeed increases completeness of impact prediction. However, and surprisingly to us, the most basic call graph gives the highest trade-off between precision and recall for impact prediction
MuDelta: Delta-Oriented Mutation Testing at Commit Time
To effectively test program changes using mutation testing, one needs to use mutants that are relevant to the altered program behaviours. In view of this, we introduce MuDelta, an approach that identifies commit-relevant mutants; mutants that affect and are affected by the changed program behaviours. Our approach uses machine learning applied on a combined scheme of graph and vector-based representations of static code features. Our results, from 50 commits in 21 Coreutils programs, demonstrate a strong prediction ability of our approach; yielding 0.80 (ROC) and 0.50 (PR Curve) AUC values with 0.63 and 0.32 precision and recall values. These predictions are significantly higher than random guesses, 0.20 (PR-Curve) AUC, 0.21 and 0.21 precision and recall, and subsequently lead to strong relevant tests that kill 45%more relevant mutants than randomly sampled mutants (either sampled from those residing on the changed component(s) or from the changed lines). Our results also show that MuDelta selects mutants with 27% higher fault revealing ability in fault introducing commits. Taken together, our results corroborate the conclusion that commit-based mutation testing is suitable and promising for evolving software
Empirical Evaluation of Mutation-based Test Prioritization Techniques
We propose a new test case prioritization technique that combines both
mutation-based and diversity-based approaches. Our diversity-aware
mutation-based technique relies on the notion of mutant distinguishment, which
aims to distinguish one mutant's behavior from another, rather than from the
original program. We empirically investigate the relative cost and
effectiveness of the mutation-based prioritization techniques (i.e., using both
the traditional mutant kill and the proposed mutant distinguishment) with 352
real faults and 553,477 developer-written test cases. The empirical evaluation
considers both the traditional and the diversity-aware mutation criteria in
various settings: single-objective greedy, hybrid, and multi-objective
optimization. The results show that there is no single dominant technique
across all the studied faults. To this end, \rev{we we show when and the reason
why each one of the mutation-based prioritization criteria performs poorly,
using a graphical model called Mutant Distinguishment Graph (MDG) that
demonstrates the distribution of the fault detecting test cases with respect to
mutant kills and distinguishment
Investment Opportunities Forecasting: Extending the Grammar of a GP-based Tool
In this paper we present a new version of a GP financial forecasting tool, called EDDIE 8. The novelty of this version is that it allows the GP to search in the space of indicators, instead of using pre-specified ones. We compare EDDIE 8 with its predecessor, EDDIE 7, and find that new and improved solutions can be found. Analysis also shows that, on average, EDDIE 8's best tree performs better than the one of EDDIE 7. The above allows us to characterize EDDIE 8 as a valuable forecasting tool
Towards Smart Hybrid Fuzzing for Smart Contracts
Smart contracts are Turing-complete programs that are executed across a
blockchain network. Unlike traditional programs, once deployed they cannot be
modified. As smart contracts become more popular and carry more value, they
become more of an interesting target for attackers. In recent years, smart
contracts suffered major exploits, costing millions of dollars, due to
programming errors. As a result, a variety of tools for detecting bugs has been
proposed. However, majority of these tools often yield many false positives due
to over-approximation or poor code coverage due to complex path constraints.
Fuzzing or fuzz testing is a popular and effective software testing technique.
However, traditional fuzzers tend to be more effective towards finding shallow
bugs and less effective in finding bugs that lie deeper in the execution. In
this work, we present CONFUZZIUS, a hybrid fuzzer that combines evolutionary
fuzzing with constraint solving in order to execute more code and find more
bugs in smart contracts. Evolutionary fuzzing is used to exercise shallow parts
of a smart contract, while constraint solving is used to generate inputs which
satisfy complex conditions that prevent the evolutionary fuzzing from exploring
deeper paths. Moreover, we use data dependency analysis to efficiently generate
sequences of transactions, that create specific contract states in which bugs
may be hidden. We evaluate the effectiveness of our fuzzing strategy, by
comparing CONFUZZIUS with state-of-the-art symbolic execution tools and
fuzzers. Our evaluation shows that our hybrid fuzzing approach produces
significantly better results than state-of-the-art symbolic execution tools and
fuzzers
- …