7,126 research outputs found
Automatic Repair of Real Bugs: An Experience Report on the Defects4J Dataset
Defects4J is a large, peer-reviewed, structured dataset of real-world Java
bugs. Each bug in Defects4J is provided with a test suite and at least one
failing test case that triggers the bug. In this paper, we report on an
experiment to explore the effectiveness of automatic repair on Defects4J. The
result of our experiment shows that 47 bugs of the Defects4J dataset can be
automatically repaired by state-of- the-art repair. This sets a baseline for
future research on automatic repair for Java. We have manually analyzed 84
different patches to assess their real correctness. In total, 9 real Java bugs
can be correctly fixed with test-suite based repair. This analysis shows that
test-suite based repair suffers from under-specified bugs, for which trivial
and incorrect patches still pass the test suite. With respect to practical
applicability, it takes in average 14.8 minutes to find a patch. The experiment
was done on a scientific grid, totaling 17.6 days of computation time. All
their systems and experimental results are publicly available on Github in
order to facilitate future research on automatic repair
Practical issues for the implementation of survivability and recovery techniques in optical networks
On Oracles for Automated Diagnosis and Repair of Software Bugs
This HDR focuses on my work on automatic diagnosis and repair done over the past years. Among my past publications, it highlights three contributions on this topic, respectively published in ACM Transactions on Software Engineering and Methodology (TOSEM), IEEE Transactions on Software Engineering (TSE) and Elsevier Information & Software Technology (IST). My goal is to show that those three contributions share something deep, that they are founded on a unifying concept, which is the one of oracle. The first contribution is about statistical oracles. In the context of object-oriented software, we have defined a notion of context and normality that is specific to a fault class: missing method calls. Those inferred regularities act as oracle and their violations are considered as bugs. The second contribution is about test case based oracles for automatic repair. We describe an automatic repair system that fixes failing test cases by generating a patch. It is founded on the idea of refining the knowledge given by the violation of the oracle of the failing test case into finer-grain information, which we call a “micro-oracle”. By considering micro-oracles, we are capable of obtaining at the same time a precise fault localization diagnostic and a well-formed input-output specification to be used for program synthesis in order to repair a bug. The third contribution discusses a novel generic oracle in the context of exception handling. A generic oracle states properties that hold for many domains. Our technique verifies the compliance to this new oracle using test suite execution and exception injection. This document concludes with a research agenda about the future of engineering ultra-dependable and antifragile software systems
BigIssue: A Realistic Bug Localization Benchmark
As machine learning tools progress, the inevitable question arises: How can
machine learning help us write better code? With significant progress being
achieved in natural language processing with models like GPT-3 and Bert, the
applications of natural language processing techniques to code are starting to
be explored. Most of the research has been focused on automatic program repair
(APR), and while the results on synthetic or highly filtered datasets are
promising, such models are hard to apply in real-world scenarios because of
inadequate bug localization. We propose BigIssue: a benchmark for realistic bug
localization. The goal of the benchmark is two-fold. We provide (1) a general
benchmark with a diversity of real and synthetic Java bugs and (2) a motivation
to improve bug localization capabilities of models through attention to the
full repository context. With the introduction of BigIssue, we hope to advance
the state of the art in bug localization, in turn improving APR performance and
increasing its applicability to the modern development cycle
Locating Faults with Program Slicing: An Empirical Analysis
Statistical fault localization is an easily deployed technique for quickly
determining candidates for faulty code locations. If a human programmer has to
search the fault beyond the top candidate locations, though, more traditional
techniques of following dependencies along dynamic slices may be better suited.
In a large study of 457 bugs (369 single faults and 88 multiple faults) in 46
open source C programs, we compare the effectiveness of statistical fault
localization against dynamic slicing. For single faults, we find that dynamic
slicing was eight percentage points more effective than the best performing
statistical debugging formula; for 66% of the bugs, dynamic slicing finds the
fault earlier than the best performing statistical debugging formula. In our
evaluation, dynamic slicing is more effective for programs with single fault,
but statistical debugging performs better on multiple faults. Best results,
however, are obtained by a hybrid approach: If programmers first examine at
most the top five most suspicious locations from statistical debugging, and
then switch to dynamic slices, on average, they will need to examine 15% (30
lines) of the code. These findings hold for 18 most effective statistical
debugging formulas and our results are independent of the number of faults
(i.e. single or multiple faults) and error type (i.e. artificial or real
errors)
- …