14,986 research outputs found
Evaluating defect prediction approaches: a benchmark and an extensive comparison
Reliably predicting software defects is one of the holy grails of software engineering. Researchers have devised and implemented a plethora of defect/bug prediction approaches varying in terms of accuracy, complexity and the input data they require. However, the absence of an established benchmark makes it hard, if not impossible, to compare approaches. We present a benchmark for defect prediction, in the form of a publicly available dataset consisting of several software systems, and provide an extensive comparison of well-known bug prediction approaches, together with novel approaches we devised. We evaluate the performance of the approaches using different performance indicators: classification of entities as defect-prone or not, ranking of the entities, with and without taking into account the effort to review an entity. We performed three sets of experiments aimed at (1) comparing the approaches across different systems, (2) testing whether the differences in performance are statistically significant, and (3) investigating the stability of approaches across different learners. Our results indicate that, while some approaches perform better than others in a statistically significant manner, external validity in defect prediction is still an open problem, as generalizing results to different contexts/learners proved to be a partially unsuccessful endeavo
What is the Connection Between Issues, Bugs, and Enhancements? (Lessons Learned from 800+ Software Projects)
Agile teams juggle multiple tasks so professionals are often assigned to
multiple projects, especially in service organizations that monitor and
maintain a large suite of software for a large user base. If we could predict
changes in project conditions changes, then managers could better adjust the
staff allocated to those projects.This paper builds such a predictor using data
from 832 open source and proprietary applications. Using a time series analysis
of the last 4 months of issues, we can forecast how many bug reports and
enhancement requests will be generated next month. The forecasts made in this
way only require a frequency count of this issue reports (and do not require an
historical record of bugs found in the project). That is, this kind of
predictive model is very easy to deploy within a project. We hence strongly
recommend this method for forecasting future issues, enhancements, and bugs in
a project.Comment: Accepted to 2018 International Conference on Software Engineering, at
the software engineering in practice track. 10 pages, 10 figure
Easy over Hard: A Case Study on Deep Learning
While deep learning is an exciting new technique, the benefits of this method
need to be assessed with respect to its computational cost. This is
particularly important for deep learning since these learners need hours (to
weeks) to train the model. Such long training time limits the ability of (a)~a
researcher to test the stability of their conclusion via repeated runs with
different random seeds; and (b)~other researchers to repeat, improve, or even
refute that original work.
For example, recently, deep learning was used to find which questions in the
Stack Overflow programmer discussion forum can be linked together. That deep
learning system took 14 hours to execute. We show here that applying a very
simple optimizer called DE to fine tune SVM, it can achieve similar (and
sometimes better) results. The DE approach terminated in 10 minutes; i.e. 84
times faster hours than deep learning method.
We offer these results as a cautionary tale to the software analytics
community and suggest that not every new innovation should be applied without
critical analysis. If researchers deploy some new and expensive process, that
work should be baselined against some simpler and faster alternatives.Comment: 12 pages, 6 figures, accepted at FSE201
Too Trivial To Test? An Inverse View on Defect Prediction to Identify Methods with Low Fault Risk
Background. Test resources are usually limited and therefore it is often not
possible to completely test an application before a release. To cope with the
problem of scarce resources, development teams can apply defect prediction to
identify fault-prone code regions. However, defect prediction tends to low
precision in cross-project prediction scenarios.
Aims. We take an inverse view on defect prediction and aim to identify
methods that can be deferred when testing because they contain hardly any
faults due to their code being "trivial". We expect that characteristics of
such methods might be project-independent, so that our approach could improve
cross-project predictions.
Method. We compute code metrics and apply association rule mining to create
rules for identifying methods with low fault risk. We conduct an empirical
study to assess our approach with six Java open-source projects containing
precise fault data at the method level.
Results. Our results show that inverse defect prediction can identify approx.
32-44% of the methods of a project to have a low fault risk; on average, they
are about six times less likely to contain a fault than other methods. In
cross-project predictions with larger, more diversified training sets,
identified methods are even eleven times less likely to contain a fault.
Conclusions. Inverse defect prediction supports the efficient allocation of
test resources by identifying methods that can be treated with less priority in
testing activities and is well applicable in cross-project prediction
scenarios.Comment: Submitted to PeerJ C
The effect of climate change on the population of sycamore lace bug (Corythuca ciliata, Say) based on a simulation model with phenological response
Climate change affects on insect populations in many ways: it can cause a shift in geographical spread, abundance, or diversity, it can change the location, the timing and the magnitude of outbreaks of pests and it can define the phenological or even the genetic properties of the species. Long-time investigations of special insect populations, simulation models and scenario studies give us very
important information about the response of the insects far away and near to our century. Getting to know
the potential responses of insect populations to climate change makes us possible to evaluate the
adaptation of pest management alternatives as well as to formulate our future management policy. In this
paper we apply two simple models, in order to introduce a complex case study for a Sycamore lace bug
population. We test how the model works in case the whether conditions are very different from those in
our days. Thus, besides we can understand the processes that happen in present, we can analyze the
effects of a possible climate change, as well
- …