1,518 research outputs found
BiLO-CPDP: Bi-Level Programming for Automated Model Discovery in Cross-Project Defect Prediction
Cross-Project Defect Prediction (CPDP), which borrows data from similar
projects by combining a transfer learner with a classifier, have emerged as a
promising way to predict software defects when the available data about the
target project is insufficient. How-ever, developing such a model is challenge
because it is difficult to determine the right combination of transfer learner
and classifier along with their optimal hyper-parameter settings. In this
paper, we propose a tool, dubbedBiLO-CPDP, which is the first of its kind to
formulate the automated CPDP model discovery from the perspective of bi-level
programming. In particular, the bi-level programming proceeds the optimization
with two nested levels in a hierarchical manner. Specifically, the upper-level
optimization routine is designed to search for the right combination of
transfer learner and classifier while the nested lower-level optimization
routine aims to optimize the corresponding hyper-parameter settings.To
evaluateBiLO-CPDP, we conduct experiments on 20 projects to compare it with a
total of 21 existing CPDP techniques, along with its single-level optimization
variant and Auto-Sklearn, a state-of-the-art automated machine learning tool.
Empirical results show that BiLO-CPDP champions better prediction performance
than all other 21 existing CPDP techniques on 70% of the projects, while be-ing
overwhelmingly superior to Auto-Sklearn and its single-level optimization
variant on all cases. Furthermore, the unique bi-level formalization
inBiLO-CPDP also permits to allocate more budget to the upper-level, which
significantly boosts the performance
Easy over Hard: A Case Study on Deep Learning
While deep learning is an exciting new technique, the benefits of this method
need to be assessed with respect to its computational cost. This is
particularly important for deep learning since these learners need hours (to
weeks) to train the model. Such long training time limits the ability of (a)~a
researcher to test the stability of their conclusion via repeated runs with
different random seeds; and (b)~other researchers to repeat, improve, or even
refute that original work.
For example, recently, deep learning was used to find which questions in the
Stack Overflow programmer discussion forum can be linked together. That deep
learning system took 14 hours to execute. We show here that applying a very
simple optimizer called DE to fine tune SVM, it can achieve similar (and
sometimes better) results. The DE approach terminated in 10 minutes; i.e. 84
times faster hours than deep learning method.
We offer these results as a cautionary tale to the software analytics
community and suggest that not every new innovation should be applied without
critical analysis. If researchers deploy some new and expensive process, that
work should be baselined against some simpler and faster alternatives.Comment: 12 pages, 6 figures, accepted at FSE201
Learning Effective Changes for Software Projects
The primary motivation of much of software analytics is decision making. How
to make these decisions? Should one make decisions based on lessons that arise
from within a particular project? Or should one generate these decisions from
across multiple projects? This work is an attempt to answer these questions.
Our work was motivated by a realization that much of the current generation
software analytics tools focus primarily on prediction. Indeed prediction is a
useful task, but it is usually followed by "planning" about what actions need
to be taken. This research seeks to address the planning task by seeking
methods that support actionable analytics that offer clear guidance on what to
do. Specifically, we propose XTREE and BELLTREE algorithms for generating a set
of actionable plans within and across projects. Each of these plans, if
followed will improve the quality of the software project.Comment: 4 pages, 2 figures. This a submission for ASE 2017 Doctoral Symposiu
Search based software engineering: Trends, techniques and applications
© ACM, 2012. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version is available from the link below.In the past five years there has been a dramatic increase in work on Search-Based Software Engineering (SBSE), an approach to Software Engineering (SE) in which Search-Based Optimization (SBO) algorithms are used to address problems in SE. SBSE has been applied to problems throughout the SE lifecycle, from requirements and project planning to maintenance and reengineering. The approach is attractive because it offers a suite of adaptive automated and semiautomated solutions in situations typified by large complex problem spaces with multiple competing and conflicting objectives.
This article provides a review and classification of literature on SBSE. The work identifies research trends and relationships between the techniques applied and the applications to which they have been applied and highlights gaps in the literature and avenues for further research.EPSRC and E
- …