33,348 research outputs found
Is "Better Data" Better than "Better Data Miners"? (On the Benefits of Tuning SMOTE for Defect Prediction)
We report and fix an important systematic error in prior studies that ranked
classifiers for software analytics. Those studies did not (a) assess
classifiers on multiple criteria and they did not (b) study how variations in
the data affect the results. Hence, this paper applies (a) multi-criteria tests
while (b) fixing the weaker regions of the training data (using SMOTUNED, which
is a self-tuning version of SMOTE). This approach leads to dramatically large
increases in software defect predictions. When applied in a 5*5
cross-validation study for 3,681 JAVA classes (containing over a million lines
of code) from open source systems, SMOTUNED increased AUC and recall by 60% and
20% respectively. These improvements are independent of the classifier used to
predict for quality. Same kind of pattern (improvement) was observed when a
comparative analysis of SMOTE and SMOTUNED was done against the most recent
class imbalance technique. In conclusion, for software analytic tasks like
defect prediction, (1) data pre-processing can be more important than
classifier choice, (2) ranking studies are incomplete without such
pre-processing, and (3) SMOTUNED is a promising candidate for pre-processing.Comment: 10 pages + 2 references. Accepted to International Conference of
Software Engineering (ICSE), 201
Is "Better Data" Better than "Better Data Miners"? (On the Benefits of Tuning SMOTE for Defect Prediction)
We report and fix an important systematic error in prior studies that ranked
classifiers for software analytics. Those studies did not (a) assess
classifiers on multiple criteria and they did not (b) study how variations in
the data affect the results. Hence, this paper applies (a) multi-criteria tests
while (b) fixing the weaker regions of the training data (using SMOTUNED, which
is a self-tuning version of SMOTE). This approach leads to dramatically large
increases in software defect predictions. When applied in a 5*5
cross-validation study for 3,681 JAVA classes (containing over a million lines
of code) from open source systems, SMOTUNED increased AUC and recall by 60% and
20% respectively. These improvements are independent of the classifier used to
predict for quality. Same kind of pattern (improvement) was observed when a
comparative analysis of SMOTE and SMOTUNED was done against the most recent
class imbalance technique. In conclusion, for software analytic tasks like
defect prediction, (1) data pre-processing can be more important than
classifier choice, (2) ranking studies are incomplete without such
pre-processing, and (3) SMOTUNED is a promising candidate for pre-processing.Comment: 10 pages + 2 references. Accepted to International Conference of
Software Engineering (ICSE), 201
Technical Debt Prioritization: State of the Art. A Systematic Literature Review
Background. Software companies need to manage and refactor Technical Debt
issues. Therefore, it is necessary to understand if and when refactoring
Technical Debt should be prioritized with respect to developing features or
fixing bugs. Objective. The goal of this study is to investigate the existing
body of knowledge in software engineering to understand what Technical Debt
prioritization approaches have been proposed in research and industry. Method.
We conducted a Systematic Literature Review among 384 unique papers published
until 2018, following a consolidated methodology applied in Software
Engineering. We included 38 primary studies. Results. Different approaches have
been proposed for Technical Debt prioritization, all having different goals and
optimizing on different criteria. The proposed measures capture only a small
part of the plethora of factors used to prioritize Technical Debt qualitatively
in practice. We report an impact map of such factors. However, there is a lack
of empirical and validated set of tools. Conclusion. We observed that technical
Debt prioritization research is preliminary and there is no consensus on what
are the important factors and how to measure them. Consequently, we cannot
consider current research conclusive and in this paper, we outline different
directions for necessary future investigations
Parametric Local Metric Learning for Nearest Neighbor Classification
We study the problem of learning local metrics for nearest neighbor
classification. Most previous works on local metric learning learn a number of
local unrelated metrics. While this "independence" approach delivers an
increased flexibility its downside is the considerable risk of overfitting. We
present a new parametric local metric learning method in which we learn a
smooth metric matrix function over the data manifold. Using an approximation
error bound of the metric matrix function we learn local metrics as linear
combinations of basis metrics defined on anchor points over different regions
of the instance space. We constrain the metric matrix function by imposing on
the linear combinations manifold regularization which makes the learned metric
matrix function vary smoothly along the geodesics of the data manifold. Our
metric learning method has excellent performance both in terms of predictive
power and scalability. We experimented with several large-scale classification
problems, tens of thousands of instances, and compared it with several state of
the art metric learning methods, both global and local, as well as to SVM with
automatic kernel selection, all of which it outperforms in a significant
manner
A Survey on Metric Learning for Feature Vectors and Structured Data
The need for appropriate ways to measure the distance or similarity between
data is ubiquitous in machine learning, pattern recognition and data mining,
but handcrafting such good metrics for specific problems is generally
difficult. This has led to the emergence of metric learning, which aims at
automatically learning a metric from data and has attracted a lot of interest
in machine learning and related fields for the past ten years. This survey
paper proposes a systematic review of the metric learning literature,
highlighting the pros and cons of each approach. We pay particular attention to
Mahalanobis distance metric learning, a well-studied and successful framework,
but additionally present a wide range of methods that have recently emerged as
powerful alternatives, including nonlinear metric learning, similarity learning
and local metric learning. Recent trends and extensions, such as
semi-supervised metric learning, metric learning for histogram data and the
derivation of generalization guarantees, are also covered. Finally, this survey
addresses metric learning for structured data, in particular edit distance
learning, and attempts to give an overview of the remaining challenges in
metric learning for the years to come.Comment: Technical report, 59 pages. Changes in v2: fixed typos and improved
presentation. Changes in v3: fixed typos. Changes in v4: fixed typos and new
method
Competing Dimensions of Energy Security: An International Perspective
How well are industrialized nations doing in terms of their energy security? Without a standardized set of metrics, it is difficult to determine the extent that countries are properly responding to the emerging energy security challenges related to climate change, growing dependence on fossil fuels, population growth and economic development. In response, we propose the creation of an Energy Security Index to inform policymakers, investors and analysts about the status of energy conditions. Using the United States and 21 other member countries of the Organization for Economic Cooperation and Development (OECD) as an example, and looking at energy security from 1970 to 2007, our index shows that only four countries¡ªBelgium, Denmark, Japan, and the United Kingdom¡ªhave made progress on multiple dimensions of the energy security problem. The remaining 18 have either made no improvement or are less secure. To make this argument, the first section of the article surveys the scholarly literature on energy security from 2003 to 2008 and argues that an index should address accessibility, affordability, efficiency, and environmental stewardship. Because each of these four components is multidimensional, the second section discusses ten metrics that comprise an Energy Security Index: oil import dependence, percentage of alternative transport fuels, on-road fuel economy for passenger vehicles, energy intensity, natural gas import dependence, electricity prices, gasoline prices, sulfur dioxide emissions, and carbon dioxide emissions. The third section analyzes the relative performance of four countries: Denmark (the top performer), Japan (which performed well), the United States (which performed poorly), and Spain (the worst performer). The article concludes by offering implications for policy. Conflicts between energy security criteria mean that advancement along any one dimension can undermine progress on another dimension. By focusing on a 10-point index, public policy can better illuminate such tradeoffs and can identify compensating policies
Solution and quality robust project scheduling: a methodological framework.
The vast majority of the research efforts in project scheduling over the past several years has concentrated on the development of exact and suboptimal procedures for the generation of a baseline schedule assuming complete information and a deterministic environment. During execution, however, projects may be the subject of considerable uncertainty, which may lead to numerous schedule disruptions. Predictive-reactive scheduling refers to the process where a baseline schedule is developed prior to the start of the project and updated if necessary during project execution. It is the objective of this paper to review possible procedures for the generation of proactive (robust) schedules, which are as well as possible protected against schedule disruptions, and for the deployment of reactive scheduling procedures that may be used to revise or re-optimize the baseline schedule when unexpected events occur. We also offer a methodological framework that should allow project management to identify the proper scheduling methodology for different project scheduling environments. Finally, we survey the basics of Critical Chain scheduling and indicate in which environments it is useful.Framework; Information; Management; Processes; Project management; Project scheduling; Project scheduling under uncertainty; Stability; Robust scheduling; Quality; Scheduling; Stability; Uncertainty;
- …