33,348 research outputs found

    Is "Better Data" Better than "Better Data Miners"? (On the Benefits of Tuning SMOTE for Defect Prediction)

    Full text link
    We report and fix an important systematic error in prior studies that ranked classifiers for software analytics. Those studies did not (a) assess classifiers on multiple criteria and they did not (b) study how variations in the data affect the results. Hence, this paper applies (a) multi-criteria tests while (b) fixing the weaker regions of the training data (using SMOTUNED, which is a self-tuning version of SMOTE). This approach leads to dramatically large increases in software defect predictions. When applied in a 5*5 cross-validation study for 3,681 JAVA classes (containing over a million lines of code) from open source systems, SMOTUNED increased AUC and recall by 60% and 20% respectively. These improvements are independent of the classifier used to predict for quality. Same kind of pattern (improvement) was observed when a comparative analysis of SMOTE and SMOTUNED was done against the most recent class imbalance technique. In conclusion, for software analytic tasks like defect prediction, (1) data pre-processing can be more important than classifier choice, (2) ranking studies are incomplete without such pre-processing, and (3) SMOTUNED is a promising candidate for pre-processing.Comment: 10 pages + 2 references. Accepted to International Conference of Software Engineering (ICSE), 201

    Is "Better Data" Better than "Better Data Miners"? (On the Benefits of Tuning SMOTE for Defect Prediction)

    Full text link
    We report and fix an important systematic error in prior studies that ranked classifiers for software analytics. Those studies did not (a) assess classifiers on multiple criteria and they did not (b) study how variations in the data affect the results. Hence, this paper applies (a) multi-criteria tests while (b) fixing the weaker regions of the training data (using SMOTUNED, which is a self-tuning version of SMOTE). This approach leads to dramatically large increases in software defect predictions. When applied in a 5*5 cross-validation study for 3,681 JAVA classes (containing over a million lines of code) from open source systems, SMOTUNED increased AUC and recall by 60% and 20% respectively. These improvements are independent of the classifier used to predict for quality. Same kind of pattern (improvement) was observed when a comparative analysis of SMOTE and SMOTUNED was done against the most recent class imbalance technique. In conclusion, for software analytic tasks like defect prediction, (1) data pre-processing can be more important than classifier choice, (2) ranking studies are incomplete without such pre-processing, and (3) SMOTUNED is a promising candidate for pre-processing.Comment: 10 pages + 2 references. Accepted to International Conference of Software Engineering (ICSE), 201

    Technical Debt Prioritization: State of the Art. A Systematic Literature Review

    Get PDF
    Background. Software companies need to manage and refactor Technical Debt issues. Therefore, it is necessary to understand if and when refactoring Technical Debt should be prioritized with respect to developing features or fixing bugs. Objective. The goal of this study is to investigate the existing body of knowledge in software engineering to understand what Technical Debt prioritization approaches have been proposed in research and industry. Method. We conducted a Systematic Literature Review among 384 unique papers published until 2018, following a consolidated methodology applied in Software Engineering. We included 38 primary studies. Results. Different approaches have been proposed for Technical Debt prioritization, all having different goals and optimizing on different criteria. The proposed measures capture only a small part of the plethora of factors used to prioritize Technical Debt qualitatively in practice. We report an impact map of such factors. However, there is a lack of empirical and validated set of tools. Conclusion. We observed that technical Debt prioritization research is preliminary and there is no consensus on what are the important factors and how to measure them. Consequently, we cannot consider current research conclusive and in this paper, we outline different directions for necessary future investigations

    Parametric Local Metric Learning for Nearest Neighbor Classification

    Full text link
    We study the problem of learning local metrics for nearest neighbor classification. Most previous works on local metric learning learn a number of local unrelated metrics. While this "independence" approach delivers an increased flexibility its downside is the considerable risk of overfitting. We present a new parametric local metric learning method in which we learn a smooth metric matrix function over the data manifold. Using an approximation error bound of the metric matrix function we learn local metrics as linear combinations of basis metrics defined on anchor points over different regions of the instance space. We constrain the metric matrix function by imposing on the linear combinations manifold regularization which makes the learned metric matrix function vary smoothly along the geodesics of the data manifold. Our metric learning method has excellent performance both in terms of predictive power and scalability. We experimented with several large-scale classification problems, tens of thousands of instances, and compared it with several state of the art metric learning methods, both global and local, as well as to SVM with automatic kernel selection, all of which it outperforms in a significant manner

    A Survey on Metric Learning for Feature Vectors and Structured Data

    Full text link
    The need for appropriate ways to measure the distance or similarity between data is ubiquitous in machine learning, pattern recognition and data mining, but handcrafting such good metrics for specific problems is generally difficult. This has led to the emergence of metric learning, which aims at automatically learning a metric from data and has attracted a lot of interest in machine learning and related fields for the past ten years. This survey paper proposes a systematic review of the metric learning literature, highlighting the pros and cons of each approach. We pay particular attention to Mahalanobis distance metric learning, a well-studied and successful framework, but additionally present a wide range of methods that have recently emerged as powerful alternatives, including nonlinear metric learning, similarity learning and local metric learning. Recent trends and extensions, such as semi-supervised metric learning, metric learning for histogram data and the derivation of generalization guarantees, are also covered. Finally, this survey addresses metric learning for structured data, in particular edit distance learning, and attempts to give an overview of the remaining challenges in metric learning for the years to come.Comment: Technical report, 59 pages. Changes in v2: fixed typos and improved presentation. Changes in v3: fixed typos. Changes in v4: fixed typos and new method

    Competing Dimensions of Energy Security: An International Perspective

    Get PDF
    How well are industrialized nations doing in terms of their energy security? Without a standardized set of metrics, it is difficult to determine the extent that countries are properly responding to the emerging energy security challenges related to climate change, growing dependence on fossil fuels, population growth and economic development. In response, we propose the creation of an Energy Security Index to inform policymakers, investors and analysts about the status of energy conditions. Using the United States and 21 other member countries of the Organization for Economic Cooperation and Development (OECD) as an example, and looking at energy security from 1970 to 2007, our index shows that only four countries¡ªBelgium, Denmark, Japan, and the United Kingdom¡ªhave made progress on multiple dimensions of the energy security problem. The remaining 18 have either made no improvement or are less secure. To make this argument, the first section of the article surveys the scholarly literature on energy security from 2003 to 2008 and argues that an index should address accessibility, affordability, efficiency, and environmental stewardship. Because each of these four components is multidimensional, the second section discusses ten metrics that comprise an Energy Security Index: oil import dependence, percentage of alternative transport fuels, on-road fuel economy for passenger vehicles, energy intensity, natural gas import dependence, electricity prices, gasoline prices, sulfur dioxide emissions, and carbon dioxide emissions. The third section analyzes the relative performance of four countries: Denmark (the top performer), Japan (which performed well), the United States (which performed poorly), and Spain (the worst performer). The article concludes by offering implications for policy. Conflicts between energy security criteria mean that advancement along any one dimension can undermine progress on another dimension. By focusing on a 10-point index, public policy can better illuminate such tradeoffs and can identify compensating policies

    Solution and quality robust project scheduling: a methodological framework.

    Get PDF
    The vast majority of the research efforts in project scheduling over the past several years has concentrated on the development of exact and suboptimal procedures for the generation of a baseline schedule assuming complete information and a deterministic environment. During execution, however, projects may be the subject of considerable uncertainty, which may lead to numerous schedule disruptions. Predictive-reactive scheduling refers to the process where a baseline schedule is developed prior to the start of the project and updated if necessary during project execution. It is the objective of this paper to review possible procedures for the generation of proactive (robust) schedules, which are as well as possible protected against schedule disruptions, and for the deployment of reactive scheduling procedures that may be used to revise or re-optimize the baseline schedule when unexpected events occur. We also offer a methodological framework that should allow project management to identify the proper scheduling methodology for different project scheduling environments. Finally, we survey the basics of Critical Chain scheduling and indicate in which environments it is useful.Framework; Information; Management; Processes; Project management; Project scheduling; Project scheduling under uncertainty; Stability; Robust scheduling; Quality; Scheduling; Stability; Uncertainty;
    corecore