16,421 research outputs found
Improved Multi-Class Cost-Sensitive Boosting via Estimation of the Minimum-Risk Class
We present a simple unified framework for multi-class cost-sensitive boosting.
The minimum-risk class is estimated directly, rather than via an approximation
of the posterior distribution. Our method jointly optimizes binary weak learners
and their corresponding output vectors, requiring classes to share features at each
iteration. By training in a cost-sensitive manner, weak learners are invested in separating
classes whose discrimination is important, at the expense of less relevant
classification boundaries. Additional contributions are a family of loss functions
along with proof that our algorithm is Boostable in the theoretical sense, as well
as an efficient procedure for growing decision trees for use as weak learners. We
evaluate our method on a variety of datasets: a collection of synthetic planar data,
common UCI datasets, MNIST digits, SUN scenes, and CUB-200 birds. Results
show state-of-the-art performance across all datasets against several strong baselines,
including non-boosting multi-class approaches
Improved Multi-Class Cost-Sensitive Boosting via Estimation of the Minimum-Risk Class
We present a simple unified framework for multi-class cost-sensitive boosting.
The minimum-risk class is estimated directly, rather than via an approximation
of the posterior distribution. Our method jointly optimizes binary weak learners
and their corresponding output vectors, requiring classes to share features at each
iteration. By training in a cost-sensitive manner, weak learners are invested in separating
classes whose discrimination is important, at the expense of less relevant
classification boundaries. Additional contributions are a family of loss functions
along with proof that our algorithm is Boostable in the theoretical sense, as well
as an efficient procedure for growing decision trees for use as weak learners. We
evaluate our method on a variety of datasets: a collection of synthetic planar data,
common UCI datasets, MNIST digits, SUN scenes, and CUB-200 birds. Results
show state-of-the-art performance across all datasets against several strong baselines,
including non-boosting multi-class approaches
A survey of cost-sensitive decision tree induction algorithms
The past decade has seen a significant interest on the problem of inducing decision trees that take account of costs of misclassification and costs of acquiring the features used for decision making. This survey identifies over 50 algorithms including approaches that are direct adaptations of accuracy based methods, use genetic algorithms, use anytime methods and utilize boosting and bagging. The survey brings together these different studies and novel approaches to cost-sensitive decision tree learning, provides a useful taxonomy, a historical timeline of how the field has developed and should provide a useful reference point for future research in this field
One-Class Classification: Taxonomy of Study and Review of Techniques
One-class classification (OCC) algorithms aim to build classification models
when the negative class is either absent, poorly sampled or not well defined.
This unique situation constrains the learning of efficient classifiers by
defining class boundary just with the knowledge of positive class. The OCC
problem has been considered and applied under many research themes, such as
outlier/novelty detection and concept learning. In this paper we present a
unified view of the general problem of OCC by presenting a taxonomy of study
for OCC problems, which is based on the availability of training data,
algorithms used and the application domains applied. We further delve into each
of the categories of the proposed taxonomy and present a comprehensive
literature review of the OCC algorithms, techniques and methodologies with a
focus on their significance, limitations and applications. We conclude our
paper by discussing some open research problems in the field of OCC and present
our vision for future research.Comment: 24 pages + 11 pages of references, 8 figure
A Survey on Metric Learning for Feature Vectors and Structured Data
The need for appropriate ways to measure the distance or similarity between
data is ubiquitous in machine learning, pattern recognition and data mining,
but handcrafting such good metrics for specific problems is generally
difficult. This has led to the emergence of metric learning, which aims at
automatically learning a metric from data and has attracted a lot of interest
in machine learning and related fields for the past ten years. This survey
paper proposes a systematic review of the metric learning literature,
highlighting the pros and cons of each approach. We pay particular attention to
Mahalanobis distance metric learning, a well-studied and successful framework,
but additionally present a wide range of methods that have recently emerged as
powerful alternatives, including nonlinear metric learning, similarity learning
and local metric learning. Recent trends and extensions, such as
semi-supervised metric learning, metric learning for histogram data and the
derivation of generalization guarantees, are also covered. Finally, this survey
addresses metric learning for structured data, in particular edit distance
learning, and attempts to give an overview of the remaining challenges in
metric learning for the years to come.Comment: Technical report, 59 pages. Changes in v2: fixed typos and improved
presentation. Changes in v3: fixed typos. Changes in v4: fixed typos and new
method
- …