15,743 research outputs found
Distance metric learning based on structural neighborhoods for dimensionality reduction and classification performance improvement
Distance metric learning can be viewed as one of the fundamental interests in
pattern recognition and machine learning, which plays a pivotal role in the
performance of many learning methods. One of the effective methods in learning
such a metric is to learn it from a set of labeled training samples. The issue
of data imbalance is the most important challenge of recent methods. This
research tries not only to preserve the local structures but also covers the
issue of imbalanced datasets. To do this, the proposed method first tries to
extract a low dimensional manifold from the input data. Then, it learns the
local neighborhood structures and the relationship of the data points in the
ambient space based on the adjacencies of the same data points on the embedded
low dimensional manifold. Using the local neighborhood relationships extracted
from the manifold space, the proposed method learns the distance metric in a
way which minimizes the distance between similar data and maximizes their
distance from the dissimilar data points. The evaluations of the proposed
method on numerous datasets from the UCI repository of machine learning, and
also the KDDCup98 dataset as the most imbalance dataset, justify the supremacy
of the proposed approach in comparison with other approaches especially when
the imbalance factor is high.Comment: 30 pages, 5 figure
Dynamic Metric Learning from Pairwise Comparisons
Recent work in distance metric learning has focused on learning
transformations of data that best align with specified pairwise similarity and
dissimilarity constraints, often supplied by a human observer. The learned
transformations lead to improved retrieval, classification, and clustering
algorithms due to the better adapted distance or similarity measures. Here, we
address the problem of learning these transformations when the underlying
constraint generation process is nonstationary. This nonstationarity can be due
to changes in either the ground-truth clustering used to generate constraints
or changes in the feature subspaces in which the class structure is apparent.
We propose Online Convex Ensemble StrongLy Adaptive Dynamic Learning (OCELAD),
a general adaptive, online approach for learning and tracking optimal metrics
as they change over time that is highly robust to a variety of nonstationary
behaviors in the changing metric. We apply the OCELAD framework to an ensemble
of online learners. Specifically, we create a retro-initialized composite
objective mirror descent (COMID) ensemble (RICE) consisting of a set of
parallel COMID learners with different learning rates, demonstrate RICE-OCELAD
on both real and synthetic data sets and show significant performance
improvements relative to previously proposed batch and online distance metric
learning algorithms.Comment: to appear Allerton 2016. arXiv admin note: substantial text overlap
with arXiv:1603.0367
Metric Embedding for Nearest Neighbor Classification
The distance metric plays an important role in nearest neighbor (NN)
classification. Usually the Euclidean distance metric is assumed or a
Mahalanobis distance metric is optimized to improve the NN performance. In this
paper, we study the problem of embedding arbitrary metric spaces into a
Euclidean space with the goal to improve the accuracy of the NN classifier. We
propose a solution by appealing to the framework of regularization in a
reproducing kernel Hilbert space and prove a representer-like theorem for NN
classification. The embedding function is then determined by solving a
semidefinite program which has an interesting connection to the soft-margin
linear binary support vector machine classifier. Although the main focus of
this paper is to present a general, theoretical framework for metric embedding
in a NN setting, we demonstrate the performance of the proposed method on some
benchmark datasets and show that it performs better than the Mahalanobis metric
learning algorithm in terms of leave-one-out and generalization errors.Comment: 9 pages, 1 tabl
Alignment Distances on Systems of Bags
Recent research in image and video recognition indicates that many visual
processes can be thought of as being generated by a time-varying generative
model. A nearby descriptive model for visual processes is thus a statistical
distribution that varies over time. Specifically, modeling visual processes as
streams of histograms generated by a kernelized linear dynamic system turns out
to be efficient. We refer to such a model as a System of Bags. In this work, we
investigate Systems of Bags with special emphasis on dynamic scenes and dynamic
textures. Parameters of linear dynamic systems suffer from ambiguities. In
order to cope with these ambiguities in the kernelized setting, we develop a
kernelized version of the alignment distance. For its computation, we use a
Jacobi-type method and prove its convergence to a set of critical points. We
employ it as a dissimilarity measure on Systems of Bags. As such, it
outperforms other known dissimilarity measures for kernelized linear dynamic
systems, in particular the Martin Distance and the Maximum Singular Value
Distance, in every tested classification setting. A considerable margin can be
observed in settings, where classification is performed with respect to an
abstract mean of video sets. For this scenario, the presented approach can
outperform state-of-the-art techniques, such as Dynamic Fractal Spectrum or
Orthogonal Tensor Dictionary Learning
Distance Metric Learning using Graph Convolutional Networks: Application to Functional Brain Networks
Evaluating similarity between graphs is of major importance in several
computer vision and pattern recognition problems, where graph representations
are often used to model objects or interactions between elements. The choice of
a distance or similarity metric is, however, not trivial and can be highly
dependent on the application at hand. In this work, we propose a novel metric
learning method to evaluate distance between graphs that leverages the power of
convolutional neural networks, while exploiting concepts from spectral graph
theory to allow these operations on irregular graphs. We demonstrate the
potential of our method in the field of connectomics, where neuronal pathways
or functional connections between brain regions are commonly modelled as
graphs. In this problem, the definition of an appropriate graph similarity
function is critical to unveil patterns of disruptions associated with certain
brain disorders. Experimental results on the ABIDE dataset show that our method
can learn a graph similarity metric tailored for a clinical application,
improving the performance of a simple k-nn classifier by 11.9% compared to a
traditional distance metric.Comment: International Conference on Medical Image Computing and
Computer-Assisted Interventions (MICCAI) 201
Survey of data mining approaches to user modeling for adaptive hypermedia
The ability of an adaptive hypermedia system to create tailored environments depends mainly on the amount and accuracy of information stored in each user model. Some of the difficulties that user modeling faces are the amount of data available to create user models, the adequacy of the data, the noise within that data, and the necessity of capturing the imprecise nature of human behavior. Data mining and machine learning techniques have the ability to handle large amounts of data and to process uncertainty. These characteristics make these techniques suitable for automatic generation of user models that simulate human decision making. This paper surveys different data mining techniques that can be used to efficiently and accurately capture user behavior. The paper also presents guidelines that show which techniques may be used more efficiently according to the task implemented by the applicatio
Metric and Kernel Learning using a Linear Transformation
Metric and kernel learning are important in several machine learning
applications. However, most existing metric learning algorithms are limited to
learning metrics over low-dimensional data, while existing kernel learning
algorithms are often limited to the transductive setting and do not generalize
to new data points. In this paper, we study metric learning as a problem of
learning a linear transformation of the input data. We show that for
high-dimensional data, a particular framework for learning a linear
transformation of the data based on the LogDet divergence can be efficiently
kernelized to learn a metric (or equivalently, a kernel function) over an
arbitrarily high dimensional space. We further demonstrate that a wide class of
convex loss functions for learning linear transformations can similarly be
kernelized, thereby considerably expanding the potential applications of metric
learning. We demonstrate our learning approach by applying it to large-scale
real world problems in computer vision and text mining
DeepSafe: A Data-driven Approach for Checking Adversarial Robustness in Neural Networks
Deep neural networks have become widely used, obtaining remarkable results in
domains such as computer vision, speech recognition, natural language
processing, audio recognition, social network filtering, machine translation,
and bio-informatics, where they have produced results comparable to human
experts. However, these networks can be easily fooled by adversarial
perturbations: minimal changes to correctly-classified inputs, that cause the
network to mis-classify them. This phenomenon represents a concern for both
safety and security, but it is currently unclear how to measure a network's
robustness against such perturbations. Existing techniques are limited to
checking robustness around a few individual input points, providing only very
limited guarantees. We propose a novel approach for automatically identifying
safe regions of the input space, within which the network is robust against
adversarial perturbations. The approach is data-guided, relying on clustering
to identify well-defined geometric regions as candidate safe regions. We then
utilize verification techniques to confirm that these regions are safe or to
provide counter-examples showing that they are not safe. We also introduce the
notion of targeted robustness which, for a given target label and region,
ensures that a NN does not map any input in the region to the target label. We
evaluated our technique on the MNIST dataset and on a neural network
implementation of a controller for the next-generation Airborne Collision
Avoidance System for unmanned aircraft (ACAS Xu). For these networks, our
approach identified multiple regions which were completely safe as well as some
which were only safe for specific labels. It also discovered several
adversarial perturbations of interest
Exploit Bounding Box Annotations for Multi-label Object Recognition
Convolutional neural networks (CNNs) have shown great performance as general
feature representations for object recognition applications. However, for
multi-label images that contain multiple objects from different categories,
scales and locations, global CNN features are not optimal. In this paper, we
incorporate local information to enhance the feature discriminative power. In
particular, we first extract object proposals from each image. With each image
treated as a bag and object proposals extracted from it treated as instances,
we transform the multi-label recognition problem into a multi-class
multi-instance learning problem. Then, in addition to extracting the typical
CNN feature representation from each proposal, we propose to make use of
ground-truth bounding box annotations (strong labels) to add another level of
local information by using nearest-neighbor relationships of local regions to
form a multi-view pipeline. The proposed multi-view multi-instance framework
utilizes both weak and strong labels effectively, and more importantly it has
the generalization ability to even boost the performance of unseen categories
by partial strong labels from other categories. Our framework is extensively
compared with state-of-the-art hand-crafted feature based methods and CNN based
methods on two multi-label benchmark datasets. The experimental results
validate the discriminative power and the generalization ability of the
proposed framework. With strong labels, our framework is able to achieve
state-of-the-art results in both datasets.Comment: Accepted in CVPR 201
Ignorance-Aware Approaches and Algorithms for Prototype Selection in Machine Learning
Operating with ignorance is an important concern of the Machine Learning
research, especially when the objective is to discover knowledge from the
imperfect data. Data mining (driven by appropriate knowledge discovery tools)
is about processing available (observed, known and understood) samples of data
aiming to build a model (e.g., a classifier) to handle data samples, which are
not yet observed, known or understood. These tools traditionally take samples
of the available data (known facts) as an input for learning. We want to
challenge the indispensability of this approach and we suggest considering the
things the other way around. What if the task would be as follows: how to learn
a model based on our ignorance, i.e. by processing the shape of 'voids' within
the available data space? Can we improve traditional classification by modeling
also the ignorance? In this paper, we provide some algorithms for the discovery
and visualizing of the ignorance zones in two-dimensional data spaces and
design two ignorance-aware smart prototype selection techniques (incremental
and adversarial) to improve the performance of the nearest neighbor
classifiers. We present experiments with artificial and real datasets to test
the concept of the usefulness of ignorance discovery in machine learning
- …