19,225 research outputs found
A Metric-learning based framework for Support Vector Machines and Multiple Kernel Learning
Most metric learning algorithms, as well as Fisher's Discriminant Analysis
(FDA), optimize some cost function of different measures of within-and
between-class distances. On the other hand, Support Vector Machines(SVMs) and
several Multiple Kernel Learning (MKL) algorithms are based on the SVM large
margin theory. Recently, SVMs have been analyzed from SVM and metric learning,
and to develop new algorithms that build on the strengths of each. Inspired by
the metric learning interpretation of SVM, we develop here a new
metric-learning based SVM framework in which we incorporate metric learning
concepts within SVM. We extend the optimization problem of SVM to include some
measure of the within-class distance and along the way we develop a new
within-class distance measure which is appropriate for SVM. In addition, we
adopt the same approach for MKL and show that it can be also formulated as a
Mahalanobis metric learning problem. Our end result is a number of SVM/MKL
algorithms that incorporate metric learning concepts. We experiment with them
on a set of benchmark datasets and observe important predictive performance
improvements
Reproducing Kernel Hilbert Space, Mercer's Theorem, Eigenfunctions, Nystr\"om Method, and Use of Kernels in Machine Learning: Tutorial and Survey
This is a tutorial and survey paper on kernels, kernel methods, and related
fields. We start with reviewing the history of kernels in functional analysis
and machine learning. Then, Mercer kernel, Hilbert and Banach spaces,
Reproducing Kernel Hilbert Space (RKHS), Mercer's theorem and its proof,
frequently used kernels, kernel construction from distance metric, important
classes of kernels (including bounded, integrally positive definite, universal,
stationary, and characteristic kernels), kernel centering and normalization,
and eigenfunctions are explained in detail. Then, we introduce types of use of
kernels in machine learning including kernel methods (such as kernel support
vector machines), kernel learning by semi-definite programming, Hilbert-Schmidt
independence criterion, maximum mean discrepancy, kernel mean embedding, and
kernel dimensionality reduction. We also cover rank and factorization of kernel
matrix as well as the approximation of eigenfunctions and kernels using the
Nystr{\"o}m method. This paper can be useful for various fields of science
including machine learning, dimensionality reduction, functional analysis in
mathematics, and mathematical physics in quantum mechanics.Comment: To appear as a part of an upcoming textbook on dimensionality
reduction and manifold learnin
A Kernel Classification Framework for Metric Learning
Learning a distance metric from the given training samples plays a crucial
role in many machine learning tasks, and various models and optimization
algorithms have been proposed in the past decade. In this paper, we generalize
several state-of-the-art metric learning methods, such as large margin nearest
neighbor (LMNN) and information theoretic metric learning (ITML), into a kernel
classification framework. First, doublets and triplets are constructed from the
training samples, and a family of degree-2 polynomial kernel functions are
proposed for pairs of doublets or triplets. Then, a kernel classification
framework is established, which can not only generalize many popular metric
learning methods such as LMNN and ITML, but also suggest new metric learning
methods, which can be efficiently implemented, interestingly, by using the
standard support vector machine (SVM) solvers. Two novel metric learning
methods, namely doublet-SVM and triplet-SVM, are then developed under the
proposed framework. Experimental results show that doublet-SVM and triplet-SVM
achieve competitive classification accuracies with state-of-the-art metric
learning methods such as ITML and LMNN but with significantly less training
time.Comment: 11 pages, 7 figure
Parametic Classification of Handvein Patterns Based on Texture Features
In this paper, we have developed Biometric recognition system adopting hand
based modality Handvein, which has the unique pattern for each individual and
it is impossible to counterfeit and fabricate as it is an internal feature. We
have opted in choosing feature extraction algorithms such as LBP-visual
descriptor ,LPQ-blur insensitive texture operator, Log-Gabor-Texture
descriptor. We have chosen well known classifiers such as KNN and SVM for
classification. We have experimented and tabulated results of single algorithm
recognition rate for Handvein under different distance measures and kernel
options. The feature level fusion is carried out which increased the
performance level.Comment: 8 pages, International Conference on Electrical, Electronics,
Materials and Applied Science (ICEEMAS). AIP: Proceedings International
Conference on Electrical, Electronics, Materials and Applied Science
(ICEEMAS),22nd and 23rd December 201
Classifying Network Data with Deep Kernel Machines
Inspired by a growing interest in analyzing network data, we study the
problem of node classification on graphs, focusing on approaches based on
kernel machines. Conventionally, kernel machines are linear classifiers in the
implicit feature space. We argue that linear classification in the feature
space of kernels commonly used for graphs is often not enough to produce good
results. When this is the case, one naturally considers nonlinear classifiers
in the feature space. We show that repeating this process produces something we
call "deep kernel machines." We provide some examples where deep kernel
machines can make a big difference in classification performance, and point out
some connections to various recent literature on deep architectures in
artificial intelligence and machine learning
Probabilistic classifiers with low rank indefinite kernels
Indefinite similarity measures can be frequently found in bio-informatics by
means of alignment scores, but are also common in other fields like shape
measures in image retrieval. Lacking an underlying vector space, the data are
given as pairwise similarities only. The few algorithms available for such data
do not scale to larger datasets. Focusing on probabilistic batch classifiers,
the Indefinite Kernel Fisher Discriminant (iKFD) and the Probabilistic
Classification Vector Machine (PCVM) are both effective algorithms for this
type of data but, with cubic complexity. Here we propose an extension of iKFD
and PCVM such that linear runtime and memory complexity is achieved for low
rank indefinite kernels. Employing the Nystr\"om approximation for indefinite
kernels, we also propose a new almost parameter free approach to identify the
landmarks, restricted to a supervised learning problem. Evaluations at several
larger similarity data from various domains show that the proposed methods
provides similar generalization capabilities while being easier to parametrize
and substantially faster for large scale data
Blindfold learning of an accurate neural metric
The brain has no direct access to physical stimuli, but only to the spiking
activity evoked in sensory organs. It is unclear how the brain can structure
its representation of the world based on differences between those noisy,
correlated responses alone. Here we show how to build a distance map of
responses from the structure of the population activity of retinal ganglion
cells, allowing for the accurate discrimination of distinct visual stimuli from
the retinal response. We introduce the Temporal Restricted Boltzmann Machine to
learn the spatiotemporal structure of the population activity, and use this
model to define a distance between spike trains. We show that this metric
outperforms existing neural distances at discriminating pairs of stimuli that
are barely distinguishable. The proposed method provides a generic and
biologically plausible way to learn to associate similar stimuli based on their
spiking responses, without any other knowledge of these stimuli
Learning Local Invariant Mahalanobis Distances
For many tasks and data types, there are natural transformations to which the
data should be invariant or insensitive. For instance, in visual recognition,
natural images should be insensitive to rotation and translation. This
requirement and its implications have been important in many machine learning
applications, and tolerance for image transformations was primarily achieved by
using robust feature vectors. In this paper we propose a novel and
computationally efficient way to learn a local Mahalanobis metric per datum,
and show how we can learn a local invariant metric to any transformation in
order to improve performance
Deep Transductive Semi-supervised Maximum Margin Clustering
Semi-supervised clustering is an very important topic in machine learning and
computer vision. The key challenge of this problem is how to learn a metric,
such that the instances sharing the same label are more likely close to each
other on the embedded space. However, little attention has been paid to learn
better representations when the data lie on non-linear manifold. Fortunately,
deep learning has led to great success on feature learning recently. Inspired
by the advances of deep learning, we propose a deep transductive
semi-supervised maximum margin clustering approach. More specifically, given
pairwise constraints, we exploit both labeled and unlabeled data to learn a
non-linear mapping under maximum margin framework for clustering analysis.
Thus, our model unifies transductive learning, feature learning and maximum
margin techniques in the semi-supervised clustering framework. We pretrain the
deep network structure with restricted Boltzmann machines (RBMs) layer by layer
greedily, and optimize our objective function with gradient descent. By
checking the most violated constraints, our approach updates the model
parameters through error backpropagation, in which deep features are learned
automatically. The experimental results shows that our model is significantly
better than the state of the art on semi-supervised clustering.Comment: 1
Multiscale CNN based Deep Metric Learning for Bioacoustic Classification: Overcoming Training Data Scarcity Using Dynamic Triplet Loss
This paper proposes multiscale convolutional neural network (CNN)-based deep
metric learning for bioacoustic classification, under low training data
conditions. The proposed CNN is characterized by the utilization of four
different filter sizes at each level to analyze input feature maps. This
multiscale nature helps in describing different bioacoustic events effectively:
smaller filters help in learning the finer details of bioacoustic events,
whereas, larger filters help in analyzing a larger context leading to global
details. A dynamic triplet loss is employed in the proposed CNN architecture to
learn a transformation from the input space to the embedding space, where
classification is performed. The triplet loss helps in learning this
transformation by analyzing three examples, referred to as triplets, at a time
where intra-class distance is minimized while maximizing the inter-class
separation by a dynamically increasing margin. The number of possible triplets
increases cubically with the dataset size, making triplet loss more suitable
than the softmax cross-entropy loss in low training data conditions.
Experiments on three different publicly available datasets show that the
proposed framework performs better than existing bioacoustic classification
frameworks. Experimental results also confirm the superiority of the triplet
loss over the cross-entropy loss in low training data conditionsComment: Under Review at JASA. Primitive version of paper. We are still
working on getting better performances out of the comparative method
- …