6,184 research outputs found
An Efficient Dual Approach to Distance Metric Learning
Distance metric learning is of fundamental interest in machine learning
because the distance metric employed can significantly affect the performance
of many learning methods. Quadratic Mahalanobis metric learning is a popular
approach to the problem, but typically requires solving a semidefinite
programming (SDP) problem, which is computationally expensive. Standard
interior-point SDP solvers typically have a complexity of (with
the dimension of input data), and can thus only practically solve problems
exhibiting less than a few thousand variables. Since the number of variables is
, this implies a limit upon the size of problem that can
practically be solved of around a few hundred dimensions. The complexity of the
popular quadratic Mahalanobis metric learning approach thus limits the size of
problem to which metric learning can be applied. Here we propose a
significantly more efficient approach to the metric learning problem based on
the Lagrange dual formulation of the problem. The proposed formulation is much
simpler to implement, and therefore allows much larger Mahalanobis metric
learning problems to be solved. The time complexity of the proposed method is
, which is significantly lower than that of the SDP approach.
Experiments on a variety of datasets demonstrate that the proposed method
achieves an accuracy comparable to the state-of-the-art, but is applicable to
significantly larger problems. We also show that the proposed method can be
applied to solve more general Frobenius-norm regularized SDP problems
approximately
Adaptive Graph via Multiple Kernel Learning for Nonnegative Matrix Factorization
Nonnegative Matrix Factorization (NMF) has been continuously evolving in
several areas like pattern recognition and information retrieval methods. It
factorizes a matrix into a product of 2 low-rank non-negative matrices that
will define parts-based, and linear representation of nonnegative data.
Recently, Graph regularized NMF (GrNMF) is proposed to find a compact
representation,which uncovers the hidden semantics and simultaneously respects
the intrinsic geometric structure. In GNMF, an affinity graph is constructed
from the original data space to encode the geometrical information. In this
paper, we propose a novel idea which engages a Multiple Kernel Learning
approach into refining the graph structure that reflects the factorization of
the matrix and the new data space. The GrNMF is improved by utilizing the graph
refined by the kernel learning, and then a novel kernel learning method is
introduced under the GrNMF framework. Our approach shows encouraging results of
the proposed algorithm in comparison to the state-of-the-art clustering
algorithms like NMF, GrNMF, SVD etc.Comment: This paper has been withdrawn by the author due to the terrible
writin
LDMNet: Low Dimensional Manifold Regularized Neural Networks
Deep neural networks have proved very successful on archetypal tasks for
which large training sets are available, but when the training data are scarce,
their performance suffers from overfitting. Many existing methods of reducing
overfitting are data-independent, and their efficacy is often limited when the
training set is very small. Data-dependent regularizations are mostly motivated
by the observation that data of interest lie close to a manifold, which is
typically hard to parametrize explicitly and often requires human input of
tangent vectors. These methods typically only focus on the geometry of the
input data, and do not necessarily encourage the networks to produce
geometrically meaningful features. To resolve this, we propose a new framework,
the Low-Dimensional-Manifold-regularized neural Network (LDMNet), which
incorporates a feature regularization method that focuses on the geometry of
both the input data and the output features. In LDMNet, we regularize the
network by encouraging the combination of the input data and the output
features to sample a collection of low dimensional manifolds, which are
searched efficiently without explicit parametrization. To achieve this, we
directly use the manifold dimension as a regularization term in a variational
functional. The resulting Euler-Lagrange equation is a Laplace-Beltrami
equation over a point cloud, which is solved by the point integral method
without increasing the computational complexity. We demonstrate two benefits of
LDMNet in the experiments. First, we show that LDMNet significantly outperforms
widely-used network regularizers such as weight decay and DropOut. Second, we
show that LDMNet can be designed to extract common features of an object imaged
via different modalities, which proves to be very useful in real-world
applications such as cross-spectral face recognition
- …