13,122 research outputs found
A Survey on Metric Learning for Feature Vectors and Structured Data
The need for appropriate ways to measure the distance or similarity between
data is ubiquitous in machine learning, pattern recognition and data mining,
but handcrafting such good metrics for specific problems is generally
difficult. This has led to the emergence of metric learning, which aims at
automatically learning a metric from data and has attracted a lot of interest
in machine learning and related fields for the past ten years. This survey
paper proposes a systematic review of the metric learning literature,
highlighting the pros and cons of each approach. We pay particular attention to
Mahalanobis distance metric learning, a well-studied and successful framework,
but additionally present a wide range of methods that have recently emerged as
powerful alternatives, including nonlinear metric learning, similarity learning
and local metric learning. Recent trends and extensions, such as
semi-supervised metric learning, metric learning for histogram data and the
derivation of generalization guarantees, are also covered. Finally, this survey
addresses metric learning for structured data, in particular edit distance
learning, and attempts to give an overview of the remaining challenges in
metric learning for the years to come.Comment: Technical report, 59 pages. Changes in v2: fixed typos and improved
presentation. Changes in v3: fixed typos. Changes in v4: fixed typos and new
method
AffinityNet: semi-supervised few-shot learning for disease type prediction
While deep learning has achieved great success in computer vision and many
other fields, currently it does not work very well on patient genomic data with
the "big p, small N" problem (i.e., a relatively small number of samples with
high-dimensional features). In order to make deep learning work with a small
amount of training data, we have to design new models that facilitate few-shot
learning. Here we present the Affinity Network Model (AffinityNet), a data
efficient deep learning model that can learn from a limited number of training
examples and generalize well. The backbone of the AffinityNet model consists of
stacked k-Nearest-Neighbor (kNN) attention pooling layers. The kNN attention
pooling layer is a generalization of the Graph Attention Model (GAM), and can
be applied to not only graphs but also any set of objects regardless of whether
a graph is given or not. As a new deep learning module, kNN attention pooling
layers can be plugged into any neural network model just like convolutional
layers. As a simple special case of kNN attention pooling layer, feature
attention layer can directly select important features that are useful for
classification tasks. Experiments on both synthetic data and cancer genomic
data from TCGA projects show that our AffinityNet model has better
generalization power than conventional neural network models with little
training data. The code is freely available at
https://github.com/BeautyOfWeb/AffinityNet .Comment: 14 pages, 6 figure
NFFT meets Krylov methods: Fast matrix-vector products for the graph Laplacian of fully connected networks
The graph Laplacian is a standard tool in data science, machine learning, and
image processing. The corresponding matrix inherits the complex structure of
the underlying network and is in certain applications densely populated. This
makes computations, in particular matrix-vector products, with the graph
Laplacian a hard task. A typical application is the computation of a number of
its eigenvalues and eigenvectors. Standard methods become infeasible as the
number of nodes in the graph is too large. We propose the use of the fast
summation based on the nonequispaced fast Fourier transform (NFFT) to perform
the dense matrix-vector product with the graph Laplacian fast without ever
forming the whole matrix. The enormous flexibility of the NFFT algorithm allows
us to embed the accelerated multiplication into Lanczos-based eigenvalues
routines or iterative linear system solvers and even consider other than the
standard Gaussian kernels. We illustrate the feasibility of our approach on a
number of test problems from image segmentation to semi-supervised learning
based on graph-based PDEs. In particular, we compare our approach with the
Nystr\"om method. Moreover, we present and test an enhanced, hybrid version of
the Nystr\"om method, which internally uses the NFFT.Comment: 28 pages, 9 figure
- …