111,076 research outputs found
Large-scale Distance Metric Learning with Uncertainty
Distance metric learning (DML) has been studied extensively in the past
decades for its superior performance with distance-based algorithms. Most of
the existing methods propose to learn a distance metric with pairwise or
triplet constraints. However, the number of constraints is quadratic or even
cubic in the number of the original examples, which makes it challenging for
DML to handle the large-scale data set. Besides, the real-world data may
contain various uncertainty, especially for the image data. The uncertainty can
mislead the learning procedure and cause the performance degradation. By
investigating the image data, we find that the original data can be observed
from a small set of clean latent examples with different distortions. In this
work, we propose the margin preserving metric learning framework to learn the
distance metric and latent examples simultaneously. By leveraging the ideal
properties of latent examples, the training efficiency can be improved
significantly while the learned metric also becomes robust to the uncertainty
in the original data. Furthermore, we can show that the metric is learned from
latent examples only, but it can preserve the large margin property even for
the original data. The empirical study on the benchmark image data sets
demonstrates the efficacy and efficiency of the proposed method.Comment: accepted by CVPR'1
Bayesian Neighbourhood Component Analysis
Learning a good distance metric in feature space potentially improves the
performance of the KNN classifier and is useful in many real-world
applications. Many metric learning algorithms are however based on the point
estimation of a quadratic optimization problem, which is time-consuming,
susceptible to overfitting, and lack a natural mechanism to reason with
parameter uncertainty, an important property useful especially when the
training set is small and/or noisy. To deal with these issues, we present a
novel Bayesian metric learning method, called Bayesian NCA, based on the
well-known Neighbourhood Component Analysis method, in which the metric
posterior is characterized by the local label consistency constraints of
observations, encoded with a similarity graph instead of independent pairwise
constraints. For efficient Bayesian optimization, we explore the variational
lower bound over the log-likelihood of the original NCA objective. Experiments
on several publicly available datasets demonstrate that the proposed method is
able to learn robust metric measures from small size dataset and/or from
challenging training set with labels contaminated by errors. The proposed
method is also shown to outperform a previous pairwise constrained Bayesian
metric learning method
The Anchors Hierachy: Using the triangle inequality to survive high dimensional data
This paper is about metric data structures in high-dimensional or
non-Euclidean space that permit cached sufficient statistics accelerations of
learning algorithms.
It has recently been shown that for less than about 10 dimensions, decorating
kd-trees with additional "cached sufficient statistics" such as first and
second moments and contingency tables can provide satisfying acceleration for a
very wide range of statistical learning tasks such as kernel regression,
locally weighted regression, k-means clustering, mixture modeling and Bayes Net
learning.
In this paper, we begin by defining the anchors hierarchy - a fast data
structure and algorithm for localizing data based only on a
triangle-inequality-obeying distance metric. We show how this, in its own
right, gives a fast and effective clustering of data. But more importantly we
show how it can produce a well-balanced structure similar to a Ball-Tree
(Omohundro, 1991) or a kind of metric tree (Uhlmann, 1991; Ciaccia, Patella, &
Zezula, 1997) in a way that is neither "top-down" nor "bottom-up" but instead
"middle-out". We then show how this structure, decorated with cached sufficient
statistics, allows a wide variety of statistical learning algorithms to be
accelerated even in thousands of dimensions.Comment: Appears in Proceedings of the Sixteenth Conference on Uncertainty in
Artificial Intelligence (UAI2000
A Bayesian Model for Supervised Clustering with the Dirichlet Process Prior
We develop a Bayesian framework for tackling the supervised clustering
problem, the generic problem encountered in tasks such as reference matching,
coreference resolution, identity uncertainty and record linkage. Our clustering
model is based on the Dirichlet process prior, which enables us to define
distributions over the countably infinite sets that naturally arise in this
problem. We add supervision to our model by positing the existence of a set of
unobserved random variables (we call these "reference types") that are generic
across all clusters. Inference in our framework, which requires integrating
over infinitely many parameters, is solved using Markov chain Monte Carlo
techniques. We present algorithms for both conjugate and non-conjugate priors.
We present a simple--but general--parameterization of our model based on a
Gaussian assumption. We evaluate this model on one artificial task and three
real-world tasks, comparing it against both unsupervised and state-of-the-art
supervised algorithms. Our results show that our model is able to outperform
other models across a variety of tasks and performance metrics
Test Selection for Deep Learning Systems
Testing of deep learning models is challenging due to the excessive number
and complexity of computations involved. As a result, test data selection is
performed manually and in an ad hoc way. This raises the question of how we can
automatically select candidate test data to test deep learning models. Recent
research has focused on adapting test selection metrics from code-based
software testing (such as coverage) to deep learning. However, deep learning
models have different attributes from code such as spread of computations
across the entire network reflecting training data properties, balance of
neuron weights and redundancy (use of many more neurons than needed). Such
differences make code-based metrics inappropriate to select data that can
challenge the models (can trigger misclassification). We thus propose a set of
test selection metrics based on the notion of model uncertainty (model
confidence on specific inputs). Intuitively, the more uncertain we are about a
candidate sample, the more likely it is that this sample triggers a
misclassification. Similarly, the samples for which we are the most uncertain,
are the most informative and should be used to improve the model by retraining.
We evaluate these metrics on two widely-used image classification problems
involving real and artificial (adversarial) data. We show that
uncertainty-based metrics have a strong ability to select data that are
misclassified and lead to major improvement in classification accuracy during
retraining: up to 80% more gain than random selection and other
state-of-the-art metrics on one dataset and up to 29% on the other
Direct Uncertainty Prediction for Medical Second Opinions
The issue of disagreements amongst human experts is a ubiquitous one in both
machine learning and medicine. In medicine, this often corresponds to doctor
disagreements on a patient diagnosis. In this work, we show that machine
learning models can be trained to give uncertainty scores to data instances
that might result in high expert disagreements. In particular, they can
identify patient cases that would benefit most from a medical second opinion.
Our central methodological finding is that Direct Uncertainty Prediction (DUP),
training a model to predict an uncertainty score directly from the raw patient
features, works better than Uncertainty Via Classification, the two-step
process of training a classifier and postprocessing the output distribution to
give an uncertainty score. We show this both with a theoretical result, and on
extensive evaluations on a large scale medical imaging application.Comment: Accepted for publication at ICML 201
CELLO-3D: Estimating the Covariance of ICP in the Real World
The fusion of Iterative Closest Point (ICP) reg- istrations in existing state
estimation frameworks relies on an accurate estimation of their uncertainty. In
this paper, we study the estimation of this uncertainty in the form of a
covariance. First, we scrutinize the limitations of existing closed-form
covariance estimation algorithms over 3D datasets. Then, we set out to estimate
the covariance of ICP registrations through a data-driven approach, with over 5
100 000 registrations on 1020 pairs from real 3D point clouds. We assess our
solution upon a wide spectrum of environments, ranging from structured to
unstructured and indoor to outdoor. The capacity of our algorithm to predict
covariances is accurately assessed, as well as the usefulness of these
estimations for uncertainty estimation over trajectories. The proposed method
estimates covariances better than existing closed-form solutions, and makes
predictions that are consistent with observed trajectories
Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces
Classical deformable registration techniques achieve impressive results and
offer a rigorous theoretical treatment, but are computationally intensive since
they solve an optimization problem for each image pair. Recently,
learning-based methods have facilitated fast registration by learning spatial
deformation functions. However, these approaches use restricted deformation
models, require supervised labels, or do not guarantee a diffeomorphic
(topology-preserving) registration. Furthermore, learning-based registration
tools have not been derived from a probabilistic framework that can offer
uncertainty estimates.
In this paper, we build a connection between classical and learning-based
methods. We present a probabilistic generative model and derive an unsupervised
learning-based inference algorithm that uses insights from classical
registration methods and makes use of recent developments in convolutional
neural networks (CNNs). We demonstrate our method on a 3D brain registration
task for both images and anatomical surfaces, and provide extensive empirical
analyses. Our principled approach results in state of the art accuracy and very
fast runtimes, while providing diffeomorphic guarantees. Our implementation is
available at http://voxelmorph.csail.mit.edu.Comment: MedIA: Medical Image Analysis (MICCAI2018 Special Issue). Expands on
MICCAI 2018 paper (arXiv:1805.04605) by introducing an extension to
anatomical surface registration, new experiments, and analysis of
diffeomorphic implementations. Keywords: medical image registration;
diffeomorphic; invertible; probabilistic modeling; variational inference.
Code available at http://voxelmorph.csail.mit.edu. arXiv admin note: text
overlap with arXiv:1805.0460
Monocular Visual Teach and Repeat Aided by Local Ground Planarity
Visual Teach and Repeat (VT\&R) allows an autonomous vehicle to repeat a
previously traversed route without a global positioning system. Existing
implementations of VT\&R typically rely on 3D sensors such as stereo cameras
for mapping and localization, but many mobile robots are equipped with only 2D
monocular vision for tasks such as teleoperated bomb disposal. While
simultaneous localization and mapping (SLAM) algorithms exist that can recover
3D structure and motion from monocular images, the scale ambiguity inherent in
these methods complicates the estimation and control of lateral path-tracking
error, which is essential for achieving high-accuracy path following. In this
paper, we propose a monocular vision pipeline that enables kilometre-scale
route repetition with centimetre-level accuracy by approximating the ground
surface near the vehicle as planar (with some uncertainty) and recovering
absolute scale from the known position and orientation of the camera relative
to the vehicle. This system provides added value to many existing robots by
allowing for high-accuracy autonomous route repetition with a simple software
upgrade and no additional sensors. We validate our system over 4.3 km of
autonomous navigation and demonstrate accuracy on par with the conventional
stereo pipeline, even in highly non-planar terrain.Comment: In: Wettergreen D., Barfoot T. (eds) Field and Service Robotics.
Springer Tracts in Advanced Robotics, vol 113. Springer, Cha
Information fusion in multi-task Gaussian processes
This paper evaluates heterogeneous information fusion using multi-task
Gaussian processes in the context of geological resource modeling.
Specifically, it empirically demonstrates that information integration across
heterogeneous information sources leads to superior estimates of all the
quantities being modeled, compared to modeling them individually. Multi-task
Gaussian processes provide a powerful approach for simultaneous modeling of
multiple quantities of interest while taking correlations between these
quantities into consideration. Experiments are performed on large scale real
sensor data.Comment: 53 pages, 33 figures; improved presentatio
- …