1,819 research outputs found
Tomographic Auto-Encoder: Unsupervised Bayesian Recovery of Corrupted Data
We propose a new probabilistic method for unsupervised recovery of corrupted
data. Given a large ensemble of degraded samples, our method recovers accurate
posteriors of clean values, allowing the exploration of the manifold of
possible reconstructed data and hence characterising the underlying
uncertainty. In this setting, direct application of classical variational
methods often gives rise to collapsed densities that do not adequately explore
the solution space. Instead, we derive our novel reduced entropy condition
approximate inference method that results in rich posteriors. We test our model
in a data recovery task under the common setting of missing values and noise,
demonstrating superior performance to existing variational methods for
imputation and de-noising with different real data sets. We further show higher
classification accuracy after imputation, proving the advantage of propagating
uncertainty to downstream tasks with our model.Comment: 8+12 page
Manifold Relevance Determination
In this paper we present a fully Bayesian latent variable model which
exploits conditional nonlinear(in)-dependence structures to learn an efficient
latent representation. The latent space is factorized to represent shared and
private information from multiple views of the data. In contrast to previous
approaches, we introduce a relaxation to the discrete segmentation and allow
for a "softly" shared latent space. Further, Bayesian techniques allow us to
automatically estimate the dimensionality of the latent spaces. The model is
capable of capturing structure underlying extremely high dimensional spaces.
This is illustrated by modelling unprocessed images with tenths of thousands of
pixels. This also allows us to directly generate novel images from the trained
model by sampling from the discovered latent spaces. We also demonstrate the
model by prediction of human pose in an ambiguous setting. Our Bayesian
framework allows us to perform disambiguation in a principled manner by
including latent space priors which incorporate the dynamic nature of the data.Comment: ICML201
Dropout Inference in Bayesian Neural Networks with Alpha-divergences
To obtain uncertainty estimates with real-world Bayesian deep learning
models, practical inference approximations are needed. Dropout variational
inference (VI) for example has been used for machine vision and medical
applications, but VI can severely underestimates model uncertainty.
Alpha-divergences are alternative divergences to VI's KL objective, which are
able to avoid VI's uncertainty underestimation. But these are hard to use in
practice: existing techniques can only use Gaussian approximating
distributions, and require existing models to be changed radically, thus are of
limited use for practitioners. We propose a re-parametrisation of the
alpha-divergence objectives, deriving a simple inference technique which,
together with dropout, can be easily implemented with existing models by simply
changing the loss of the model. We demonstrate improved uncertainty estimates
and accuracy compared to VI in dropout networks. We study our model's epistemic
uncertainty far away from the data using adversarial images, showing that these
can be distinguished from non-adversarial images by examining our model's
uncertainty
- …