15,497 research outputs found
Hierarchical Implicit Models and Likelihood-Free Variational Inference
Implicit probabilistic models are a flexible class of models defined by a
simulation process for data. They form the basis for theories which encompass
our understanding of the physical world. Despite this fundamental nature, the
use of implicit models remains limited due to challenges in specifying complex
latent structure in them, and in performing inferences in such models with
large data sets. In this paper, we first introduce hierarchical implicit models
(HIMs). HIMs combine the idea of implicit densities with hierarchical Bayesian
modeling, thereby defining models via simulators of data with rich hidden
structure. Next, we develop likelihood-free variational inference (LFVI), a
scalable variational inference algorithm for HIMs. Key to LFVI is specifying a
variational family that is also implicit. This matches the model's flexibility
and allows for accurate approximation of the posterior. We demonstrate diverse
applications: a large-scale physical simulator for predator-prey populations in
ecology; a Bayesian generative adversarial network for discrete data; and a
deep implicit model for text generation.Comment: Appears in Neural Information Processing Systems, 201
Maximum-a-posteriori estimation with Bayesian confidence regions
Solutions to inverse problems that are ill-conditioned or ill-posed may have
significant intrinsic uncertainty. Unfortunately, analysing and quantifying
this uncertainty is very challenging, particularly in high-dimensional
problems. As a result, while most modern mathematical imaging methods produce
impressive point estimation results, they are generally unable to quantify the
uncertainty in the solutions delivered. This paper presents a new general
methodology for approximating Bayesian high-posterior-density credibility
regions in inverse problems that are convex and potentially very
high-dimensional. The approximations are derived by using recent concentration
of measure results related to information theory for log-concave random
vectors. A remarkable property of the approximations is that they can be
computed very efficiently, even in large-scale problems, by using standard
convex optimisation techniques. In particular, they are available as a
by-product in problems solved by maximum-a-posteriori estimation. The
approximations also have favourable theoretical properties, namely they
outer-bound the true high-posterior-density credibility regions, and they are
stable with respect to model dimension. The proposed methodology is illustrated
on two high-dimensional imaging inverse problems related to tomographic
reconstruction and sparse deconvolution, where the approximations are used to
perform Bayesian hypothesis tests and explore the uncertainty about the
solutions, and where proximal Markov chain Monte Carlo algorithms are used as
benchmark to compute exact credible regions and measure the approximation
error
Receiver Architectures for MIMO-OFDM Based on a Combined VMP-SP Algorithm
Iterative information processing, either based on heuristics or analytical
frameworks, has been shown to be a very powerful tool for the design of
efficient, yet feasible, wireless receiver architectures. Within this context,
algorithms performing message-passing on a probabilistic graph, such as the
sum-product (SP) and variational message passing (VMP) algorithms, have become
increasingly popular.
In this contribution, we apply a combined VMP-SP message-passing technique to
the design of receivers for MIMO-ODFM systems. The message-passing equations of
the combined scheme can be obtained from the equations of the stationary points
of a constrained region-based free energy approximation. When applied to a
MIMO-OFDM probabilistic model, we obtain a generic receiver architecture
performing iterative channel weight and noise precision estimation,
equalization and data decoding. We show that this generic scheme can be
particularized to a variety of different receiver structures, ranging from
high-performance iterative structures to low complexity receivers. This allows
for a flexible design of the signal processing specially tailored for the
requirements of each specific application. The numerical assessment of our
solutions, based on Monte Carlo simulations, corroborates the high performance
of the proposed algorithms and their superiority to heuristic approaches
Bregman Cost for Non-Gaussian Noise
One of the tasks of the Bayesian inverse problem is to find a good estimate
based on the posterior probability density. The most common point estimators
are the conditional mean (CM) and maximum a posteriori (MAP) estimates, which
correspond to the mean and the mode of the posterior, respectively. From a
theoretical point of view it has been argued that the MAP estimate is only in
an asymptotic sense a Bayes estimator for the uniform cost function, while the
CM estimate is a Bayes estimator for the means squared cost function. Recently,
it has been proven that the MAP estimate is a proper Bayes estimator for the
Bregman cost if the image is corrupted by Gaussian noise. In this work we
extend this result to other noise models with log-concave likelihood density,
by introducing two related Bregman cost functions for which the CM and the MAP
estimates are proper Bayes estimators. Moreover, we also prove that the CM
estimate outperforms the MAP estimate, when the error is measured in a certain
Bregman distance, a result previously unknown also in the case of additive
Gaussian noise
- …