9,433 research outputs found
Fast Gibbs sampling for high-dimensional Bayesian inversion
Solving ill-posed inverse problems by Bayesian inference has recently
attracted considerable attention. Compared to deterministic approaches, the
probabilistic representation of the solution by the posterior distribution can
be exploited to explore and quantify its uncertainties. In applications where
the inverse solution is subject to further analysis procedures, this can be a
significant advantage. Alongside theoretical progress, various new
computational techniques allow to sample very high dimensional posterior
distributions: In [Lucka2012], a Markov chain Monte Carlo (MCMC) posterior
sampler was developed for linear inverse problems with -type priors. In
this article, we extend this single component Gibbs-type sampler to a wide
range of priors used in Bayesian inversion, such as general priors
with additional hard constraints. Besides a fast computation of the
conditional, single component densities in an explicit, parameterized form, a
fast, robust and exact sampling from these one-dimensional densities is key to
obtain an efficient algorithm. We demonstrate that a generalization of slice
sampling can utilize their specific structure for this task and illustrate the
performance of the resulting slice-within-Gibbs samplers by different computed
examples. These new samplers allow us to perform sample-based Bayesian
inference in high-dimensional scenarios with certain priors for the first time,
including the inversion of computed tomography (CT) data with the popular
isotropic total variation (TV) prior.Comment: submitted to "Inverse Problems
Robust Classification of Functional and Quantitative Image Data Using Functional Mixed Models
This paper describes how to perform classification of complex, high-dimensional functional data using the functional mixed model (FMM) framework. The FMM relates a functional response to a set of predictors through functional fixed and random effects, which allows it to account for various factors and between-function correlations. Classification is performed through training the model treating class as one of the fixed effects, and then predicting on the test data using posterior predictive probabilities of class. Through a Bayesian scheme, we are able to adjust for factors affecting both the functions and the class designations. While the method we present can be applied to any FMM-based method, we provide details for two specific Bayesian approaches: the Gaussian, wavelet-based functional mixed model (G-WFMM) and the robust, wavelet-based functional mixed model (R-WFMM). Both methods perform modeling in the wavelet space, which yields parsimonious representations for the functions, and can naturally adapt to local features and complex nonstationarities in the functions. The R-WFMM allows potentially heavier tails for features of the functions indexed by particular wavelet coefficients, leading to a down weighting of outliers that makes the method robust to outlying functions or regions of functions. The models are applied to a pancreatic cancer mass spectroscopy data set and compared with some other recently developed functional classification methods
Fast Markov chain Monte Carlo sampling for sparse Bayesian inference in high-dimensional inverse problems using L1-type priors
Sparsity has become a key concept for solving of high-dimensional inverse
problems using variational regularization techniques. Recently, using similar
sparsity-constraints in the Bayesian framework for inverse problems by encoding
them in the prior distribution has attracted attention. Important questions
about the relation between regularization theory and Bayesian inference still
need to be addressed when using sparsity promoting inversion. A practical
obstacle for these examinations is the lack of fast posterior sampling
algorithms for sparse, high-dimensional Bayesian inversion: Accessing the full
range of Bayesian inference methods requires being able to draw samples from
the posterior probability distribution in a fast and efficient way. This is
usually done using Markov chain Monte Carlo (MCMC) sampling algorithms. In this
article, we develop and examine a new implementation of a single component
Gibbs MCMC sampler for sparse priors relying on L1-norms. We demonstrate that
the efficiency of our Gibbs sampler increases when the level of sparsity or the
dimension of the unknowns is increased. This property is contrary to the
properties of the most commonly applied Metropolis-Hastings (MH) sampling
schemes: We demonstrate that the efficiency of MH schemes for L1-type priors
dramatically decreases when the level of sparsity or the dimension of the
unknowns is increased. Practically, Bayesian inversion for L1-type priors using
MH samplers is not feasible at all. As this is commonly believed to be an
intrinsic feature of MCMC sampling, the performance of our Gibbs sampler also
challenges common beliefs about the applicability of sample based Bayesian
inference.Comment: 33 pages, 14 figure
Latent Gaussian modeling and INLA: A review with focus on space-time applications
Bayesian hierarchical models with latent Gaussian layers have proven very
flexible in capturing complex stochastic behavior and hierarchical structures
in high-dimensional spatial and spatio-temporal data. Whereas simulation-based
Bayesian inference through Markov Chain Monte Carlo may be hampered by slow
convergence and numerical instabilities, the inferential framework of
Integrated Nested Laplace Approximation (INLA) is capable to provide accurate
and relatively fast analytical approximations to posterior quantities of
interest. It heavily relies on the use of Gauss-Markov dependence structures to
avoid the numerical bottleneck of high-dimensional nonsparse matrix
computations. With a view towards space-time applications, we here review the
principal theoretical concepts, model classes and inference tools within the
INLA framework. Important elements to construct space-time models are certain
spatial Mat\'ern-like Gauss-Markov random fields, obtained as approximate
solutions to a stochastic partial differential equation. Efficient
implementation of statistical inference tools for a large variety of models is
available through the INLA package of the R software. To showcase the practical
use of R-INLA and to illustrate its principal commands and syntax, a
comprehensive simulation experiment is presented using simulated non Gaussian
space-time count data with a first-order autoregressive dependence structure in
time
Inference via low-dimensional couplings
We investigate the low-dimensional structure of deterministic transformations
between random variables, i.e., transport maps between probability measures. In
the context of statistics and machine learning, these transformations can be
used to couple a tractable "reference" measure (e.g., a standard Gaussian) with
a target measure of interest. Direct simulation from the desired measure can
then be achieved by pushing forward reference samples through the map. Yet
characterizing such a map---e.g., representing and evaluating it---grows
challenging in high dimensions. The central contribution of this paper is to
establish a link between the Markov properties of the target measure and the
existence of low-dimensional couplings, induced by transport maps that are
sparse and/or decomposable. Our analysis not only facilitates the construction
of transformations in high-dimensional settings, but also suggests new
inference methodologies for continuous non-Gaussian graphical models. For
instance, in the context of nonlinear state-space models, we describe new
variational algorithms for filtering, smoothing, and sequential parameter
inference. These algorithms can be understood as the natural
generalization---to the non-Gaussian case---of the square-root
Rauch-Tung-Striebel Gaussian smoother.Comment: 78 pages, 25 figure
Adaptive Mixture of Student-t Distributions as a Flexible Candidate Distribution for Efficient Simulation: The R Package AdMit
This paper presents the R package AdMit which provides flexible functions to approximate a certain target distribution and to efficiently generate a sample of random draws from it, given only a kernel of the target density function. The core algorithm consists of the function AdMit which fits an adaptive mixture of Student-t distributions to the density of interest. Then, importance sampling or the independence chain Metropolis-Hastings algorithm is used to obtain quantities of interest for the target density, using the fitted mixture as the importance or candidate density. The estimation procedure is fully automatic and thus avoids the time-consuming and difficult task of tuning a sampling algorithm. The relevance of the package is shown in two examples. The first aims at illustrating in detail the use of the functions provided by the package in a bivariate bimodal distribution. The second shows the relevance of the adaptive mixture procedure through the Bayesian estimation of a mixture of ARCH model fitted to foreign exchange log-returns data. The methodology is compared to standard cases of importance sampling and the Metropolis-Hastings algorithm using a naive candidate and with the Griddy-Gibbs approach.
- …