6,570 research outputs found
Efficient Gaussian Sampling for Solving Large-Scale Inverse Problems using MCMC Methods
The resolution of many large-scale inverse problems using MCMC methods
requires a step of drawing samples from a high dimensional Gaussian
distribution. While direct Gaussian sampling techniques, such as those based on
Cholesky factorization, induce an excessive numerical complexity and memory
requirement, sequential coordinate sampling methods present a low rate of
convergence. Based on the reversible jump Markov chain framework, this paper
proposes an efficient Gaussian sampling algorithm having a reduced computation
cost and memory usage. The main feature of the algorithm is to perform an
approximate resolution of a linear system with a truncation level adjusted
using a self-tuning adaptive scheme allowing to achieve the minimal computation
cost. The connection between this algorithm and some existing strategies is
discussed and its efficiency is illustrated on a linear inverse problem of
image resolution enhancement.Comment: 20 pages, 10 figures, under review for journal publicatio
Parallel Markov Chain Monte Carlo Methods for Large Scale Statistical Inverse Problems
The Bayesian method has proven to be a powerful way of modeling inverse problems.
The solution to Bayesian inverse problems is the posterior distribution of estimated
parameters which can provide not only estimates for the inferred parameters
but also the uncertainty of these estimations. Markov chain Monte Carlo (MCMC)
is a useful technique to sample the posterior distribution and information can be
extracted from the sampled ensemble. However, MCMC is very expensive to compute,
especially in inverse problems where the underlying forward problems involve
solving differential equations. Even worse, MCMC is difficult to parallelize due to
its sequential nature|that is, under the current framework, we can barely accelerate
MCMC with parallel computing.
We develop a new framework of parallel MCMC algorithms-the Markov chain
preconditioned Monte Carlo (MCPMC) method-for sampling Bayesian inverse problems.
With the help of a fast auxiliary MCMC chain running on computationally
cheaper approximate models, which serves as a stochastic preconditioner to the target
distribution, the sampler randomly selects candidates from the preconditioning
chain for further processing on the accurate model. As this accurate model processing
can be executed in parallel, the algorithm is suitable for parallel systems.
We implement it using a modified master-slave architecture, analyze its potential
to accelerate sampling and apply it to three examples. A two dimensional Gaussian
mixture example shows that the new sampler can bring statistical efficiency in
addition to increasing sampling speed. Through a 2D inverse problem with an elliptic
equation as the forward model, we demonstrate the use of an enhanced error
model to build the preconditioner. With a 3D optical tomography problem we use
adaptive finite element methods to build the approximate model. In both examples,
the MCPMC successfully samples the posterior distributions with multiple processors,
demonstrating efficient speedups comparing to traditional MCMC algorithms.
In addition, the 3D optical tomography example shows the feasibility of applying
MCPMC towards real world, large scale, statistical inverse problems
A computational framework for infinite-dimensional Bayesian inverse problems: Part II. Stochastic Newton MCMC with application to ice sheet flow inverse problems
We address the numerical solution of infinite-dimensional inverse problems in
the framework of Bayesian inference. In the Part I companion to this paper
(arXiv.org:1308.1313), we considered the linearized infinite-dimensional
inverse problem. Here in Part II, we relax the linearization assumption and
consider the fully nonlinear infinite-dimensional inverse problem using a
Markov chain Monte Carlo (MCMC) sampling method. To address the challenges of
sampling high-dimensional pdfs arising from Bayesian inverse problems governed
by PDEs, we build on the stochastic Newton MCMC method. This method exploits
problem structure by taking as a proposal density a local Gaussian
approximation of the posterior pdf, whose construction is made tractable by
invoking a low-rank approximation of its data misfit component of the Hessian.
Here we introduce an approximation of the stochastic Newton proposal in which
we compute the low-rank-based Hessian at just the MAP point, and then reuse
this Hessian at each MCMC step. We compare the performance of the proposed
method to the original stochastic Newton MCMC method and to an independence
sampler. The comparison of the three methods is conducted on a synthetic ice
sheet inverse problem. For this problem, the stochastic Newton MCMC method with
a MAP-based Hessian converges at least as rapidly as the original stochastic
Newton MCMC method, but is far cheaper since it avoids recomputing the Hessian
at each step. On the other hand, it is more expensive per sample than the
independence sampler; however, its convergence is significantly more rapid, and
thus overall it is much cheaper. Finally, we present extensive analysis and
interpretation of the posterior distribution, and classify directions in
parameter space based on the extent to which they are informed by the prior or
the observations.Comment: 31 page
Parameter estimation by implicit sampling
Implicit sampling is a weighted sampling method that is used in data
assimilation, where one sequentially updates estimates of the state of a
stochastic model based on a stream of noisy or incomplete data. Here we
describe how to use implicit sampling in parameter estimation problems, where
the goal is to find parameters of a numerical model, e.g.~a partial
differential equation (PDE), such that the output of the numerical model is
compatible with (noisy) data. We use the Bayesian approach to parameter
estimation, in which a posterior probability density describes the probability
of the parameter conditioned on data and compute an empirical estimate of this
posterior with implicit sampling. Our approach generates independent samples,
so that some of the practical difficulties one encounters with Markov Chain
Monte Carlo methods, e.g.~burn-in time or correlations among dependent samples,
are avoided. We describe a new implementation of implicit sampling for
parameter estimation problems that makes use of multiple grids (coarse to fine)
and BFGS optimization coupled to adjoint equations for the required gradient
calculations. The implementation is "dimension independent", in the sense that
a well-defined finite dimensional subspace is sampled as the mesh used for
discretization of the PDE is refined. We illustrate the algorithm with an
example where we estimate a diffusion coefficient in an elliptic equation from
sparse and noisy pressure measurements. In the example, dimension\slash
mesh-independence is achieved via Karhunen-Lo\`{e}ve expansions
Informed Proposal Monte Carlo
Any search or sampling algorithm for solution of inverse problems needs
guidance to be efficient. Many algorithms collect and apply information about
the problem on the fly, and much improvement has been made in this way.
However, as a consequence of the the No-Free-Lunch Theorem, the only way we can
ensure a significantly better performance of search and sampling algorithms is
to build in as much information about the problem as possible. In the special
case of Markov Chain Monte Carlo sampling (MCMC) we review how this is done
through the choice of proposal distribution, and we show how this way of adding
more information about the problem can be made particularly efficient when
based on an approximate physics model of the problem. A highly nonlinear
inverse scattering problem with a high-dimensional model space serves as an
illustration of the gain of efficiency through this approach
Scalable iterative methods for sampling from massive Gaussian random vectors
Sampling from Gaussian Markov random fields (GMRFs), that is multivariate
Gaussian ran- dom vectors that are parameterised by the inverse of their
covariance matrix, is a fundamental problem in computational statistics. In
this paper, we show how we can exploit arbitrarily accu- rate approximations to
a GMRF to speed up Krylov subspace sampling methods. We also show that these
methods can be used when computing the normalising constant of a large
multivariate Gaussian distribution, which is needed for both any
likelihood-based inference method. The method we derive is also applicable to
other structured Gaussian random vectors and, in particu- lar, we show that
when the precision matrix is a perturbation of a (block) circulant matrix, it
is still possible to derive O(n log n) sampling schemes.Comment: 17 Pages, 4 Figure
- …