1,283 research outputs found
Multilevel Sequential Monte Carlo with Dimension-Independent Likelihood-Informed Proposals
In this article we develop a new sequential Monte Carlo (SMC) method for
multilevel (ML) Monte Carlo estimation. In particular, the method can be used
to estimate expectations with respect to a target probability distribution over
an infinite-dimensional and non-compact space as given, for example, by a
Bayesian inverse problem with Gaussian random field prior. Under suitable
assumptions the MLSMC method has the optimal bound on the
cost to obtain a mean-square error of . The algorithm is
accelerated by dimension-independent likelihood-informed (DILI) proposals
designed for Gaussian priors, leveraging a novel variation which uses empirical
sample covariance information in lieu of Hessian information, hence eliminating
the requirement for gradient evaluations. The efficiency of the algorithm is
illustrated on two examples: inversion of noisy pressure measurements in a PDE
model of Darcy flow to recover the posterior distribution of the permeability
field, and inversion of noisy measurements of the solution of an SDE to recover
the posterior path measure
Multilevel Sequential Monte Carlo with Dimension-Independent Likelihood-Informed Proposals
In this article we develop a new sequential Monte Carlo method for multilevel Monte Carlo estimation.
In particular, the method can be used to estimate expectations with respect to a target
probability distribution over an infinite-dimensional and noncompact space—as produced, for example,
by a Bayesian inverse problem with a Gaussian random field prior. Under suitable assumptions
the MLSMC method has the optimal O(ε
−2
) bound on the cost to obtain a mean-square error of
O(ε
2
). The algorithm is accelerated by dimension-independent likelihood-informed proposals [T. Cui,
K. J. Law, and Y. M. Marzouk, (2016), J. Comput. Phys., 304, pp. 109–137] designed for Gaussian
priors, leveraging a novel variation which uses empirical covariance information in lieu of Hessian
information, hence eliminating the requirement for gradient evaluations. The efficiency of the algorithm
is illustrated on two examples: (i) inversion of noisy pressure measurements in a PDE model
of Darcy flow to recover the posterior distribution of the permeability field and (ii) inversion of noisy
measurements of the solution of an SDE to recover the posterior path measure
Multilevel Delayed Acceptance MCMC with Applications to Hydrogeological Inverse Problems
Quantifying the uncertainty of model predictions is a critical task for engineering decision support systems. This is a particularly challenging effort in the context of statistical inverse problems, where the model parameters are unknown or poorly constrained, and where the data is often scarce. Many such problems emerge in the fields of hydrology and hydro--environmental engineering in general, and in hydrogeology in particular. While methods for rigorously quantifying the uncertainty of such problems exist, they are often prohibitively computationally expensive, particularly when the forward model is high--dimensional and expensive to evaluate. In this thesis, I present a Metropolis--Hastings algorithm, namely the Multilevel Delayed Acceptance (MLDA) algorithm, which exploits a hierarchy of forward models of increasing computational cost to significantly reduce the total cost of quantifying the uncertainty of high--dimensional, expensive forward models. The algorithm is shown to be in detailed balance with the posterior distribution of parameters, and the computational gains of the algorithm is demonstrated on multiple examples. Additionally, I present an approach for exploiting a deep neural network as an ultra--fast model approximation in an MLDA model hierarchy. This method is demonstrated in the context of both 2D and 3D groundwater flow modelling. Finally, I present a novel approach to adaptive optimal design of groundwater surveying, in which MLDA is employed to construct the posterior Monte Carlo estimates. This method utilises the posterior uncertainty of the primary problem in conjunction with the expected solution to an adjoint problem to sequentially determine the optimal location of the next datapoint.Engineering and Physical Sciences Research Council (EPSRC)Alan Turing InstituteEngineering and Physical Sciences Research Council (EPSRC
Multilevel Delayed Acceptance MCMC
We develop a novel Markov chain Monte Carlo (MCMC) method that exploits a
hierarchy of models of increasing complexity to efficiently generate samples
from an unnormalized target distribution. Broadly, the method rewrites the
Multilevel MCMC approach of Dodwell et al. (2015) in terms of the Delayed
Acceptance (DA) MCMC of Christen & Fox (2005). In particular, DA is extended to
use a hierarchy of models of arbitrary depth, and allow subchains of arbitrary
length. We show that the algorithm satisfies detailed balance, hence is ergodic
for the target distribution. Furthermore, multilevel variance reduction is
derived that exploits the multiple levels and subchains, and an adaptive
multilevel correction to coarse-level biases is developed. Three numerical
examples of Bayesian inverse problems are presented that demonstrate the
advantages of these novel methods. The software and examples are available in
PyMC3.Comment: 29 pages, 12 figure
Kernel Sequential Monte Carlo
We propose kernel sequential Monte Carlo (KSMC), a framework for sampling from static target densities. KSMC is a family of sequential Monte Carlo algorithms that are based on building emulator models of the current particle system in a reproducing kernel Hilbert space. We here focus on modelling nonlinear covariance structure and gradients of the target. The emulator's geometry is adaptively updated and subsequently used to inform local proposals. Unlike in adaptive Markov chain Monte Carlo, continuous adaptation does not compromise convergence of the sampler. KSMC combines the strengths of sequental Monte Carlo and kernel methods: superior performance for multimodal targets and the ability to estimate model evidence as compared to Markov chain Monte Carlo, and the emulator's ability to represent targets that exhibit high degrees of nonlinearity. As KSMC does not require access to target gradients, it is particularly applicable on targets whose gradients are unknown or prohibitively expensive. We describe necessary tuning details and demonstrate the benefits of the the proposed methodology on a series of challenging synthetic and real-world examples
Large deviation theory-based adaptive importance sampling for rare events in high dimensions
We propose a method for the accurate estimation of rare event or failure
probabilities for expensive-to-evaluate numerical models in high dimensions.
The proposed approach combines ideas from large deviation theory and adaptive
importance sampling. The importance sampler uses a cross-entropy method to find
an optimal Gaussian biasing distribution, and reuses all samples made
throughout the process for both, the target probability estimation and for
updating the biasing distributions. Large deviation theory is used to find a
good initial biasing distribution through the solution of an optimization
problem. Additionally, it is used to identify a low-dimensional subspace that
is most informative of the rare event probability. This subspace is used for
the cross-entropy method, which is known to lose efficiency in higher
dimensions. The proposed method does not require smoothing of indicator
functions nor does it involve numerical tuning parameters. We compare the
method with a state-of-the-art cross-entropy-based importance sampling scheme
using three examples: a high-dimensional failure probability estimation
benchmark, a problem governed by a diffusion equation, and a tsunami problem
governed by the time-dependent shallow water system in one spatial dimension
- …