2,018 research outputs found
Accelerating Asymptotically Exact MCMC for Computationally Intensive Models via Local Approximations
We construct a new framework for accelerating Markov chain Monte Carlo in
posterior sampling problems where standard methods are limited by the
computational cost of the likelihood, or of numerical models embedded therein.
Our approach introduces local approximations of these models into the
Metropolis-Hastings kernel, borrowing ideas from deterministic approximation
theory, optimization, and experimental design. Previous efforts at integrating
approximate models into inference typically sacrifice either the sampler's
exactness or efficiency; our work seeks to address these limitations by
exploiting useful convergence characteristics of local approximations. We prove
the ergodicity of our approximate Markov chain, showing that it samples
asymptotically from the \emph{exact} posterior distribution of interest. We
describe variations of the algorithm that employ either local polynomial
approximations or local Gaussian process regressors. Our theoretical results
reinforce the key observation underlying this paper: when the likelihood has
some \emph{local} regularity, the number of model evaluations per MCMC step can
be greatly reduced without biasing the Monte Carlo average. Numerical
experiments demonstrate multiple order-of-magnitude reductions in the number of
forward model evaluations used in representative ODE and PDE inference
problems, with both synthetic and real data.Comment: A major update of the theory and example
Coordinate Transformation and Polynomial Chaos for the Bayesian Inference of a Gaussian Process with Parametrized Prior Covariance Function
This paper addresses model dimensionality reduction for Bayesian inference
based on prior Gaussian fields with uncertainty in the covariance function
hyper-parameters. The dimensionality reduction is traditionally achieved using
the Karhunen-\Loeve expansion of a prior Gaussian process assuming covariance
function with fixed hyper-parameters, despite the fact that these are uncertain
in nature. The posterior distribution of the Karhunen-Lo\`{e}ve coordinates is
then inferred using available observations. The resulting inferred field is
therefore dependent on the assumed hyper-parameters. Here, we seek to
efficiently estimate both the field and covariance hyper-parameters using
Bayesian inference. To this end, a generalized Karhunen-Lo\`{e}ve expansion is
derived using a coordinate transformation to account for the dependence with
respect to the covariance hyper-parameters. Polynomial Chaos expansions are
employed for the acceleration of the Bayesian inference using similar
coordinate transformations, enabling us to avoid expanding explicitly the
solution dependence on the uncertain hyper-parameters. We demonstrate the
feasibility of the proposed method on a transient diffusion equation by
inferring spatially-varying log-diffusivity fields from noisy data. The
inferred profiles were found closer to the true profiles when including the
hyper-parameters' uncertainty in the inference formulation.Comment: 34 pages, 17 figure
A hybrid adaptive MCMC algorithm in function spaces
The preconditioned Crank-Nicolson (pCN) method is a Markov Chain Monte Carlo
(MCMC) scheme, specifically designed to perform Bayesian inferences in function
spaces. Unlike many standard MCMC algorithms, the pCN method can preserve the
sampling efficiency under the mesh refinement, a property referred to as being
dimension independent. In this work we consider an adaptive strategy to further
improve the efficiency of pCN. In particular we develop a hybrid adaptive MCMC
method: the algorithm performs an adaptive Metropolis scheme in a chosen finite
dimensional subspace, and a standard pCN algorithm in the complement space of
the chosen subspace. We show that the proposed algorithm satisfies certain
important ergodicity conditions. Finally with numerical examples we demonstrate
that the proposed method has competitive performance with existing adaptive
algorithms.Comment: arXiv admin note: text overlap with arXiv:1511.0583
Hamiltonian Monte Carlo Acceleration Using Surrogate Functions with Random Bases
For big data analysis, high computational cost for Bayesian methods often
limits their applications in practice. In recent years, there have been many
attempts to improve computational efficiency of Bayesian inference. Here we
propose an efficient and scalable computational technique for a
state-of-the-art Markov Chain Monte Carlo (MCMC) methods, namely, Hamiltonian
Monte Carlo (HMC). The key idea is to explore and exploit the structure and
regularity in parameter space for the underlying probabilistic model to
construct an effective approximation of its geometric properties. To this end,
we build a surrogate function to approximate the target distribution using
properly chosen random bases and an efficient optimization process. The
resulting method provides a flexible, scalable, and efficient sampling
algorithm, which converges to the correct target distribution. We show that by
choosing the basis functions and optimization process differently, our method
can be related to other approaches for the construction of surrogate functions
such as generalized additive models or Gaussian process models. Experiments
based on simulated and real data show that our approach leads to substantially
more efficient sampling algorithms compared to existing state-of-the art
methods
The ROMES method for statistical modeling of reduced-order-model error
This work presents a technique for statistically modeling errors introduced
by reduced-order models. The method employs Gaussian-process regression to
construct a mapping from a small number of computationally inexpensive `error
indicators' to a distribution over the true error. The variance of this
distribution can be interpreted as the (epistemic) uncertainty introduced by
the reduced-order model. To model normed errors, the method employs existing
rigorous error bounds and residual norms as indicators; numerical experiments
show that the method leads to a near-optimal expected effectivity in contrast
to typical error bounds. To model errors in general outputs, the method uses
dual-weighted residuals---which are amenable to uncertainty control---as
indicators. Experiments illustrate that correcting the reduced-order-model
output with this surrogate can improve prediction accuracy by an order of
magnitude; this contrasts with existing `multifidelity correction' approaches,
which often fail for reduced-order models and suffer from the curse of
dimensionality. The proposed error surrogates also lead to a notion of
`probabilistic rigor', i.e., the surrogate bounds the error with specified
probability
- …