316 research outputs found
Algorithms for Kullback-Leibler Approximation of Probability Measures in Infinite Dimensions
In this paper we study algorithms to find a Gaussian approximation to a
target measure defined on a Hilbert space of functions; the target measure
itself is defined via its density with respect to a reference Gaussian measure.
We employ the Kullback-Leibler divergence as a distance and find the best
Gaussian approximation by minimizing this distance. It then follows that the
approximate Gaussian must be equivalent to the Gaussian reference measure,
defining a natural function space setting for the underlying calculus of
variations problem. We introduce a computational algorithm which is
well-adapted to the required minimization, seeking to find the mean as a
function, and parameterizing the covariance in two different ways: through low
rank perturbations of the reference covariance; and through Schr\"odinger
potential perturbations of the inverse reference covariance. Two applications
are shown: to a nonlinear inverse problem in elliptic PDEs, and to a
conditioned diffusion process. We also show how the Gaussian approximations we
obtain may be used to produce improved pCN-MCMC methods which are not only
well-adapted to the high-dimensional setting, but also behave well with respect
to small observational noise (resp. small temperatures) in the inverse problem
(resp. conditioned diffusion).Comment: 28 page
A Function Space HMC Algorithm With Second Order Langevin Diffusion Limit
We describe a new MCMC method optimized for the sampling of probability
measures on Hilbert space which have a density with respect to a Gaussian; such
measures arise in the Bayesian approach to inverse problems, and in conditioned
diffusions. Our algorithm is based on two key design principles: (i) algorithms
which are well-defined in infinite dimensions result in methods which do not
suffer from the curse of dimensionality when they are applied to approximations
of the infinite dimensional target measure on \bbR^N; (ii) non-reversible
algorithms can have better mixing properties compared to their reversible
counterparts. The method we introduce is based on the hybrid Monte Carlo
algorithm, tailored to incorporate these two design principles. The main result
of this paper states that the new algorithm, appropriately rescaled, converges
weakly to a second order Langevin diffusion on Hilbert space; as a consequence
the algorithm explores the approximate target measures on \bbR^N in a number
of steps which is independent of . We also present the underlying theory for
the limiting non-reversible diffusion on Hilbert space, including
characterization of the invariant measure, and we describe numerical
simulations demonstrating that the proposed method has favourable mixing
properties as an MCMC algorithm.Comment: 41 pages, 2 figures. This is the final version, with more comments
and an extra appendix adde
Kullback--Leibler approximation for probability measures on infinite dimensional spaces
In a variety of applications it is important to extract information from a probability measure on an infinite dimensional space. Examples include the Bayesian approach to inverse problems and (possibly conditioned) continuous time Markov processes. It may then be of interest to find a measure , from within a simple class of measures, which approximates . This problem is studied in the case where the Kullback--Leibler divergence is employed to measure the quality of the approximation. A calculus of variations viewpoint is adopted, and the particular case where is chosen from the set of Gaussian measures is studied in detail. Basic existence and uniqueness theorems are established, together with properties of minimizing sequences. Furthermore, parameterization of the class of Gaussians through the mean and inverse covariance is introduced, the need for regularization is explained, and a regularized minimization is studied in detail. The calculus of variations framework resulting from this work provides the appropriate underpinning for computational algorithms
Kullback--Leibler approximation for probability measures on infinite dimensional spaces
In a variety of applications it is important to extract information from a probability measure on an infinite dimensional space. Examples include the Bayesian approach to inverse problems and (possibly conditioned) continuous time Markov processes. It may then be of interest to find a measure , from within a simple class of measures, which approximates . This problem is studied in the case where the Kullback--Leibler divergence is employed to measure the quality of the approximation. A calculus of variations viewpoint is adopted, and the particular case where is chosen from the set of Gaussian measures is studied in detail. Basic existence and uniqueness theorems are established, together with properties of minimizing sequences. Furthermore, parameterization of the class of Gaussians through the mean and inverse covariance is introduced, the need for regularization is explained, and a regularized minimization is studied in detail. The calculus of variations framework resulting from this work provides the appropriate underpinning for computational algorithms
Electronic structure and x-ray magnetic dichroism in random substitutional alloys of f-electron elements
The Koringa-Kohn-Rostoker ācoherent-potential-approximation method combines multiple-scattering theory and the coherent-potential approximation to calculate the electronic structure of random substitutional alloys of transition metals. In this paper we describe the generalization of this theory to describe f-electron alloys. The theory is illustrated with a calculation of the electronic structure and magnetic dichroism curves for a random substitutional alloy containing rare-earth or actinide elements from first principles
- ā¦