29 research outputs found
Subsampling in ensemble Kalman inversion
We consider the Ensemble Kalman Inversion which has been recently introduced
as an efficient, gradient-free optimisation method to estimate unknown
parameters in an inverse setting. In the case of large data sets, the Ensemble
Kalman Inversion becomes computationally infeasible as the data misfit needs to
be evaluated for each particle in each iteration. Here, randomised algorithms
like stochastic gradient descent have been demonstrated to successfully
overcome this issue by using only a random subset of the data in each
iteration, so-called subsampling techniques. Based on a recent analysis of a
continuous-time representation of stochastic gradient methods, we propose,
analyse, and apply subsampling-techniques within Ensemble Kalman Inversion.
Indeed, we propose two different subsampling techniques: either every particle
observes the same data subset (single subsampling) or every particle observes a
different data subset (batch subsampling)
Subsampling in ensemble Kalman inversion
We consider the ensemble Kalman inversion (EKI) which has been recently introduced as an efficient, gradient-free optimisation method to estimate unknown parameters in an inverse setting. In the case of large data sets, the EKI becomes computationally infeasible as the data misfit needs to be evaluated for each particle in each iteration. Here, randomised algorithms like stochastic gradient descent have been demonstrated to successfully overcome this issue by using only a random subset of the data in each iteration, so-called subsampling techniques. Based on a recent analysis of a continuous-time representation of stochastic gradient methods, we propose, analyse, and apply subsampling-techniques within EKI. Indeed, we propose two different subsampling techniques: either every particle observes the same data subset (single subsampling) or every particle observes a different data subset (batch subsampling)
Subsampling Error in Stochastic Gradient Langevin Diffusions
The Stochastic Gradient Langevin Dynamics (SGLD) are popularly used to
approximate Bayesian posterior distributions in statistical learning procedures
with large-scale data. As opposed to many usual Markov chain Monte Carlo (MCMC)
algorithms, SGLD is not stationary with respect to the posterior distribution;
two sources of error appear: The first error is introduced by an
Euler--Maruyama discretisation of a Langevin diffusion process, the second
error comes from the data subsampling that enables its use in large-scale data
settings. In this work, we consider an idealised version of SGLD to analyse the
method's pure subsampling error that we then see as a best-case error for
diffusion-based subsampling MCMC methods. Indeed, we introduce and study the
Stochastic Gradient Langevin Diffusion (SGLDiff), a continuous-time Markov
process that follows the Langevin diffusion corresponding to a data subset and
switches this data subset after exponential waiting times. There, we show that
the Wasserstein distance between the posterior and the limiting distribution of
SGLDiff is bounded above by a fractional power of the mean waiting time.
Importantly, this fractional power does not depend on the dimension of the
state space. We bring our results into context with other analyses of SGLD
Nested Sampling for Uncertainty Quantification and Rare Event Estimation
Nested Sampling is a method for computing the Bayesian evidence, also called
the marginal likelihood, which is the integral of the likelihood with respect
to the prior. More generally, it is a numerical probabilistic quadrature rule.
The main idea of Nested Sampling is to replace a high-dimensional likelihood
integral over parameter space with an integral over the unit line by employing
a push-forward with respect to a suitable transformation. Practically, a set of
active samples ascends the level sets of the integrand function, with the
measure contraction of the super-level sets being statistically estimated. We
justify the validity of this approach for integrands with non-negligible
plateaus, and demonstrate Nested Sampling's practical effectiveness in
estimating the (log-)probability of rare events.Comment: 24 page
Gaussian random fields on non-separable Banach spaces
We study Gaussian random fields on certain Banach spaces and investigate
conditions for their existence. Our results apply inter alia to spaces of Radon
measures and H\"older functions. In the former case, we are able to define
Gaussian white noise on the space of measures directly, avoiding, e.g., an
embedding into a negative-order Sobolev space. In the latter case, we
demonstrate how H\"older regularity of the samples is controlled by that of the
covariance kernel and, thus, show a connection to the Theorem of
Kolmogorov-Chentsov
Certified and fast computations with shallow covariance kernels
Many techniques for data science and uncertainty quantification demand
efficient tools to handle Gaussian random fields, which are defined in terms of
their mean functions and covariance operators. Recently, parameterized Gaussian
random fields have gained increased attention, due to their higher degree of
flexibility. However, especially if the random field is parameterized through
its covariance operator, classical random field discretization techniques fail
or become inefficient. In this work we introduce and analyze a new and
certified algorithm for the low-rank approximation of a parameterized family of
covariance operators which represents an extension of the adaptive cross
approximation method for symmetric positive definite matrices. The algorithm
relies on an affine linear expansion of the covariance operator with respect to
the parameters, which needs to be computed in a preprocessing step using, e.g.,
the empirical interpolation method. We discuss and test our new approach for
isotropic covariance kernels, such as Mat\'ern kernels. The numerical results
demonstrate the advantages of our approach in terms of computational time and
confirm that the proposed algorithm provides the basis of a fast sampling
procedure for parameter dependent Gaussian random fields