Central limit theorems for multilevel Monte Carlo methods

In this work, we show that uniform integrability is not a necessary condition for central limit theorems (CLT) to hold for normalized multilevel Monte Carlo (MLMC) estimators and we provide near optimal weaker conditions under which the CLT is achieved. In particular, if the variance decay rate dominates the computational cost rate (i.e., $\beta > \gamma$), we prove that the CLT applies to the standard (variance minimizing) MLMC estimator. For other settings where the CLT may not apply to the standard MLMC estimator, we propose an alternative estimator, called the mass-shifted MLMC estimator, to which the CLT always applies. This comes at a small efficiency loss: the computational cost of achieving mean square approximation error $\mathcal{O}(\epsilon^2)$ is at worst a factor $\mathcal{O}(\log(1/\epsilon))$ higher with the mass-shifted estimator than with the standard one

Statistical and numerical methods for diffusion processes with multiple scales

In this thesis we address the problem of data-driven coarse-graining, i.e. the process of inferring simplified models, which describe the evolution of the essential characteristics of a complex system, from available data (e.g. experimental observation or simulation data). Specifically, we consider the case where the coarse-grained model can be formulated as a stochastic differential equation. The main part of this work is concerned with data-driven coarse-graining when the underlying complex system is characterised by processes occurring across two widely separated time scales. It is known that in this setting commonly used statistical techniques fail to obtain reasonable estimators for parameters in the coarse-grained model, due to the multiscale structure of the data. To enable reliable data-driven coarse-graining techniques for diffusion processes with multiple time scales, we develop a novel estimation procedure which decisively relies on combining techniques from mathematical statistics and numerical analysis. We demonstrate, both rigorously and by means of extensive simulations, that this methodology yields accurate approximations of coarse-grained SDE models. In the final part of this work, we then discuss a systematic framework to analyse and predict complex systems using observations. Specifically, we use data-driven techniques to identify simple, yet adequate, coarse-grained models, which in turn allow to study statistical properties that cannot be investigated directly from the time series. The value of this generic framework is exemplified through two seemingly unrelated data sets of real world phenomena.Open Acces

A new framework for extracting coarse-grained models from time series with multiscale structure

In many applications it is desirable to infer coarse-grained models from observational data. The observed process often corresponds only to a few selected degrees of freedom of a high-dimensional dynamical system with multiple time scales. In this work we consider the inference problem of identifying an appropriate coarse-grained model from a single time series of a multiscale system. It is known that estimators such as the maximum likelihood estimator or the quadratic variation of the path estimator can be strongly biased in this setting. Here we present a novel parametric inference methodology for problems with linear parameter dependency that does not suffer from this drawback. Furthermore, we demonstrate through a wide spectrum of examples that our methodology can be used to derive appropriate coarse-grained models from time series of partial observations of a multiscale system in an effective and systematic fashion

Sequential Estimation using Hierarchically Stratified Domains with Latin Hypercube Sampling

Quantifying the effect of uncertainties in systems where only point evaluations in the stochastic domain but no regularity conditions are available is limited to sampling-based techniques. This work presents an adaptive sequential stratification estimation method that uses Latin Hypercube Sampling within each stratum. The adaptation is achieved through a sequential hierarchical refinement of the stratification, guided by previous estimators using local (i.e., stratum-dependent) variability indicators based on generalized polynomial chaos expansions and Sobol decompositions. For a given total number of samples $N$, the corresponding hierarchically constructed sequence of Stratified Sampling estimators combined with Latin Hypercube sampling is adequately averaged to provide a final estimator with reduced variance. Numerical experiments illustrate the procedure's efficiency, indicating that it can offer a variance decay proportional to $N^{-2}$ in some cases

A Fully Parallelized and Budgeted Multi-Level Monte Carlo Method and the Application to Acoustic Waves

We present a novel variant of the multi-level Monte Carlo method that effectively utilizes a reserved computational budget on a high-performance computing system to minimize the mean squared error. Our approach combines concepts of the continuation multi-level Monte Carlo method with dynamic programming techniques following Bellman's optimality principle, and a new parallelization strategy based on a single distributed data structure. Additionally, we establish a theoretical bound on the error reduction on a parallel computing cluster and provide empirical evidence that the proposed method adheres to this bound. We implement, test, and benchmark the approach on computationally demanding problems, focusing on its application to acoustic wave propagation in high-dimensional random media

Plateau Proposal Distributions for Adaptive Component-wise Multiple-Try Metropolis

Markov chain Monte Carlo (MCMC) methods are sampling methods that have become a commonly used tool in statistics, for example to perform Monte Carlo integration. As a consequence of the increase in computational power, many variations of MCMC methods exist for generating samples from arbitrary, possibly complex, target distributions. The performance of an MCMC method is predominately governed by the choice of the so-called proposal distribution used. In this paper, we introduce a new type of proposal distribution for the use in MCMC methods that operates component-wise and with multiple trials per iteration. Specifically, the novel class of proposal distributions, called Plateau distributions, do not overlap, thus ensuring that the multiple trials are drawn from different regions of the state space. Furthermore, the Plateau proposal distributions allow for a bespoke adaptation procedure that lends itself to a Markov chain with efficient problem dependent state space exploration and improved burn-in properties. Simulation studies show that our novel MCMC algorithm outperforms competitors when sampling from distributions with a complex shape, highly correlated components or multiple modes.Comment: 24 pages, 12 figure

MATHICSE Technical Report : Multilevel Monte Carlo approximation of functions

Many applications across sciences and technologies require a careful quantification of non-deterministic effects to a system output, for example when evaluating the system's reliability or when gearing it towards more robust operation conditions. At the heart of these considerations lies an accurate characterization of uncertain system outputs. In this work we introduce and analyze novel multilevel Monte Carlo techniques for an efficient characterization of an uncertain system output's distribution. These techniques rely on accurately approximating general parametric expectations, i.e. expectations that depend on a parameter, uniformly on an interval. Applications of interest include, for example, the approximation of the characteristic function and of the cumulative distribution function of an uncertain system output. A further important consequence of the introduced approximation techniques for parametric expectations (i.e. for functions) is that they allow to construct multilevel Monte Carlo estimators for various robustness indicators, such as for a quantile (also known as value-at-risk) and for the conditional value-at-risk. These robustness indicators cannot be expressed as moments and are thus not easily accessible usually. In fact, here we provide a framework that allows to simultaneously estimate a cumulative distribution function, a quantile, and the associated conditional value-at-risk of an uncertain system output at the cost of a single multilevel Monte Carlo simulation, while each estimated quantity satisfies a prescribed tolerance goal