Towards probabilistic time-parallel algorithms for solving initial value problems

This thesis concerns the development of probabilistic time-parallel algorithms for solving initial value problems (IVPs) that are computationally expensive to simulate using traditional (serial) time-stepping methods. We begin by considering Parareal, a well-studied deterministic time-parallel algorithm that combines solutions from cheap (coarse) and expensive (fine) time-steppers within a predictor-corrector (PC) scheme, to solve the IVP in parallel. Our goal is to derive, analyse, and test our own probabilistic time-parallel algorithms that incorporate sampling- and learning-based techniques from the _eld of probabilistic numerics into Parareal. These techniques enable us to exploit valuable information contained within the _ne and coarse solution data generated during a Parareal simulation. We aim to accelerate the convergence of Parareal (i.e. increase numerical speedup), generate probabilistic solutions to the IVPs (to quantify numerical uncertainty explicitly), and verify the accuracy of these solutions both numerically and analytically. We first propose SParareal, a sampling-based algorithm that provides the PC with candidate solution values drawn from probability distributions constructed using the most recent fine and coarse solution data. Increased sampling in SParareal leads to accelerated convergence vs. Parareal for low-dimensional IVPs, returning stochastic solutions that are accurate (in the mean-square sense) with respect to the (exact) serially obtained _ne solver solution. Next, we propose GParareal, a learning-based algorithm that models part of the PC using a Gaussian process emulator, trained on all previously collected _ne and coarse solution data. GParareal achieves accelerated convergence for low to moderately sized IVPs, attains accurate solutions, and has the ability to re-use legacy solution data from prior simulations|something that existing time-parallel methods do not do. After introducing both algorithms, we investigate their performance and analyse their limitations, assessing whether or not they are viable methods for solving large-scale IVPs in parallel and discussing what can be done to improve them in their current form

Error bound analysis of the stochastic parareal algorithm

Stochastic parareal (SParareal) is a probabilistic variant of the popular parallel-in-time algorithm known as parareal. Similarly to parareal, it combines fine- and coarse-grained solutions to an ordinary differential equation (ODE) using a predictor-corrector (PC) scheme. The key difference is that carefully chosen random perturbations are added to the PC to try to accelerate the location of a stochastic solution to the ODE. In this paper, we derive superlinear and linear mean-square error bounds for SParareal applied to nonlinear systems of ODEs using different types of perturbations. We illustrate these bounds numerically on a linear system of ODEs and a scalar nonlinear ODE, showing a good match between theory and numerics

Stochastic parareal: an application of probabilistic methods to time-parallelisation

Parareal is a well-studied algorithm for numerically integrating systems of time-dependent differential equations by parallelising the temporal domain. Given approximate initial values at each temporal sub-interval, the algorithm locates a solution in a fixed number of iterations using a predictor-corrector, stopping once a tolerance is met. This iterative process combines solutions located by inexpensive (coarse resolution) and expensive (fine resolution) numerical integrators. In this paper, we introduce a stochastic parareal algorithm with the aim of accelerating the convergence of the deterministic parareal algorithm. Instead of providing the predictor-corrector with a deterministically located set of initial values, the stochastic algorithm samples initial values from dynamically varying probability distributions in each temporal sub-interval. All samples are then propagated by the numerical method in parallel. The initial values yielding the most continuous (smoothest) trajectory across consecutive sub-intervals are chosen as the new, more accurate, set of initial values. These values are fed into the predictor-corrector, converging in fewer iterations than the deterministic algorithm with a given probability. The performance of the stochastic algorithm, implemented using various probability distributions, is illustrated on systems of ordinary differential equations. When the number of sampled initial values is large enough, we show that stochastic parareal converges almost certainly in fewer iterations than the deterministic algorithm while maintaining solution accuracy. Additionally, it is shown that the expected value of the convergence rate decreases with increasing numbers of samples

GParareal: A time-parallel ODE solver using Gaussian process emulation

Sequential numerical methods for integrating initial value problems (IVPs) can be prohibitively expensive when high numerical accuracy is required over the entire interval of integration. One remedy is to integrate in a parallel fashion, "predicting" the solution serially using a cheap (coarse) solver and "correcting" these values using an expensive (fine) solver that runs in parallel on a number of temporal subintervals. In this work, we propose a time-parallel algorithm (GParareal) that solves IVPs by modelling the correction term, i.e. the difference between fine and coarse solutions, using a Gaussian process emulator. This approach compares favourably with the classic parareal algorithm and we demonstrate, on a number of IVPs, that GParareal can converge in fewer iterations than parareal, leading to an increase in parallel speed-up. GParareal also manages to locate solutions to certain IVPs where parareal fails and has the additional advantage of being able to use archives of legacy solutions, e.g. solutions from prior runs of the IVP for different initial conditions, to further accelerate convergence of the method -- something that existing time-parallel methods do not do