Postprocessed integrators for the high order integration of ergodic SDEs

The concept of effective order is a popular methodology in the deterministic literature for the construction of efficient and accurate integrators for differential equations over long times. The idea is to enhance the accuracy of a numerical method by using an appropriate change of variables called the processor. We show that this technique can be extended to the stochastic context for the construction of new high order integrators for the sampling of the invariant measure of ergodic systems. The approach is illustrated with modifications of the stochastic $\theta$-method applied to Brownian dynamics, where postprocessors achieving order two are introduced. Numerical experiments, including stiff ergodic systems, illustrate the efficiency and versatility of the approach.Comment: 21 pages, to appear in SIAM J. Sci. Compu

On the Numerical Stationary Distribution of Overdamped Langevin Equation in Harmonic System

Efficient numerical algorithm for stochastic differential equation has been an important object in the research of statistical physics and mathematics for a long time. In this paper we study the highly accurate numerical algorithm of the overdamped Langevin equation. In particular, our interest is the behaviour of the numerical schemes for solving the overdamped Langevin equation in the harmonic system. Three algorithms are obtained for overdamped Langevin equation, from the large friction limit of the schemes for underdamped Langevin dynamics. We derive the explicit expression of the stationary distribution of each algorithm by analysing the discrete time trajectory, for both one-dimensional and multi-dimensional cases. The accuracy of the stationary distribution of each algorithm is illustrated by comparing to the exact Boltzmann distribution. Our results demonstrate that, the "BAOA-limit" algorithm generates the exact distribution for the harmonic system in the canonical ensemble, within the stable regime of the time interval. The other algorithms do not produce the exact distribution of the harmonic system.Comment: 19 page

On the long-time integration of stochastic gradient systems

This article addresses the weak convergence of numerical methods for Brownian dynamics. Typical analyses of numerical methods for stochastic differential equations focus on properties such as the weak order which estimates the asymptotic (stepsize h → 0) convergence behavior of the error of finite time averages. Recently it has been demonstrated, by study of Fokker-Planck operators, that a non-Markovian numerical method [Leimkuhler and Matthews, 2013] generates approximations in the long time limit with higher accuracy order (2nd order) than would be expected from its weak convergence analysis (finite-time averages are 1st order accurate). In this article we describe the transition from the transient to the steady-state regime of this numerical method by estimating the time-dependency of the coefficients in an asymptotic expansion for the weak error, demonstrating that the convergence to 2nd order is exponentially rapid in time. Moreover, we provide numerical tests of the theory, including comparisons of the efficiencies of the Euler-Maruyama method, the popular 2nd order Heun method, and the non-Markovian method

Time correlation functions of equilibrium and nonequilibrium Langevin dynamics: Derivations and numerics using random numbers

We study the time correlation functions of coupled linear Langevin dynamics without and with inertia effects, both analytically and numerically. The model equation represents the physical behavior of a harmonic oscillator in two or three dimensions in the presence of friction, noise, and an external field with both rotational and deformational components. This simple model plays pivotal roles in understanding more complicated processes. The presented analytical solution serves as a test of numerical integration schemes, its derivation is presented in a fashion that allows to be repeated directly in a classroom. While the results in the absence of fields (equilibrium) or confinement (free particle) are omnipresent in the literature, we write down, apparently for the first time, the full nonequilibrium results that may correspond, e.g., to a Hookean dumbbell embedded in a macroscopically homogeneous shear or mixed flow field. We demonstrate how the inertia results reduce to their noninertia counterparts in the nontrivial limit of vanishing mass. While the results are derived using basic integrations over Dirac delta distributions, we mention its relationship with alternative approaches involving (i) Fourier transforms, that seems advantageous only if the measured quantities also reside in Fourier space, and (ii) a Fokker--Planck equation and the moments of the probability distribution. The results, verified by numerical experiments, provide additional means of measuring the performance of numerical methods for such systems. It should be emphasized that this manuscript provides specific details regarding the derivations of the time correlation functions as well as the implementations of various numerical methods, so that it can serve as a standalone piece as part of education in the framework of stochastic differential equations and calculus.Comment: 35 pages, 5 figure

Exotic aromatic B-series for the study of long time integrators for a class of ergodic SDEs

We introduce a new algebraic framework based on a modification (called exotic) of aromatic Butcher-series for the systematic study of the accuracy of numerical integrators for the invariant measure of a class of ergodic stochastic differential equations (SDEs) with additive noise. The proposed analysis covers Runge-Kutta type schemes including the cases of partitioned methods and postprocessed methods. We also show that the introduced exotic aromatic B-series satisfy an isometric equivariance property.Comment: 33 page

Accelerated Bayesian imaging by relaxed proximal-point Langevin sampling

This paper presents a new accelerated proximal Markov chain Monte Carlo methodology to perform Bayesian inference in imaging inverse problems with an underlying convex geometry. The proposed strategy takes the form of a stochastic relaxed proximal-point iteration that admits two complementary interpretations. For models that are smooth or regularised by Moreau-Yosida smoothing, the algorithm is equivalent to an implicit midpoint discretisation of an overdamped Langevin diffusion targeting the posterior distribution of interest. This discretisation is asymptotically unbiased for Gaussian targets and shown to converge in an accelerated manner for any target that is $\kappa$-strongly log-concave (i.e., requiring in the order of $\sqrt{\kappa}$ iterations to converge, similarly to accelerated optimisation schemes), comparing favorably to [M. Pereyra, L. Vargas Mieles, K.C. Zygalakis, SIAM J. Imaging Sciences, 13,2 (2020), pp. 905-935] which is only provably accelerated for Gaussian targets and has bias. For models that are not smooth, the algorithm is equivalent to a Leimkuhler-Matthews discretisation of a Langevin diffusion targeting a Moreau-Yosida approximation of the posterior distribution of interest, and hence achieves a significantly lower bias than conventional unadjusted Langevin strategies based on the Euler-Maruyama discretisation. For targets that are $\kappa$-strongly log-concave, the provided non-asymptotic convergence analysis also identifies the optimal time step which maximizes the convergence speed. The proposed methodology is demonstrated through a range of experiments related to image deconvolution with Gaussian and Poisson noise, with assumption-driven and data-driven convex priors. Source codes for the numerical experiments of this paper are available from https://github.com/MI2G/accelerated-langevin-imla.Comment: 34 pages, 13 figure

A uniformly accurate scheme for the numerical integration of penalized Langevin dynamics

In molecular dynamics, penalized overdamped Langevin dynamics are used to model the motion of a set of particles that follow constraints up to a parameter ε. The most used schemes for simulating these dynamics are the Euler integrator in Rd and the constrained Euler integrator. Both have weak order one of accuracy, but work properly only in specific regimes depending on the size of the parameter ε. We propose in this paper a new consistent method with an accuracy independent of ε for solving penalized dynamics on a manifold of any dimension. Moreover, this method converges to the constrained Euler scheme when ε goes to zero. The numerical experiments confirm the theoretical findings, in the context of weak convergence and for the invariant measure, on a torus and on the orthogonal group in high dimension and high codimension.publishedVersio

Order conditions for sampling the invariant measure of ergodic stochastic differential equations on manifolds

We derive a new methodology for the construction of high order integrators for sampling the invariant measure of ergodic stochastic differential equations with dynamics constrained on a manifold. We obtain the order conditions for sampling the invariant measure for a class of Runge-Kutta methods applied to the constrained overdamped Langevin equation. The analysis is valid for arbitrarily high order and relies on an extension of the exotic aromatic Butcher-series formalism. To illustrate the methodology, a method of order two is introduced, and numerical experiments on the sphere, the torus and the special linear group confirm the theoretical findings.Comment: 40 page