The connections between Lyapunov functions for some optimization algorithms and differential equations

In this manuscript, we study the properties of a family of second-order differential equations with damping, its discretizations and their connections with accelerated optimization algorithms for $m$-strongly convex and $L$-smooth functions. In particular, using the Linear Matrix Inequality LMI framework developed by \emph{Fazlyab et. al. $(2018)$}, we derive analytically a (discrete) Lyapunov function for a two-parameter family of Nesterov optimization methods, which allows for the complete characterization of their convergence rate. In the appropriate limit, this family of methods may be seen as a discretization of a family of second-order ordinary differential equations for which we construct(continuous) Lyapunov functions by means of the LMI framework. The continuous Lyapunov functions may alternatively, be obtained by studying the limiting behaviour of their discrete counterparts. Finally, we show that the majority of typical discretizations of the family of ODEs, such as the Heavy ball method, do not possess Lyapunov functions with properties similar to those of the Lyapunov function constructed here for the Nesterov method.Comment: 21 pages, 1 figur

Wasserstein distance estimates for the distributions of numerical approximations to ergodic stochastic differential equations

We present a framework that allows for the non-asymptotic study of the 2 -Wasserstein distance between the invariant distribution of an ergodic stochastic differential equation and the distribution of its numerical approximation in the strongly log-concave case. This allows us to study in a unified way a number of different integrators proposed in the literature for the overdamped and underdamped Langevin dynamics. In addition, we analyze a novel splitting method for the underdamped Langevin dynamics which only requires one gradient evaluation per time step. Under an additional smoothness assumption on a d --dimensional strongly log-concave distribution with condition number κ , the algorithm is shown to produce with an O(κ5/4d1/4ϵ−1/2) complexity samples from a distribution that, in Wasserstein distance, is at most ϵ>0 away from the target distribution

Are Gauss-Legendre methods useful in molecular dynamics?

AbstractWe apply the two-stage Gauss-Legendre method to the numerical simulation of liquid argon, a typical problem in molecular dynamics. It is found that the scheme is less efficient than the Verlet/leapfrog method, standard in this sort of simulation