Hypocoercivity in metastable settings and kinetic simulated annealing

Combining classical arguments for the analysis of the simulated annealing algorithm with the more recent hypocoercive method of distorted entropy, we prove the convergence for large time of the kinetic Langevin annealing with logarithmic cooling schedule

On the generalised Langevin equation for simulated annealing

In this paper, we consider the generalised (higher order) Langevin equation for the purpose of simulated annealing and optimisation of nonconvex functions. Our approach modifies the underdamped Langevin equation by replacing the Brownian noise with an appropriate Ornstein-Uhlenbeck process to account for memory in the system. Under reasonable conditions on the loss function and the annealing schedule, we establish convergence of the continuous time dynamics to a global minimum. In addition, we investigate the performance numerically and show better performance and higher exploration of the state space compared to the underdamped and overdamped Langevin dynamics with the same annealing schedule

On the generalized langevin equation for simulated annealing

In this paper, we consider the generalized (higher order) Langevin equation for the purpose of simulated annealing and optimization of nonconvex functions. Our approach modifies the underdamped Langevin equation by replacing the Brownian noise with an appropriate Ornstein–Uhlenbeck process to account for memory in the system. Under reasonable conditions on the loss function and the annealing schedule, we establish convergence of the continuous time dynamics to a global minimum. In addition, we investigate the performance numerically and show better performance and higher exploration of the state space compared to the underdamped Langevin dynamics with the same annealing schedule

Convergence and variance reduction for stochastic differential equations in sampling and optimisation

Three problems that are linked by way of motivation are addressed in this work. In the first part of the thesis, we study the generalised Langevin equation for simulated annealing with the underlying goal of improving continuous-time dynamics for the problem of global optimisation of nonconvex functions. The main result in this part is on the convergence to the global optimum, which is shown using techniques from hypocoercivity given suitable assumptions on the nonconvex function. Alongside, we investigate numerically the problem of parameter tuning in the continuous-time equation. In the second part of the thesis, this last problem is addressed rigorously for the underdamped Langevin dynamics. In particular, a systematic procedure for finding the optimal friction matrix in the sampling problem is presented. We give an expression for the gradient of the asymptotic variance in terms of solutions to Poisson equations and present a working algorithm for approximating its value. Lastly, regularity of an associated semigroup, twice differentiable-in-space solutions to the Kolmogorov equation and weak numerical convergence rates of order one are shown for a class of stochastic differential equations with superlinearly growing, non-globally monotone coefficients. In the relation to the previous part, the results allow the use of Poisson equations for variations of Langevin dynamics not permissible before.Open Acces