Quadrature Points via Heat Kernel Repulsion

We discuss the classical problem of how to pick $N$ weighted points on a $d-$dimensional manifold so as to obtain a reasonable quadrature rule $$\frac{1}{|M|}\int_{M}{f(x) dx} \simeq \frac{1}{N} \sum_{n=1}^{N}{a_i f(x_i)}.$$ This problem, naturally, has a long history; the purpose of our paper is to propose selecting points and weights so as to minimize the energy functional \sum_{i,j =1}^{N}{ a_i a_j \exp\left(-\frac{d(x_i,x_j)^2}{4t}\right) } \rightarrow \min, \quad \mbox{where}~t \sim N^{-2/d}, $d(x,y)$ is the geodesic distance and $d$ is the dimension of the manifold. This yields point sets that are theoretically guaranteed, via spectral theoretic properties of the Laplacian $-\Delta$, to have good properties. One nice aspect is that the energy functional is universal and independent of the underlying manifold; we show several numerical examples

Efficient posterior sampling for high-dimensional imbalanced logistic regression

High-dimensional data are routinely collected in many areas. We are particularly interested in Bayesian classification models in which one or more variables are imbalanced. Current Markov chain Monte Carlo algorithms for posterior computation are inefficient as $n$ and/or $p$ increase due to worsening time per step and mixing rates. One strategy is to use a gradient-based sampler to improve mixing while using data sub-samples to reduce per-step computational complexity. However, usual sub-sampling breaks down when applied to imbalanced data. Instead, we generalize piece-wise deterministic Markov chain Monte Carlo algorithms to include importance-weighted and mini-batch sub-sampling. These approaches maintain the correct stationary distribution with arbitrarily small sub-samples, and substantially outperform current competitors. We provide theoretical support and illustrate gains in simulated and real data applications.Comment: 4 figure

Hypocoercivity properties of adaptive Langevin dynamics

International audienceAdaptive Langevin dynamics is a method for sampling the Boltzmann-Gibbs distribution at prescribed temperature in cases where the potential gradient is subject to stochastic perturbation of unknown magnitude. The method replaces the friction in underdamped Langevin dynamics with a dynamical variable, updated according to a negative feedback loop control law as in the Nose-Hoover thermostat. Using a hypocoercivity analysis we show that the law of Adaptive Langevin dynamics converges exponentially rapidly to the stationary distribution, with a rate that can be quantified in terms of the key parameters of the dynamics. This allows us in particular to obtain a central limit theorem with respect to the time averages computed along a stochastic path. Our theoretical findings are illustrated by numerical simulations involving classification of the MNIST data set of handwritten digits using Bayesian logistic regression

Posterior computation with the Gibbs zig-zag sampler

An intriguing new class of piecewise deterministic Markov processes (PDMPs) has recently been proposed as an alternative to Markov chain Monte Carlo (MCMC). In order to facilitate the application to a larger class of problems, we propose a new class of PDMPs termed Gibbs zig-zag samplers, which allow parameters to be updated in blocks with a zig-zag sampler applied to certain parameters and traditional MCMC-style updates to others. We demonstrate the flexibility of this framework on posterior sampling for logistic models with shrinkage priors for high-dimensional regression and random effects and provide conditions for geometric ergodicity and the validity of a central limit theorem.Comment: 29 pages, 4 figure

Generalised Langevin equation: asymptotic properties and numerical analysis

In this thesis we concentrate on instances of the GLE which can be represented in a Markovian form in an extended phase space. We extend previous results on the geometric ergodicity of this class of GLEs using Lyapunov techniques, which allows us to conclude ergodicity for a large class of GLEs relevant to molecular dynamics applications. The main body of this thesis concerns the numerical discretisation of the GLE in the extended phase space representation. We generalise numerical discretisation schemes which have been previously proposed for the underdamped Langevin equation and which are based on a decomposition of the vector field into a Hamiltonian part and a linear SDE. Certain desirable properties regarding the accuracy of configurational averages of these schemes are inherited in the GLE context. We also rigorously prove geometric ergodicity on bounded domains by showing that a uniform minorisation condition and a uniform Lyapunov condition are satisfied for sufficiently small timestep size. We show that the discretisation schemes which we propose behave consistently in the white noise and overdamped limits, hence we provide a family of universal integrators for Langevin dynamics. Finally, we consider multiple-time stepping schemes making use of a decomposition of the fluctuation-dissipation term into a reversible and non-reversible part. These schemes are designed to efficiently integrate instances of the GLE whose Markovian representation involves a high number of auxiliary variables or a configuration dependent fluctuation-dissipation term. We also consider an application of dynamics based on the GLE in the context of large scale Bayesian inference as an extension of previously proposed adaptive thermostat methods. In these methods the gradient of the log posterior density is only evaluated on a subset (minibatch) of the whole dataset, which is randomly selected at each timestep. Incorporating a memory kernel in the adaptive thermostat formulation ensures that time-correlated gradient noise is dissipated in accordance with the fluctuation-dissipation theorem. This allows us to relax the requirement of using i.i.d. minibatches, and explore a variety of minibatch sampling approaches

Non-reversible Markov chain Monte Carlo for sampling of districting maps

Evaluating the degree of partisan districting (Gerrymandering) in a statistical framework typically requires an ensemble of districting plans which are drawn from a prescribed probability distribution that adheres to a realistic and non-partisan criteria. In this article we introduce novel non-reversible Markov chain Monte-Carlo (MCMC) methods for the sampling of such districting plans which have improved mixing properties in comparison to previously used (reversible) MCMC algorithms. In doing so we extend the current framework for construction of non-reversible Markov chains on discrete sampling spaces by considering a generalization of skew detailed balance. We provide a detailed description of the proposed algorithms and evaluate their performance in numerical experiments.Comment: 38 page

Hyperactive Learning (HAL) for Data-Driven Interatomic Potentials

Data-driven interatomic potentials have emerged as a powerful class of surrogate models for {\it ab initio} potential energy surfaces that are able to reliably predict macroscopic properties with experimental accuracy. In generating accurate and transferable potentials the most time-consuming and arguably most important task is generating the training set, which still requires significant expert user input. To accelerate this process, this work presents \text{\it hyperactive learning} (HAL), a framework for formulating an accelerated sampling algorithm specifically for the task of training database generation. The key idea is to start from a physically motivated sampler (e.g., molecular dynamics) and add a biasing term that drives the system towards high uncertainty and thus to unseen training configurations. Building on this framework, general protocols for building training databases for alloys and polymers leveraging the HAL framework will be presented. For alloys, ACE potentials for AlSi10 are created by fitting to a minimal HAL-generated database containing 88 configurations (32 atoms each) with fast evaluation times of <100 microsecond/atom/cpu-core. These potentials are demonstrated to predict the melting temperature with excellent accuracy. For polymers, a HAL database is built using ACE, able to determine the density of a long polyethylene glycol (PEG) polymer formed of 200 monomer units with experimental accuracy by only fitting to small isolated PEG polymers with sizes ranging from 2 to 32.Comment: 21 pages, 11 figure

Langevin dynamics with variable coefficients and nonconservative forces: from stationary states to numerical methods

Langevin dynamics is a versatile stochastic model used in biology, chemistry, engineering, physics and computer science. Traditionally, in thermal equilibrium, one assumes (i) the forces are given as the gradient of a potential and (ii) a fluctuation-dissipation relation holds between stochastic and dissipative forces; these assumptions ensure that the system samples a prescribed invariant Gibbs-Boltzmann distribution for a specified target temperature. In this article, we relax these assumptions, incorporating variable friction and temperature parameters and allowing nonconservative force fields, for which the form of the stationary state is typically not known a priori. We examine theoretical issues such as stability of the steady state and ergodic properties, as well as practical aspects such as the design of numerical methods for stochastic particle models. Applications to nonequilibrium systems with thermal gradients and active particles are discussed