Multi-stage splitting integrators for sampling with modified Hamiltonian Monte Carlo methods

Modified Hamiltonian Monte Carlo (MHMC) methods combine the ideas behind two popular sampling approaches: Hamiltonian Monte Carlo (HMC) and importance sampling. As in the HMC case, the bulk of the computational cost of MHMC algorithms lies in the numerical integration of a Hamiltonian system of differential equations. We suggest novel integrators designed to enhance accuracy and sampling performance of MHMC methods. The novel integrators belong to families of splitting algorithms and are therefore easily implemented. We identify optimal integrators within the families by minimizing the energy error or the average energy error. We derive and discuss in detail the modified Hamiltonians of the new integrators, as the evaluation of those Hamiltonians is key to the efficiency of the overall algorithms. Numerical experiments show that the use of the new integrators may improve very significantly the sampling performance of MHMC methods, in both statistical and molecular dynamics problems.MTM2013-46553-C3-1-P, MTM2016-77660-P, VA024P17, BES-2014-06864

Modified Hamiltonian Monte Carlo for Bayesian Inference

The Hamiltonian Monte Carlo (HMC) method has been recognized as a powerful sampling tool in computational statistics. We show that performance of HMC can be significantly improved by incorporating importance sampling and an irreversible part of the dynamics into a chain. This is achieved by replacing Hamiltonians in the Metropolis test with modified Hamiltonians, and a complete momentum update with a partial momentum refreshment. We call the resulting generalized HMC importance sampler—Mix & Match Hamiltonian Monte Carlo (MMHMC). The method is irreversible by construction and further benefits from (i) the efficient algorithms for computation of modified Hamiltonians; (ii) the implicit momentum update procedure and (iii) the multi-stage splitting integrators specially derived for the methods sampling with modified Hamiltonians. MMHMC has been implemented, tested on the popular statistical models and compared in sampling efficiency with HMC, Riemann Manifold Hamiltonian Monte Carlo, Generalized Hybrid Monte Carlo, Generalized Shadow Hybrid Monte Carlo, Metropolis Adjusted Langevin Algorithm and Random Walk Metropolis-Hastings. To make a fair comparison, we propose a metric that accounts for correlations among samples and weights, and can be readily used for all methods which generate such samples. The experiments reveal the superiority of MMHMC over popular sampling techniques, especially in solving high dimensional problems.Agile BioFoundry (http://agilebiofoundry.org) supported by the U.S. Department of Energy, Energy Efficiency and Renewable Energy, Bioenergy Technologies Office, through contract DE-AC02-05CH11231 between Lawrence Berkeley National Laboratory and the U.S. Department of Energy

Adapting Hybrid Monte Carlo methods for solving complex problems in life and materials sciences

Efficient sampling is the key to success of molecular simulation of complex physical systems. Still, a unique recipe for achieving this goal is unavailable. Hybrid Monte Carlo (HMC) is a promising sampling tool offering a smart, free of discretization errors, propagation in phase space, rigorous temperature control, and flexibility. However, its inability to provide dynamical information and its weakness in simulations of reasonably large systems do not allow HMC to become a sampler of choice in molecular simulation of complex systems. In this thesis, we show that performance of HMC can be dramatically improved by introducing in the method the splitting numerical integrators and importance sampling. We propose a novel splitting integration scheme called Adaptive Integration Approach or AIA, which leads to very promising improvements in accuracy and sampling in HMC simulations. Given a simulation problem and a time step, AIA automatically chooses the optimal scheme out of the family of two-stage splitting integrators. A system-specific integrator identified by our approach is optimal in the sense that it provides the best conservation of energy for harmonic forces. The role of importance sampling on the performance of HMC is studied through the modified Hamiltonian Monte Carlo (MHMC) methods, sampling with respect to a modified or shadow Hamiltonian. The particular attention is paid to Generalized Shadow Hybrid Monte Carlo (GSHMC), introduced by Akhmatskaya and Reich in 2008. To improve the performance of MHMC in general and GSHMC in particular, we develop and test the new multi-stage splitting integrators, specially formulated for sampling with respect to modified Hamiltonians. The novel adaptive two-stage integration approach or MAIA, specifically derived for MHMC is presented. We also discuss in detail the adaptation of GSHMC to the NPT ensemble and provide the thorough analysis of its performance. Moreover, for the first time, we formulate GSHMC in the grand canonical ensemble. A general framework, useful for an extension of other Hybrid Monte Carlo methods to the grand canonical ensemble, is also provided. The software development is another fundamental part of the present work. The algorithms presented in this thesis are implemented in MultiHMC-GROMACS, an in-house version of the popular software package GROMACS. We explain the details of such implementation and give useful recommendations and hints for implementation of the new algorithms in other software packages. In summary, in this thesis, we propose the new numerical algorithms that are capable of improving the accuracy and sampling efficiency of molecular simulations with Hybrid Monte Carlo methods. We show that equipping the Hybrid Monte Carlo algorithm with extra features makes it even a “smarter” sampler and, no doubts, a strong competitor to the well-established molecular simulation techniques such as molecular dynamics (MD) and Monte Carlo. The up to 60 times increase in sampling efficiency of GSHMC over MD, due to the new algorithms in simulations of selected systems, supports such a belief.MTM2013-46553-C3-1-

Palindromic 3-stage splitting integrators, a roadmap

The implementation of multi-stage splitting integrators is essentially the same as the implementation of the familiar Strang/Verlet method. Therefore multi-stage formulas may be easily incorporated into software that now uses the Strang/Verlet integrator. We study in detail the two-parameter family of palindromic, three-stage splitting formulas and identify choices of parameters that may outperform the Strang/Verlet method. One of these choices leads to a method of effective order four suitable to integrate in time some partial differential equations. Other choices may be seen as perturbations of the Strang method that increase efficiency in molecular dynamics simulations and in Hybrid Monte Carlo sampling.Comment: 20 pages, 8 figures, 2 table

Adaptive multi-stage integrators for optimal energy conservation in molecular simulations

We introduce a new Adaptive Integration Approach (AIA) to be used in a wide range of molecular simulations. Given a simulation problem and a step size, the method automatically chooses the optimal scheme out of an available family of numerical integrators. Although we focus on two-stage splitting integrators, the idea may be used with more general families. In each instance, the system-specific integrating scheme identified by our approach is optimal in the sense that it provides the best conservation of energy for harmonic forces. The AIA method has been implemented in the BCAM-modified GROMACS software package. Numerical tests in molecular dynamics and hybrid Monte Carlo simulations of constrained and unconstrained physical systems show that the method successfully realises the fail-safe strategy. In all experiments, and for each of the criteria employed, the AIA is at least as good as, and often significantly outperforms the standard Verlet scheme, as well as fixed parameter, optimized two-stage integrators. In particular, the sampling efficiency found in simulations using the AIA is up to 5 times better than the one achieved with other tested schemes

Enhancing Sampling in Computational Statistics Using Modified Hamiltonians

The Hamiltonian Monte Carlo (HMC) method has been recognized as a powerful sampling tool in computational statistics. In this thesis, we show that performance of HMC can be dramatically improved by replacing Hamiltonians in the Metropolis test with modified Hamiltonians, and a complete momentum update with a partial momentum refreshment. The resulting generalized HMC importance sampler, which we called Mix & Match Hamiltonian Monte Carlo (MMHMC), arose as an extension of the Generalized Shadow Hybrid Monte Carlo (GSHMC) method, previously proposed for molecular simulation. The MMHMC method adapts GSHMC specifically to computational statistics and enriches it with new essential features: (i) the efficient algorithms for computation of modified Hamiltonians; (ii) the implicit momentum update procedure and (iii) the two-stage splitting integration schemes specially derived for the methods sampling with modified Hamiltonians. In addition, different optional strategies for momentum update and flipping are introduced as well as algorithms for adaptive tuning of parameters and efficient sampling of multimodal distributions are developed. MMHMC has been implemented in the in-house software package HaiCS (Hamiltonians in Computational Statistics) written in C, tested on the popular statistical models and compared in sampling efficiency with HMC, Generalized Hybrid Monte Carlo, Riemann Manifold Hamiltonian Monte Carlo, Metropolis Adjusted Langevin Algorithm and Random Walk Metropolis-Hastings. The analysis of time-normalized effective sample size reveals the superiority of MMHMC over popular sampling techniques, especially in solving high-dimensional problems.FPU12/05209, MTM2013–46553–C3–1–

Enhancing sampling in computational statistics using modified hamiltonians

154 p.The Hamiltonian Monte Carlo (HMC) method has been recognized as a powerful sampling tool in computational statistics. In this thesis,we showthat performance ofHMCcan be dramatically improved by replacing Hamiltonians in theMetropolis test with modified Hamiltonians, and a complete momentum update with a partial momentum refreshment. The resulting generalized HMC importance sampler, whichwe called Mix & Match Hamiltonian Monte Carlo (MMHMC), arose as an extension of the Generalized Shadow Hybrid Monte Carlo (GSHMC) method, previously proposed for molecular simulation. The MMHMC method adapts GSHMC specifically to computational statistics and enriches it with new essential features: (i) the e icient algorithms for computation of modified Hamiltonians; (ii) the implicit momentum update procedure and (iii) the two-stage splitting integration schemes specially derived for the methods sampling with modified Hamiltonians. In addition, di erent optional strategies formomentumupdate and flipping are introduced as well as algorithms for adaptive tuning of parameters and e icient sampling of multimodal distributions are developed. MMHMChas been implemented in the in-house so ware package HaiCS (Hamiltonians in Computational Statistics) written in C, tested on the popular statistical models and compared in sampling e iciency with HMC, Generalized Hybrid Monte Carlo, Riemann Manifold Hamiltonian Monte Carlo, Metropolis Adjusted Langevin Algorithm and RandomWalk Metropolis-Hastings. The analysis of time-normalized e ective sample size reveals the superiority of MMHMC over popular sampling techniques, especially in solving high-dimensional problems.Basque Center for Applied Mathematic

Symmetrically Processed Splitting Integrators for Enhanced Hamiltonian Monte Carlo Sampling

[EN] We construct integrators to be used in Hamiltonian (or Hybrid) Monte Carlo sampling. The new integrators are easily implementable and, for a given computational budget, may deliver five times as many accepted proposals as standard leapfrog/Verlet without impairing in any way the quality of the samples. They are based on a suitable modification of the processing technique first introduced by Butcher. The idea of modified processing may also be useful for other purposes, like the construction of high-order splitting integrators with positive coefficients.The first, third, and fourth authors were supported by project PID2019-104927GB-C21 (AEI/FEDER, UE) . The second author was supported by projects PID2019-104927GB-C22 (GNI-QUAMC) , (AEI/FEDER, UE) VA105G18, and VA169P20 (Junta de Castilla y Leon, ES) co-financed by FEDER funds.Blanes Zamora, S.; Calvo, M.; Casas, F.; Sanz-Serna, JM. (2021). Symmetrically Processed Splitting Integrators for Enhanced Hamiltonian Monte Carlo Sampling. SIAM Journal on Scientific Computing. 43(5):A3357-A3371. https://doi.org/10.1137/20M137940X30SA3357A337143

Symmetrically processed splitting integrators for enhanced hamiltonian monte carlo sampling

We construct integrators to be used in Hamiltonian (or Hybrid) Monte Carlo sampling. The new integrators are easily implementable and, for a given computational budget, may deliver five times as many accepted proposals as standard leapfrog/Verlet without impairing in any way the quality of the samples. They are based on a suitable modification of the processing technique first introduced by Butcher. The idea of modified processing may also be useful for other purposes, like the construction of high-order splitting integrators with positive coefficients

Adaptive Splitting Integrators for Enhancing Sampling Efficiency of Modified Hamiltonian Monte Carlo Methods in Molecular Simulation

The modified Hamiltonian Monte Carlo (MHMC) methods, i.e., importance sampling methods that use modified Hamiltonians within a Hybrid Monte Carlo (HMC) framework, often outperform in sampling efficiency standard techniques such as molecular dynamics (MD) and HMC. The performance of MHMC may be enhanced further through the rational choice of the simulation parameters and by replacing the standard Verlet integrator with more sophisticated splitting algorithms. Unfortunately, it is not easy to identify the appropriate values of the parameters that appear in those algorithms. We propose a technique, that we call MAIA (Modified Adaptive Integration Approach), which, for a given simulation system and a given time step, automatically selects the optimal integrator within a useful family of two-stage splitting formulas. Extended MAIA (or e-MAIA) is an enhanced version of MAIA, which additionally supplies a value of the method-specific parameter that, for the problem under consideration, keeps the momentum acceptance rate at a user-desired level. The MAIA and e-MAIA algorithms have been implemented, with no computational overhead during simulations, in MultiHMC-GROMACS, a modified version of the popular software package GROMACS. Tests performed on well-known molecular models demonstrate the superiority of the suggested approaches over a range of integrators (both standard and recently developed), as well as their capacity to improve the sampling efficiency of GSHMC, a noticeable method for molecular simulation in the MHMC family. GSHMC combined with e-MAIA shows a remarkably good performance when compared to MD and HMC coupled with the appropriate adaptive integrators.MTM2013-46553-C3-1-P MTM2016-77660-