Bayesian Adaptive Hamiltonian Monte Carlo with an Application to High-Dimensional BEKK GARCH Models

Hamiltonian Monte Carlo (HMC) is a recent statistical procedure to sample from complex distributions. Distant proposal draws are taken in a equence of steps following the Hamiltonian dynamics of the underlying parameter space, often yielding superior mixing properties of the resulting Markov chain. However, its performance can deteriorate sharply with the degree of irregularity of the underlying likelihood due to its lack of local adaptability in the parameter space. Riemann Manifold HMC (RMHMC), a locally adaptive version of HMC, alleviates this problem, but at a substantially increased computational cost that can become prohibitive in high-dimensional scenarios. In this paper we propose the Adaptive HMC (AHMC), an alternative inferential method based on HMC that is both fast and locally adaptive, combining the advantages of both HMC and RMHMC. The benefits become more pronounced with higher dimensionality of the parameter space and with the degree of irregularity of the underlying likelihood surface. We show that AHMC satisfies detailed balance for a valid MCMC scheme and provide a comparison with RMHMC in terms of effective sample size, highlighting substantial efficiency gains of AHMC. Simulation examples and an application of the BEKK GARCH model show the usefulness of the new posterior sampler.High-dimensional joint sampling; Markov chain Monte Carlo; Multivariate GARCH

Bayesian Adaptive Hamiltonian Monte Carlo with an Application to High-Dimensional BEKK GARCH Models

Hamiltonian Monte Carlo (HMC) is a recent statistical procedure to sample from complex distributions. Distant proposal draws are taken in a equence of steps following the Hamiltonian dynamics of the underlying parameter space, often yielding superior mixing properties of the resulting Markov chain. However, its performance can deteriorate sharply with the degree of irregularity of the underlying likelihood due to its lack of local adaptability in the parameter space. Riemann Manifold HMC (RMHMC), a locally adaptive version of HMC, alleviates this problem, but at a substantially increased computational cost that can become prohibitive in high-dimensional scenarios. In this paper we propose the Adaptive HMC (AHMC), an alternative inferential method based on HMC that is both fast and locally adaptive, combining the advantages of both HMC and RMHMC. The benefits become more pronounced with higher dimensionality of the parameter space and with the degree of irregularity of the underlying likelihood surface. We show that AHMC satisfies detailed balance for a valid MCMC scheme and provide a comparison with RMHMC in terms of effective sample size, highlighting substantial efficiency gains of AHMC. Simulation examples and an application of the BEKK GARCH model show the usefulness of the new posterior sampler.High-dimensional joint sampling; Markov chain Monte Carlo; Multivariate GARCH

Metropolis Integration Schemes for Self-Adjoint Diffusions

We present explicit methods for simulating diffusions whose generator is self-adjoint with respect to a known (but possibly not normalizable) density. These methods exploit this property and combine an optimized Runge-Kutta algorithm with a Metropolis-Hastings Monte-Carlo scheme. The resulting numerical integration scheme is shown to be weakly accurate at finite noise and to gain higher order accuracy in the small noise limit. It also permits to avoid computing explicitly certain terms in the equation, such as the divergence of the mobility tensor, which can be tedious to calculate. Finally, the scheme is shown to be ergodic with respect to the exact equilibrium probability distribution of the diffusion when it exists. These results are illustrated on several examples including a Brownian dynamics simulation of DNA in a solvent. In this example, the proposed scheme is able to accurately compute dynamics at time step sizes that are an order of magnitude (or more) larger than those permitted with commonly used explicit predictor-corrector schemes.Comment: 54 pages, 8 figures, To appear in MM

Optimal tuning of the hybrid Monte Carlo algorithm

We investigate the properties of the hybrid Monte Carlo algorithm (HMC) in high dimensions. HMC develops a Markov chain reversible with respect to a given target distribution Π using separable Hamiltonian dynamics with potential −logΠ−log⁡Π. The additional momentum variables are chosen at random from the Boltzmann distribution, and the continuous-time Hamiltonian dynamics are then discretised using the leapfrog scheme. The induced bias is removed via a Metropolis–Hastings accept/reject rule. In the simplified scenario of independent, identically distributed components, we prove that, to obtain an O(1) acceptance probability as the dimension dd of the state space tends to ∞, the leapfrog step size hh should be scaled as h=l×d^(−1/4). Therefore, in high dimensions, HMC requires O(d^(1/4)) steps to traverse the state space. We also identify analytically the asymptotically optimal acceptance probability, which turns out to be 0.651 (to three decimal places). This value optimally balances the cost of generating a proposal, which decreases as l increases (because fewer steps are required to reach the desired final integration time), against the cost related to the average number of proposals required to obtain acceptance, which increases as l increases

Towards More Efficient Enhanced Sampling Methods To Study Phase Transitions

The most familiar phase transitions observed in nature are associated with a change in the state of matter (solid, liquid, and gas). In some rare cases this may involve the plasma phase. Such transitions are often referred to as first order phase transitions and often occur commonly such as during the melting of snow or freezing of lakes and rivers during winter. This project focuses on the most ubiquitous phase changes such as, liquid-solid and vapor-liquid as well as the less prevalent vapor-solid transitions. These types of phase transitions are also known as classical phase transitions. They usually involve symmetry breaking and can be identified by a singularity in the free energy or one of its derivatives. More modern classification of phase transitions relies on the order parameters as exemplified by the Landau\u27s theory. An order parameter is a quantity that takes a value of zero in the disordered phase and assumes finite values in the ordered phase. In the case of liquid-vapor transition, the order parameter is the density. The study of phase transitions is often complicated by the amount of time required by these phase changes and the presence of a high free energy barrier. Consequently, changes occurring close to coexistence are hard or even impossible to follow via conventional experimental techniques. Molecular simulation is therefore the method of choice to study these processes. Molecular simulations are numerical experiments carried out on model systems and have a number of advantages over traditional experiments. Simulations do not have any limitation as to the type of molecules or conditions under which they can be applied. Current simulation methods used to accomplish this task, such as the grand canonical and Gibbs ensemble Monte Carlo methods, employ the concept of particles insertion and deletion moves or requires the knowledge of at least one point at coexistence. These types of moves are extremely inefficient when dense fluids are involved and limit the accuracy of these methods. To circumvent these difficulties, non-Boltzmann sampling methods such as the umbrella sampling and Wang-Landau sampling techniques, have been employed to study these phase transitions. Vapor-solid and liquid-solid phase transitions were studied using a combination of hybrid Monte Carlo (HMC) and the umbrella sampling on a system of C60 molecules. The crystallization process occurs in two steps, nucleation and growth. The nucleation step is an activated process that involves a high free energy barrier. The free energy barrier is overcome through a series of HMC steps. The growth step on the other hand is studied by means of unconstrained molecular dynamics (MD). This study illustrates that the body centered cubic structure plays no role in the crystallization of C60. This is because only the face centered cubic and the hexagonal closed parked crystal structures were observed in both the nucleation and growth steps. In addition, the growth process is observed to follow a complex mechanism known as cross nucleation. The process of cross nucleation has also been observed in model fluids such as Lennard-Jones fluid and in the experimental study of D-mannitol. Hybrid Monte Carlo and configurational bias Monte Carlo (CBMC) were combined with the Wang-Landau (WL) sample method to study the vapor-liquid equilibria of Polycyclic aromatic hydrocarbons (PAHs) with four fused benzene rings and &alpha-olefins (C2 - C6 respectively. These studies are conducted in the isothermal-isobaric (NPT) ensemble to avoid the particle insertion and deletion moves that resulted in low acceptance rates in previous simulations. These studies led to the prediction of the critical temperatures, pressures and densities of both systems

Comparative Study of Nonlinear Acoustic and Guided Wave Methods for Structural Damage Detection

The overall purpose of this research work is to use nonlinear acoustic techniques and Lamb wave methods for Structural Health Monitoring (SHM). The work constitutes fatigue crack detection studies on glass and aluminium plates as well as low-velocity impact damage and compression damages on carbon fibre reinforced polymer. In addition, the SHM techniques were evaluated by detecting damage on a hammer impacted wind turbine blade. For nonlinear acoustic tests, Finite Element (FE) modeling was used to calculate the crack edge divergence for three different crack modes. After that, FE modeling extracted the modal parameters (e.g. natural frequencies and mode shapes) of vibration modes for the corresponding crack modes. These selected vibration modes were used for low frequency excitation in nonlinear acoustic experiments. Experimental work was performed to analyse the effect of nonlinear acoustics by signal wave excitation, Frequency Response Functions (FRFs) with varying excitation levels and Vibro-acoustic excitation. Various physical mechanisms to account for these effects have been investigated. The experimental results present three main nonlinear effects. These effects are non-classical Luxemburg-Gorky (L-G) type dissipation, the dissipation mechanism related to crack-wave interaction and nonlinear elasticity. The application of outlier analysis on Lamb wave tests is a novelty detection method. This method has indicated successful classification for undamaged and damaged data in fatigue tests, compression tests and impact tests. In addition, outlier analysis is able to give an indication of damage severity in the glass plate test. Moreover, outlier analysis gives the information to localise damage in the wind turbine blade test

Molecular Dynamics Simulation

Condensed matter systems, ranging from simple fluids and solids to complex multicomponent materials and even biological matter, are governed by well understood laws of physics, within the formal theoretical framework of quantum theory and statistical mechanics. On the relevant scales of length and time, the appropriate ‘first-principles’ description needs only the Schroedinger equation together with Gibbs averaging over the relevant statistical ensemble. However, this program cannot be carried out straightforwardly—dealing with electron correlations is still a challenge for the methods of quantum chemistry. Similarly, standard statistical mechanics makes precise explicit statements only on the properties of systems for which the many-body problem can be effectively reduced to one of independent particles or quasi-particles. [...