Bayesian Adaptive Hamiltonian Monte Carlo with an Application to High-Dimensional BEKK GARCH Models

Hamiltonian Monte Carlo (HMC) is a recent statistical procedure to sample from complex distributions. Distant proposal draws are taken in a equence of steps following the Hamiltonian dynamics of the underlying parameter space, often yielding superior mixing properties of the resulting Markov chain. However, its performance can deteriorate sharply with the degree of irregularity of the underlying likelihood due to its lack of local adaptability in the parameter space. Riemann Manifold HMC (RMHMC), a locally adaptive version of HMC, alleviates this problem, but at a substantially increased computational cost that can become prohibitive in high-dimensional scenarios. In this paper we propose the Adaptive HMC (AHMC), an alternative inferential method based on HMC that is both fast and locally adaptive, combining the advantages of both HMC and RMHMC. The benefits become more pronounced with higher dimensionality of the parameter space and with the degree of irregularity of the underlying likelihood surface. We show that AHMC satisfies detailed balance for a valid MCMC scheme and provide a comparison with RMHMC in terms of effective sample size, highlighting substantial efficiency gains of AHMC. Simulation examples and an application of the BEKK GARCH model show the usefulness of the new posterior sampler.High-dimensional joint sampling; Markov chain Monte Carlo; Multivariate GARCH

Bayesian Adaptive Hamiltonian Monte Carlo with an Application to High-Dimensional BEKK GARCH Models

Hamiltonian Monte Carlo (HMC) is a recent statistical procedure to sample from complex distributions. Distant proposal draws are taken in a equence of steps following the Hamiltonian dynamics of the underlying parameter space, often yielding superior mixing properties of the resulting Markov chain. However, its performance can deteriorate sharply with the degree of irregularity of the underlying likelihood due to its lack of local adaptability in the parameter space. Riemann Manifold HMC (RMHMC), a locally adaptive version of HMC, alleviates this problem, but at a substantially increased computational cost that can become prohibitive in high-dimensional scenarios. In this paper we propose the Adaptive HMC (AHMC), an alternative inferential method based on HMC that is both fast and locally adaptive, combining the advantages of both HMC and RMHMC. The benefits become more pronounced with higher dimensionality of the parameter space and with the degree of irregularity of the underlying likelihood surface. We show that AHMC satisfies detailed balance for a valid MCMC scheme and provide a comparison with RMHMC in terms of effective sample size, highlighting substantial efficiency gains of AHMC. Simulation examples and an application of the BEKK GARCH model show the usefulness of the new posterior sampler.High-dimensional joint sampling; Markov chain Monte Carlo; Multivariate GARCH

Essays in Semiparametric Econometrics and Panel Data Analysis

Limited dependent variable (LDV) panel data models pose substantial challenges in maximum likelihood estimation. The likelihood function in such models typically contains multivariate integrals that are often analytically intractable. To overcome such problem in a panel probit model with unobserved individual heterogeneity and autocorrelated errors, in Chapter 1 - co-authored with Roman Liesenfeld and Jean-François Richard - we perform classical and Bayesian analysis of the model based on the Efficient Importance Sampling (EIS) technique (Richard and Zhang, 2006). We apply our method to the product innovation activity of a panel of German manufacturing firms in response to imports and foreign direct investment confirming their positive effects. Nonetheless, our key coefficient estimates are smaller than found in previous literature which can be explained by our flexible model assumptions. The remaining two chapters present my work on new estimation methods for models based on conditional moment restrictions. Such models are frequently stipulated by economic theory but only a few estimators based directly on them have so far been analyzed in the literature. Indeed, estimation of parameters therein poses a difficult ill-posed inverse problem. Rather, these models are typically converted into unconditional moment restrictions that are easier to handle. However, such conversion results in a loss of information compared to the original specification. Using the information-theoretic framework of so-called Generalized Minimum Contrast (GMC) estimation, in Chapter 2 I propose a new class of estimators based directly on conditional moment restrictions that encompasses the entire GMC family. Moreover, I show that previous literature covering a few special cases of the GMC class use an arbitrary uniform weighting scheme over the space of exogenous variables that can be improved upon with optimal local weighting. All currently available GMC estimators are based on moments containing finite-dimensional Euclidean parameters. To alleviate a potential misspecification problem resulting from strong parametric assumptions, in Chapter 3 I propose a new Sieve-based Locally Weighted Conditional Empirical Likelihood (SLWCEL) estimator containing also infinite dimensional unknown functions, thus extending a special case of Chapter 2 to the semiparametric environment. Much of Chapter 3 is devoted to analysis of SLWCEL's asymptotic properties

Bayesian Analysis of a Probit Panel Data Model with Unobserved Individual Heterogeneity and Autocorrelated Errors

In this paper, we perform Bayesian analysis of a panel probit model with unobserved individual heterogeneity and serially correlated errors. We augment the data with latent variables and sample the unobserved heterogeneity component as one Gibbs block per individual using a flexible piecewise linear approximation to the marginal posterior density. The latent time effects are simulated as another Gibbs block. For this purpose we develop a new user-friendly form of the Efficient Importance Sampling proposal density for an Acceptance-Rejection Metropolis-Hastings step. We apply our method to the analysis of product innovation activity of a panel of German manufacturing firms in response to imports, foreign direct investment and other control variables. The dataset used here was analyzed under more restrictive assumptions by Bertschek and Lechner (1998) and Greene (2004). Although our results differ to a certain degree from these benchmark studies, we confirm the positive effect of imports and FDI on firms' innovation activity. Moreover, unobserved firm heterogeneity is shown to play a far more significant role in the application than the latent time effects.Dynamic latent variables; Markov Chain Monte Carlo; importance sampling

Adaptive networks of trading agents

Multi-agent models have been used in many contexts to study generic collective behavior. Similarly, complex networks have become very popular because of the diversity of growth rules giving rise to scale-free behavior. Here we study adaptive networks where the agents trade ``wealth'' when they are linked together while links can appear and disappear according to the wealth of the corresponding agents; thus the agents influence the network dynamics and vice-versa. Our framework generalizes a multi-agent model of Bouchand and Mezard, and leads to a steady state with fluctuating connectivities. The system spontaneously self-organizes into a critical state where the wealth distribution has a fat tail and the network is scale-free; in addition, network heterogeneities lead to enhanced wealth condensation.Comment: 7 figure

An a posteriori error estimate for the Stokes-Brinkman problem in a polygonal domain

summary:We derive a residual based a posteriori error estimate for the Stokes-Brinkman problem on a two-dimensional polygonal domain. We use Taylor-Hood triangular elements. The link to the possible information on the regularity of the problem is discussed

An application of the BDDC method to the Navier-Stokes equations in 3-D cavity

summary:We deal with numerical simulation of incompressible flow governed by the Navier-Stokes equations. The problem is discretised using the finite element method, and the arising system of nonlinear equations is solved by Picard iteration. We explore the applicability of the Balancing Domain Decomposition by Constraints (BDDC) method to nonsymmetric problems arising from such linearisation. One step of BDDC is applied as the preconditioner for the stabilized variant of the biconjugate gradient (BiCGstab) method. We present results for a 3-D cavity problem computed on 32 cores of a parallel supercomputer

Network of inherent structures in spin glasses: scaling and scale-free distributions

The local minima (inherent structures) of a system and their associated transition links give rise to a network. Here we consider the topological and distance properties of such a network in the context of spin glasses. We use steepest descent dynamics, determining for each disorder sample the transition links appearing within a given barrier height. We find that differences between linked inherent structures are typically associated with local clusters of spins; we interpret this within a framework based on droplets in which the characteristic ``length scale'' grows with the barrier height. We also consider the network connectivity and the degrees of its nodes. Interestingly, for spin glasses based on random graphs, the degree distribution of the network of inherent structures exhibits a non-trivial scale-free tail.Comment: minor changes and references adde