36 research outputs found
Connecting the Dots: Towards Continuous Time Hamiltonian Monte Carlo
Continuous time Hamiltonian Monte Carlo is introduced, as a powerful
alternative to Markov chain Monte Carlo methods for continuous target
distributions. The method is constructed in two steps: First Hamiltonian
dynamics are chosen as the deterministic dynamics in a continuous time
piecewise deterministic Markov process. Under very mild restrictions, such a
process will have the desired target distribution as an invariant distribution.
Secondly, the numerical implementation of such processes, based on adaptive
numerical integration of second order ordinary differential equations is
considered. The numerical implementation yields an approximate, yet highly
robust algorithm that, unlike conventional Hamiltonian Monte Carlo, enables the
exploitation of the complete Hamiltonian trajectories (hence the title). The
proposed algorithm may yield large speedups and improvements in stability
relative to relevant benchmarks, while incurring numerical errors that are
negligible relative to the overall Monte Carlo errors
Connecting the Dots: Numerical Randomized Hamiltonian Monte Carlo with State-Dependent Event Rates
Numerical generalized randomized Hamiltonian Monte Carlo is introduced, as a robust, easy to use and computationally fast alternative to conventional Markov chain Monte Carlo methods for continuous target distributions. A wide class of piecewise deterministic Markov processes generalizing Randomized HMC (Bou-Rabee and Sanz-Serna) by allowing for state-dependent event rates is defined. Under very mild restrictions, such processes will have the desired target distribution as an invariant distribution. Second, the numerical implementation of such processes, based on adaptive numerical integration of second order ordinary differential equations (ODEs) is considered. The numerical implementation yields an approximate, yet highly robust algorithm that, unlike conventional Hamiltonian Monte Carlo, enables the exploitation of the complete Hamiltonian trajectories (hence, the title). The proposed algorithm may yield large speedups and improvements in stability relative to relevant benchmarks, while incurring numerical biases that are negligible relative to the overall Monte Carlo errors. Granted access to a high-quality ODE code, the proposed methodology is both easy to implement and use, even for highly challenging and high-dimensional target distributions. Supplementary materials for this article are available online.publishedVersio
Log-density gradient covariance and automatic metric tensors for Riemann manifold Monte Carlo methods
A metric tensor for Riemann manifold Monte Carlo particularly suited for
non-linear Bayesian hierarchical models is proposed. The metric tensor is built
from symmetric positive semidefinite log-density gradient covariance (LGC)
matrices, which are also proposed and further explored here. The LGCs
generalize the Fisher information matrix by measuring the joint information
content and dependence structure of both a random variable and the parameters
of said variable. Consequently, positive definite Fisher/LGC-based metric
tensors may be constructed not only from the observation likelihoods as is
current practice, but also from arbitrarily complicated non-linear prior/latent
variable structures, provided the LGC may be derived for each conditional
distribution used to construct said structures. The proposed methodology is
highly automatic and allows for exploitation of any sparsity associated with
the model in question. When implemented in conjunction with a Riemann manifold
variant of the recently proposed numerical generalized randomized Hamiltonian
Monte Carlo processes, the proposed methodology is highly competitive, in
particular for the more challenging target distributions associated with
Bayesian hierarchical models
Simulated maximum likelihood for general stochastic volatility models: a change of variable approach
Maximum likelihood has proved to be a valuable tool for fitting the log-normal stochastic volatility model to financial returns time series. Using a sequential change of variable framework, we are able to cast more general stochastic volatility models into a form appropriate for importance samplers based on the Laplace approximation. We apply the methodology to two example models, showing that efficient importance samplers can be constructed even for highly non-Gaussian latent processes such as square-root diffusions.Change of Variable; Heston Model; Laplace Importance Sampler; Simulated Maximum Likelihood; Stochastic Volatility
Efficient high-dimensional importance sampling in mixture frameworks
This paper provides high-dimensional and flexible importance sampling procedures for the likelihood evaluation of dynamic latent variable models involving finite or infinite mixtures leading to possibly heavy tailed and/or multi-modal target densities. Our approach is based upon the efficient importance sampling (EIS) approach of Richard and Zhang (2007) and exploits the mixture structure of the model when constructing importance sampling distributions as mixture of distributions. The proposed mixture EIS procedures are illustrated with ML estimation of a student-t state space model for realized volatilities and a stochastic volatility model with leverage effects and jumps for asset returns. --dynamic latent variable model,importance sampling,marginalized likelihood,mixture,Monte Carlo,realized volatility,stochastic volatility
The Gibbs Sampler with Particle Efficient Importance Sampling for State-Space Models
We consider Particle Gibbs (PG) as a tool for Bayesian analysis of non-linear
non-Gaussian state-space models. PG is a Monte Carlo (MC) approximation of the
standard Gibbs procedure which uses sequential MC (SMC) importance sampling
inside the Gibbs procedure to update the latent and potentially
high-dimensional state trajectories. We propose to combine PG with a generic
and easily implementable SMC approach known as Particle Efficient Importance
Sampling (PEIS). By using SMC importance sampling densities which are
approximately fully globally adapted to the targeted density of the states,
PEIS can substantially improve the mixing and the efficiency of the PG draws
from the posterior of the states and the parameters relative to existing PG
implementations. The efficiency gains achieved by PEIS are illustrated in PG
applications to a univariate stochastic volatility model for asset returns, a
non-Gaussian nonlinear local-level model for interest rates, and a multivariate
stochastic volatility model for the realized covariance matrix of asset
returns
Simulated Maximum Likelihood Estimation for Latent Diffusion Models
In this paper a method is developed and implemented to provide the simulated maximum likelihood estimation of latent diffusions based on discrete data. The method is applicable to diffusions that either have latent elements in the state vector or are only observed at discrete time with a noise. Latent diffusions are very important in practical applications in nancial economics. The proposed approach synthesizes the closed form method of Aït-Sahalia (2008) and the ecient importance sampler of Richard and Zhang (2007). It does not require any inll observations to be introduced and hence is computationally tractable. The Monte Carlo study shows that the method works well in finite sample. The empirical applications illustrate usefulness of the method and find no evidence of infinite variance in the importance sampler.Closed-form approximation; Diusion Model; Ecient importance sampler
Estimating the GARCH Diffusion: Simulated Maximum Likelihood in Continuous Time
A new algorithm is developed to provide a simulated maximum likelihood estimation of the GARCH diffusion model of Nelson (1990) based on return data only. The method combines two accurate approximation procedures, namely, the polynomial expansion of Aït-Sahalia (2008) to approximate the transition probability density of return and volatility, and the Efficient Importance Sampler (EIS) of Richard and Zhang (2007) to integrate out the volatility. The first and second order terms in the polynomial expansion are used to generate a base-line importance density for an EIS algorithm. The higher order terms are included when evaluating the importance weights. Monte Carlo experiments show that the new method works well and the discretization error is well controlled by the polynomial expansion. In the empirical application, we fit the GARCH diffusion to equity data, perform diagnostics on the model fit, and test the finiteness of the importance weights.Ecient importance sampling; GARCH diusion model; Simulated Maximum likelihood; Stochastic volatility