Integrated Nested Laplace Approximations for Large-Scale Spatial-Temporal Bayesian Modeling

Bayesian inference tasks continue to pose a computational challenge. This especially holds for spatial-temporal modeling where high-dimensional latent parameter spaces are ubiquitous. The methodology of integrated nested Laplace approximations (INLA) provides a framework for performing Bayesian inference applicable to a large subclass of additive Bayesian hierarchical models. In combination with the stochastic partial differential equations (SPDE) approach it gives rise to an efficient method for spatial-temporal modeling. In this work we build on the INLA-SPDE approach, by putting forward a performant distributed memory variant, INLA-DIST, for large-scale applications. To perform the arising computational kernel operations, consisting of Cholesky factorizations, solving linear systems, and selected matrix inversions, we present two numerical solver options, a sparse CPU-based library and a novel blocked GPU-accelerated approach which we propose. We leverage the recurring nonzero block structure in the arising precision (inverse covariance) matrices, which allows us to employ dense subroutines within a sparse setting. Both versions of INLA-DIST are highly scalable, capable of performing inference on models with millions of latent parameters. We demonstrate their accuracy and performance on synthetic as well as real-world climate dataset applications.Comment: 22 pages, 14 figure

Structured prior distributions for the covariance matrix in latent factor models

Factor models are widely used for dimension reduction in the analysis of multivariate data. This is achieved through decomposition of a p x p covariance matrix into the sum of two components. Through a latent factor representation, they can be interpreted as a diagonal matrix of idiosyncratic variances and a shared variation matrix, that is, the product of a p x k factor loadings matrix and its transpose. If k << p, this defines a sparse factorisation of the covariance matrix. Historically, little attention has been paid to incorporating prior information in Bayesian analyses using factor models where, at best, the prior for the factor loadings is order invariant. In this work, a class of structured priors is developed that can encode ideas of dependence structure about the shared variation matrix. The construction allows data-informed shrinkage towards sensible parametric structures while also facilitating inference over the number of factors. Using an unconstrained reparameterisation of stationary vector autoregressions, the methodology is extended to stationary dynamic factor models. For computational inference, parameter-expanded Markov chain Monte Carlo samplers are proposed, including an efficient adaptive Gibbs sampler. Two substantive applications showcase the scope of the methodology and its inferential benefits

Bayesian inference and model selection for multi-dimensional diffusion process models with non-parametric drift and constant diffusivity

For a multi-dimensional, partially observed diffusion process model with unknown drift and variable-independent diffusivity, we construct a composite methodology to perform Bayesian inference for the coefficients. Recent development of non-parametric Bayesian estimation of the drift has been restricted to dimension one, since the local time process is unavailable in the multi-dimensional case. We involve the empirical measure instead and show that the drift likelihood has a quadratic form, which allows a conjugate Gaussian measure prior whose precision operator is chosen to be a high order differential operator. We detail a computationally efficient pseudo-spectral method for solving the posterior mean, and describe how inference for the drift can be constrained to allow only conservative drifts. We also adapt a Langevin MCMC approach to sampling from diffusion bridges as a data augmentation scheme. To sample from the diffusivity, we specify an Inverse Wishart prior and implement a random walk Metropolis-Hastings algorithm. Evaluation of model fit for diffusion processes historically involved frequentist goodness-of-fit testing for fully parametric null models. We extend an existing transition density-based omnibus test to the null model case with non-parametric drift. We study the finite-sample behaviour of the test statistic and show that existing asymptotic results are inappropriate for settings involving real data. We implement the Bayesian discrepancy p-value to complement our inference methodology. With the goal of model improvement in mind, we describe how outlier removal and systematic sub-sampling of the data can be beneficial

Gaussian Process Vector Autoregressions and Macroeconomic Uncertainty

We develop a non-parametric multivariate time series model that remains agnostic on the precise relationship between a (possibly) large set of macroeconomic time series and their lagged values. The main building block of our model is a Gaussian process prior on the functional relationship that determines the conditional mean of the model, hence the name of Gaussian process vector autoregression (GP-VAR). A flexible stochastic volatility specification is used to provide additional flexibility and control for heteroskedasticity. Markov chain Monte Carlo (MCMC) estimation is carried out through an efficient and scalable algorithm which can handle large models. The GP-VAR is illustrated by means of simulated data and in a forecasting exercise with US data. Moreover, we use the GP-VAR to analyze the effects of macroeconomic uncertainty, with a particular emphasis on time variation and asymmetries in the transmission mechanisms.Comment: JEL: C11, C14, C32, E32; KEYWORDS: Bayesian non-parametrics, non-linear vector autoregressions, asymmetric uncertainty shock