4,194 research outputs found
Improving the INLA approach for approximate Bayesian inference for latent Gaussian models
We introduce a new copula-based correction for generalized linear mixed
models (GLMMs) within the integrated nested Laplace approximation (INLA)
approach for approximate Bayesian inference for latent Gaussian models. While
INLA is usually very accurate, some (rather extreme) cases of GLMMs with e.g.
binomial or Poisson data have been seen to be problematic. Inaccuracies can
occur when there is a very low degree of smoothing or "borrowing strength"
within the model, and we have therefore developed a correction aiming to push
the boundaries of the applicability of INLA. Our new correction has been
implemented as part of the R-INLA package, and adds only negligible
computational cost. Empirical evaluations on both real and simulated data
indicate that the method works well
Latent Gaussian modeling and INLA: A review with focus on space-time applications
Bayesian hierarchical models with latent Gaussian layers have proven very
flexible in capturing complex stochastic behavior and hierarchical structures
in high-dimensional spatial and spatio-temporal data. Whereas simulation-based
Bayesian inference through Markov Chain Monte Carlo may be hampered by slow
convergence and numerical instabilities, the inferential framework of
Integrated Nested Laplace Approximation (INLA) is capable to provide accurate
and relatively fast analytical approximations to posterior quantities of
interest. It heavily relies on the use of Gauss-Markov dependence structures to
avoid the numerical bottleneck of high-dimensional nonsparse matrix
computations. With a view towards space-time applications, we here review the
principal theoretical concepts, model classes and inference tools within the
INLA framework. Important elements to construct space-time models are certain
spatial Mat\'ern-like Gauss-Markov random fields, obtained as approximate
solutions to a stochastic partial differential equation. Efficient
implementation of statistical inference tools for a large variety of models is
available through the INLA package of the R software. To showcase the practical
use of R-INLA and to illustrate its principal commands and syntax, a
comprehensive simulation experiment is presented using simulated non Gaussian
space-time count data with a first-order autoregressive dependence structure in
time
Projection predictive model selection for Gaussian processes
We propose a new method for simplification of Gaussian process (GP) models by
projecting the information contained in the full encompassing model and
selecting a reduced number of variables based on their predictive relevance.
Our results on synthetic and real world datasets show that the proposed method
improves the assessment of variable relevance compared to the automatic
relevance determination (ARD) via the length-scale parameters. We expect the
method to be useful for improving explainability of the models, reducing the
future measurement costs and reducing the computation time for making new
predictions.Comment: A few minor changes in tex
Understanding and Comparing Scalable Gaussian Process Regression for Big Data
As a non-parametric Bayesian model which produces informative predictive
distribution, Gaussian process (GP) has been widely used in various fields,
like regression, classification and optimization. The cubic complexity of
standard GP however leads to poor scalability, which poses challenges in the
era of big data. Hence, various scalable GPs have been developed in the
literature in order to improve the scalability while retaining desirable
prediction accuracy. This paper devotes to investigating the methodological
characteristics and performance of representative global and local scalable GPs
including sparse approximations and local aggregations from four main
perspectives: scalability, capability, controllability and robustness. The
numerical experiments on two toy examples and five real-world datasets with up
to 250K points offer the following findings. In terms of scalability, most of
the scalable GPs own a time complexity that is linear to the training size. In
terms of capability, the sparse approximations capture the long-term spatial
correlations, the local aggregations capture the local patterns but suffer from
over-fitting in some scenarios. In terms of controllability, we could improve
the performance of sparse approximations by simply increasing the inducing
size. But this is not the case for local aggregations. In terms of robustness,
local aggregations are robust to various initializations of hyperparameters due
to the local attention mechanism. Finally, we highlight that the proper hybrid
of global and local scalable GPs may be a promising way to improve both the
model capability and scalability for big data.Comment: 25 pages, 15 figures, preprint submitted to KB
Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression
We propose a general algorithm for approximating nonstandard Bayesian
posterior distributions. The algorithm minimizes the Kullback-Leibler
divergence of an approximating distribution to the intractable posterior
distribution. Our method can be used to approximate any posterior distribution,
provided that it is given in closed form up to the proportionality constant.
The approximation can be any distribution in the exponential family or any
mixture of such distributions, which means that it can be made arbitrarily
precise. Several examples illustrate the speed and accuracy of our
approximation method in practice
- …