7,441 research outputs found
Sequential Gaussian Processes for Online Learning of Nonstationary Functions
Many machine learning problems can be framed in the context of estimating
functions, and often these are time-dependent functions that are estimated in
real-time as observations arrive. Gaussian processes (GPs) are an attractive
choice for modeling real-valued nonlinear functions due to their flexibility
and uncertainty quantification. However, the typical GP regression model
suffers from several drawbacks: i) Conventional GP inference scales
with respect to the number of observations; ii) updating a GP model
sequentially is not trivial; and iii) covariance kernels often enforce
stationarity constraints on the function, while GPs with non-stationary
covariance kernels are often intractable to use in practice. To overcome these
issues, we propose an online sequential Monte Carlo algorithm to fit mixtures
of GPs that capture non-stationary behavior while allowing for fast,
distributed inference. By formulating hyperparameter optimization as a
multi-armed bandit problem, we accelerate mixing for real time inference. Our
approach empirically improves performance over state-of-the-art methods for
online GP estimation in the context of prediction for simulated non-stationary
data and hospital time series data
Smoothing and mean-covariance estimation of functional data with a Bayesian hierarchical model
Functional data, with basic observational units being functions (e.g.,
curves, surfaces) varying over a continuum, are frequently encountered in
various applications. While many statistical tools have been developed for
functional data analysis, the issue of smoothing all functional observations
simultaneously is less studied. Existing methods often focus on smoothing each
individual function separately, at the risk of removing important systematic
patterns common across functions. We propose a nonparametric Bayesian approach
to smooth all functional observations simultaneously and nonparametrically. In
the proposed approach, we assume that the functional observations are
independent Gaussian processes subject to a common level of measurement errors,
enabling the borrowing of strength across all observations. Unlike most
Gaussian process regression models that rely on pre-specified structures for
the covariance kernel, we adopt a hierarchical framework by assuming a Gaussian
process prior for the mean function and an Inverse-Wishart process prior for
the covariance function. These prior assumptions induce an automatic
mean-covariance estimation in the posterior inference in addition to the
simultaneous smoothing of all observations. Such a hierarchical framework is
flexible enough to incorporate functional data with different characteristics,
including data measured on either common or uncommon grids, and data with
either stationary or nonstationary covariance structures. Simulations and real
data analysis demonstrate that, in comparison with alternative methods, the
proposed Bayesian approach achieves better smoothing accuracy and comparable
mean-covariance estimation results. Furthermore, it can successfully retain the
systematic patterns in the functional observations that are usually neglected
by the existing functional data analyses based on individual-curve smoothing.Comment: Submitted to Bayesian Analysi
High-Dimensional Bayesian Geostatistics
With the growing capabilities of Geographic Information Systems (GIS) and
user-friendly software, statisticians today routinely encounter geographically
referenced data containing observations from a large number of spatial
locations and time points. Over the last decade, hierarchical spatiotemporal
process models have become widely deployed statistical tools for researchers to
better understand the complex nature of spatial and temporal variability.
However, fitting hierarchical spatiotemporal models often involves expensive
matrix computations with complexity increasing in cubic order for the number of
spatial locations and temporal points. This renders such models unfeasible for
large data sets. This article offers a focused review of two methods for
constructing well-defined highly scalable spatiotemporal stochastic processes.
Both these processes can be used as "priors" for spatiotemporal random fields.
The first approach constructs a low-rank process operating on a
lower-dimensional subspace. The second approach constructs a Nearest-Neighbor
Gaussian Process (NNGP) that ensures sparse precision matrices for its finite
realizations. Both processes can be exploited as a scalable prior embedded
within a rich hierarchical modeling framework to deliver full Bayesian
inference. These approaches can be described as model-based solutions for big
spatiotemporal datasets. The models ensure that the algorithmic complexity has
floating point operations (flops), where the number of spatial
locations (per iteration). We compare these methods and provide some insight
into their methodological underpinnings
Bayesian Nonstationary Spatial Modeling for Very Large Datasets
With the proliferation of modern high-resolution measuring instruments
mounted on satellites, planes, ground-based vehicles and monitoring stations, a
need has arisen for statistical methods suitable for the analysis of large
spatial datasets observed on large spatial domains. Statistical analyses of
such datasets provide two main challenges: First, traditional
spatial-statistical techniques are often unable to handle large numbers of
observations in a computationally feasible way. Second, for large and
heterogeneous spatial domains, it is often not appropriate to assume that a
process of interest is stationary over the entire domain.
We address the first challenge by using a model combining a low-rank
component, which allows for flexible modeling of medium-to-long-range
dependence via a set of spatial basis functions, with a tapered remainder
component, which allows for modeling of local dependence using a compactly
supported covariance function. Addressing the second challenge, we propose two
extensions to this model that result in increased flexibility: First, the model
is parameterized based on a nonstationary Matern covariance, where the
parameters vary smoothly across space. Second, in our fully Bayesian model, all
components and parameters are considered random, including the number,
locations, and shapes of the basis functions used in the low-rank component.
Using simulated data and a real-world dataset of high-resolution soil
measurements, we show that both extensions can result in substantial
improvements over the current state-of-the-art.Comment: 16 pages, 2 color figure
Distributed Gaussian Processes
To scale Gaussian processes (GPs) to large data sets we introduce the robust
Bayesian Committee Machine (rBCM), a practical and scalable product-of-experts
model for large-scale distributed GP regression. Unlike state-of-the-art sparse
GP approximations, the rBCM is conceptually simple and does not rely on
inducing or variational parameters. The key idea is to recursively distribute
computations to independent computational units and, subsequently, recombine
them to form an overall result. Efficient closed-form inference allows for
straightforward parallelisation and distributed computations with a small
memory footprint. The rBCM is independent of the computational graph and can be
used on heterogeneous computing infrastructures, ranging from laptops to
clusters. With sufficient computing resources our distributed GP model can
handle arbitrarily large data sets.Comment: 10 pages, 5 figures. Appears in Proceedings of ICML 201
- …