966 research outputs found
Large-scale Heteroscedastic Regression via Gaussian Process
Heteroscedastic regression considering the varying noises among observations
has many applications in the fields like machine learning and statistics. Here
we focus on the heteroscedastic Gaussian process (HGP) regression which
integrates the latent function and the noise function together in a unified
non-parametric Bayesian framework. Though showing remarkable performance, HGP
suffers from the cubic time complexity, which strictly limits its application
to big data. To improve the scalability, we first develop a variational sparse
inference algorithm, named VSHGP, to handle large-scale datasets. Furthermore,
two variants are developed to improve the scalability and capability of VSHGP.
The first is stochastic VSHGP (SVSHGP) which derives a factorized evidence
lower bound, thus enhancing efficient stochastic variational inference. The
second is distributed VSHGP (DVSHGP) which (i) follows the Bayesian committee
machine formalism to distribute computations over multiple local VSHGP experts
with many inducing points; and (ii) adopts hybrid parameters for experts to
guard against over-fitting and capture local variety. The superiority of DVSHGP
and SVSHGP as compared to existing scalable heteroscedastic/homoscedastic GPs
is then extensively verified on various datasets.Comment: 14 pages, 15 figure
Understanding and Comparing Scalable Gaussian Process Regression for Big Data
As a non-parametric Bayesian model which produces informative predictive
distribution, Gaussian process (GP) has been widely used in various fields,
like regression, classification and optimization. The cubic complexity of
standard GP however leads to poor scalability, which poses challenges in the
era of big data. Hence, various scalable GPs have been developed in the
literature in order to improve the scalability while retaining desirable
prediction accuracy. This paper devotes to investigating the methodological
characteristics and performance of representative global and local scalable GPs
including sparse approximations and local aggregations from four main
perspectives: scalability, capability, controllability and robustness. The
numerical experiments on two toy examples and five real-world datasets with up
to 250K points offer the following findings. In terms of scalability, most of
the scalable GPs own a time complexity that is linear to the training size. In
terms of capability, the sparse approximations capture the long-term spatial
correlations, the local aggregations capture the local patterns but suffer from
over-fitting in some scenarios. In terms of controllability, we could improve
the performance of sparse approximations by simply increasing the inducing
size. But this is not the case for local aggregations. In terms of robustness,
local aggregations are robust to various initializations of hyperparameters due
to the local attention mechanism. Finally, we highlight that the proper hybrid
of global and local scalable GPs may be a promising way to improve both the
model capability and scalability for big data.Comment: 25 pages, 15 figures, preprint submitted to KB
A probabilistic data-driven model for planar pushing
This paper presents a data-driven approach to model planar pushing
interaction to predict both the most likely outcome of a push and its expected
variability. The learned models rely on a variation of Gaussian processes with
input-dependent noise called Variational Heteroscedastic Gaussian processes
(VHGP) that capture the mean and variance of a stochastic function. We show
that we can learn accurate models that outperform analytical models after less
than 100 samples and saturate in performance with less than 1000 samples. We
validate the results against a collected dataset of repeated trajectories, and
use the learned models to study questions such as the nature of the variability
in pushing, and the validity of the quasi-static assumption.Comment: 8 pages, 11 figures, ICRA 201
Heteroscedastic Gaussian processes for uncertainty modeling in large-scale crowdsourced traffic data
Accurately modeling traffic speeds is a fundamental part of efficient
intelligent transportation systems. Nowadays, with the widespread deployment of
GPS-enabled devices, it has become possible to crowdsource the collection of
speed information to road users (e.g. through mobile applications or dedicated
in-vehicle devices). Despite its rather wide spatial coverage, crowdsourced
speed data also brings very important challenges, such as the highly variable
measurement noise in the data due to a variety of driving behaviors and sample
sizes. When not properly accounted for, this noise can severely compromise any
application that relies on accurate traffic data. In this article, we propose
the use of heteroscedastic Gaussian processes (HGP) to model the time-varying
uncertainty in large-scale crowdsourced traffic data. Furthermore, we develop a
HGP conditioned on sample size and traffic regime (SRC-HGP), which makes use of
sample size information (probe vehicles per minute) as well as previous
observed speeds, in order to more accurately model the uncertainty in observed
speeds. Using 6 months of crowdsourced traffic data from Copenhagen, we
empirically show that the proposed heteroscedastic models produce significantly
better predictive distributions when compared to current state-of-the-art
methods for both speed imputation and short-term forecasting tasks.Comment: 22 pages, Transportation Research Part C: Emerging Technologies
(Elsevier
- …