3 research outputs found
Scalable computation of predictive probabilities in probit models with Gaussian process priors
Predictive models for binary data are fundamental in various fields, and the
growing complexity of modern applications has motivated several flexible
specifications for modeling the relationship between the observed predictors
and the binary responses. A widely-implemented solution is to express the
probability parameter via a probit mapping of a Gaussian process indexed by
predictors. However, unlike for continuous settings, there is a lack of
closed-form results for predictive distributions in binary models with Gaussian
process priors. Markov chain Monte Carlo methods and approximation strategies
provide common solutions to this problem, but state-of-the-art algorithms are
either computationally intractable or inaccurate in moderate-to-high
dimensions. In this article, we aim to cover this gap by deriving closed-form
expressions for the predictive probabilities in probit Gaussian processes that
rely either on cumulative distribution functions of multivariate Gaussians or
on functionals of multivariate truncated normals. To evaluate these quantities
we develop novel scalable solutions based on tile-low-rank Monte Carlo methods
for computing multivariate Gaussian probabilities, and on mean-field
variational approximations of multivariate truncated normals. Closed-form
expressions for the marginal likelihood and for the posterior distribution of
the Gaussian process are also discussed. As shown in simulated and real-world
empirical studies, the proposed methods scale to dimensions where
state-of-the-art solutions are impractical.Comment: 21 pages, 4 figure
Skew gaussian processes for classification
Gaussian processes (GPs) are distributions over functions, which provide a Bayesian
nonparametric approach to regression and classification. In spite of their success, GPs
have limited use in some applications, for example, in some cases a symmetric distribution
with respect to its mean is an unreasonable model. This implies, for instance, that the
mean and the median coincide, while the mean and median in an asymmetric (skewed)
distribution can be different numbers. In this paper, we propose skew-Gaussian processes
(SkewGPs) as a non-parametric prior over functions. A SkewGP extends the multivariate
unified skew normal distribution over finite dimensional vectors to a stochastic processes.
The SkewGP class of distributions includes GPs and, therefore, SkewGPs inherit all good
properties of GPs and increase their flexibility by allowing asymmetry in the probabilistic
model. By exploiting the fact that SkewGP and probit likelihood are conjugate model, we
derive closed form expressions for the marginal likelihood and predictive distribution of
this new nonparametric classifier. We verify empirically that the proposed SkewGP
classifier provides a better performance than a GP classifier based on either Laplace’s
method or expectation propagation