732 research outputs found
Computation of Gaussian orthant probabilities in high dimension
We study the computation of Gaussian orthant probabilities, i.e. the
probability that a Gaussian falls inside a quadrant. The
Geweke-Hajivassiliou-Keane (GHK) algorithm [Genz, 1992; Geweke, 1991;
Hajivassiliou et al., 1996; Keane, 1993], is currently used for integrals of
dimension greater than 10. In this paper we show that for Markovian covariances
GHK can be interpreted as the estimator of the normalizing constant of a state
space model using sequential importance sampling (SIS). We show for an AR(1)
the variance of the GHK, properly normalized, diverges exponentially fast with
the dimension. As an improvement we propose using a particle filter (PF). We
then generalize this idea to arbitrary covariance matrices using Sequential
Monte Carlo (SMC) with properly tailored MCMC moves. We show empirically that
this can lead to drastic improvements on currently used algorithms. We also
extend the framework to orthants of mixture of Gaussians (Student, Cauchy
etc.), and to the simulation of truncated Gaussians
A New Monte Carlo Based Algorithm for the Gaussian Process Classification Problem
Gaussian process is a very promising novel technology that has been applied
to both the regression problem and the classification problem. While for the
regression problem it yields simple exact solutions, this is not the case for
the classification problem, because we encounter intractable integrals. In this
paper we develop a new derivation that transforms the problem into that of
evaluating the ratio of multivariate Gaussian orthant integrals. Moreover, we
develop a new Monte Carlo procedure that evaluates these integrals. It is based
on some aspects of bootstrap sampling and acceptancerejection. The proposed
approach has beneficial properties compared to the existing Markov Chain Monte
Carlo approach, such as simplicity, reliability, and speed
Selecting universities: personal preference and rankings
Polyhedral geometry can be used to quantitatively assess the dependence of
rankings on personal preference, and provides a tool for both students and
universities to assess US News and World Report rankings
Integrals over Gaussians under Linear Domain Constraints
Integrals of linearly constrained multivariate Gaussian densities are a
frequent problem in machine learning and statistics, arising in tasks like
generalized linear models and Bayesian optimization. Yet they are notoriously
hard to compute, and to further complicate matters, the numerical values of
such integrals may be very small. We present an efficient black-box algorithm
that exploits geometry for the estimation of integrals over a small, truncated
Gaussian volume, and to simulate therefrom. Our algorithm uses the
Holmes-Diaconis-Ross (HDR) method combined with an analytic version of
elliptical slice sampling (ESS). Adapted to the linear setting, ESS allows for
rejection-free sampling, because intersections of ellipses and domain
boundaries have closed-form solutions. The key idea of HDR is to decompose the
integral into easier-to-compute conditional probabilities by using a sequence
of nested domains. Remarkably, it allows for direct computation of the
logarithm of the integral value and thus enables the computation of extremely
small probability masses. We demonstrate the effectiveness of our tailored
combination of HDR and ESS on high-dimensional integrals and on entropy search
for Bayesian optimization
- …