29 research outputs found
The Gaussian process density sampler
We present the Gaussian Process Density Sampler (GPDS), an exchangeable generative model for use in nonparametric Bayesian density estimation. Samples drawn from the GPDS are consistent with exact, independent samples from a fixed density function that is a transformation of a function drawn from a Gaussian process prior. Our formulation allows us to infer an unknown density from data using Markov chain Monte Carlo, which gives samples from the posterior distribution over density functions and from the predictive distribution on data space. We can also infer the hyperparameters of the Gaussian process. We compare this density modeling technique to several existing techniques on a toy problem and a skullreconstruction task.
Data augmentation for models based on rejection sampling
We present a data augmentation scheme to perform Markov chain Monte Carlo
inference for models where data generation involves a rejection sampling
algorithm. Our idea, which seems to be missing in the literature, is a simple
scheme to instantiate the rejected proposals preceding each data point. The
resulting joint probability over observed and rejected variables can be much
simpler than the marginal distribution over the observed variables, which often
involves intractable integrals. We consider three problems, the first being the
modeling of flow-cytometry measurements subject to truncation. The second is a
Bayesian analysis of the matrix Langevin distribution on the Stiefel manifold,
and the third, Bayesian inference for a nonparametric Gaussian process density
model. The latter two are instances of problems where Markov chain Monte Carlo
inference is doubly-intractable. Our experiments demonstrate superior
performance over state-of-the-art sampling algorithms for such problems.Comment: 6 figures. arXiv admin note: text overlap with arXiv:1311.090
Neural Likelihoods via Cumulative Distribution Functions
We leverage neural networks as universal approximators of monotonic functions
to build a parameterization of conditional cumulative distribution functions
(CDFs). By the application of automatic differentiation with respect to
response variables and then to parameters of this CDF representation, we are
able to build black box CDF and density estimators. A suite of families is
introduced as alternative constructions for the multivariate case. At one
extreme, the simplest construction is a competitive density estimator against
state-of-the-art deep learning methods, although it does not provide an easily
computable representation of multivariate CDFs. At the other extreme, we have a
flexible construction from which multivariate CDF evaluations and
marginalizations can be obtained by a simple forward pass in a deep neural net,
but where the computation of the likelihood scales exponentially with
dimensionality. Alternatives in between the extremes are discussed. We evaluate
the different representations empirically on a variety of tasks involving tail
area probabilities, tail dependence and (partial) density estimation.Comment: 10 page