180 research outputs found
Latent Gaussian processes for distribution estimation of multivariate categorical data
Multivariate categorical data occur in many applications of machine learning.
One of the main difficulties with these vectors of categorical variables is
sparsity. The number of possible observations grows exponentially with vector
length, but dataset diversity might be poor in comparison. Recent models have
gained significant improvement in supervised tasks with this data. These models
embed observations in a continuous space to capture similarities between them.
Building on these ideas we propose a Bayesian model for the unsupervised task
of distribution estimation of multivariate categorical data. We model vectors
of categorical variables as generated from a non-linear transformation of a
continuous latent space. Non-linearity captures multi-modality in the
distribution. The continuous representation addresses sparsity. Our model ties
together many existing models, linking the linear categorical latent Gaussian
model, the Gaussian process latent variable model, and Gaussian process
classification. We derive inference for our model based on recent developments
in sampling based variational inference. We show empirically that the model
outperforms its linear and discrete counterparts in imputation tasks of sparse
data.YG is supported by the Google European fellowship in Machine Learning.This is the final version of the article. It first appeared from Microtome Publishing via http://jmlr.org/proceedings/papers/v37/gala15.htm
A Class of Conjugate Priors for Multinomial Probit Models which Includes the Multivariate Normal One
Multinomial probit models are widely-implemented representations which allow
both classification and inference by learning changes in vectors of class
probabilities with a set of p observed predictors. Although various frequentist
methods have been developed for estimation, inference and classification within
such a class of models, Bayesian inference is still lagging behind. This is due
to the apparent absence of a tractable class of conjugate priors, that may
facilitate posterior inference on the multinomial probit coefficients. Such an
issue has motivated increasing efforts toward the development of effective
Markov chain Monte Carlo methods, but state-of-the-art solutions still face
severe computational bottlenecks, especially in large p settings. In this
article, we prove that the entire class of unified skew-normal (SUN)
distributions is conjugate to a wide variety of multinomial probit models, and
we exploit the SUN properties to improve upon state-of-art-solutions for
posterior inference and classification both in terms of closed-form results for
key functionals of interest, and also by developing novel computational methods
relying either on independent and identically distributed samples from the
exact posterior or on scalable and accurate variational approximations based on
blocked partially-factorized representations. As illustrated in a
gastrointestinal lesions application, the magnitude of the improvements
relative to current methods is particularly evident, in practice, when the
focus is on large p applications
Discrete-Continuous ADMM for Transductive Inference in Higher-Order MRFs
This paper introduces a novel algorithm for transductive inference in
higher-order MRFs, where the unary energies are parameterized by a variable
classifier. The considered task is posed as a joint optimization problem in the
continuous classifier parameters and the discrete label variables. In contrast
to prior approaches such as convex relaxations, we propose an advantageous
decoupling of the objective function into discrete and continuous subproblems
and a novel, efficient optimization method related to ADMM. This approach
preserves integrality of the discrete label variables and guarantees global
convergence to a critical point. We demonstrate the advantages of our approach
in several experiments including video object segmentation on the DAVIS data
set and interactive image segmentation
- …