30,499 research outputs found
Stochastic Variational Inference
We develop stochastic variational inference, a scalable algorithm for
approximating posterior distributions. We develop this technique for a large
class of probabilistic models and we demonstrate it with two probabilistic
topic models, latent Dirichlet allocation and the hierarchical Dirichlet
process topic model. Using stochastic variational inference, we analyze several
large collections of documents: 300K articles from Nature, 1.8M articles from
The New York Times, and 3.8M articles from Wikipedia. Stochastic inference can
easily handle data sets of this size and outperforms traditional variational
inference, which can only handle a smaller subset. (We also show that the
Bayesian nonparametric topic model outperforms its parametric counterpart.)
Stochastic variational inference lets us apply complex Bayesian models to
massive data sets
Stochastic Collapsed Variational Inference for Sequential Data
Stochastic variational inference for collapsed models has recently been
successfully applied to large scale topic modelling. In this paper, we propose
a stochastic collapsed variational inference algorithm in the sequential data
setting. Our algorithm is applicable to both finite hidden Markov models and
hierarchical Dirichlet process hidden Markov models, and to any datasets
generated by emission distributions in the exponential family. Our experiment
results on two discrete datasets show that our inference is both more efficient
and more accurate than its uncollapsed version, stochastic variational
inference.Comment: NIPS Workshop on Advances in Approximate Bayesian Inference, 201
Practical Collapsed Stochastic Variational Inference for the HDP
Recent advances have made it feasible to apply the stochastic variational
paradigm to a collapsed representation of latent Dirichlet allocation (LDA).
While the stochastic variational paradigm has successfully been applied to an
uncollapsed representation of the hierarchical Dirichlet process (HDP), no
attempts to apply this type of inference in a collapsed setting of
non-parametric topic modeling have been put forward so far. In this paper we
explore such a collapsed stochastic variational Bayes inference for the HDP.
The proposed online algorithm is easy to implement and accounts for the
inference of hyper-parameters. First experiments show a promising improvement
in predictive performance.Comment: NIPS Workshop; Topic Models: Computation, Application, and Evaluatio
- …