1 research outputs found
Nested Variational Autoencoder for Topic Modeling on Microtexts with Word Vectors
Most of the information on the Internet is represented in the form of
microtexts, which are short text snippets such as news headlines or tweets.
These sources of information are abundant, and mining these data could uncover
meaningful insights. Topic modeling is one of the popular methods to extract
knowledge from a collection of documents; however, conventional topic models
such as latent Dirichlet allocation (LDA) are unable to perform well on short
documents, mostly due to the scarcity of word co-occurrence statistics embedded
in the data. The objective of our research is to create a topic model that can
achieve great performances on microtexts while requiring a small runtime for
scalability to large datasets. To solve the lack of information of microtexts,
we allow our method to take advantage of word embeddings for additional
knowledge of relationships between words. For speed and scalability, we apply
autoencoding variational Bayes, an algorithm that can perform efficient
black-box inference in probabilistic models. The result of our work is a novel
topic model called the nested variational autoencoder, which is a distribution
that takes into account word vectors and is parameterized by a neural network
architecture. For optimization, the model is trained to approximate the
posterior distribution of the original LDA model. Experiments show the
improvements of our model on microtexts as well as its runtime advantage.Comment: 27 pages, 9 figures, under review at Expert System