Dropout is a widely used regularization trick to resolve the overfitting
issue in large feedforward neural networks trained on a small dataset, which
performs poorly on the held-out test subset. Although the effectiveness of this
regularization trick has been extensively studied for convolutional neural
networks, there is a lack of analysis of it for unsupervised models and in
particular, VAE-based neural topic models. In this paper, we have analyzed the
consequences of dropout in the encoder as well as in the decoder of the VAE
architecture in three widely used neural topic models, namely, contextualized
topic model (CTM), ProdLDA, and embedded topic model (ETM) using four publicly
available datasets. We characterize the dropout effect on these models in terms
of the quality and predictive performance of the generated topics.Comment: Accepted at EACL 202