57 research outputs found
Combining Sentiment Lexica with a Multi-View Variational Autoencoder
When assigning quantitative labels to a dataset, different methodologies may
rely on different scales. In particular, when assigning polarities to words in
a sentiment lexicon, annotators may use binary, categorical, or continuous
labels. Naturally, it is of interest to unify these labels from disparate
scales to both achieve maximal coverage over words and to create a single, more
robust sentiment lexicon while retaining scale coherence. We introduce a
generative model of sentiment lexica to combine disparate scales into a common
latent representation. We realize this model with a novel multi-view
variational autoencoder (VAE), called SentiVAE. We evaluate our approach via a
downstream text classification task involving nine English-Language sentiment
analysis datasets; our representation outperforms six individual sentiment
lexica, as well as a straightforward combination thereof.Comment: To appear in NAACL-HLT 201
Bivariate Beta-LSTM
Long Short-Term Memory (LSTM) infers the long term dependency through a cell
state maintained by the input and the forget gate structures, which models a
gate output as a value in [0,1] through a sigmoid function. However, due to the
graduality of the sigmoid function, the sigmoid gate is not flexible in
representing multi-modality or skewness. Besides, the previous models lack
modeling on the correlation between the gates, which would be a new method to
adopt inductive bias for a relationship between previous and current input.
This paper proposes a new gate structure with the bivariate Beta distribution.
The proposed gate structure enables probabilistic modeling on the gates within
the LSTM cell so that the modelers can customize the cell state flow with
priors and distributions. Moreover, we theoretically show the higher upper
bound of the gradient compared to the sigmoid function, and we empirically
observed that the bivariate Beta distribution gate structure provides higher
gradient values in training. We demonstrate the effectiveness of bivariate Beta
gate structure on the sentence classification, image classification, polyphonic
music modeling, and image caption generation.Comment: AAAI 202
Neural ODEs with stochastic vector field mixtures
It was recently shown that neural ordinary differential equation models
cannot solve fundamental and seemingly straightforward tasks even with
high-capacity vector field representations. This paper introduces two other
fundamental tasks to the set that baseline methods cannot solve, and proposes
mixtures of stochastic vector fields as a model class that is capable of
solving these essential problems. Dynamic vector field selection is of critical
importance for our model, and our approach is to propagate component
uncertainty over the integration interval with a technique based on forward
filtering. We also formalise several loss functions that encourage desirable
properties on the trajectory paths, and of particular interest are those that
directly encourage fewer expected function evaluations. Experimentally, we
demonstrate that our model class is capable of capturing the natural dynamics
of human behaviour; a notoriously volatile application area. Baseline
approaches cannot adequately model this problem
- …