2 research outputs found
Variational Semi-supervised Aspect-term Sentiment Analysis via Transformer
Aspect-term sentiment analysis (ATSA) is a longstanding challenge in natural
language understanding. It requires fine-grained semantical reasoning about a
target entity appeared in the text. As manual annotation over the aspects is
laborious and time-consuming, the amount of labeled data is limited for
supervised learning. This paper proposes a semi-supervised method for the ATSA
problem by using the Variational Autoencoder based on Transformer (VAET), which
models the latent distribution via variational inference. By disentangling the
latent representation into the aspect-specific sentiment and the lexical
context, our method induces the underlying sentiment prediction for the
unlabeled data, which then benefits the ATSA classifier. Our method is
classifier agnostic, i.e., the classifier is an independent module and various
advanced supervised models can be integrated. Experimental results are obtained
on the SemEval 2014 task 4 and show that our method is effective with four
classical classifiers. The proposed method outperforms two general
semisupervised methods and achieves state-of-the-art performance.Comment: Accepted by CoNLL 201
Transformer-based Conditional Variational Autoencoder for Controllable Story Generation
We investigate large-scale latent variable models (LVMs) for neural story
generation -- an under-explored application for open-domain long text -- with
objectives in two threads: generation effectiveness and controllability. LVMs,
especially the variational autoencoder (VAE), have achieved both effective and
controllable generation through exploiting flexible distributional latent
representations. Recently, Transformers and its variants have achieved
remarkable effectiveness without explicit latent representation learning, thus
lack satisfying controllability in generation. In this paper, we advocate to
revive latent variable modeling, essentially the power of representation
learning, in the era of Transformers to enhance controllability without hurting
state-of-the-art generation effectiveness. Specifically, we integrate latent
representation vectors with a Transformer-based pre-trained architecture to
build conditional variational autoencoder (CVAE). Model components such as
encoder, decoder and the variational posterior are all built on top of
pre-trained language models -- GPT2 specifically in this paper. Experiments
demonstrate state-of-the-art conditional generation ability of our model, as
well as its excellent representation learning capability and controllability