40 research outputs found
A Hybrid Convolutional Variational Autoencoder for Text Generation
In this paper we explore the effect of architectural choices on learning a
Variational Autoencoder (VAE) for text generation. In contrast to the
previously introduced VAE model for text where both the encoder and decoder are
RNNs, we propose a novel hybrid architecture that blends fully feed-forward
convolutional and deconvolutional components with a recurrent language model.
Our architecture exhibits several attractive properties such as faster run time
and convergence, ability to better handle long sequences and, more importantly,
it helps to avoid some of the major difficulties posed by training VAE models
on textual data
A Conditional Variational Framework for Dialog Generation
Deep latent variable models have been shown to facilitate the response
generation for open-domain dialog systems. However, these latent variables are
highly randomized, leading to uncontrollable generated responses. In this
paper, we propose a framework allowing conditional response generation based on
specific attributes. These attributes can be either manually assigned or
automatically detected. Moreover, the dialog states for both speakers are
modeled separately in order to reflect personal features. We validate this
framework on two different scenarios, where the attribute refers to genericness
and sentiment states respectively. The experiment result testified the
potential of our model, where meaningful responses can be generated in
accordance with the specified attributes.Comment: Accepted by ACL201
Language Transfer of Audio Word2Vec: Learning Audio Segment Representations without Target Language Data
Audio Word2Vec offers vector representations of fixed dimensionality for
variable-length audio segments using Sequence-to-sequence Autoencoder (SA).
These vector representations are shown to describe the sequential phonetic
structures of the audio segments to a good degree, with real world applications
such as query-by-example Spoken Term Detection (STD). This paper examines the
capability of language transfer of Audio Word2Vec. We train SA from one
language (source language) and use it to extract the vector representation of
the audio segments of another language (target language). We found that SA can
still catch phonetic structure from the audio segments of the target language
if the source and target languages are similar. In query-by-example STD, we
obtain the vector representations from the SA learned from a large amount of
source language data, and found them surpass the representations from naive
encoder and SA directly learned from a small amount of target language data.
The result shows that it is possible to learn Audio Word2Vec model from
high-resource languages and use it on low-resource languages. This further
expands the usability of Audio Word2Vec.Comment: arXiv admin note: text overlap with arXiv:1603.0098
Generating Sentences Using a Dynamic Canvas
We introduce the Attentive Unsupervised Text (W)riter (AUTR), which is a word
level generative model for natural language. It uses a recurrent neural network
with a dynamic attention and canvas memory mechanism to iteratively construct
sentences. By viewing the state of the memory at intermediate stages and where
the model is placing its attention, we gain insight into how it constructs
sentences. We demonstrate that AUTR learns a meaningful latent representation
for each sentence, and achieves competitive log-likelihood lower bounds whilst
being computationally efficient. It is effective at generating and
reconstructing sentences, as well as imputing missing words.Comment: AAAI 201
A Deep Generative Framework for Paraphrase Generation
Paraphrase generation is an important problem in NLP, especially in question
answering, information retrieval, information extraction, conversation systems,
to name a few. In this paper, we address the problem of generating paraphrases
automatically. Our proposed method is based on a combination of deep generative
models (VAE) with sequence-to-sequence models (LSTM) to generate paraphrases,
given an input sentence. Traditional VAEs when combined with recurrent neural
networks can generate free text but they are not suitable for paraphrase
generation for a given sentence. We address this problem by conditioning the
both, encoder and decoder sides of VAE, on the original sentence, so that it
can generate the given sentence's paraphrases. Unlike most existing models, our
model is simple, modular and can generate multiple paraphrases, for a given
sentence. Quantitative evaluation of the proposed method on a benchmark
paraphrase dataset demonstrates its efficacy, and its performance improvement
over the state-of-the-art methods by a significant margin, whereas qualitative
human evaluation indicate that the generated paraphrases are well-formed,
grammatically correct, and are relevant to the input sentence. Furthermore, we
evaluate our method on a newly released question paraphrase dataset, and
establish a new baseline for future research