47,258 research outputs found
A Deep Generative Framework for Paraphrase Generation
Paraphrase generation is an important problem in NLP, especially in question
answering, information retrieval, information extraction, conversation systems,
to name a few. In this paper, we address the problem of generating paraphrases
automatically. Our proposed method is based on a combination of deep generative
models (VAE) with sequence-to-sequence models (LSTM) to generate paraphrases,
given an input sentence. Traditional VAEs when combined with recurrent neural
networks can generate free text but they are not suitable for paraphrase
generation for a given sentence. We address this problem by conditioning the
both, encoder and decoder sides of VAE, on the original sentence, so that it
can generate the given sentence's paraphrases. Unlike most existing models, our
model is simple, modular and can generate multiple paraphrases, for a given
sentence. Quantitative evaluation of the proposed method on a benchmark
paraphrase dataset demonstrates its efficacy, and its performance improvement
over the state-of-the-art methods by a significant margin, whereas qualitative
human evaluation indicate that the generated paraphrases are well-formed,
grammatically correct, and are relevant to the input sentence. Furthermore, we
evaluate our method on a newly released question paraphrase dataset, and
establish a new baseline for future research
Sequential Copying Networks
Copying mechanism shows effectiveness in sequence-to-sequence based neural
network models for text generation tasks, such as abstractive sentence
summarization and question generation. However, existing works on modeling
copying or pointing mechanism only considers single word copying from the
source sentences. In this paper, we propose a novel copying framework, named
Sequential Copying Networks (SeqCopyNet), which not only learns to copy single
words, but also copies sequences from the input sentence. It leverages the
pointer networks to explicitly select a sub-span from the source side to target
side, and integrates this sequential copying mechanism to the generation
process in the encoder-decoder paradigm. Experiments on abstractive sentence
summarization and question generation tasks show that the proposed SeqCopyNet
can copy meaningful spans and outperforms the baseline models.Comment: In AAAI 201
A Hierarchical Neural Autoencoder for Paragraphs and Documents
Natural language generation of coherent long texts like paragraphs or longer
documents is a challenging problem for recurrent networks models. In this
paper, we explore an important step toward this generation task: training an
LSTM (Long-short term memory) auto-encoder to preserve and reconstruct
multi-sentence paragraphs. We introduce an LSTM model that hierarchically
builds an embedding for a paragraph from embeddings for sentences and words,
then decodes this embedding to reconstruct the original paragraph. We evaluate
the reconstructed paragraph using standard metrics like ROUGE and Entity Grid,
showing that neural models are able to encode texts in a way that preserve
syntactic, semantic, and discourse coherence. While only a first step toward
generating coherent text units from neural models, our work has the potential
to significantly impact natural language generation and
summarization\footnote{Code for the three models described in this paper can be
found at www.stanford.edu/~jiweil/
CGMH: Constrained Sentence Generation by Metropolis-Hastings Sampling
In real-world applications of natural language generation, there are often
constraints on the target sentences in addition to fluency and naturalness
requirements. Existing language generation techniques are usually based on
recurrent neural networks (RNNs). However, it is non-trivial to impose
constraints on RNNs while maintaining generation quality, since RNNs generate
sentences sequentially (or with beam search) from the first word to the last.
In this paper, we propose CGMH, a novel approach using Metropolis-Hastings
sampling for constrained sentence generation. CGMH allows complicated
constraints such as the occurrence of multiple keywords in the target
sentences, which cannot be handled in traditional RNN-based approaches.
Moreover, CGMH works in the inference stage, and does not require parallel
corpora for training. We evaluate our method on a variety of tasks, including
keywords-to-sentence generation, unsupervised sentence paraphrasing, and
unsupervised sentence error correction. CGMH achieves high performance compared
with previous supervised methods for sentence generation. Our code is released
at https://github.com/NingMiao/CGMHComment: AAAI1
- …