4,353 research outputs found
Putting the Horse Before the Cart:A Generator-Evaluator Framework for Question Generation from Text
Automatic question generation (QG) is a useful yet challenging task in NLP.
Recent neural network-based approaches represent the state-of-the-art in this
task. In this work, we attempt to strengthen them significantly by adopting a
holistic and novel generator-evaluator framework that directly optimizes
objectives that reward semantics and structure. The {\it generator} is a
sequence-to-sequence model that incorporates the {\it structure} and {\it
semantics} of the question being generated. The generator predicts an answer in
the passage that the question can pivot on. Employing the copy and coverage
mechanisms, it also acknowledges other contextually important (and possibly
rare) keywords in the passage that the question needs to conform to, while not
redundantly repeating words. The {\it evaluator} model evaluates and assigns a
reward to each predicted question based on its conformity to the {\it
structure} of ground-truth questions. We propose two novel QG-specific reward
functions for text conformity and answer conformity of the generated question.
The evaluator also employs structure-sensitive rewards based on evaluation
measures such as BLEU, GLEU, and ROUGE-L, which are suitable for QG. In
contrast, most of the previous works only optimize the cross-entropy loss,
which can induce inconsistencies between training (objective) and testing
(evaluation) measures. Our evaluation shows that our approach significantly
outperforms state-of-the-art systems on the widely-used SQuAD benchmark as per
both automatic and human evaluation.Comment: 10 pages, The SIGNLL Conference on Computational Natural Language
Learning (CoNLL 2019
Learning to Extract Coherent Summary via Deep Reinforcement Learning
Coherence plays a critical role in producing a high-quality summary from a
document. In recent years, neural extractive summarization is becoming
increasingly attractive. However, most of them ignore the coherence of
summaries when extracting sentences. As an effort towards extracting coherent
summaries, we propose a neural coherence model to capture the cross-sentence
semantic and syntactic coherence patterns. The proposed neural coherence model
obviates the need for feature engineering and can be trained in an end-to-end
fashion using unlabeled data. Empirical results show that the proposed neural
coherence model can efficiently capture the cross-sentence coherence patterns.
Using the combined output of the neural coherence model and ROUGE package as
the reward, we design a reinforcement learning method to train a proposed
neural extractive summarizer which is named Reinforced Neural Extractive
Summarization (RNES) model. The RNES model learns to optimize coherence and
informative importance of the summary simultaneously. Experimental results show
that the proposed RNES outperforms existing baselines and achieves
state-of-the-art performance in term of ROUGE on CNN/Daily Mail dataset. The
qualitative evaluation indicates that summaries produced by RNES are more
coherent and readable.Comment: 8 pages, 1 figure, presented at AAAI-201
- …