3,006 research outputs found
Conditional Neural Generation using Sub-Aspect Functions for Extractive News Summarization
Much progress has been made in text summarization, fueled by neural
architectures using large-scale training corpora. However, in the news domain,
neural models easily overfit by leveraging position-related features due to the
prevalence of the inverted pyramid writing style. In addition, there is an
unmet need to generate a variety of summaries for different users. In this
paper, we propose a neural framework that can flexibly control summary
generation by introducing a set of sub-aspect functions (i.e. importance,
diversity, position). These sub-aspect functions are regulated by a set of
control codes to decide which sub-aspect to focus on during summary generation.
We demonstrate that extracted summaries with minimal position bias is
comparable with those generated by standard models that take advantage of
position preference. We also show that news summaries generated with a focus on
diversity can be more preferred by human raters. These results suggest that a
more flexible neural summarization framework providing more control options
could be desirable in tailoring to different user preferences, which is useful
since it is often impractical to articulate such preferences for different
applications a priori.Comment: Accepted to Findings of EMNLP 202
Automatic Generation of Text Descriptive Comments for Code Blocks
We propose a framework to automatically generate descriptive comments for
source code blocks. While this problem has been studied by many researchers
previously, their methods are mostly based on fixed template and achieves poor
results. Our framework does not rely on any template, but makes use of a new
recursive neural network called Code-RNN to extract features from the source
code and embed them into one vector. When this vector representation is input
to a new recurrent neural network (Code-GRU), the overall framework generates
text descriptions of the code with accuracy (Rouge-2 value) significantly
higher than other learning-based approaches such as sequence-to-sequence model.
The Code-RNN model can also be used in other scenario where the representation
of code is required.Comment: aaai 201
An Integrated Approach for Keyphrase Generation via Exploring the Power of Retrieval and Extraction
In this paper, we present a novel integrated approach for keyphrase
generation (KG). Unlike previous works which are purely extractive or
generative, we first propose a new multi-task learning framework that jointly
learns an extractive model and a generative model. Besides extracting
keyphrases, the output of the extractive model is also employed to rectify the
copy probability distribution of the generative model, such that the generative
model can better identify important contents from the given document. Moreover,
we retrieve similar documents with the given document from training data and
use their associated keyphrases as external knowledge for the generative model
to produce more accurate keyphrases. For further exploiting the power of
extraction and retrieval, we propose a neural-based merging module to combine
and re-rank the predicted keyphrases from the enhanced generative model, the
extractive model, and the retrieved keyphrases. Experiments on the five KG
benchmarks demonstrate that our integrated approach outperforms the
state-of-the-art methods.Comment: NAACL 1
Unifying Human and Statistical Evaluation for Natural Language Generation
How can we measure whether a natural language generation system produces both
high quality and diverse outputs? Human evaluation captures quality but not
diversity, as it does not catch models that simply plagiarize from the training
set. On the other hand, statistical evaluation (i.e., perplexity) captures
diversity but not quality, as models that occasionally emit low quality samples
would be insufficiently penalized. In this paper, we propose a unified
framework which evaluates both diversity and quality, based on the optimal
error rate of predicting whether a sentence is human- or machine-generated. We
demonstrate that this error rate can be efficiently estimated by combining
human and statistical evaluation, using an evaluation metric which we call
HUSE. On summarization and chit-chat dialogue, we show that (i) HUSE detects
diversity defects which fool pure human evaluation and that (ii) techniques
such as annealing for improving quality actually decrease HUSE due to decreased
diversity.Comment: NAACL Camera Ready Submissio
Generative Adversarial Nets for Information Retrieval: Fundamentals and Advances
Generative adversarial nets (GANs) have been widely studied during the recent
development of deep learning and unsupervised learning. With an adversarial
training mechanism, GAN manages to train a generative model to fit the
underlying unknown real data distribution under the guidance of the
discriminative model estimating whether a data instance is real or generated.
Such a framework is originally proposed for fitting continuous data
distribution such as images, thus it is not straightforward to be directly
applied to information retrieval scenarios where the data is mostly discrete,
such as IDs, text and graphs. In this tutorial, we focus on discussing the GAN
techniques and the variants on discrete data fitting in various information
retrieval scenarios. (i) We introduce the fundamentals of GAN framework and its
theoretic properties; (ii) we carefully study the promising solutions to extend
GAN onto discrete data generation; (iii) we introduce IRGAN, the fundamental
GAN framework of fitting single ID data distribution and the direct application
on information retrieval; (iv) we further discuss the task of sequential
discrete data generation tasks, e.g., text generation, and the corresponding
GAN solutions; (v) we present the most recent work on graph/network data
fitting with node embedding techniques by GANs. Meanwhile, we also introduce
the relevant open-source platforms such as IRGAN and Texygen to help audience
conduct research experiments on GANs in information retrieval. Finally, we
conclude this tutorial with a comprehensive summarization and a prospect of
further research directions for GANs in information retrieval.Comment: 4 pages, SIGIR 2018 tutoria
Code Attention: Translating Code to Comments by Exploiting Domain Features
Appropriate comments of code snippets provide insight for code functionality,
which are helpful for program comprehension. However, due to the great cost of
authoring with the comments, many code projects do not contain adequate
comments. Automatic comment generation techniques have been proposed to
generate comments from pieces of code in order to alleviate the human efforts
in annotating the code. Most existing approaches attempt to exploit certain
correlations (usually manually given) between code and generated comments,
which could be easily violated if the coding patterns change and hence the
performance of comment generation declines. In this paper, we first build
C2CGit, a large dataset from open projects in GitHub, which is more than
20 larger than existing datasets. Then we propose a new attention
module called Code Attention to translate code to comments, which is able to
utilize the domain features of code snippets, such as symbols and identifiers.
We make ablation studies to determine effects of different parts in Code
Attention. Experimental results demonstrate that the proposed module has better
performance over existing approaches in both BLEU and METEOR
Towards Storytelling from Visual Lifelogging: An Overview
Visual lifelogging consists of acquiring images that capture the daily
experiences of the user by wearing a camera over a long period of time. The
pictures taken offer considerable potential for knowledge mining concerning how
people live their lives, hence, they open up new opportunities for many
potential applications in fields including healthcare, security, leisure and
the quantified self. However, automatically building a story from a huge
collection of unstructured egocentric data presents major challenges. This
paper provides a thorough review of advances made so far in egocentric data
analysis, and in view of the current state of the art, indicates new lines of
research to move us towards storytelling from visual lifelogging.Comment: 16 pages, 11 figures, Submitted to IEEE Transactions on Human-Machine
System
Guiding Extractive Summarization with Question-Answering Rewards
Highlighting while reading is a natural behavior for people to track salient
content of a document. It would be desirable to teach an extractive summarizer
to do the same. However, a major obstacle to the development of a supervised
summarizer is the lack of ground-truth. Manual annotation of extraction units
is cost-prohibitive, whereas acquiring labels by automatically aligning human
abstracts and source documents can yield inferior results. In this paper we
describe a novel framework to guide a supervised, extractive summarization
system with question-answering rewards. We argue that quality summaries should
serve as a document surrogate to answer important questions, and such
question-answer pairs can be conveniently obtained from human abstracts. The
system learns to promote summaries that are informative, fluent, and perform
competitively on question-answering. Our results compare favorably with those
reported by strong summarization baselines as evaluated by automatic metrics
and human assessors.Comment: NAACL 201
Recent advances in conversational NLP : Towards the standardization of Chatbot building
Dialogue systems have become recently essential in our life. Their use is
getting more and more fluid and easy throughout the time. This boils down to
the improvements made in NLP and AI fields. In this paper, we try to provide an
overview to the current state of the art of dialogue systems, their categories
and the different approaches to build them. We end up with a discussion that
compares all the techniques and analyzes the strengths and weaknesses of each.
Finally, we present an opinion piece suggesting to orientate the research
towards the standardization of dialogue systems building.Comment: 8 pages with references, 1 figur
Video Summarization with Attention-Based Encoder-Decoder Networks
This paper addresses the problem of supervised video summarization by
formulating it as a sequence-to-sequence learning problem, where the input is a
sequence of original video frames, the output is a keyshot sequence. Our key
idea is to learn a deep summarization network with attention mechanism to mimic
the way of selecting the keyshots of human. To this end, we propose a novel
video summarization framework named Attentive encoder-decoder networks for
Video Summarization (AVS), in which the encoder uses a Bidirectional Long
Short-Term Memory (BiLSTM) to encode the contextual information among the input
video frames. As for the decoder, two attention-based LSTM networks are
explored by using additive and multiplicative objective functions,
respectively. Extensive experiments are conducted on three video summarization
benchmark datasets, i.e., SumMe, and TVSum. The results demonstrate the
superiority of the proposed AVS-based approaches against the state-of-the-art
approaches,with remarkable improvements from 0.8% to 3% on two
datasets,respectively..Comment: 9 pages, 7 figure
- …