2,347 research outputs found
Automatic Summarization
It has now been 50 years since the publication of Luhn’s seminal paper on automatic summarization. During these years the practical need for automatic summarization has become increasingly urgent and numerous papers have been published on the topic. As a result, it has become harder to find a single reference that gives an overview of past efforts or a complete view of summarization tasks and necessary system components. This article attempts to fill this void by providing a comprehensive overview of research in summarization, including the more traditional efforts in sentence extraction as well as the most novel recent approaches for determining important content, for domain and genre specific summarization and for evaluation of summarization. We also discuss the challenges that remain open, in particular the need for language generation and deeper semantic understanding of language that would be necessary for future advances in the field
Multiple Alternative Sentene Compressions as a Tool for Automatic Summarization Tasks
Automatic summarization is the distillation of important information from a source into an abridged form for a particular user or task.
Many current systems summarize texts by selecting sentences with important content. The limitation of extraction at the sentence level
is that highly relevant sentences may also contain non-relevant and
redundant content.
This thesis presents a novel framework for text summarization that
addresses the limitations of sentence-level extraction. Under this
framework text summarization is performed by generating Multiple
Alternative Sentence Compressions (MASC) as candidate summary
components and using weighted features of the candidates to construct
summaries from them. Sentence compression is the rewriting of a
sentence in a shorter form. This framework provides an environment in
which hypotheses about summarization techniques can be tested.
Three approaches to sentence compression were developed under this
framework. The first approach, HMM Hedge, uses the Noisy Channel
Model to calculate the most likely compressions of a sentence. The
second approach, Trimmer, uses syntactic trimming rules that are
linguistically motivated by Headlinese, a form of compressed English
associated with newspaper headlines. The third approach, Topiary, is
a combination of fluent text with topic terms.
The MASC framework for automatic text summarization has been applied
to the tasks of headline generation and multi-document summarization,
and has been used for initial work in summarization of novel genres
and applications, including broadcast news, email threads,
cross-language, and structured queries. The framework supports
combinations of component techniques, fostering collaboration between
development teams.
Three results will be demonstrated under the MASC framework. The first is
that an extractive summarization system can produce better summaries
by automatically selecting from a pool of compressed sentence
candidates than by automatically selecting from unaltered source
sentences. The second result is that sentence selectors can construct
better summaries from pools of compressed candidates when they make
use of larger candidate feature sets. The third result is that for
the task of Headline Generation, a combination of topic terms and
compressed sentences performs better then either approach alone.
Experimental evidence supports all three results
Structural Features for Predicting the Linguistic Quality of Text: Applications to Machine Translation, Automatic Summarization and Human-Authored Text
Sentence structure is considered to be an important component of the overall linguistic quality of text. Yet few empirical studies have sought to characterize how and to what extent structural features determine fluency and linguistic quality. We report the results of experiments on the predictive power of syntactic phrasing statistics and other structural features for these aspects of text. Manual assessments of sentence fluency for machine translation evaluation and text quality for summarization evaluation are used as gold-standard. We find that many structural features related to phrase length are weakly but significantly correlated with fluency and classifiers based on the entire suite of structural features can achieve high accuracy in pairwise comparison of sentence fluency and in distinguishing machine translations from human translations. We also test the hypothesis that the learned models capture general fluency properties applicable to human-authored text. The results from our experiments do not support the hypothesis. At the same time structural features and models based on them prove to be robust for automatic evaluation of the linguistic quality of multi-document summaries
A Novel ILP Framework for Summarizing Content with High Lexical Variety
Summarizing content contributed by individuals can be challenging, because
people make different lexical choices even when describing the same events.
However, there remains a significant need to summarize such content. Examples
include the student responses to post-class reflective questions, product
reviews, and news articles published by different news agencies related to the
same events. High lexical diversity of these documents hinders the system's
ability to effectively identify salient content and reduce summary redundancy.
In this paper, we overcome this issue by introducing an integer linear
programming-based summarization framework. It incorporates a low-rank
approximation to the sentence-word co-occurrence matrix to intrinsically group
semantically-similar lexical items. We conduct extensive experiments on
datasets of student responses, product reviews, and news documents. Our
approach compares favorably to a number of extractive baselines as well as a
neural abstractive summarization system. The paper finally sheds light on when
and why the proposed framework is effective at summarizing content with high
lexical variety.Comment: Accepted for publication in the journal of Natural Language
Engineering, 201
- …