3,184 research outputs found
Unsupervised Dual-Cascade Learning with Pseudo-Feedback Distillation for Query-based Extractive Summarization
We propose Dual-CES -- a novel unsupervised, query-focused, multi-document
extractive summarizer. Dual-CES is designed to better handle the tradeoff
between saliency and focus in summarization. To this end, Dual-CES employs a
two-step dual-cascade optimization approach with saliency-based pseudo-feedback
distillation. Overall, Dual-CES significantly outperforms all other
state-of-the-art unsupervised alternatives. Dual-CES is even shown to be able
to outperform strong supervised summarizers
Addressing Complex and Subjective Product-Related Queries with Customer Reviews
Online reviews are often our first port of call when considering products and
purchases online. When evaluating a potential purchase, we may have a specific
query in mind, e.g. `will this baby seat fit in the overhead compartment of a
747?' or `will I like this album if I liked Taylor Swift's 1989?'. To answer
such questions we must either wade through huge volumes of consumer reviews
hoping to find one that is relevant, or otherwise pose our question directly to
the community via a Q/A system.
In this paper we hope to fuse these two paradigms: given a large volume of
previously answered queries about products, we hope to automatically learn
whether a review of a product is relevant to a given query. We formulate this
as a machine learning problem using a mixture-of-experts-type framework---here
each review is an `expert' that gets to vote on the response to a particular
query; simultaneously we learn a relevance function such that `relevant'
reviews are those that vote correctly. At test time this learned relevance
function allows us to surface reviews that are relevant to new queries
on-demand. We evaluate our system, Moqa, on a novel corpus of 1.4 million
questions (and answers) and 13 million reviews. We show quantitatively that it
is effective at addressing both binary and open-ended queries, and
qualitatively that it surfaces reviews that human evaluators consider to be
relevant.Comment: WWW 2016; 14 pages, 5 figure
Revisiting Summarization Evaluation for Scientific Articles
Evaluation of text summarization approaches have been mostly based on metrics
that measure similarities of system generated summaries with a set of human
written gold-standard summaries. The most widely used metric in summarization
evaluation has been the ROUGE family. ROUGE solely relies on lexical overlaps
between the terms and phrases in the sentences; therefore, in cases of
terminology variations and paraphrasing, ROUGE is not as effective. Scientific
article summarization is one such case that is different from general domain
summarization (e.g. newswire data). We provide an extensive analysis of ROUGE's
effectiveness as an evaluation metric for scientific summarization; we show
that, contrary to the common belief, ROUGE is not much reliable in evaluating
scientific summaries. We furthermore show how different variants of ROUGE
result in very different correlations with the manual Pyramid scores. Finally,
we propose an alternative metric for summarization evaluation which is based on
the content relevance between a system generated summary and the corresponding
human written summaries. We call our metric SERA (Summarization Evaluation by
Relevance Analysis). Unlike ROUGE, SERA consistently achieves high correlations
with manual scores which shows its effectiveness in evaluation of scientific
article summarization.Comment: LREC 201
Rookie: A unique approach for exploring news archives
News archives are an invaluable primary source for placing current events in
historical context. But current search engine tools do a poor job at uncovering
broad themes and narratives across documents. We present Rookie: a practical
software system which uses natural language processing (NLP) to help readers,
reporters and editors uncover broad stories in news archives. Unlike prior
work, Rookie's design emerged from 18 months of iterative development in
consultation with editors and computational journalists. This process lead to a
dramatically different approach from previous academic systems with similar
goals. Our efforts offer a generalizable case study for others building
real-world journalism software using NLP.Comment: Presented at KDD 2017: Data Science + Journalism worksho
PageRank without hyperlinks: Structural re-ranking using links induced by language models
Inspired by the PageRank and HITS (hubs and authorities) algorithms for Web
search, we propose a structural re-ranking approach to ad hoc information
retrieval: we reorder the documents in an initially retrieved set by exploiting
asymmetric relationships between them. Specifically, we consider generation
links, which indicate that the language model induced from one document assigns
high probability to the text of another; in doing so, we take care to prevent
bias against long documents. We study a number of re-ranking criteria based on
measures of centrality in the graphs formed by generation links, and show that
integrating centrality into standard language-model-based retrieval is quite
effective at improving precision at top ranks
BiRank: Towards Ranking on Bipartite Graphs
The bipartite graph is a ubiquitous data structure that can model the
relationship between two entity types: for instance, users and items, queries
and webpages. In this paper, we study the problem of ranking vertices of a
bipartite graph, based on the graph's link structure as well as prior
information about vertices (which we term a query vector). We present a new
solution, BiRank, which iteratively assigns scores to vertices and finally
converges to a unique stationary ranking. In contrast to the traditional random
walk-based methods, BiRank iterates towards optimizing a regularization
function, which smooths the graph under the guidance of the query vector.
Importantly, we establish how BiRank relates to the Bayesian methodology,
enabling the future extension in a probabilistic way. To show the rationale and
extendability of the ranking methodology, we further extend it to rank for the
more generic n-partite graphs. BiRank's generic modeling of both the graph
structure and vertex features enables it to model various ranking hypotheses
flexibly. To illustrate its functionality, we apply the BiRank and TriRank
(ranking for tripartite graphs) algorithms to two real-world applications: a
general ranking scenario that predicts the future popularity of items, and a
personalized ranking scenario that recommends items of interest to users.
Extensive experiments on both synthetic and real-world datasets demonstrate
BiRank's soundness (fast convergence), efficiency (linear in the number of
graph edges) and effectiveness (achieving state-of-the-art in the two
real-world tasks).Comment: 15 pages, 8 figure
Scientific Article Summarization Using Citation-Context and Article's Discourse Structure
We propose a summarization approach for scientific articles which takes
advantage of citation-context and the document discourse model. While citations
have been previously used in generating scientific summaries, they lack the
related context from the referenced article and therefore do not accurately
reflect the article's content. Our method overcomes the problem of
inconsistency between the citation summary and the article's content by
providing context for each citation. We also leverage the inherent scientific
article's discourse for producing better summaries. We show that our proposed
method effectively improves over existing summarization approaches (greater
than 30% improvement over the best performing baseline) in terms of
\textsc{Rouge} scores on TAC2014 scientific summarization dataset. While the
dataset we use for evaluation is in the biomedical domain, most of our
approaches are general and therefore adaptable to other domains.Comment: EMNLP 201
Real-Time Web Scale Event Summarization Using Sequential Decision Making
We present a system based on sequential decision making for the online
summarization of massive document streams, such as those found on the web.
Given an event of interest (e.g. "Boston marathon bombing"), our system is able
to filter the stream for relevance and produce a series of short text updates
describing the event as it unfolds over time. Unlike previous work, our
approach is able to jointly model the relevance, comprehensiveness, novelty,
and timeliness required by time-sensitive queries. We demonstrate a 28.3%
improvement in summary F1 and a 43.8% improvement in time-sensitive F1 metrics.Comment: in Proceedings of the 25th International Joint Conference on
Artificial Intelligence 201
State of the Art, Evaluation and Recommendations regarding "Document Processing and Visualization Techniques"
Several Networks of Excellence have been set up in the framework of the
European FP5 research program. Among these Networks of Excellence, the NEMIS
project focuses on the field of Text Mining.
Within this field, document processing and visualization was identified as
one of the key topics and the WG1 working group was created in the NEMIS
project, to carry out a detailed survey of techniques associated with the text
mining process and to identify the relevant research topics in related research
areas.
In this document we present the results of this comprehensive survey. The
report includes a description of the current state-of-the-art and practice, a
roadmap for follow-up research in the identified areas, and recommendations for
anticipated technological development in the domain of text mining.Comment: 54 pages, Report of Working Group 1 for the European Network of
Excellence (NoE) in Text Mining and its Applications in Statistics (NEMIS
Joint Lifelong Topic Model and Manifold Ranking for Document Summarization
Due to the manifold ranking method has a significant effect on the ranking of
unknown data based on known data by using a weighted network, many researchers
use the manifold ranking method to solve the document summarization task.
However, their models only consider the original features but ignore the
semantic features of sentences when they construct the weighted networks for
the manifold ranking method. To solve this problem, we proposed two improved
models based on the manifold ranking method. One is combining the topic model
and manifold ranking method (JTMMR) to solve the document summarization task.
This model not only uses the original feature, but also uses the semantic
feature to represent the document, which can improve the accuracy of the
manifold ranking method. The other one is combining the lifelong topic model
and manifold ranking method (JLTMMR). On the basis of the JTMMR, this model
adds the constraint of knowledge to improve the quality of the topic. At the
same time, we also add the constraint of the relationship between documents to
dig out a better document semantic features. The JTMMR model can improve the
effect of the manifold ranking method by using the better semantic feature.
Experiments show that our models can achieve a better result than other
baseline models for multi-document summarization task. At the same time, our
models also have a good performance on the single document summarization task.
After combining with a few basic surface features, our model significantly
outperforms some model based on deep learning in recent years. After that, we
also do an exploring work for lifelong machine learning by analyzing the effect
of adding feedback. Experiments show that the effect of adding feedback to our
model is significant.Comment: 28 pages, 7 figure
- …