3,227 research outputs found
GLEAKE: Global and Local Embedding Automatic Keyphrase Extraction
Automated methods for granular categorization of large corpora of text
documents have become increasingly more important with the rate scientific,
news, medical, and web documents are growing in the last few years. Automatic
keyphrase extraction (AKE) aims to automatically detect a small set of single
or multi-words from within a single textual document that captures the main
topics of the document. AKE plays an important role in various NLP and
information retrieval tasks such as document summarization and categorization,
full-text indexing, and article recommendation. Due to the lack of sufficient
human-labeled data in different textual contents, supervised learning
approaches are not ideal for automatic detection of keyphrases from the content
of textual bodies. With the state-of-the-art advances in text embedding
techniques, NLP researchers have focused on developing unsupervised methods to
obtain meaningful insights from raw datasets. In this work, we introduce Global
and Local Embedding Automatic Keyphrase Extractor (GLEAKE) for the task of AKE.
GLEAKE utilizes single and multi-word embedding techniques to explore the
syntactic and semantic aspects of the candidate phrases and then combines them
into a series of embedding-based graphs. Moreover, GLEAKE applies network
analysis techniques on each embedding-based graph to refine the most
significant phrases as a final set of keyphrases. We demonstrate the high
performance of GLEAKE by evaluating its results on five standard AKE datasets
from different domains and writing styles and by showing its superiority with
regards to other state-of-the-art methods
Learning Symmetric Collaborative Dialogue Agents with Dynamic Knowledge Graph Embeddings
We study a symmetric collaborative dialogue setting in which two agents, each
with private knowledge, must strategically communicate to achieve a common
goal. The open-ended dialogue state in this setting poses new challenges for
existing dialogue systems. We collected a dataset of 11K human-human dialogues,
which exhibits interesting lexical, semantic, and strategic elements. To model
both structured knowledge and unstructured language, we propose a neural model
with dynamic knowledge graph embeddings that evolve as the dialogue progresses.
Automatic and human evaluations show that our model is both more effective at
achieving the goal and more human-like than baseline neural and rule-based
models.Comment: ACL 201
Cross-Sentence N-ary Relation Extraction with Graph LSTMs
Past work in relation extraction has focused on binary relations in single
sentences. Recent NLP inroads in high-value domains have sparked interest in
the more general setting of extracting n-ary relations that span multiple
sentences. In this paper, we explore a general relation extraction framework
based on graph long short-term memory networks (graph LSTMs) that can be easily
extended to cross-sentence n-ary relation extraction. The graph formulation
provides a unified way of exploring different LSTM approaches and incorporating
various intra-sentential and inter-sentential dependencies, such as sequential,
syntactic, and discourse relations. A robust contextual representation is
learned for the entities, which serves as input to the relation classifier.
This simplifies handling of relations with arbitrary arity, and enables
multi-task learning with related relations. We evaluate this framework in two
important precision medicine settings, demonstrating its effectiveness with
both conventional supervised learning and distant supervision. Cross-sentence
extraction produced larger knowledge bases. and multi-task learning
significantly improved extraction accuracy. A thorough analysis of various LSTM
approaches yielded useful insight the impact of linguistic analysis on
extraction accuracy.Comment: Conditional accepted by TACL in December 2016; published in April
2017; presented at ACL in August 201
An end-to-end Neural Network Framework for Text Clustering
The unsupervised text clustering is one of the major tasks in natural
language processing (NLP) and remains a difficult and complex problem.
Conventional \mbox{methods} generally treat this task using separated steps,
including text representation learning and clustering the representations. As
an improvement, neural methods have also been introduced for continuous
representation learning to address the sparsity problem. However, the
multi-step process still deviates from the unified optimization target.
Especially the second step of cluster is generally performed with conventional
methods such as k-Means. We propose a pure neural framework for text clustering
in an end-to-end manner. It jointly learns the text representation and the
clustering model. Our model works well when the context can be obtained, which
is nearly always the case in the field of NLP. We have our method
\mbox{evaluated} on two widely used benchmarks: IMDB movie reviews for
sentiment classification and -Newsgroup for topic categorization. Despite
its simplicity, experiments show the model outperforms previous clustering
methods by a large margin. Furthermore, the model is also verified on English
wiki dataset as a large corpus
Investigating the Working of Text Classifiers
Text classification is one of the most widely studied tasks in natural
language processing. Motivated by the principle of compositionality, large
multilayer neural network models have been employed for this task in an attempt
to effectively utilize the constituent expressions. Almost all of the reported
work train large networks using discriminative approaches, which come with a
caveat of no proper capacity control, as they tend to latch on to any signal
that may not generalize. Using various recent state-of-the-art approaches for
text classification, we explore whether these models actually learn to compose
the meaning of the sentences or still just focus on some keywords or lexicons
for classifying the document. To test our hypothesis, we carefully construct
datasets where the training and test splits have no direct overlap of such
lexicons, but overall language structure would be similar. We study various
text classifiers and observe that there is a big performance drop on these
datasets. Finally, we show that even simple models with our proposed
regularization techniques, which disincentivize focusing on key lexicons, can
substantially improve classification accuracy.Comment: Proceedings of COLING 2018, the 27th International Conference on
Computational Linguistics: Technical Papers (COLING 2018), NIPS 2017 Workshop
on Deep Learning: Bridging Theory and Practic
Deep Multitask Learning for Semantic Dependency Parsing
We present a deep neural architecture that parses sentences into three
semantic dependency graph formalisms. By using efficient, nearly arc-factored
inference and a bidirectional-LSTM composed with a multi-layer perceptron, our
base system is able to significantly improve the state of the art for semantic
dependency parsing, without using hand-engineered features or syntax. We then
explore two multitask learning approaches---one that shares parameters across
formalisms, and one that uses higher-order structures to predict the graphs
jointly. We find that both approaches improve performance across formalisms on
average, achieving a new state of the art. Our code is open-source and
available at https://github.com/Noahs-ARK/NeurboParser.Comment: Proceedings of ACL 201
More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction
Relational facts are an important component of human knowledge, which are
hidden in vast amounts of text. In order to extract these facts from text,
people have been working on relation extraction (RE) for years. From early
pattern matching to current neural networks, existing RE methods have achieved
significant progress. Yet with explosion of Web text and emergence of new
relations, human knowledge is increasing drastically, and we thus require
"more" from RE: a more powerful RE system that can robustly utilize more data,
efficiently learn more relations, easily handle more complicated context, and
flexibly generalize to more open domains. In this paper, we look back at
existing RE methods, analyze key challenges we are facing nowadays, and show
promising directions towards more powerful RE. We hope our view can advance
this field and inspire more efforts in the community
Modelling Context with User Embeddings for Sarcasm Detection in Social Media
We introduce a deep neural network for automated sarcasm detection. Recent
work has emphasized the need for models to capitalize on contextual features,
beyond lexical and syntactic cues present in utterances. For example, different
speakers will tend to employ sarcasm regarding different subjects and, thus,
sarcasm detection models ought to encode such speaker information. Current
methods have achieved this by way of laborious feature engineering. By
contrast, we propose to automatically learn and then exploit user embeddings,
to be used in concert with lexical signals to recognize sarcasm. Our approach
does not require elaborate feature engineering (and concomitant data scraping);
fitting user embeddings requires only the text from their previous posts. The
experimental results show that our model outperforms a state-of-the-art
approach leveraging an extensive set of carefully crafted features.Comment: published as a conference paper at CONLL 201
Out of the Box: Reasoning with Graph Convolution Nets for Factual Visual Question Answering
Accurately answering a question about a given image requires combining
observations with general knowledge. While this is effortless for humans,
reasoning with general knowledge remains an algorithmic challenge. To advance
research in this direction a novel `fact-based' visual question answering
(FVQA) task has been introduced recently along with a large set of curated
facts which link two entities, i.e., two possible answers, via a relation.
Given a question-image pair, deep network techniques have been employed to
successively reduce the large set of facts until one of the two entities of the
final remaining fact is predicted as the answer. We observe that a successive
process which considers one fact at a time to form a local decision is
sub-optimal. Instead, we develop an entity graph and use a graph convolutional
network to `reason' about the correct answer by jointly considering all
entities. We show on the challenging FVQA dataset that this leads to an
improvement in accuracy of around 7% compared to the state of the art.Comment: Accepted to NIPS 201
Generating Logical Forms from Graph Representations of Text and Entities
Structured information about entities is critical for many semantic parsing
tasks. We present an approach that uses a Graph Neural Network (GNN)
architecture to incorporate information about relevant entities and their
relations during parsing. Combined with a decoder copy mechanism, this approach
provides a conceptually simple mechanism to generate logical forms with
entities. We demonstrate that this approach is competitive with the
state-of-the-art across several tasks without pre-training, and outperforms
existing approaches when combined with BERT pre-training.Comment: ACL 201
- …