1,354 research outputs found
Event Coreference Resolution Using Neural Network Classifiers
This paper presents a neural network classifier approach to detecting both
within- and cross- document event coreference effectively using only event
mention based features. Our approach does not (yet) rely on any event argument
features such as semantic roles or spatiotemporal arguments. Experimental
results on the ECB+ dataset show that our approach produces F1 scores that
significantly outperform the state-of-the-art methods for both within-document
and cross-document event coreference resolution when we use B3 and CEAFe
evaluation measures, but gets worse F1 score with the MUC measure. However,
when we use the CoNLL measure, which is the average of these three scores, our
approach has slightly better F1 for within- document event coreference
resolution but is significantly better for cross-document event coreference
resolution
An Integrated, Conditional Model of Information Extraction and Coreference with Applications to Citation Matching
Although information extraction and coreference resolution appear together in
many applications, most current systems perform them as ndependent steps. This
paper describes an approach to integrated inference for extraction and
coreference based on conditionally-trained undirected graphical models. We
discuss the advantages of conditional probability training, and of a
coreference model structure based on graph partitioning. On a data set of
research paper citations, we show significant reduction in error by using
extraction uncertainty to improve coreference citation matching accuracy, and
using coreference to improve the accuracy of the extracted fields.Comment: Appears in Proceedings of the Twentieth Conference on Uncertainty in
Artificial Intelligence (UAI2004
GLoMo: Unsupervisedly Learned Relational Graphs as Transferable Representations
Modern deep transfer learning approaches have mainly focused on learning
generic feature vectors from one task that are transferable to other tasks,
such as word embeddings in language and pretrained convolutional features in
vision. However, these approaches usually transfer unary features and largely
ignore more structured graphical representations. This work explores the
possibility of learning generic latent relational graphs that capture
dependencies between pairs of data units (e.g., words or pixels) from
large-scale unlabeled data and transferring the graphs to downstream tasks. Our
proposed transfer learning framework improves performance on various tasks
including question answering, natural language inference, sentiment analysis,
and image classification. We also show that the learned graphs are generic
enough to be transferred to different embeddings on which the graphs have not
been trained (including GloVe embeddings, ELMo embeddings, and task-specific
RNN hidden unit), or embedding-free units such as image pixels
A Deterministic Algorithm for Bridging Anaphora Resolution
Previous work on bridging anaphora resolution (Poesio et al., 2004; Hou et
al., 2013b) use syntactic preposition patterns to calculate word relatedness.
However, such patterns only consider NPs' head nouns and hence do not fully
capture the semantics of NPs. Recently, Hou (2018) created word embeddings
(embeddings_PP) to capture associative similarity (ie, relatedness) between
nouns by exploring the syntactic structure of noun phrases. But embeddings_PP
only contains word representations for nouns. In this paper, we create new word
vectors by combining embeddings_PP with GloVe. This new word embeddings
(embeddings_bridging) are a more general lexical knowledge resource for
bridging and allow us to represent the meaning of an NP beyond its head easily.
We therefore develop a deterministic approach for bridging anaphora resolution,
which represents the semantics of an NP based on its head noun and
modifications. We show that this simple approach achieves the competitive
results compared to the best system in Hou et al.(2013b) which explores Markov
Logic Networks to model the problem. Additionally, we further improve the
results for bridging anaphora resolution reported in Hou (2018) by combining
our simple deterministic approach with Hou et al.(2013b)'s best system MLN II.Comment: 11 page
Triad-based Neural Network for Coreference Resolution
We propose a triad-based neural network system that generates affinity scores
between entity mentions for coreference resolution. The system simultaneously
accepts three mentions as input, taking mutual dependency and logical
constraints of all three mentions into account, and thus makes more accurate
predictions than the traditional pairwise approach. Depending on system
choices, the affinity scores can be further used in clustering or mention
ranking. Our experiments show that a standard hierarchical clustering using the
scores produces state-of-art results with gold mentions on the English portion
of CoNLL 2012 Shared Task. The model does not rely on many handcrafted features
and is easy to train and use. The triads can also be easily extended to polyads
of higher orders. To our knowledge, this is the first neural network system to
model mutual dependency of more than two members at mention level
Unsupervised Methods for Determining Object and Relation Synonyms on the Web
The task of identifying synonymous relations and objects, or synonym
resolution, is critical for high-quality information extraction. This paper
investigates synonym resolution in the context of unsupervised information
extraction, where neither hand-tagged training examples nor domain knowledge is
available. The paper presents a scalable, fully-implemented system that runs in
O(KN log N) time in the number of extractions, N, and the maximum number of
synonyms per word, K. The system, called Resolver, introduces a probabilistic
relational model for predicting whether two strings are co-referential based on
the similarity of the assertions containing them. On a set of two million
assertions extracted from the Web, Resolver resolves objects with 78% precision
and 68% recall, and resolves relations with 90% precision and 35% recall.
Several variations of resolvers probabilistic model are explored, and
experiments demonstrate that under appropriate conditions these variations can
improve F1 by 5%. An extension to the basic Resolver system allows it to handle
polysemous names with 97% precision and 95% recall on a data set from the TREC
corpus
Probabilistic Reasoning via Deep Learning: Neural Association Models
In this paper, we propose a new deep learning approach, called neural
association model (NAM), for probabilistic reasoning in artificial
intelligence. We propose to use neural networks to model association between
any two events in a domain. Neural networks take one event as input and compute
a conditional probability of the other event to model how likely these two
events are to be associated. The actual meaning of the conditional
probabilities varies between applications and depends on how the models are
trained. In this work, as two case studies, we have investigated two NAM
structures, namely deep neural networks (DNN) and relation-modulated neural
nets (RMNN), on several probabilistic reasoning tasks in AI, including
recognizing textual entailment, triple classification in multi-relational
knowledge bases and commonsense reasoning. Experimental results on several
popular datasets derived from WordNet, FreeBase and ConceptNet have all
demonstrated that both DNNs and RMNNs perform equally well and they can
significantly outperform the conventional methods available for these reasoning
tasks. Moreover, compared with DNNs, RMNNs are superior in knowledge transfer,
where a pre-trained model can be quickly extended to an unseen relation after
observing only a few training samples. To further prove the effectiveness of
the proposed models, in this work, we have applied NAMs to solving challenging
Winograd Schema (WS) problems. Experiments conducted on a set of WS problems
prove that the proposed models have the potential for commonsense reasoning.Comment: Probabilistic reasoning, Winograd Schema Challenge, Deep learning,
Neural Networks, Distributed Representatio
A Large-Scale Exploration of Effective Global Features for a Joint Entity Detection and Tracking Model
Entity detection and tracking (EDT) is the task of identifying textual
mentions of real-world entities in documents, extending the named entity
detection and coreference resolution task by considering mentions other than
names (pronouns, definite descriptions, etc.). Like NE tagging and coreference
resolution, most solutions to the EDT task separate out the mention detection
aspect from the coreference aspect. By doing so, these solutions are limited to
using only local features for learning. In contrast, by modeling both aspects
of the EDT task simultaneously, we are able to learn using highly complex,
non-local features. We develop a new joint EDT model and explore the utility of
many features, demonstrating their effectiveness on this task
A Tidy Data Model for Natural Language Processing using cleanNLP
The package cleanNLP provides a set of fast tools for converting a textual
corpus into a set of normalized tables. The underlying natural language
processing pipeline utilizes Stanford's CoreNLP library, exposing a number of
annotation tasks for text written in English, French, German, and Spanish.
Annotators include tokenization, part of speech tagging, named entity
recognition, entity linking, sentiment analysis, dependency parsing,
coreference resolution, and information extraction.Comment: 20 pages; 4 figure
Reasoning in Vector Space: An Exploratory Study of Question Answering
Question answering tasks have shown remarkable progress with distributed
vector representation. In this paper, we investigate the recently proposed
Facebook bAbI tasks which consist of twenty different categories of questions
that require complex reasoning. Because the previous work on bAbI are all
end-to-end models, errors could come from either an imperfect understanding of
semantics or in certain steps of the reasoning. For clearer analysis, we
propose two vector space models inspired by Tensor Product Representation (TPR)
to perform knowledge encoding and logical reasoning based on common-sense
inference. They together achieve near-perfect accuracy on all categories
including positional reasoning and path finding that have proved difficult for
most of the previous approaches. We hypothesize that the difficulties in these
categories are due to the multi-relations in contrast to uni-relational
characteristic of other categories. Our exploration sheds light on designing
more sophisticated dataset and moving one step toward integrating transparent
and interpretable formalism of TPR into existing learning paradigms
- …