27,826 research outputs found
CANDiS: Coupled & Attention-Driven Neural Distant Supervision
Distant Supervision for Relation Extraction uses heuristically aligned text
data with an existing knowledge base as training data. The unsupervised nature
of this technique allows it to scale to web-scale relation extraction tasks, at
the expense of noise in the training data. Previous work has explored
relationships among instances of the same entity-pair to reduce this noise, but
relationships among instances across entity-pairs have not been fully
exploited. We explore the use of inter-instance couplings based on verb-phrase
and entity type similarities. We propose a novel technique, CANDiS, which casts
distant supervision using inter-instance coupling into an end-to-end neural
network model. CANDiS incorporates an attention module at the instance-level to
model the multi-instance nature of this problem. CANDiS outperforms existing
state-of-the-art techniques on a standard benchmark dataset.Comment: WiNLP 201
Relation Extraction : A Survey
With the advent of the Internet, large amount of digital text is generated
everyday in the form of news articles, research publications, blogs, question
answering forums and social media. It is important to develop techniques for
extracting information automatically from these documents, as lot of important
information is hidden within them. This extracted information can be used to
improve access and management of knowledge hidden in large text corpora.
Several applications such as Question Answering, Information Retrieval would
benefit from this information. Entities like persons and organizations, form
the most basic unit of the information. Occurrences of entities in a sentence
are often linked through well-defined relations; e.g., occurrences of person
and organization in a sentence may be linked through relations such as employed
at. The task of Relation Extraction (RE) is to identify such relations
automatically. In this paper, we survey several important supervised,
semi-supervised and unsupervised RE techniques. We also cover the paradigms of
Open Information Extraction (OIE) and Distant Supervision. Finally, we describe
some of the recent trends in the RE techniques and possible future research
directions. This survey would be useful for three kinds of readers - i)
Newcomers in the field who want to quickly learn about RE; ii) Researchers who
want to know how the various RE techniques evolved over time and what are
possible future research directions and iii) Practitioners who just need to
know which RE technique works best in various settings
Cross-Sentence N-ary Relation Extraction with Graph LSTMs
Past work in relation extraction has focused on binary relations in single
sentences. Recent NLP inroads in high-value domains have sparked interest in
the more general setting of extracting n-ary relations that span multiple
sentences. In this paper, we explore a general relation extraction framework
based on graph long short-term memory networks (graph LSTMs) that can be easily
extended to cross-sentence n-ary relation extraction. The graph formulation
provides a unified way of exploring different LSTM approaches and incorporating
various intra-sentential and inter-sentential dependencies, such as sequential,
syntactic, and discourse relations. A robust contextual representation is
learned for the entities, which serves as input to the relation classifier.
This simplifies handling of relations with arbitrary arity, and enables
multi-task learning with related relations. We evaluate this framework in two
important precision medicine settings, demonstrating its effectiveness with
both conventional supervised learning and distant supervision. Cross-sentence
extraction produced larger knowledge bases. and multi-task learning
significantly improved extraction accuracy. A thorough analysis of various LSTM
approaches yielded useful insight the impact of linguistic analysis on
extraction accuracy.Comment: Conditional accepted by TACL in December 2016; published in April
2017; presented at ACL in August 201
Towards Time-Aware Distant Supervision for Relation Extraction
Distant supervision for relation extraction heavily suffers from the wrong
labeling problem. To alleviate this issue in news data with the timestamp, we
take a new factor time into consideration and propose a novel time-aware
distant supervision framework (Time-DS). Time-DS is composed of a time series
instance-popularity and two strategies. Instance-popularity is to encode the
strong relevance of time and true relation mention. Therefore,
instance-popularity would be an effective clue to reduce the noises generated
through distant supervision labeling. The two strategies, i.e., hard filter and
curriculum learning are both ways to implement instance-popularity for better
relation extraction in the manner of Time-DS. The curriculum learning is a more
sophisticated and flexible way to exploit instance-popularity to eliminate the
bad effects of noises, thus get better relation extraction performance.
Experiments on our collected multi-source news corpus show that Time-DS
achieves significant improvements for relation extraction
DSReg: Using Distant Supervision as a Regularizer
In this paper, we aim at tackling a general issue in NLP tasks where some of
the negative examples are highly similar to the positive examples, i.e.,
hard-negative examples. We propose the distant supervision as a regularizer
(DSReg) approach to tackle this issue. The original task is converted to a
multi-task learning problem, in which distant supervision is used to retrieve
hard-negative examples. The obtained hard-negative examples are then used as a
regularizer. The original target objective of distinguishing positive examples
from negative examples is jointly optimized with the auxiliary task objective
of distinguishing softened positive (i.e., hard-negative examples plus positive
examples) from easy-negative examples. In the neural context, this can be done
by outputting the same representation from the last neural layer to different
functions. Using this strategy, we can improve the performance of
baseline models in a range of different NLP tasks, including text
classification, sequence labeling and reading comprehension
Combining Distant and Direct Supervision for Neural Relation Extraction
In relation extraction with distant supervision, noisy labels make it
difficult to train quality models. Previous neural models addressed this
problem using an attention mechanism that attends to sentences that are likely
to express the relations. We improve such models by combining the distant
supervision data with an additional directly-supervised data, which we use as
supervision for the attention weights. We find that joint training on both
types of supervision leads to a better model because it improves the model's
ability to identify noisy sentences. In addition, we find that sigmoidal
attention weights with max pooling achieves better performance over the
commonly used weighted average attention in this setup. Our proposed method
achieves a new state-of-the-art result on the widely used FB-NYT dataset
OpenKI: Integrating Open Information Extraction and Knowledge Bases with Relation Inference
In this paper, we consider advancing web-scale knowledge extraction and
alignment by integrating OpenIE extractions in the form of (subject, predicate,
object) triples with Knowledge Bases (KB). Traditional techniques from
universal schema and from schema mapping fall in two extremes: either they
perform instance-level inference relying on embedding for (subject, object)
pairs, thus cannot handle pairs absent in any existing triples; or they perform
predicate-level mapping and completely ignore background evidence from
individual entities, thus cannot achieve satisfying quality. We propose OpenKI
to handle sparsity of OpenIE extractions by performing instance-level
inference: for each entity, we encode the rich information in its neighborhood
in both KB and OpenIE extractions, and leverage this information in relation
inference by exploring different methods of aggregation and attention. In order
to handle unseen entities, our model is designed without creating
entity-specific parameters. Extensive experiments show that this method not
only significantly improves state-of-the-art for conventional OpenIE
extractions like ReVerb, but also boosts the performance on OpenIE from
semi-structured data, where new entity pairs are abundant and data are fairly
sparse
Machine Learning with World Knowledge: The Position and Survey
Machine learning has become pervasive in multiple domains, impacting a wide
variety of applications, such as knowledge discovery and data mining, natural
language processing, information retrieval, computer vision, social and health
informatics, ubiquitous computing, etc. Two essential problems of machine
learning are how to generate features and how to acquire labels for machines to
learn. Particularly, labeling large amount of data for each domain-specific
problem can be very time consuming and costly. It has become a key obstacle in
making learning protocols realistic in applications. In this paper, we will
discuss how to use the existing general-purpose world knowledge to enhance
machine learning processes, by enriching the features or reducing the labeling
work. We start from the comparison of world knowledge with domain-specific
knowledge, and then introduce three key problems in using world knowledge in
learning processes, i.e., explicit and implicit feature representation,
inference for knowledge linking and disambiguation, and learning with direct or
indirect supervision. Finally we discuss the future directions of this research
topic
Relation Discovery with Out-of-Relation Knowledge Base as Supervision
Unsupervised relation discovery aims to discover new relations from a given
text corpus without annotated data. However, it does not consider existing
human annotated knowledge bases even when they are relevant to the relations to
be discovered. In this paper, we study the problem of how to use
out-of-relation knowledge bases to supervise the discovery of unseen relations,
where out-of-relation means that relations to discover from the text corpus and
those in knowledge bases are not overlapped. We construct a set of constraints
between entity pairs based on the knowledge base embedding and then incorporate
constraints into the relation discovery by a variational auto-encoder based
algorithm. Experiments show that our new approach can improve the
state-of-the-art relation discovery performance by a large margin.Comment: Aceepted by NAACL-HLT 201
A Data-driven Approach for Noise Reduction in Distantly Supervised Biomedical Relation Extraction
Fact triples are a common form of structured knowledge used within the
biomedical domain. As the amount of unstructured scientific texts continues to
grow, manual annotation of these texts for the task of relation extraction
becomes increasingly expensive. Distant supervision offers a viable approach to
combat this by quickly producing large amounts of labeled, but considerably
noisy, data. We aim to reduce such noise by extending an entity-enriched
relation classification BERT model to the problem of multiple instance
learning, and defining a simple data encoding scheme that significantly reduces
noise, reaching state-of-the-art performance for distantly-supervised
biomedical relation extraction. Our approach further encodes knowledge about
the direction of relation triples, allowing for increased focus on relation
learning by reducing noise and alleviating the need for joint learning with
knowledge graph completion
- …