323 research outputs found
Effective Piecewise CNN with attention mechanism for distant supervision on relation extraction task
Relation Extraction is an important sub-task in the field of information extraction. Its goal is to identify entities from text and extract semantic relationships between entities. However, the current Relationship Extraction task based on deep learning methods generally have practical problems such as insufficient amount of manually labeled data, so training under weak supervision has become a big challenge. Distant Supervision is a novel idea that can automatically annotate a large number of unlabeled data based on a small amount of labeled data. Based on this idea, this paper proposes a method combining the Piecewise Convolutional Neural Networks and Attention mechanism for automatically annotating the data of Relation Extraction task. The experiments proved that the proposed method achieved the highest precision is 76.24% on NYT-FB (New York Times-Freebase) dataset (top 100 relation categories). The results show that the proposed method performed better than CNN-based models in most cases
Effective Piecewise CNN with attention mechanism for distant supervision on relation extraction task
Cross-relation Cross-bag Attention for Distantly-supervised Relation Extraction
Distant supervision leverages knowledge bases to automatically label
instances, thus allowing us to train relation extractor without human
annotations. However, the generated training data typically contain massive
noise, and may result in poor performances with the vanilla supervised
learning. In this paper, we propose to conduct multi-instance learning with a
novel Cross-relation Cross-bag Selective Attention (CSA), which leads to
noise-robust training for distant supervised relation extractor. Specifically,
we employ the sentence-level selective attention to reduce the effect of noisy
or mismatched sentences, while the correlation among relations were captured to
improve the quality of attention weights. Moreover, instead of treating all
entity-pairs equally, we try to pay more attention to entity-pairs with a
higher quality. Similarly, we adopt the selective attention mechanism to
achieve this goal. Experiments with two types of relation extractor demonstrate
the superiority of the proposed approach over the state-of-the-art, while
further ablation studies verify our intuitions and demonstrate the
effectiveness of our proposed two techniques.Comment: AAAI 201
Attention-Based Capsule Networks with Dynamic Routing for Relation Extraction
A capsule is a group of neurons, whose activity vector represents the
instantiation parameters of a specific type of entity. In this paper, we
explore the capsule networks used for relation extraction in a multi-instance
multi-label learning framework and propose a novel neural approach based on
capsule networks with attention mechanisms. We evaluate our method with
different benchmarks, and it is demonstrated that our method improves the
precision of the predicted relations. Particularly, we show that capsule
networks improve multiple entity pairs relation extraction.Comment: To be published in EMNLP 201
Looking Beyond Label Noise: Shifted Label Distribution Matters in Distantly Supervised Relation Extraction
In recent years there is a surge of interest in applying distant supervision
(DS) to automatically generate training data for relation extraction (RE). In
this paper, we study the problem what limits the performance of DS-trained
neural models, conduct thorough analyses, and identify a factor that can
influence the performance greatly, shifted label distribution. Specifically, we
found this problem commonly exists in real-world DS datasets, and without
special handing, typical DS-RE models cannot automatically adapt to this shift,
thus achieving deteriorated performance. To further validate our intuition, we
develop a simple yet effective adaptation method for DS-trained models, bias
adjustment, which updates models learned over the source domain (i.e., DS
training set) with a label distribution estimated on the target domain (i.e.,
test set). Experiments demonstrate that bias adjustment achieves consistent
performance gains on DS-trained models, especially on neural models, with an up
to 23% relative F1 improvement, which verifies our assumptions. Our code and
data can be found at
\url{https://github.com/INK-USC/shifted-label-distribution}.Comment: 13 pages: 10 pages paper, 3 pages appendix. Appears at EMNLP 201
Improving Neural Relation Extraction with Implicit Mutual Relations
Relation extraction (RE) aims at extracting the relation between two entities
from the text corpora. It is a crucial task for Knowledge Graph (KG)
construction. Most existing methods predict the relation between an entity pair
by learning the relation from the training sentences, which contain the
targeted entity pair. In contrast to existing distant supervision approaches
that suffer from insufficient training corpora to extract relations, our
proposal of mining implicit mutual relation from the massive unlabeled corpora
transfers the semantic information of entity pairs into the RE model, which is
more expressive and semantically plausible. After constructing an entity
proximity graph based on the implicit mutual relations, we preserve the
semantic relations of entity pairs via embedding each vertex of the graph into
a low-dimensional space. As a result, we can easily and flexibly integrate the
implicit mutual relations and other entity information, such as entity types,
into the existing RE methods.
Our experimental results on a New York Times and another Google Distant
Supervision datasets suggest that our proposed neural RE framework provides a
promising improvement for the RE task, and significantly outperforms the
state-of-the-art methods. Moreover, the component for mining implicit mutual
relations is so flexible that can help to improve the performance of both
CNN-based and RNN-based RE models significant.Comment: 12 page
- …