600 research outputs found
Looking Beyond Label Noise: Shifted Label Distribution Matters in Distantly Supervised Relation Extraction
In recent years there is a surge of interest in applying distant supervision
(DS) to automatically generate training data for relation extraction (RE). In
this paper, we study the problem what limits the performance of DS-trained
neural models, conduct thorough analyses, and identify a factor that can
influence the performance greatly, shifted label distribution. Specifically, we
found this problem commonly exists in real-world DS datasets, and without
special handing, typical DS-RE models cannot automatically adapt to this shift,
thus achieving deteriorated performance. To further validate our intuition, we
develop a simple yet effective adaptation method for DS-trained models, bias
adjustment, which updates models learned over the source domain (i.e., DS
training set) with a label distribution estimated on the target domain (i.e.,
test set). Experiments demonstrate that bias adjustment achieves consistent
performance gains on DS-trained models, especially on neural models, with an up
to 23% relative F1 improvement, which verifies our assumptions. Our code and
data can be found at
\url{https://github.com/INK-USC/shifted-label-distribution}.Comment: 13 pages: 10 pages paper, 3 pages appendix. Appears at EMNLP 201
Cross-relation Cross-bag Attention for Distantly-supervised Relation Extraction
Distant supervision leverages knowledge bases to automatically label
instances, thus allowing us to train relation extractor without human
annotations. However, the generated training data typically contain massive
noise, and may result in poor performances with the vanilla supervised
learning. In this paper, we propose to conduct multi-instance learning with a
novel Cross-relation Cross-bag Selective Attention (CSA), which leads to
noise-robust training for distant supervised relation extractor. Specifically,
we employ the sentence-level selective attention to reduce the effect of noisy
or mismatched sentences, while the correlation among relations were captured to
improve the quality of attention weights. Moreover, instead of treating all
entity-pairs equally, we try to pay more attention to entity-pairs with a
higher quality. Similarly, we adopt the selective attention mechanism to
achieve this goal. Experiments with two types of relation extractor demonstrate
the superiority of the proposed approach over the state-of-the-art, while
further ablation studies verify our intuitions and demonstrate the
effectiveness of our proposed two techniques.Comment: AAAI 201
RESIDE: Improving Distantly-Supervised Neural Relation Extraction using Side Information
Distantly-supervised Relation Extraction (RE) methods train an extractor by
automatically aligning relation instances in a Knowledge Base (KB) with
unstructured text. In addition to relation instances, KBs often contain other
relevant side information, such as aliases of relations (e.g., founded and
co-founded are aliases for the relation founderOfCompany). RE models usually
ignore such readily available side information. In this paper, we propose
RESIDE, a distantly-supervised neural relation extraction method which utilizes
additional side information from KBs for improved relation extraction. It uses
entity type and relation alias information for imposing soft constraints while
predicting relations. RESIDE employs Graph Convolution Networks (GCN) to encode
syntactic information from text and improves performance even when limited side
information is available. Through extensive experiments on benchmark datasets,
we demonstrate RESIDE's effectiveness. We have made RESIDE's source code
available to encourage reproducible research.Comment: 10 pages, 6 figures, EMNLP 201
- …