202,731 research outputs found
Improving Relation Extraction with Knowledge-attention
While attention mechanisms have been proven to be effective in many NLP
tasks, majority of them are data-driven. We propose a novel knowledge-attention
encoder which incorporates prior knowledge from external lexical resources into
deep neural networks for relation extraction task. Furthermore, we present
three effective ways of integrating knowledge-attention with self-attention to
maximize the utilization of both knowledge and data. The proposed relation
extraction system is end-to-end and fully attention-based. Experiment results
show that the proposed knowledge-attention mechanism has complementary
strengths with self-attention, and our integrated models outperform existing
CNN, RNN, and self-attention based models. State-of-the-art performance is
achieved on TACRED, a complex and large-scale relation extraction dataset.Comment: Paper presented at 2019 Conference on Empirical Methods in Natural
Language Processing (EMNLP 2019
Cross-relation Cross-bag Attention for Distantly-supervised Relation Extraction
Distant supervision leverages knowledge bases to automatically label
instances, thus allowing us to train relation extractor without human
annotations. However, the generated training data typically contain massive
noise, and may result in poor performances with the vanilla supervised
learning. In this paper, we propose to conduct multi-instance learning with a
novel Cross-relation Cross-bag Selective Attention (CSA), which leads to
noise-robust training for distant supervised relation extractor. Specifically,
we employ the sentence-level selective attention to reduce the effect of noisy
or mismatched sentences, while the correlation among relations were captured to
improve the quality of attention weights. Moreover, instead of treating all
entity-pairs equally, we try to pay more attention to entity-pairs with a
higher quality. Similarly, we adopt the selective attention mechanism to
achieve this goal. Experiments with two types of relation extractor demonstrate
the superiority of the proposed approach over the state-of-the-art, while
further ablation studies verify our intuitions and demonstrate the
effectiveness of our proposed two techniques.Comment: AAAI 201
- …