39 research outputs found
Denoising Distantly Supervised Named Entity Recognition via a Hypergeometric Probabilistic Model
Denoising is the essential step for distant supervision based named entity
recognition. Previous denoising methods are mostly based on instance-level
confidence statistics, which ignore the variety of the underlying noise
distribution on different datasets and entity types. This makes them difficult
to be adapted to high noise rate settings. In this paper, we propose
Hypergeometric Learning (HGL), a denoising algorithm for distantly supervised
NER that takes both noise distribution and instance-level confidence into
consideration. Specifically, during neural network training, we naturally model
the noise samples in each batch following a hypergeometric distribution
parameterized by the noise-rate. Then each instance in the batch is regarded as
either correct or noisy one according to its label confidence derived from
previous training step, as well as the noise distribution in this sampled
batch. Experiments show that HGL can effectively denoise the weakly-labeled
data retrieved from distant supervision, and therefore results in significant
improvements on the trained models.Comment: Accepted to AAAI202
Improving Distantly-Supervised Named Entity Recognition with Self-Collaborative Denoising Learning
Distantly supervised named entity recognition (DS-NER) efficiently reduces
labor costs but meanwhile intrinsically suffers from the label noise due to the
strong assumption of distant supervision. Typically, the wrongly labeled
instances comprise numbers of incomplete and inaccurate annotation noise, while
most prior denoising works are only concerned with one kind of noise and fail
to fully explore useful information in the whole training set. To address this
issue, we propose a robust learning paradigm named Self-Collaborative Denoising
Learning (SCDL), which jointly trains two teacher-student networks in a
mutually-beneficial manner to iteratively perform noisy label refinery. Each
network is designed to exploit reliable labels via self denoising, and two
networks communicate with each other to explore unreliable annotations by
collaborative denoising. Extensive experimental results on five real-world
datasets demonstrate that SCDL is superior to state-of-the-art DS-NER denoising
methods.Comment: EMNLP (12 pages, 4 figures, 6 tables
Looking Beyond Label Noise: Shifted Label Distribution Matters in Distantly Supervised Relation Extraction
In recent years there is a surge of interest in applying distant supervision
(DS) to automatically generate training data for relation extraction (RE). In
this paper, we study the problem what limits the performance of DS-trained
neural models, conduct thorough analyses, and identify a factor that can
influence the performance greatly, shifted label distribution. Specifically, we
found this problem commonly exists in real-world DS datasets, and without
special handing, typical DS-RE models cannot automatically adapt to this shift,
thus achieving deteriorated performance. To further validate our intuition, we
develop a simple yet effective adaptation method for DS-trained models, bias
adjustment, which updates models learned over the source domain (i.e., DS
training set) with a label distribution estimated on the target domain (i.e.,
test set). Experiments demonstrate that bias adjustment achieves consistent
performance gains on DS-trained models, especially on neural models, with an up
to 23% relative F1 improvement, which verifies our assumptions. Our code and
data can be found at
\url{https://github.com/INK-USC/shifted-label-distribution}.Comment: 13 pages: 10 pages paper, 3 pages appendix. Appears at EMNLP 201