13 research outputs found
Reference Matters: Benchmarking Factual Error Correction for Dialogue Summarization with Fine-grained Evaluation Framework
Factuality is important to dialogue summarization. Factual error correction
(FEC) of model-generated summaries is one way to improve factuality. Current
FEC evaluation that relies on factuality metrics is not reliable and detailed
enough. To address this problem, we are the first to manually annotate a FEC
dataset for dialogue summarization containing 4000 items and propose FERRANTI,
a fine-grained evaluation framework based on reference correction that
automatically evaluates the performance of FEC models on different error
categories. Using this evaluation framework, we conduct sufficient experiments
with FEC approaches under a variety of settings and find the best training
modes and significant differences in the performance of the existing approaches
on different factual error categories.Comment: Accepted to ACL 2023 Main Conferenc
Distantly-Supervised Named Entity Recognition with Adaptive Teacher Learning and Fine-grained Student Ensemble
Distantly-Supervised Named Entity Recognition (DS-NER) effectively alleviates
the data scarcity problem in NER by automatically generating training samples.
Unfortunately, the distant supervision may induce noisy labels, thus
undermining the robustness of the learned models and restricting the practical
application. To relieve this problem, recent works adopt self-training
teacher-student frameworks to gradually refine the training labels and improve
the generalization ability of NER models. However, we argue that the
performance of the current self-training frameworks for DS-NER is severely
underestimated by their plain designs, including both inadequate student
learning and coarse-grained teacher updating. Therefore, in this paper, we make
the first attempt to alleviate these issues by proposing: (1) adaptive teacher
learning comprised of joint training of two teacher-student networks and
considering both consistent and inconsistent predictions between two teachers,
thus promoting comprehensive student learning. (2) fine-grained student
ensemble that updates each fragment of the teacher model with a temporal moving
average of the corresponding fragment of the student, which enhances consistent
predictions on each model fragment against noise. To verify the effectiveness
of our proposed method, we conduct experiments on four DS-NER datasets. The
experimental results demonstrate that our method significantly surpasses
previous SOTA methods.Comment: Accepted at AAAI 202
A Survey on Arabic Named Entity Recognition: Past, Recent Advances, and Future Trends
As more and more Arabic texts emerged on the Internet, extracting important
information from these Arabic texts is especially useful. As a fundamental
technology, Named entity recognition (NER) serves as the core component in
information extraction technology, while also playing a critical role in many
other Natural Language Processing (NLP) systems, such as question answering and
knowledge graph building. In this paper, we provide a comprehensive review of
the development of Arabic NER, especially the recent advances in deep learning
and pre-trained language model. Specifically, we first introduce the background
of Arabic NER, including the characteristics of Arabic and existing resources
for Arabic NER. Then, we systematically review the development of Arabic NER
methods. Traditional Arabic NER systems focus on feature engineering and
designing domain-specific rules. In recent years, deep learning methods achieve
significant progress by representing texts via continuous vector
representations. With the growth of pre-trained language model, Arabic NER
yields better performance. Finally, we conclude the method gap between Arabic
NER and NER methods from other languages, which helps outline future directions
for Arabic NER.Comment: Accepted by IEEE TKD
Mining Word Boundaries in Speech as Naturally Annotated Word Segmentation Data
Inspired by early research on exploring naturally annotated data for Chinese
word segmentation (CWS), and also by recent research on integration of speech
and text processing, this work for the first time proposes to mine word
boundaries from parallel speech/text data. First we collect parallel
speech/text data from two Internet sources that are related with CWS data used
in our experiments. Then, we obtain character-level alignments and design
simple heuristic rules for determining word boundaries according to pause
duration between adjacent characters. Finally, we present an effective
complete-then-train strategy that can better utilize extra naturally annotated
data for model training. Experiments demonstrate our approach can significantly
boost CWS performance in both cross-domain and low-resource scenarios.Comment: latest versio
Efficient Document-level Event Extraction via Pseudo-Trigger-aware Pruned Complete Graph
Most previous studies of document-level event extraction mainly focus on
building argument chains in an autoregressive way, which achieves a certain
success but is inefficient in both training and inference. In contrast to the
previous studies, we propose a fast and lightweight model named as PTPCG. In
our model, we design a novel strategy for event argument combination together
with a non-autoregressive decoding algorithm via pruned complete graphs, which
are constructed under the guidance of the automatically selected pseudo
triggers. Compared to the previous systems, our system achieves competitive
results with 19.8\% of parameters and much lower resource consumption, taking
only 3.8\% GPU hours for training and up to 8.5 times faster for inference.
Besides, our model shows superior compatibility for the datasets with (or
without) triggers and the pseudo triggers can be the supplements for annotated
triggers to make further improvements. Codes are available at
https://github.com/Spico197/DocEE .Comment: Accepted to IJCAI'202
Mirror: A Universal Framework for Various Information Extraction Tasks
Sharing knowledge between information extraction tasks has always been a
challenge due to the diverse data formats and task variations. Meanwhile, this
divergence leads to information waste and increases difficulties in building
complex applications in real scenarios. Recent studies often formulate IE tasks
as a triplet extraction problem. However, such a paradigm does not support
multi-span and n-ary extraction, leading to weak versatility. To this end, we
reorganize IE problems into unified multi-slot tuples and propose a universal
framework for various IE tasks, namely Mirror. Specifically, we recast existing
IE tasks as a multi-span cyclic graph extraction problem and devise a
non-autoregressive graph decoding algorithm to extract all spans in a single
step. It is worth noting that this graph structure is incredibly versatile, and
it supports not only complex IE tasks, but also machine reading comprehension
and classification tasks. We manually construct a corpus containing 57 datasets
for model pretraining, and conduct experiments on 30 datasets across 8
downstream tasks. The experimental results demonstrate that our model has
decent compatibility and outperforms or reaches competitive performance with
SOTA systems under few-shot and zero-shot settings. The code, model weights,
and pretraining corpus are available at https://github.com/Spico197/Mirror .Comment: Accepted to EMNLP23 main conferenc
Read, Retrospect, Select: An MRC Framework to Short Text Entity Linking
Entity linking (EL) for the rapidly growing short text (e.g. search queries
and news titles) is critical to industrial applications. Most existing
approaches relying on adequate context for long text EL are not effective for
the concise and sparse short text. In this paper, we propose a novel framework
called Multi-turn Multiple-choice Machine reading comprehension (M3}) to solve
the short text EL from a new perspective: a query is generated for each
ambiguous mention exploiting its surrounding context, and an option selection
module is employed to identify the golden entity from candidates using the
query. In this way, M3 framework sufficiently interacts limited context with
candidate entities during the encoding process, as well as implicitly considers
the dissimilarities inside the candidate bunch in the selection stage. In
addition, we design a two-stage verifier incorporated into M3 to address the
commonly existed unlinkable problem in short text. To further consider the
topical coherence and interdependence among referred entities, M3 leverages a
multi-turn fashion to deal with mentions in a sequence manner by retrospecting
historical cues. Evaluation shows that our M3 framework achieves the
state-of-the-art performance on five Chinese and English datasets for the
real-world short text EL.Comment: Accepted at AAAI 202