3,430 research outputs found
Information Retrieval with Entity Linking
Despite the advantages of their low-resource settings, traditional sparse retrievers depend on exact matching approaches between high-dimensional bag-of-words (BoW) representations of both the queries and the collection. As a result, retrieval performance is restricted by semantic discrepancies and vocabulary gaps. On the other hand, transformer-based dense retrievers introduce significant improvements in information retrieval tasks by exploiting low-dimensional contextualized representations of the corpus. While dense retrievers are known for their relative effectiveness, they suffer from lower efficiency and lack of generalization issues, when compared to sparse retrievers. For a lightweight retrieval task, high computational resources and time consumption are major barriers encouraging the renunciation of dense models despite potential gains. In this work, I propose boosting the performance of sparse retrievers by expanding both the queries and the documents with linked entities in two formats for the entity names: 1) explicit and 2) hashed. A zero-shot end-to-end dense entity linking system is employed for entity recognition and disambiguation to augment the corpus. By leveraging the advanced entity linking methods, I believe that the effectiveness gap between sparse and dense retrievers can be narrowed. Experiments are conducted on the MS MARCO passage dataset using the original qrel set, the re-ranked qrels favoured by MonoT5 and the latter set further re-ranked by DuoT5. Since I am concerned with the early stage retrieval in cascaded ranking architectures of large information retrieval systems, the results are evaluated using recall@1000. The suggested approach is also capable of retrieving documents for query subsets judged to be particularly difficult in prior work. In addition, it is demonstrated that the non-expanded and the expanded runs with both explicit and hashed entities retrieve complementary results. Consequently, run combination methods such as run fusion and classifier selection are experimented to maximize the benefits of entity linking. Due to the success of entity methods for sparse retrieval, the proposed approach is also tested on dense retrievers. The corresponding results are reported in MRR@10
Modeling Fine-grained Information via Knowledge-aware Hierarchical Graph for Zero-shot Entity Retrieval
Zero-shot entity retrieval, aiming to link mentions to candidate entities
under the zero-shot setting, is vital for many tasks in Natural Language
Processing. Most existing methods represent mentions/entities via the sentence
embeddings of corresponding context from the Pre-trained Language Model.
However, we argue that such coarse-grained sentence embeddings can not fully
model the mentions/entities, especially when the attention scores towards
mentions/entities are relatively low. In this work, we propose GER, a
\textbf{G}raph enhanced \textbf{E}ntity \textbf{R}etrieval framework, to
capture more fine-grained information as complementary to sentence embeddings.
We extract the knowledge units from the corresponding context and then
construct a mention/entity centralized graph. Hence, we can learn the
fine-grained information about mention/entity by aggregating information from
these knowledge units. To avoid the graph information bottleneck for the
central mention/entity node, we construct a hierarchical graph and design a
novel Hierarchical Graph Attention Network~(HGAN). Experimental results on
popular benchmarks demonstrate that our proposed GER framework performs better
than previous state-of-the-art models. The code has been available at
https://github.com/wutaiqiang/GER-WSDM2023.Comment: 9 pages, 5 figure
Revisiting Sparse Retrieval for Few-shot Entity Linking
Entity linking aims to link ambiguous mentions to their corresponding
entities in a knowledge base. One of the key challenges comes from insufficient
labeled data for specific domains. Although dense retrievers have achieved
excellent performance on several benchmarks, their performance decreases
significantly when only a limited amount of in-domain labeled data is
available. In such few-shot setting, we revisit the sparse retrieval method,
and propose an ELECTRA-based keyword extractor to denoise the mention context
and construct a better query expression. For training the extractor, we propose
a distant supervision method to automatically generate training data based on
overlapping tokens between mention contexts and entity descriptions.
Experimental results on the ZESHEL dataset demonstrate that the proposed method
outperforms state-of-the-art models by a significant margin across all test
domains, showing the effectiveness of keyword-enhanced sparse retrieval.Comment: EMNLP 202
Towards Better Entity Linking with Multi-View Enhanced Distillation
Dense retrieval is widely used for entity linking to retrieve entities from
large-scale knowledge bases. Mainstream techniques are based on a dual-encoder
framework, which encodes mentions and entities independently and calculates
their relevances via rough interaction metrics, resulting in difficulty in
explicitly modeling multiple mention-relevant parts within entities to match
divergent mentions. Aiming at learning entity representations that can match
divergent mentions, this paper proposes a Multi-View Enhanced Distillation
(MVD) framework, which can effectively transfer knowledge of multiple
fine-grained and mention-relevant parts within entities from cross-encoders to
dual-encoders. Each entity is split into multiple views to avoid irrelevant
information being over-squashed into the mention-relevant view. We further
design cross-alignment and self-alignment mechanisms for this framework to
facilitate fine-grained knowledge distillation from the teacher model to the
student model. Meanwhile, we reserve a global-view that embeds the entity as a
whole to prevent dispersal of uniform information. Experiments show our method
achieves state-of-the-art performance on several entity linking benchmarks.Comment: Accepted by ACL 2023 Main Conferenc
Find the Funding: Entity Linking with Incomplete Funding Knowledge Bases
Automatic extraction of funding information from academic articles adds
significant value to industry and research communities, such as tracking
research outcomes by funding organizations, profiling researchers and
universities based on the received funding, and supporting open access
policies. Two major challenges of identifying and linking funding entities are:
(i) sparse graph structure of the Knowledge Base (KB), which makes the commonly
used graph-based entity linking approaches suboptimal for the funding domain,
(ii) missing entities in KB, which (unlike recent zero-shot approaches)
requires marking entity mentions without KB entries as NIL. We propose an
entity linking model that can perform NIL prediction and overcome data scarcity
issues in a time and data-efficient manner. Our model builds on a
transformer-based mention detection and bi-encoder model to perform entity
linking. We show that our model outperforms strong existing baselines.Comment: Accepted at COLING 202
Effective Few-Shot Named Entity Linking by Meta-Learning
Entity linking aims to link ambiguous mentions to their corresponding
entities in a knowledge base, which is significant and fundamental for various
downstream applications, e.g., knowledge base completion, question answering,
and information extraction. While great efforts have been devoted to this task,
most of these studies follow the assumption that large-scale labeled data is
available. However, when the labeled data is insufficient for specific domains
due to labor-intensive annotation work, the performance of existing algorithms
will suffer an intolerable decline. In this paper, we endeavor to solve the
problem of few-shot entity linking, which only requires a minimal amount of
in-domain labeled data and is more practical in real situations. Specifically,
we firstly propose a novel weak supervision strategy to generate non-trivial
synthetic entity-mention pairs based on mention rewriting. Since the quality of
the synthetic data has a critical impact on effective model training, we
further design a meta-learning mechanism to assign different weights to each
synthetic entity-mention pair automatically. Through this way, we can
profoundly exploit rich and precious semantic information to derive a
well-trained entity linking model under the few-shot setting. The experiments
on real-world datasets show that the proposed method can extensively improve
the state-of-the-art few-shot entity linking model and achieve impressive
performance when only a small amount of labeled data is available. Moreover, we
also demonstrate the outstanding ability of the model's transferability.Comment: 14 pages, 4 figures. Accepted at IEEE ICDE 202
A Read-and-Select Framework for Zero-shot Entity Linking
Zero-shot entity linking (EL) aims at aligning entity mentions to unseen
entities to challenge the generalization ability. Previous methods largely
focus on the candidate retrieval stage and ignore the essential candidate
ranking stage, which disambiguates among entities and makes the final linking
prediction. In this paper, we propose a read-and-select (ReS) framework by
modeling the main components of entity disambiguation, i.e., mention-entity
matching and cross-entity comparison. First, for each candidate, the reading
module leverages mention context to output mention-aware entity
representations, enabling mention-entity matching. Then, in the selecting
module, we frame the choice of candidates as a sequence labeling problem, and
all candidate representations are fused together to enable cross-entity
comparison. Our method achieves the state-of-the-art performance on the
established zero-shot EL dataset ZESHEL with a 2.55% micro-average accuracy
gain, with no need for laborious multi-phase pre-training used in most of the
previous work, showing the effectiveness of both mention-entity and
cross-entity interaction.Comment: EMNLP 2023 Finding
- …