20,194 research outputs found
Entity Linking in the Job Market Domain
In Natural Language Processing, entity linking (EL) has centered around
Wikipedia, but yet remains underexplored for the job market domain.
Disambiguating skill mentions can help us get insight into the current labor
market demands. In this work, we are the first to explore EL in this domain,
specifically targeting the linkage of occupational skills to the ESCO taxonomy
(le Vrang et al., 2014). Previous efforts linked coarse-grained (full)
sentences to a corresponding ESCO skill. In this work, we link more
fine-grained span-level mentions of skills. We tune two high-performing neural
EL models, a bi-encoder (Wu et al., 2020) and an autoregressive model (Cao et
al., 2021), on a synthetically generated mention--skill pair dataset and
evaluate them on a human-annotated skill-linking benchmark. Our findings reveal
that both models are capable of linking implicit mentions of skills to their
correct taxonomy counterparts. Empirically, BLINK outperforms GENRE in strict
evaluation, but GENRE performs better in loose evaluation (accuracy@).Comment: Accepted at EACL 2024 Finding
On the Importance of Drill-Down Analysis for Assessing Gold Standards and Named Entity Linking Performance
Rigorous evaluations and analyses of evaluation results are key towards improving Named Entity Linking systems. Nevertheless, most current evaluation tools are focused on benchmarking and comparative evaluations. Therefore, they only provide aggregated statistics such as precision, recall and F1-measure to assess system performance and no means for conducting detailed analyses up to the level of individual annotations.
This paper addresses the need for transparent benchmarking and fine-grained error analysis by introducing Orbis, an extensible framework that supports drill-down analysis, multiple annotation tasks and resource versioning. Orbis complements approaches like those deployed through the GERBIL and TAC KBP tools and helps developers to better understand and address shortcomings in their Named Entity Linking tools.
We present three uses cases in order to demonstrate the usefulness of Orbis for both research and production systems: (i)improving Named Entity Linking tools; (ii) detecting gold standard errors; and (iii) performing Named Entity Linking evaluations with multiple versions of the included resources
MAG: A Multilingual, Knowledge-base Agnostic and Deterministic Entity Linking Approach
Entity linking has recently been the subject of a significant body of
research. Currently, the best performing approaches rely on trained
mono-lingual models. Porting these approaches to other languages is
consequently a difficult endeavor as it requires corresponding training data
and retraining of the models. We address this drawback by presenting a novel
multilingual, knowledge-based agnostic and deterministic approach to entity
linking, dubbed MAG. MAG is based on a combination of context-based retrieval
on structured knowledge bases and graph algorithms. We evaluate MAG on 23 data
sets and in 7 languages. Our results show that the best approach trained on
English datasets (PBOH) achieves a micro F-measure that is up to 4 times worse
on datasets in other languages. MAG, on the other hand, achieves
state-of-the-art performance on English datasets and reaches a micro F-measure
that is up to 0.6 higher than that of PBOH on non-English languages.Comment: Accepted in K-CAP 2017: Knowledge Capture Conferenc
- …