37,993 research outputs found
Investigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linking
A typical architecture for end-to-end entity linking systems consists of
three steps: mention detection, candidate generation and entity disambiguation.
In this study we investigate the following questions: (a) Can all those steps
be learned jointly with a model for contextualized text-representations, i.e.
BERT (Devlin et al., 2019)? (b) How much entity knowledge is already contained
in pretrained BERT? (c) Does additional entity knowledge improve BERT's
performance in downstream tasks? To this end, we propose an extreme
simplification of the entity linking setup that works surprisingly well: simply
cast it as a per token classification over the entire entity vocabulary (over
700K classes in our case). We show on an entity linking benchmark that (i) this
model improves the entity representations over plain BERT, (ii) that it
outperforms entity linking architectures that optimize the tasks separately and
(iii) that it only comes second to the current state-of-the-art that does
mention detection and entity disambiguation jointly. Additionally, we
investigate the usefulness of entity-aware token-representations in the
text-understanding benchmark GLUE, as well as the question answering benchmarks
SQUAD V2 and SWAG and also the EN-DE WMT14 machine translation benchmark. To
our surprise, we find that most of those benchmarks do not benefit from
additional entity knowledge, except for a task with very small training data,
the RTE task in GLUE, which improves by 2%.Comment: Published at CoNLL 201
Improving Entity Linking by Modeling Latent Relations between Mentions
Entity linking involves aligning textual mentions of named entities to their
corresponding entries in a knowledge base. Entity linking systems often exploit
relations between textual mentions in a document (e.g., coreference) to decide
if the linking decisions are compatible. Unlike previous approaches, which
relied on supervised systems or heuristics to predict these relations, we treat
relations as latent variables in our neural entity-linking model. We induce the
relations without any supervision while optimizing the entity-linking system in
an end-to-end fashion. Our multi-relational model achieves the best reported
scores on the standard benchmark (AIDA-CoNLL) and substantially outperforms its
relation-agnostic version. Its training also converges much faster, suggesting
that the injected structural bias helps to explain regularities in the training
data.Comment: ACL 201
Recommended from our members
Encoding Knowledge Graph Entity Aliases in Attentive Neural Network for Wikidata Entity Linking
The collaborative knowledge graphs such as Wikidata excessively rely on the crowd to author the information. Since the crowd is not bound to a standard protocol for assigning entity titles, the knowledge graph is populated by non-standard, noisy, long or even sometimes awkward titles. The issue of long, implicit, and nonstandard entity representations is a challenge in Entity Linking (EL) approaches for gaining high precision and recall. Underlying KG in general is the source of target entities for EL approaches, however, it often contains other relevant information, such as aliases of entities (e.g., Obama and Barack Hussein Obama are aliases for the entity Barack Obama). EL models usually ignore such readily available entity attributes. In this paper, we examine the role of knowledge graph context on an attentive neural network approach for entity linking on Wikidata. Our approach contributes by exploiting the sufficient context from a KG as a source of background knowledge, which is then fed into the neural network. This approach demonstrates merit to address challenges associated with entity titles (multi-word, long, implicit, case-sensitive). Our experimental study shows â8% improvements over the baseline approach, and significantly outperform an end to end approach for Wikidata entity linking
Encoding Knowledge Graph Entity Aliases in Attentive Neural Network for Wikidata Entity Linking
The collaborative knowledge graphs such as Wikidata excessively rely on the
crowd to author the information. Since the crowd is not bound to a standard
protocol for assigning entity titles, the knowledge graph is populated by
non-standard, noisy, long or even sometimes awkward titles. The issue of long,
implicit, and nonstandard entity representations is a challenge in Entity
Linking (EL) approaches for gaining high precision and recall. Underlying KG,
in general, is the source of target entities for EL approaches, however, it
often contains other relevant information, such as aliases of entities (e.g.,
Obama and Barack Hussein Obama are aliases for the entity Barack Obama). EL
models usually ignore such readily available entity attributes. In this paper,
we examine the role of knowledge graph context on an attentive neural network
approach for entity linking on Wikidata. Our approach contributes by exploiting
the sufficient context from a KG as a source of background knowledge, which is
then fed into the neural network. This approach demonstrates merit to address
challenges associated with entity titles (multi-word, long, implicit,
case-sensitive). Our experimental study shows approx 8% improvements over the
baseline approach, and significantly outperform an end to end approach for
Wikidata entity linking.Comment: 15 page
- âŠ