21 research outputs found
Knowledge Base Completion: Baseline strikes back (Again)
Knowledge Base Completion has been a very active area recently, where
multiplicative models have generally outperformed additive and other deep
learning methods -- like GNN, CNN, path-based models. Several recent KBC papers
propose architectural changes, new training methods, or even a new problem
reformulation. They evaluate their methods on standard benchmark datasets -
FB15k, FB15k-237, WN18, WN18RR, and Yago3-10. Recently, some papers discussed
how 1-N scoring can speed up training and evaluation. In this paper, we discuss
how by just applying this training regime to a basic model like Complex gives
near SOTA performance on all the datasets -- we call this model COMPLEX-V2. We
also highlight how various multiplicative methods recently proposed in
literature benefit from this trick and become indistinguishable in terms of
performance on most datasets. This paper calls for a reassessment of their
individual value, in light of these findings
InteractE: Improving Convolution-based Knowledge Graph Embeddings by Increasing Feature Interactions
Most existing knowledge graphs suffer from incompleteness, which can be
alleviated by inferring missing links based on known facts. One popular way to
accomplish this is to generate low-dimensional embeddings of entities and
relations, and use these to make inferences. ConvE, a recently proposed
approach, applies convolutional filters on 2D reshapings of entity and relation
embeddings in order to capture rich interactions between their components.
However, the number of interactions that ConvE can capture is limited. In this
paper, we analyze how increasing the number of these interactions affects link
prediction performance, and utilize our observations to propose InteractE.
InteractE is based on three key ideas -- feature permutation, a novel feature
reshaping, and circular convolution. Through extensive experiments, we find
that InteractE outperforms state-of-the-art convolutional link prediction
baselines on FB15k-237. Further, InteractE achieves an MRR score that is 9%,
7.5%, and 23% better than ConvE on the FB15k-237, WN18RR and YAGO3-10 datasets
respectively. The results validate our central hypothesis -- that increasing
feature interaction is beneficial to link prediction performance. We make the
source code of InteractE available to encourage reproducible research.Comment: Accepted at AAAI 202
MEIM: Multi-partition Embedding Interaction Beyond Block Term Format for Efficient and Expressive Link Prediction
Knowledge graph embedding aims to predict the missing relations between
entities in knowledge graphs. Tensor-decomposition-based models, such as
ComplEx, provide a good trade-off between efficiency and expressiveness, that
is crucial because of the large size of real world knowledge graphs. The recent
multi-partition embedding interaction (MEI) model subsumes these models by
using the block term tensor format and provides a systematic solution for the
trade-off. However, MEI has several drawbacks, some of which carried from its
subsumed tensor-decomposition-based models. In this paper, we address these
drawbacks and introduce the Multi-partition Embedding Interaction iMproved
beyond block term format (MEIM) model, with independent core tensor for
ensemble effects and soft orthogonality for max-rank mapping, in addition to
multi-partition embedding. MEIM improves expressiveness while still being
highly efficient, helping it to outperform strong baselines and achieve
state-of-the-art results on difficult link prediction benchmarks using fairly
small embedding sizes. The source code is released at
https://github.com/tranhungnghiep/MEIM-KGE.Comment: Accepted at the International Joint Conference on Artificial
Intelligence (IJCAI), 2022; add appendix with extra experiment
Relation Prediction as an Auxiliary Training Objective for Improving Multi-Relational Graph Representations
Learning good representations on multi-relational graphs is essential to knowledge base completion (KBC). In this paper, we propose a new self-supervised training objective for multi-relational
graph representation learning, via simply incorporating relation prediction into the commonly used
1vsAll objective. The new training objective contains not only terms for predicting the subject
and object of a given triple, but also a term for predicting the relation type. We analyse how this
new objective impacts multi-relational learning in KBC: experiments on a variety of datasets and
models show that relation prediction can significantly improve entity ranking, the most widely
used evaluation task for KBC, yielding a 6.1% increase in MRR and 9.9% increase in Hits@1
on FB15k-237 as well as a 3.1% increase in MRR and 3.4% in Hits@1 on Aristo-v4. Moreover,
we observe that the proposed objective is especially effective on highly multi-relational datasets,
i.e. datasets with a large number of predicates, and generates better representations when larger
embedding sizes are used
Contextual Dictionary Lookup for Knowledge Graph Completion
Knowledge graph completion (KGC) aims to solve the incompleteness of
knowledge graphs (KGs) by predicting missing links from known triples, numbers
of knowledge graph embedding (KGE) models have been proposed to perform KGC by
learning embeddings. Nevertheless, most existing embedding models map each
relation into a unique vector, overlooking the specific fine-grained semantics
of them under different entities. Additionally, the few available fine-grained
semantic models rely on clustering algorithms, resulting in limited performance
and applicability due to the cumbersome two-stage training process. In this
paper, we present a novel method utilizing contextual dictionary lookup,
enabling conventional embedding models to learn fine-grained semantics of
relations in an end-to-end manner. More specifically, we represent each
relation using a dictionary that contains multiple latent semantics. The
composition of a given entity and the dictionary's central semantics serves as
the context for generating a lookup, thus determining the fine-grained
semantics of the relation adaptively. The proposed loss function optimizes both
the central and fine-grained semantics simultaneously to ensure their semantic
consistency. Besides, we introduce two metrics to assess the validity and
accuracy of the dictionary lookup operation. We extend several KGE models with
the method, resulting in substantial performance improvements on widely-used
benchmark datasets