1,292 research outputs found
Convolutional 2D Knowledge Graph Embeddings
Link prediction for knowledge graphs is the task of predicting missing
relationships between entities. Previous work on link prediction has focused on
shallow, fast models which can scale to large knowledge graphs. However, these
models learn less expressive features than deep, multi-layer models -- which
potentially limits performance. In this work, we introduce ConvE, a multi-layer
convolutional network model for link prediction, and report state-of-the-art
results for several established datasets. We also show that the model is highly
parameter efficient, yielding the same performance as DistMult and R-GCN with
8x and 17x fewer parameters. Analysis of our model suggests that it is
particularly effective at modelling nodes with high indegree -- which are
common in highly-connected, complex knowledge graphs such as Freebase and
YAGO3. In addition, it has been noted that the WN18 and FB15k datasets suffer
from test set leakage, due to inverse relations from the training set being
present in the test set -- however, the extent of this issue has so far not
been quantified. We find this problem to be severe: a simple rule-based model
can achieve state-of-the-art results on both WN18 and FB15k. To ensure that
models are evaluated on datasets where simply exploiting inverse relations
cannot yield competitive results, we investigate and validate several commonly
used datasets -- deriving robust variants where necessary. We then perform
experiments on these robust datasets for our own and several previously
proposed models and find that ConvE achieves state-of-the-art Mean Reciprocal
Rank across most datasets.Comment: Extended AAAI2018 pape
KBGAN: Adversarial Learning for Knowledge Graph Embeddings
We introduce KBGAN, an adversarial learning framework to improve the
performances of a wide range of existing knowledge graph embedding models.
Because knowledge graphs typically only contain positive facts, sampling useful
negative training examples is a non-trivial task. Replacing the head or tail
entity of a fact with a uniformly randomly selected entity is a conventional
method for generating negative facts, but the majority of the generated
negative facts can be easily discriminated from positive facts, and will
contribute little towards the training. Inspired by generative adversarial
networks (GANs), we use one knowledge graph embedding model as a negative
sample generator to assist the training of our desired model, which acts as the
discriminator in GANs. This framework is independent of the concrete form of
generator and discriminator, and therefore can utilize a wide variety of
knowledge graph embedding models as its building blocks. In experiments, we
adversarially train two translation-based models, TransE and TransD, each with
assistance from one of the two probability-based models, DistMult and ComplEx.
We evaluate the performances of KBGAN on the link prediction task, using three
knowledge base completion datasets: FB15k-237, WN18 and WN18RR. Experimental
results show that adversarial training substantially improves the performances
of target embedding models under various settings.Comment: To appear at NAACL HLT 201
Hypernetwork Knowledge Graph Embeddings
Knowledge graphs are graphical representations of large databases of facts,
which typically suffer from incompleteness. Inferring missing relations (links)
between entities (nodes) is the task of link prediction. A recent
state-of-the-art approach to link prediction, ConvE, implements a convolutional
neural network to extract features from concatenated subject and relation
vectors. Whilst results are impressive, the method is unintuitive and poorly
understood. We propose a hypernetwork architecture that generates simplified
relation-specific convolutional filters that (i) outperforms ConvE and all
previous approaches across standard datasets; and (ii) can be framed as tensor
factorization and thus set within a well established family of factorization
models for link prediction. We thus demonstrate that convolution simply offers
a convenient computational means of introducing sparsity and parameter tying to
find an effective trade-off between non-linear expressiveness and the number of
parameters to learn
Universal Knowledge Graph Embeddings
A variety of knowledge graph embedding approaches have been developed. Most
of them obtain embeddings by learning the structure of the knowledge graph
within a link prediction setting. As a result, the embeddings reflect only the
semantics of a single knowledge graph, and embeddings for different knowledge
graphs are not aligned, e.g., they cannot be used to find similar entities
across knowledge graphs via nearest neighbor search. However, knowledge graph
embedding applications such as entity disambiguation require a more global
representation, i.e., a representation that is valid across multiple sources.
We propose to learn universal knowledge graph embeddings from large-scale
interlinked knowledge sources. To this end, we fuse large knowledge graphs
based on the owl:sameAs relation such that every entity is represented by a
unique identity. We instantiate our idea by computing universal embeddings
based on DBpedia and Wikidata yielding embeddings for about 180 million
entities, 15 thousand relations, and 1.2 billion triples. Moreover, we develop
a convenient API to provide embeddings as a service. Experiments on link
prediction show that universal knowledge graph embeddings encode better
semantics compared to embeddings computed on a single knowledge graph. For
reproducibility purposes, we provide our source code and datasets open access
at https://github.com/dice-group/Universal_EmbeddingsComment: 5 pages, 3 table
- …