16 research outputs found
STransE: a novel embedding model of entities and relationships in knowledge bases
Knowledge bases of real-world facts about entities and their relationships
are useful resources for a variety of natural language processing tasks.
However, because knowledge bases are typically incomplete, it is useful to be
able to perform link prediction or knowledge base completion, i.e., predict
whether a relationship not in the knowledge base is likely to be true. This
paper combines insights from several previous link prediction models into a new
embedding model STransE that represents each entity as a low-dimensional
vector, and each relation by two matrices and a translation vector. STransE is
a simple combination of the SE and TransE models, but it obtains better link
prediction performance on two benchmark datasets than previous embedding
models. Thus, STransE can serve as a new baseline for the more complex models
in the link prediction task.Comment: V1: In Proceedings of the 2016 Conference of the North American
Chapter of the Association for Computational Linguistics: Human Language
Technologies, NAACL HLT 2016. V2: Corrected citation to (Krompa{\ss} et al.,
2015). V3: A revised version of our NAACL-HLT 2016 paper with additional
experimental results and latest related wor
On Multi-Relational Link Prediction with Bilinear Models
We study bilinear embedding models for the task of multi-relational link
prediction and knowledge graph completion. Bilinear models belong to the most
basic models for this task, they are comparably efficient to train and use, and
they can provide good prediction performance. The main goal of this paper is to
explore the expressiveness of and the connections between various bilinear
models proposed in the literature. In particular, a substantial number of
models can be represented as bilinear models with certain additional
constraints enforced on the embeddings. We explore whether or not these
constraints lead to universal models, which can in principle represent every
set of relations, and whether or not there are subsumption relationships
between various models. We report results of an independent experimental study
that evaluates recent bilinear models in a common experimental setup. Finally,
we provide evidence that relation-level ensembles of multiple bilinear models
can achieve state-of-the art prediction performance
Convolutional 2D Knowledge Graph Embeddings
Link prediction for knowledge graphs is the task of predicting missing
relationships between entities. Previous work on link prediction has focused on
shallow, fast models which can scale to large knowledge graphs. However, these
models learn less expressive features than deep, multi-layer models -- which
potentially limits performance. In this work, we introduce ConvE, a multi-layer
convolutional network model for link prediction, and report state-of-the-art
results for several established datasets. We also show that the model is highly
parameter efficient, yielding the same performance as DistMult and R-GCN with
8x and 17x fewer parameters. Analysis of our model suggests that it is
particularly effective at modelling nodes with high indegree -- which are
common in highly-connected, complex knowledge graphs such as Freebase and
YAGO3. In addition, it has been noted that the WN18 and FB15k datasets suffer
from test set leakage, due to inverse relations from the training set being
present in the test set -- however, the extent of this issue has so far not
been quantified. We find this problem to be severe: a simple rule-based model
can achieve state-of-the-art results on both WN18 and FB15k. To ensure that
models are evaluated on datasets where simply exploiting inverse relations
cannot yield competitive results, we investigate and validate several commonly
used datasets -- deriving robust variants where necessary. We then perform
experiments on these robust datasets for our own and several previously
proposed models and find that ConvE achieves state-of-the-art Mean Reciprocal
Rank across most datasets.Comment: Extended AAAI2018 pape
Adversarial Sets for Regularising Neural Link Predictors
In adversarial training, a set of models learn together by pursuing competing
goals, usually defined on single data instances. However, in relational
learning and other non-i.i.d domains, goals can also be defined over sets of
instances. For example, a link predictor for the is-a relation needs to be
consistent with the transitivity property: if is-a(x_1, x_2) and is-a(x_2, x_3)
hold, is-a(x_1, x_3) needs to hold as well. Here we use such assumptions for
deriving an inconsistency loss, measuring the degree to which the model
violates the assumptions on an adversarially-generated set of examples. The
training objective is defined as a minimax problem, where an adversary finds
the most offending adversarial examples by maximising the inconsistency loss,
and the model is trained by jointly minimising a supervised loss and the
inconsistency loss on the adversarial examples. This yields the first method
that can use function-free Horn clauses (as in Datalog) to regularise any
neural link predictor, with complexity independent of the domain size. We show
that for several link prediction models, the optimisation problem faced by the
adversary has efficient closed-form solutions. Experiments on link prediction
benchmarks indicate that given suitable prior knowledge, our method can
significantly improve neural link predictors on all relevant metrics.Comment: Proceedings of the 33rd Conference on Uncertainty in Artificial
Intelligence (UAI), 201
Relational Neural Machines
Deep learning has been shown to achieve impressive results in several tasks
where a large amount of training data is available. However, deep learning
solely focuses on the accuracy of the predictions, neglecting the reasoning
process leading to a decision, which is a major issue in life-critical
applications. Probabilistic logic reasoning allows to exploit both statistical
regularities and specific domain expertise to perform reasoning under
uncertainty, but its scalability and brittle integration with the layers
processing the sensory data have greatly limited its applications. For these
reasons, combining deep architectures and probabilistic logic reasoning is a
fundamental goal towards the development of intelligent agents operating in
complex environments. This paper presents Relational Neural Machines, a novel
framework allowing to jointly train the parameters of the learners and of a
First--Order Logic based reasoner. A Relational Neural Machine is able to
recover both classical learning from supervised data in case of pure
sub-symbolic learning, and Markov Logic Networks in case of pure symbolic
reasoning, while allowing to jointly train and perform inference in hybrid
learning tasks. Proper algorithmic solutions are devised to make learning and
inference tractable in large-scale problems. The experiments show promising
results in different relational tasks