53 research outputs found
Open-World Knowledge Graph Completion
Knowledge Graphs (KGs) have been applied to many tasks including Web search,
link prediction, recommendation, natural language processing, and entity
linking. However, most KGs are far from complete and are growing at a rapid
pace. To address these problems, Knowledge Graph Completion (KGC) has been
proposed to improve KGs by filling in its missing connections. Unlike existing
methods which hold a closed-world assumption, i.e., where KGs are fixed and new
entities cannot be easily added, in the present work we relax this assumption
and propose a new open-world KGC task. As a first attempt to solve this task we
introduce an open-world KGC model called ConMask. This model learns embeddings
of the entity's name and parts of its text-description to connect unseen
entities to the KG. To mitigate the presence of noisy text descriptions,
ConMask uses a relationship-dependent content masking to extract relevant
snippets and then trains a fully convolutional neural network to fuse the
extracted snippets with entities in the KG. Experiments on large data sets,
both old and new, show that ConMask performs well in the open-world KGC task
and even outperforms existing KGC models on the standard closed-world KGC task.Comment: 8 pages, accepted to AAAI 201
One-Shot Relational Learning for Knowledge Graphs
Knowledge graphs (KGs) are the key components of various natural language
processing applications. To further expand KGs' coverage, previous studies on
knowledge graph completion usually require a large number of training instances
for each relation. However, we observe that long-tail relations are actually
more common in KGs and those newly added relations often do not have many known
triples for training. In this work, we aim at predicting new facts under a
challenging setting where only one training instance is available. We propose a
one-shot relational learning framework, which utilizes the knowledge extracted
by embedding models and learns a matching metric by considering both the
learned embeddings and one-hop graph structures. Empirically, our model yields
considerable performance improvements over existing embedding models, and also
eliminates the need of re-training the embedding models when dealing with newly
added relations.Comment: EMNLP 201
Dynamic Parameter Allocation in Parameter Servers
To keep up with increasing dataset sizes and model complexity, distributed
training has become a necessity for large machine learning tasks. Parameter
servers ease the implementation of distributed parameter management---a key
concern in distributed training---, but can induce severe communication
overhead. To reduce communication overhead, distributed machine learning
algorithms use techniques to increase parameter access locality (PAL),
achieving up to linear speed-ups. We found that existing parameter servers
provide only limited support for PAL techniques, however, and therefore prevent
efficient training. In this paper, we explore whether and to what extent PAL
techniques can be supported, and whether such support is beneficial. We propose
to integrate dynamic parameter allocation into parameter servers, describe an
efficient implementation of such a parameter server called Lapse, and
experimentally compare its performance to existing parameter servers across a
number of machine learning tasks. We found that Lapse provides near-linear
scaling and can be orders of magnitude faster than existing parameter servers
Generalized Relation Learning with Semantic Correlation Awareness for Link Prediction
Developing link prediction models to automatically complete knowledge graphs
has recently been the focus of significant research interest. The current
methods for the link prediction taskhavetwonaturalproblems:1)the relation
distributions in KGs are usually unbalanced, and 2) there are many unseen
relations that occur in practical situations. These two problems limit the
training effectiveness and practical applications of the existing link
prediction models. We advocate a holistic understanding of KGs and we propose
in this work a unified Generalized Relation Learning framework GRL to address
the above two problems, which can be plugged into existing link prediction
models. GRL conducts a generalized relation learning, which is aware of
semantic correlations between relations that serve as a bridge to connect
semantically similar relations. After training with GRL, the closeness of
semantically similar relations in vector space and the discrimination of
dissimilar relations are improved. We perform comprehensive experiments on six
benchmarks to demonstrate the superior capability of GRL in the link prediction
task. In particular, GRL is found to enhance the existing link prediction
models making them insensitive to unbalanced relation distributions and capable
of learning unseen relations.Comment: Preprint of accepted AAAI2021 pape
An Open-World Extension to Knowledge Graph Completion Models
We present a novel extension to embedding-based knowledge graph completion
models which enables them to perform open-world link prediction, i.e. to
predict facts for entities unseen in training based on their textual
description. Our model combines a regular link prediction model learned from a
knowledge graph with word embeddings learned from a textual corpus. After
training both independently, we learn a transformation to map the embeddings of
an entity's name and description to the graph-based embedding space. In
experiments on several datasets including FB20k, DBPedia50k and our new dataset
FB15k-237-OWE, we demonstrate competitive results. Particularly, our approach
exploits the full knowledge graph structure even when textual descriptions are
scarce, does not require a joint training on graph and text, and can be applied
to any embedding-based link prediction model, such as TransE, ComplEx and
DistMult.Comment: 8 pages, accepted to AAAI-201
- …