709 research outputs found
Interpreting Embedding Models of Knowledge Bases: A Pedagogical Approach
Knowledge bases are employed in a variety of applications from natural
language processing to semantic web search; alas, in practice their usefulness
is hurt by their incompleteness. Embedding models attain state-of-the-art
accuracy in knowledge base completion, but their predictions are notoriously
hard to interpret. In this paper, we adapt "pedagogical approaches" (from the
literature on neural networks) so as to interpret embedding models by
extracting weighted Horn rules from them. We show how pedagogical approaches
have to be adapted to take upon the large-scale relational aspects of knowledge
bases and show experimentally their strengths and weaknesses.Comment: presented at 2018 ICML Workshop on Human Interpretability in Machine
Learning (WHI 2018), Stockholm, Swede
Context-aware Path Ranking for Knowledge Base Completion
Knowledge base (KB) completion aims to infer missing facts from existing ones
in a KB. Among various approaches, path ranking (PR) algorithms have received
increasing attention in recent years. PR algorithms enumerate paths between
entity pairs in a KB and use those paths as features to train a model for
missing fact prediction. Due to their good performances and high model
interpretability, several methods have been proposed. However, most existing
methods suffer from scalability (high RAM consumption) and feature explosion
(trains on an exponentially large number of features) problems. This paper
proposes a Context-aware Path Ranking (C-PR) algorithm to solve these problems
by introducing a selective path exploration strategy. C-PR learns global
semantics of entities in the KB using word embedding and leverages the
knowledge of entity semantics to enumerate contextually relevant paths using
bidirectional random walk. Experimental results on three large KBs show that
the path features (fewer in number) discovered by C-PR not only improve
predictive performance but also are more interpretable than existing baselines
Open-Vocabulary Semantic Parsing with both Distributional Statistics and Formal Knowledge
Traditional semantic parsers map language onto compositional, executable
queries in a fixed schema. This mapping allows them to effectively leverage the
information contained in large, formal knowledge bases (KBs, e.g., Freebase) to
answer questions, but it is also fundamentally limiting---these semantic
parsers can only assign meaning to language that falls within the KB's
manually-produced schema. Recently proposed methods for open vocabulary
semantic parsing overcome this limitation by learning execution models for
arbitrary language, essentially using a text corpus as a kind of knowledge
base. However, all prior approaches to open vocabulary semantic parsing replace
a formal KB with textual information, making no use of the KB in their models.
We show how to combine the disparate representations used by these two
approaches, presenting for the first time a semantic parser that (1) produces
compositional, executable representations of language, (2) can successfully
leverage the information contained in both a formal KB and a large corpus, and
(3) is not limited to the schema of the underlying KB. We demonstrate
significantly improved performance over state-of-the-art baselines on an
open-domain natural language question answering task.Comment: Re-written abstract and intro, other minor changes throughout. This
version published at AAAI 201
Modeling relation paths for knowledge base completion via joint adversarial training
Knowledge Base Completion (KBC), which aims at determining the missing
relations between entity pairs, has received increasing attention in recent
years. Most existing KBC methods focus on either embedding the Knowledge Base
(KB) into a specific semantic space or leveraging the joint probability of
Random Walks (RWs) on multi-hop paths. Only a few unified models take both
semantic and path-related features into consideration with adequacy. In this
paper, we propose a novel method to explore the intrinsic relationship between
the single relation (i.e. 1-hop path) and multi-hop paths between paired
entities. We use Hierarchical Attention Networks (HANs) to select important
relations in multi-hop paths and encode them into low-dimensional vectors. By
treating relations and multi-hop paths as two different input sources, we use a
feature extractor, which is shared by two downstream components (i.e. relation
classifier and source discriminator), to capture shared/similar information
between them. By joint adversarial training, we encourage our model to extract
features from the multi-hop paths which are representative for relation
completion. We apply the trained model (except for the source discriminator) to
several large-scale KBs for relation completion. Experimental results show that
our method outperforms existing path information-based approaches. Since each
sub-module of our model can be well interpreted, our model can be applied to a
large number of relation learning tasks.Comment: Accepted by Knowledge-Based System
Path Ranking with Attention to Type Hierarchies
The objective of the knowledge base completion problem is to infer missing
information from existing facts in a knowledge base. Prior work has
demonstrated the effectiveness of path-ranking based methods, which solve the
problem by discovering observable patterns in knowledge graphs, consisting of
nodes representing entities and edges representing relations. However, these
patterns either lack accuracy because they rely solely on relations or cannot
easily generalize due to the direct use of specific entity information. We
introduce Attentive Path Ranking, a novel path pattern representation that
leverages type hierarchies of entities to both avoid ambiguity and maintain
generalization. Then, we present an end-to-end trained attention-based RNN
model to discover the new path patterns from data. Experiments conducted on
benchmark knowledge base completion datasets WN18RR and FB15k-237 demonstrate
that the proposed model outperforms existing methods on the fact prediction
task by statistically significant margins of 26% and 10%, respectively.
Furthermore, quantitative and qualitative analyses show that the path patterns
balance between generalization and discrimination.Comment: Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20
A Retrieve-and-Read Framework for Knowledge Graph Link Prediction
Knowledge graph (KG) link prediction aims to infer new facts based on
existing facts in the KG. Recent studies have shown that using the graph
neighborhood of a node via graph neural networks (GNNs) provides more useful
information compared to just using the query information. Conventional GNNs for
KG link prediction follow the standard message-passing paradigm on the entire
KG, which leads to superfluous computation, over-smoothing of node
representations, and also limits their expressive power. On a large scale, it
becomes computationally expensive to aggregate useful information from the
entire KG for inference. To address the limitations of existing KG link
prediction frameworks, we propose a novel retrieve-and-read framework, which
first retrieves a relevant subgraph context for the query and then jointly
reasons over the context and the query with a high-capacity reader. As part of
our exemplar instantiation for the new framework, we propose a novel
Transformer-based GNN as the reader, which incorporates graph-based attention
structure and cross-attention between query and context for deep fusion. This
simple yet effective design enables the model to focus on salient context
information relevant to the query. Empirical results on two standard KG link
prediction datasets demonstrate the competitive performance of the proposed
method. Furthermore, our analysis yields valuable insights for designing
improved retrievers within the framework.Comment: Accepted to CIKM'23; Published version DOI:
https://doi.org/10.1145/3583780.3614769 ;12 pages, 4 figure
Communicative Message Passing for Inductive Relation Reasoning
Relation prediction for knowledge graphs aims at predicting missing
relationships between entities. Despite the importance of inductive relation
prediction, most previous works are limited to a transductive setting and
cannot process previously unseen entities. The recent proposed subgraph-based
relation reasoning models provided alternatives to predict links from the
subgraph structure surrounding a candidate triplet inductively. However, we
observe that these methods often neglect the directed nature of the extracted
subgraph and weaken the role of relation information in the subgraph modeling.
As a result, they fail to effectively handle the asymmetric/anti-symmetric
triplets and produce insufficient embeddings for the target triplets. To this
end, we introduce a \textbf{C}\textbf{o}mmunicative \textbf{M}essage
\textbf{P}assing neural network for \textbf{I}nductive re\textbf{L}ation
r\textbf{E}asoning, \textbf{CoMPILE}, that reasons over local directed subgraph
structures and has a vigorous inductive bias to process entity-independent
semantic relations. In contrast to existing models, CoMPILE strengthens the
message interactions between edges and entitles through a communicative kernel
and enables a sufficient flow of relation information. Moreover, we demonstrate
that CoMPILE can naturally handle asymmetric/anti-symmetric relations without
the need for explosively increasing the number of model parameters by
extracting the directed enclosing subgraphs. Extensive experiments show
substantial performance gains in comparison to state-of-the-art methods on
commonly used benchmark datasets with variant inductive settings.Comment: Accepted by AAAI-202
- …