8,222 research outputs found
Iteratively Learning Embeddings and Rules for Knowledge Graph Reasoning
Reasoning is essential for the development of large knowledge graphs,
especially for completion, which aims to infer new triples based on existing
ones. Both rules and embeddings can be used for knowledge graph reasoning and
they have their own advantages and difficulties. Rule-based reasoning is
accurate and explainable but rule learning with searching over the graph always
suffers from efficiency due to huge search space. Embedding-based reasoning is
more scalable and efficient as the reasoning is conducted via computation
between embeddings, but it has difficulty learning good representations for
sparse entities because a good embedding relies heavily on data richness. Based
on this observation, in this paper we explore how embedding and rule learning
can be combined together and complement each other's difficulties with their
advantages. We propose a novel framework IterE iteratively learning embeddings
and rules, in which rules are learned from embeddings with proper pruning
strategy and embeddings are learned from existing triples and new triples
inferred by rules. Evaluations on embedding qualities of IterE show that rules
help improve the quality of sparse entity embeddings and their link prediction
results. We also evaluate the efficiency of rule learning and quality of rules
from IterE compared with AMIE+, showing that IterE is capable of generating
high quality rules more efficiently. Experiments show that iteratively learning
embeddings and rules benefit each other during learning and prediction.Comment: This paper is accepted by WWW'1
Neurosymbolic AI for Reasoning on Graph Structures: A Survey
Neurosymbolic AI is an increasingly active area of research which aims to
combine symbolic reasoning methods with deep learning to generate models with
both high predictive performance and some degree of human-level
comprehensibility. As knowledge graphs are becoming a popular way to represent
heterogeneous and multi-relational data, methods for reasoning on graph
structures have attempted to follow this neurosymbolic paradigm. Traditionally,
such approaches have utilized either rule-based inference or generated
representative numerical embeddings from which patterns could be extracted.
However, several recent studies have attempted to bridge this dichotomy in ways
that facilitate interpretability, maintain performance, and integrate expert
knowledge. Within this article, we survey a breadth of methods that perform
neurosymbolic reasoning tasks on graph structures. To better compare the
various methods, we propose a novel taxonomy by which we can classify them.
Specifically, we propose three major categories: (1) logically-informed
embedding approaches, (2) embedding approaches with logical constraints, and
(3) rule-learning approaches. Alongside the taxonomy, we provide a tabular
overview of the approaches and links to their source code, if available, for
more direct comparison. Finally, we discuss the applications on which these
methods were primarily used and propose several prospective directions toward
which this new field of research could evolve.Comment: 21 pages, 8 figures, 1 table, currently under review. Corresponding
GitHub page here: https://github.com/NeSymGraph
A Comparison of Algorithms for Learning Hidden Variables in Normal Graphs
A Bayesian factor graph reduced to normal form consists in the
interconnection of diverter units (or equal constraint units) and
Single-Input/Single-Output (SISO) blocks. In this framework localized
adaptation rules are explicitly derived from a constrained maximum likelihood
(ML) formulation and from a minimum KL-divergence criterion using KKT
conditions. The learning algorithms are compared with two other updating
equations based on a Viterbi-like and on a variational approximation
respectively. The performance of the various algorithm is verified on synthetic
data sets for various architectures. The objective of this paper is to provide
the programmer with explicit algorithms for rapid deployment of Bayesian graphs
in the applications.Comment: Submitted for journal publicatio
Iteratively Learning Representations for Unseen Entities with Inter-Rule Correlations
Recent work on knowledge graph completion (KGC) focused on learning
embeddings of entities and relations in knowledge graphs. These embedding
methods require that all test entities are observed at training time, resulting
in a time-consuming retraining process for out-of-knowledge-graph (OOKG)
entities. To address this issue, current inductive knowledge embedding methods
employ graph neural networks (GNNs) to represent unseen entities by aggregating
information of known neighbors. They face three important challenges: (i) data
sparsity, (ii) the presence of complex patterns in knowledge graphs (e.g.,
inter-rule correlations), and (iii) the presence of interactions among rule
mining, rule inference, and embedding. In this paper, we propose a virtual
neighbor network with inter-rule correlations (VNC) that consists of three
stages: (i) rule mining, (ii) rule inference, and (iii) embedding. In the rule
mining process, to identify complex patterns in knowledge graphs, both logic
rules and inter-rule correlations are extracted from knowledge graphs based on
operations over relation embeddings. To reduce data sparsity, virtual neighbors
for OOKG entities are predicted and assigned soft labels by optimizing a
rule-constrained problem. We also devise an iterative framework to capture the
underlying relations between rule learning and embedding learning. In our
experiments, results on both link prediction and triple classification tasks
show that the proposed VNC framework achieves state-of-the-art performance on
four widely-used knowledge graphs. Further analysis reveals that VNC is robust
to the proportion of unseen entities and effectively mitigates data sparsity.Comment: Accepted at CIKM 202
Discrete Multi-modal Hashing with Canonical Views for Robust Mobile Landmark Search
Mobile landmark search (MLS) recently receives increasing attention for its
great practical values. However, it still remains unsolved due to two important
challenges. One is high bandwidth consumption of query transmission, and the
other is the huge visual variations of query images sent from mobile devices.
In this paper, we propose a novel hashing scheme, named as canonical view based
discrete multi-modal hashing (CV-DMH), to handle these problems via a novel
three-stage learning procedure. First, a submodular function is designed to
measure visual representativeness and redundancy of a view set. With it,
canonical views, which capture key visual appearances of landmark with limited
redundancy, are efficiently discovered with an iterative mining strategy.
Second, multi-modal sparse coding is applied to transform visual features from
multiple modalities into an intermediate representation. It can robustly and
adaptively characterize visual contents of varied landmark images with certain
canonical views. Finally, compact binary codes are learned on intermediate
representation within a tailored discrete binary embedding model which
preserves visual relations of images measured with canonical views and removes
the involved noises. In this part, we develop a new augmented Lagrangian
multiplier (ALM) based optimization method to directly solve the discrete
binary codes. We can not only explicitly deal with the discrete constraint, but
also consider the bit-uncorrelated constraint and balance constraint together.
Experiments on real world landmark datasets demonstrate the superior
performance of CV-DMH over several state-of-the-art methods
- …