14 research outputs found
Learning Reasoning Strategies in End-to-End Differentiable Proving
Attempts to render deep learning models interpretable, data-efficient, and
robust have seen some success through hybridisation with rule-based systems,
for example, in Neural Theorem Provers (NTPs). These neuro-symbolic models can
induce interpretable rules and learn representations from data via
back-propagation, while providing logical explanations for their predictions.
However, they are restricted by their computational complexity, as they need to
consider all possible proof paths for explaining a goal, thus rendering them
unfit for large-scale applications. We present Conditional Theorem Provers
(CTPs), an extension to NTPs that learns an optimal rule selection strategy via
gradient-based optimisation. We show that CTPs are scalable and yield
state-of-the-art results on the CLUTRR dataset, which tests systematic
generalisation of neural models by learning to reason over smaller graphs and
evaluating on larger ones. Finally, CTPs show better link prediction results on
standard benchmarks in comparison with other neural-symbolic models, while
being explainable. All source code and datasets are available online, at
https://github.com/uclnlp/ctp.Comment: Proceedings of the 37th International Conference on Machine Learning
(ICML 2020
Complex Query Answering on Eventuality Knowledge Graph with Implicit Logical Constraints
Querying incomplete knowledge graphs (KGs) using deep learning approaches can
naturally leverage the reasoning and generalization ability to learn to infer
better answers. Traditional neural complex query answering (CQA) approaches
mostly work on entity-centric KGs. However, in the real world, we also need to
make logical inferences about events, states, and activities (i.e.,
eventualities or situations) to push learning systems from System I to System
II, as proposed by Yoshua Bengio. Querying logically from an
EVentuality-centric KG (EVKG) can naturally provide references to such kind of
intuitive and logical inference. Thus, in this paper, we propose a new
framework to leverage neural methods to answer complex logical queries based on
an EVKG, which can satisfy not only traditional first-order logic constraints
but also implicit logical constraints over eventualities concerning their
occurrences and orders. For instance, if we know that ``Food is bad'' happens
before ``PersonX adds soy sauce,'' then ``PersonX adds soy sauce'' is unlikely
to be the cause of ``Food is bad'' due to implicit temporal constraint. To
facilitate consistent reasoning on EVKGs, we propose Complex Eventuality Query
Answering (CEQA), a more rigorous definition of CQA that considers the implicit
logical constraints governing the temporal order and occurrence of
eventualities. In this manner, we propose to leverage theorem provers for
constructing benchmark datasets to ensure the answers satisfy implicit logical
constraints. We also propose a Memory-Enhanced Query Encoding (MEQE) approach
to significantly improve the performance of state-of-the-art neural query
encoders on the CEQA task
On the Aggregation of Rules for Knowledge Graph Completion
Rule learning approaches for knowledge graph completion are efficient,
interpretable and competitive to purely neural models. The rule aggregation
problem is concerned with finding one plausibility score for a candidate fact
which was simultaneously predicted by multiple rules. Although the problem is
ubiquitous, as data-driven rule learning can result in noisy and large
rulesets, it is underrepresented in the literature and its theoretical
foundations have not been studied before in this context. In this work, we
demonstrate that existing aggregation approaches can be expressed as marginal
inference operations over the predicting rules. In particular, we show that the
common Max-aggregation strategy, which scores candidates based on the rule with
the highest confidence, has a probabilistic interpretation. Finally, we propose
an efficient and overlooked baseline which combines the previous strategies and
is competitive to computationally more expensive approaches.Comment: KLR Workshop@ICML202
Modular Design Patterns for Hybrid Learning and Reasoning Systems: a taxonomy, patterns and use cases
The unification of statistical (data-driven) and symbolic (knowledge-driven)
methods is widely recognised as one of the key challenges of modern AI. Recent
years have seen large number of publications on such hybrid neuro-symbolic AI
systems. That rapidly growing literature is highly diverse and mostly
empirical, and is lacking a unifying view of the large variety of these hybrid
systems. In this paper we analyse a large body of recent literature and we
propose a set of modular design patterns for such hybrid, neuro-symbolic
systems. We are able to describe the architecture of a very large number of
hybrid systems by composing only a small set of elementary patterns as building
blocks.
The main contributions of this paper are: 1) a taxonomically organised
vocabulary to describe both processes and data structures used in hybrid
systems; 2) a set of 15+ design patterns for hybrid AI systems, organised in a
set of elementary patterns and a set of compositional patterns; 3) an
application of these design patterns in two realistic use-cases for hybrid AI
systems. Our patterns reveal similarities between systems that were not
recognised until now. Finally, our design patterns extend and refine Kautz'
earlier attempt at categorising neuro-symbolic architectures.Comment: 20 pages, 22 figures, accepted for publication in the International
Journal of Applied Intelligenc
On the aggregation of rules for knowledge graph completion
Rule learning approaches for knowledge graph completion are efficient, interpretable and competitive to purely neural models. The rule aggregation problem is concerned with finding one plausibility score for a candidate fact which was simultaneously predicted by multiple rules. Although the problem is ubiquitous, as data-driven rule learning can result in noisy and large rule sets, it is underrepresented in the literature and its theoretical foundations have not been studied before in this context. In this work, we demonstrate that existing aggregation approaches can be expressed as marginal inference operations over the predicting rules. In particular, we show that the common Max-aggregation strategy, which scores candidates based on the rule with the highest confidence, has a probabilistic interpretation. Finally, we propose an efficient and overlooked baseline which combines the previous strategies and is competitive to computationally more expensive approaches