12 research outputs found
Implicitly Learning to Reason in First-Order Logic
We consider the problem of answering queries about formulas of first-order
logic based on background knowledge partially represented explicitly as other
formulas, and partially represented as examples independently drawn from a
fixed probability distribution. PAC semantics, introduced by Valiant, is one
rigorous, general proposal for learning to reason in formal languages: although
weaker than classical entailment, it allows for a powerful model theoretic
framework for answering queries while requiring minimal assumptions about the
form of the distribution in question. To date, however, the most significant
limitation of that approach, and more generally most machine learning
approaches with robustness guarantees, is that the logical language is
ultimately essentially propositional, with finitely many atoms. Indeed, the
theoretical findings on the learning of relational theories in such generality
have been resoundingly negative. This is despite the fact that first-order
logic is widely argued to be most appropriate for representing human knowledge.
In this work, we present a new theoretical approach to robustly learning to
reason in first-order logic, and consider universally quantified clauses over a
countably infinite domain. Our results exploit symmetries exhibited by
constants in the language, and generalize the notion of implicit learnability
to show how queries can be computed against (implicitly) learned first-order
background knowledge.Comment: In Fourth International Workshop on Declarative Learning Based
Programming (DeLBP 2019
Learnability with PAC Semantics for Multi-agent Beliefs
The tension between deduction and induction is perhaps the most fundamental
issue in areas such as philosophy, cognition and artificial intelligence. In an
influential paper, Valiant recognised that the challenge of learning should be
integrated with deduction. In particular, he proposed a semantics to capture
the quality possessed by the output of Probably Approximately Correct (PAC)
learning algorithms when formulated in a logic. Although weaker than classical
entailment, it allows for a powerful model-theoretic framework for answering
queries. In this paper, we provide a new technical foundation to demonstrate
PAC learning with multi-agent epistemic logics. To circumvent the negative
results in the literature on the difficulty of robust learning with the PAC
semantics, we consider so-called implicit learning where we are able to
incorporate observations to the background theory in service of deciding the
entailment of an epistemic query. We prove correctness of the learning
procedure and discuss results on the sample complexity, that is how many
observations we will need to provably assert that the query is entailed given a
user-specified error bound. Finally, we investigate under what circumstances
this algorithm can be made efficient. On the last point, given that reasoning
in epistemic logics especially in multi-agent epistemic logics is
PSPACE-complete, it might seem like there is no hope for this problem. We
leverage some recent results on the so-called Representation Theorem explored
for single-agent and multi-agent epistemic logics with the only knowing
operator to reduce modal reasoning to propositional reasoning
PAC Quasi-automatizability of Resolution over Restricted Distributions
We consider principled alternatives to unsupervised learning in data mining
by situating the learning task in the context of the subsequent analysis task.
Specifically, we consider a query-answering (hypothesis-testing) task: In the
combined task, we decide whether an input query formula is satisfied over a
background distribution by using input examples directly, rather than invoking
a two-stage process in which (i) rules over the distribution are learned by an
unsupervised learning algorithm and (ii) a reasoning algorithm decides whether
or not the query formula follows from the learned rules. In a previous work
(2013), we observed that the learning task could satisfy numerous desirable
criteria in this combined context -- effectively matching what could be
achieved by agnostic learning of CNFs from partial information -- that are not
known to be achievable directly. In this work, we show that likewise, there are
reasoning tasks that are achievable in such a combined context that are not
known to be achievable directly (and indeed, have been seriously conjectured to
be impossible, cf. (Alekhnovich and Razborov, 2008)). Namely, we test for a
resolution proof of the query formula of a given size in quasipolynomial time
(that is, "quasi-automatizing" resolution). The learning setting we consider is
a partial-information, restricted-distribution setting that generalizes
learning parities over the uniform distribution from partial information,
another task that is known not to be achievable directly in various models (cf.
(Ben-David and Dichterman, 1998) and (Michael, 2010))