PAC Quasi-automatizability of Resolution over Restricted Distributions

We consider principled alternatives to unsupervised learning in data mining by situating the learning task in the context of the subsequent analysis task. Specifically, we consider a query-answering (hypothesis-testing) task: In the combined task, we decide whether an input query formula is satisfied over a background distribution by using input examples directly, rather than invoking a two-stage process in which (i) rules over the distribution are learned by an unsupervised learning algorithm and (ii) a reasoning algorithm decides whether or not the query formula follows from the learned rules. In a previous work (2013), we observed that the learning task could satisfy numerous desirable criteria in this combined context -- effectively matching what could be achieved by agnostic learning of CNFs from partial information -- that are not known to be achievable directly. In this work, we show that likewise, there are reasoning tasks that are achievable in such a combined context that are not known to be achievable directly (and indeed, have been seriously conjectured to be impossible, cf. (Alekhnovich and Razborov, 2008)). Namely, we test for a resolution proof of the query formula of a given size in quasipolynomial time (that is, "quasi-automatizing" resolution). The learning setting we consider is a partial-information, restricted-distribution setting that generalizes learning parities over the uniform distribution from partial information, another task that is known not to be achievable directly in various models (cf. (Ben-David and Dichterman, 1998) and (Michael, 2010))

Learnability with PAC Semantics for Multi-agent Beliefs

The tension between deduction and induction is perhaps the most fundamental issue in areas such as philosophy, cognition and artificial intelligence. In an influential paper, Valiant recognised that the challenge of learning should be integrated with deduction. In particular, he proposed a semantics to capture the quality possessed by the output of Probably Approximately Correct (PAC) learning algorithms when formulated in a logic. Although weaker than classical entailment, it allows for a powerful model-theoretic framework for answering queries. In this paper, we provide a new technical foundation to demonstrate PAC learning with multi-agent epistemic logics. To circumvent the negative results in the literature on the difficulty of robust learning with the PAC semantics, we consider so-called implicit learning where we are able to incorporate observations to the background theory in service of deciding the entailment of an epistemic query. We prove correctness of the learning procedure and discuss results on the sample complexity, that is how many observations we will need to provably assert that the query is entailed given a user-specified error bound. Finally, we investigate under what circumstances this algorithm can be made efficient. On the last point, given that reasoning in epistemic logics especially in multi-agent epistemic logics is PSPACE-complete, it might seem like there is no hope for this problem. We leverage some recent results on the so-called Representation Theorem explored for single-agent and multi-agent epistemic logics with the only knowing operator to reduce modal reasoning to propositional reasoning

On the implicit learnability of knowledge

The deployment of knowledge-based systems in the real world requires addressing the challenge of knowledge acquisition. While knowledge engineering by hand is a daunting task, machine learning has been proposed as an alternative. However, learning explicit representations for real-world knowledge that feature a desirable level of expressiveness remains difficult and often leads to heuristics without robustness guarantees. Probably Approximately Correct (PAC) Semantics offers strong guarantees, however learning explicit representations is not tractable, even in propositional logic. Previous works have proposed solutions to these challenges by learning to reason directly, without producing an explicit representation of the learned knowledge. Recent work on so-called implicit learning has shown tremendous promise in obtaining polynomial-time results for fragments of first-order logic, bypassing the intractable step of producing an explicit representation of learned knowledge. This thesis extends these ideas to richer logical languages such as arithmetic theories and multi-agent logics. We demonstrate that it is possible to learn to reason efficiently for standard fragments of linear arithmetic, and we establish a general finding that provides an efficient reduction from the learning-to-reason problem for any logic to any sound and complete solver for that logic. We then extend implicit learning in PAC Semantics to handle noisy data in the form of intervals and threshold uncertainty in the language of linear arithmetic. We prove that our extended framework maintains existing polynomial-time complexity guarantees. Furthermore, we provide the first empirical investigation of this purely theoretical framework. Using benchmark problems, we show that our implicit approach to learning optimal linear programming objective constraints significantly outperforms an explicit approach in practice. Our results demonstrate the effectiveness of PAC Semantics and implicit learning for real-world problems with noisy data and provide a path towards robust learning in expressive languages. Development in reasoning about knowledge and interactions in complex multi-agent systems spans domains such as artificial intelligence, smart traffic, and robotics. In these systems, epistemic logic serves as a formal language for expressing and reasoning about knowledge, beliefs, and communication among agents, yet integrating learning algorithms within multi-agent epistemic logic is challenging due to the inherent complexity of distributed knowledge reasoning. We provide proof of correctness for our learning procedure and analyse the sample complexity required to assert the entailment of an epistemic query. Overall, our work offers a promising approach to integrating learning and deduction in a range of logical languages from linear arithmetic to multi-agent epistemic logics