Nesting Probabilistic Inference

When doing inference in ProbLog, a probabilistic extension of Prolog, we extend SLD resolution with some additional bookkeeping. This additional information is used to compute the probabilistic results for a probabilistic query. In Prolog's SLD, goals are nested very naturally. In ProbLog's SLD, nesting probabilistic queries interferes with the probabilistic bookkeeping. In order to support nested probabilistic inference we propose the notion of a parametrised ProbLog engine. Nesting becomes possible by suspending and resuming instances of ProbLog engines. With our approach we realise several extensions of ProbLog such as meta-calls, negation, and answers of probabilistic goals.Comment: Online Proceedings of the 11th International Colloquium on Implementation of Constraint LOgic Programming Systems (CICLOPS 2011), Lexington, KY, U.S.A., July 10, 201

DNF Sampling for ProbLog Inference

Inference in probabilistic logic languages such as ProbLog, an extension of Prolog with probabilistic facts, is often based on a reduction to a propositional formula in DNF. Calculating the probability of such a formula involves the disjoint-sum-problem, which is computationally hard. In this work we introduce a new approximation method for ProbLog inference which exploits the DNF to focus sampling. While this DNF sampling technique has been applied to a variety of tasks before, to the best of our knowledge it has not been used for inference in probabilistic logic systems. The paper also presents an experimental comparison with another sampling based inference method previously introduced for ProbLog.Comment: Online proceedings of the Joint Workshop on Implementation of Constraint Logic Programming Systems and Logic-based Methods in Programming Environments (CICLOPS-WLPE 2010), Edinburgh, Scotland, U.K., July 15, 201

SkILL - a Stochastic Inductive Logic Learner

Probabilistic Inductive Logic Programming (PILP) is a rel- atively unexplored area of Statistical Relational Learning which extends classic Inductive Logic Programming (ILP). This work introduces SkILL, a Stochastic Inductive Logic Learner, which takes probabilistic annotated data and produces First Order Logic theories. Data in several domains such as medicine and bioinformatics have an inherent degree of uncer- tainty, that can be used to produce models closer to reality. SkILL can not only use this type of probabilistic data to extract non-trivial knowl- edge from databases, but it also addresses efficiency issues by introducing a novel, efficient and effective search strategy to guide the search in PILP environments. The capabilities of SkILL are demonstrated in three dif- ferent datasets: (i) a synthetic toy example used to validate the system, (ii) a probabilistic adaptation of a well-known biological metabolism ap- plication, and (iii) a real world medical dataset in the breast cancer domain. Results show that SkILL can perform as well as a deterministic ILP learner, while also being able to incorporate probabilistic knowledge that would otherwise not be considered

Efficient Algorithms for Prolog Based Probabilistic Logic Programming (Efficiënte algoritmen voor prolog gebaseerd probabilistisch logisch programmeren)

The integration of probabilistic reasoning with logic programming has become one of the challenges in Artificial Intelligence. Lately, a lot of Probabilistic Logic Programming (PLP) formalisms have surfaced. Given that PLP is the combination of logic programming and probabilities which are two very different fields, it is expected that researchers from several fields come up with different approaches to tackle the presented challenges. This has resulted in a new discipline called Probabilistic Logic Learning (PLL) or Statistical Relational Learning (SRL) and a very active research community.Within this community, ProbLog a probabilistic extension of Prolog, has appeared. ProbLog was motivated by the task of mining links in large probabilistic graphs. The simple but powerful ProbLog formulation was extended in order to support inference on several different models. Soon ProbLog evolved into a general purpose probabilistic programming language that provides infrastructure for many PLL/SRL tasks. The two most critical aspects of a PLP language are its expression power, and its scalability over common PLL problems.This thesis focuses on the extension and implementation of ProbLog. Tabling is a well known method for avoiding re-computation in logic programming; we present how tabling can be implemented for ProbLog. Tabling has significant benefits for performance and also allows ProbLog to handle cyclic programs. In order for the ProbLog system to perform inference, it manipulates large Boolean formulae. We present the manipulation of Boolean formulae and several novel algorithms that improve the performance of this task. Furthermore, we present patterns, called AND/OR-clusters, that can be used to reduce a Boolean formula to an equivalent one. Besides performance improvements of the ProbLog system, we also present several novel extensions to the ProbLog language, such as: general negation, probabilistic meta calls and ProbLog answers. Finally, we present two applications of ProbLog that use several of the features we have contributed to ProbLog.Abstract iii Contents v List of Figures xi List of Tables xv List of Algorithms xvii 1 Introduction 1 2 ProbLog 7 2.1 ProbLog and the Distribution Semantics . . . . . . . . . . . . . . . . . . . 7 2.1.1 Possible Worlds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.2 Success Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.3 Knowledge Compilation Approaches . . . . . . . . . . . . . . . . . . . . . . 10 2.3.1 SLD Resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3.2 Tries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.3.3 Boolean Formulae Preprocessing . . . . . . . . . . . . . . . . . . . . . . 14 2.3.4 Reduced Order Binary Decision Diagrams (ROBDDs) . . . . . . . . . . . . . . 16 2.3.5 Boolean Formulae Compilation and SimpleCUDD . . . . . . . . . . . . . . . . 17 2.3.6 Syntax of the Script Language . . . . . . . . . . . . . . . . . . . . . . . 18 2.3.7 ROBDD Generation and Probability Calculation . . . . . . . . . . . . . . . 20 2.4 Other Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.4.1 Approximate Inference: Program Sampling . . . . . . . . . . . . . . . . . . 22 2.4.2 Approximate Inference: DNF Sampling . . . . . . . . . . . . . . . . . . . . 23 2.4.3 Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.4.4 Weighted CNF ProbLog . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3 Tabling of Probabilistic Logic Programs 29 3.1 SLG Resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 3.2 ProbLog & Tabling Preliminaries . . . . . . . . . . . . . . . . . . . . . . . 35 3.2.1 ProbLog Tabling Example . . . . . . . . . . . . . . . . . . . . . . . . . . 35 3.2.2 Ground Goal Assumption . . . . . . . . . . . . . . . . . . . . . . . . . . 36 3.2.3 Nested Tries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.3 ProbLog Tabling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.3.1 Cycles in a Probabilistic Setting . . . . . . . . . . . . . . . . . . . . . 39 3.4 Nested Tries to ROBDD Definitions . . . . . . . . . . . . . . . . . . . . . . 41 3.4.1 Handling the Simple Case . . . . . . . . . . . . . . . . . . . . . . . . . 41 3.4.2 Handling Cycles and the Ancestor List . . . . . . . . . . . . . . . . . . . 43 3.4.3 Optimization I: Subset . . . . . . . . . . . . . . . . . . . . . . . . . . 48 3.4.4 Optimization II: Ancestor List Refine . . . . . . . . . . . . . . . . . . . 49 3.4.5 Optimization III: Pre-process Step . . . . . . . . . . . . . . . . . . . . 51 3.4.6 Optimizing the Representation . . . . . . . . . . . . . . . . . . . . . . . 54 3.5 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.5.1 Light Weight Tabling Implementation . . . . . . . . . . . . . . . . . . . . 55 3.5.2 Built-in Tabling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 3.6 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 3.6.1 Benchmark Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 3.6.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 3.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 4 Preprocessing I: Boolean Formulae to ROBDD Definitions 71 4.1 Preprocessing Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 4.2 Naive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 4.3 Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 4.4 Recursive Node Merging . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 4.5 Depth Breadth Trie . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 4.6 Complexity of Preprocessing Methods . . . . . . . . . . . . . . . . . . . . . 83 4.6.1 Naive Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 4.6.2 Decomposition Method . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 4.6.3 Recursive Node Merging . . . . . . . . . . . . . . . . . . . . . . . . . . 85 4.7 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 4.8 Nested Tries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 4.9 Negation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 4.9.1 Negated Literals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 4.9.2 General Negation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 4.10 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . 95 5 Preprocessing II: Variable Compression 97 5.1 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 5.2 Cluster Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 5.2.1 Using the Clusters for Variable Compression . . . . . . . . . . . . . . . . 103 5.3 Discovering AND-clusters . . . . . . . . . . . . . . . . . . . . . . . . . . 104 5.3.1 The Book Marking Algorithm for AND-clusters . . . . . . . . . . . . . . . . 105 5.3.2 An Example of the Book Marking Algorithm for AND-clusters . . . . . . . . . 107 5.4 Discovering OR-clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 5.4.1 The Book Marking Algorithm for OR-clusters . . . . . . . . . . . . . . . . 109 5.4.2 An Example of the Book Marking Algorithm for OR-clusters . . . . . . . . . 111 5.5 Experiments for AND-clusters . . . . . . . . . . . . . . . . . . . . . . . . 111 5.6 Complexity Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 5.7 Related Work and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . 117 6 A New Implementation: MetaProbLog 119 6.1 Why Probabilistic Meta-calls . . . . . . . . . . . . . . . . . . . . . . . . 121 6.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 6.3 Technical Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 6.3.1 ProbLog Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 6.3.2 Parameters of the ProbLog Engine . . . . . . . . . . . . . . . . . . . . . 124 6.3.3 Inference Method: Exact . . . . . . . . . . . . . . . . . . . . . . . . . . 125 6.3.4 Inference Method: Program Sampling . . . . . . . . . . . . . . . . . . . . 125 6.3.5 ProbLog Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 6.3.6 Nesting ProbLog Engines . . . . . . . . . . . . . . . . . . . . . . . . . . 128 6.3.7 Calling the ProbLog Engine . . . . . . . . . . . . . . . . . . . . . . . . 128 6.4 Nested Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 6.4.1 Nested Inference Returning Success Probability . . . . . . . . . . . . . . 129 6.4.2 Nested Inference Returning Information & ProbLog Negation . . . . . . . . . 129 6.4.3 Nested Inference Returning Answers & ProbLog Answers 131 6.5 Modularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 6.6 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 6.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 7 Applications 141 7.1 First Order ProbLog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 7.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 7.1.2 First Order ProbLog and its Semantics . . . . . . . . . . . . . . . . . . . 143 7.1.3 Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 7.1.4 A Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 7.1.5 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 7.1.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 7.1.7 Related Work and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . 156 7.2 Analysing a Publish/Subscribe System for MANETs with ProbLog 157 7.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 7.2.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 7.2.3 ProbLog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 7.2.4 Fadip Model in ProbLog . . . . . . . . . . . . . . . . . . . . . . . . . . 158 7.2.5 Analysing the Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 7.2.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 7.3 Appendix of: Analysing a Publish/Subscribe System for MANETs with ProbLog . . 160 7.3.1 Example Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 7.3.2 Fadip Model in Problog . . . . . . . . . . . . . . . . . . . . . . . . . . 161 7.3.3 Options Used for Optimization . . . . . . . . . . . . . . . . . . . . . . . 162 7.3.4 Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 7.3.5 Analysis Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 8 Conclusion 167 8.1 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 Bibliography 171 List of Publications 181nrpages: 208status: publishe

Tabling relevant parts of SLD proofs for ground goals in a probabilistic setting

ProbLog is a probabilistic framework that extends Prolog with probabilistic facts. Inference in ProbLog is based on calculations over the SLD proof tree of a query. Tabling is a well known technique to avoid re-computation. We investigate how the idea of tabling can be used in ProbLog: the challenge is that we are not just interested in the answers of a query but in retaining the SLD proof tree. Luckily, only the relevant probabilistic parts of the SLD proof tree are required. Tabling not only improves the execution time of ProbLog programs, but also decreases accordingly the memory consumption. In addition to tabling, we also study how tabling affects the next computational steps. We explore this technique for ground queries with some standard ProbLog programs using exact probability inference and report our promising results. This successful first step outlines a direction to tabling ProbLog programs.status: publishe

Dedicated tabling for a probabilistic setting

Abstract. ProbLog is a probabilistic framework that extends Prolog with probabilistic facts. To compute the probability of a query, the complete SLD proof tree of the query is collected as a sum of products. ProbLog applies advanced techniques to make this feasible and to assess the correct probability. Tabling is a well-known technique to avoid repeated subcomputations and to terminate loops. We investigate how tabling can be used in ProbLog. The challenge is that we have to reconcile tabling with the advanced ProbLog techniques. While standard tabling collects only the answers for the calls, we do need the SLD proof tree. Finally we discuss how to deal with loops in our probabilistic framework. By avoiding repeated subcomputations, our tabling approach not only improves the execution time of ProbLog programs, but also decreases accordingly the memory consumption. We obtain promising results for ProbLog programs using exact probability inference. 1

A simplified fast interface for the use of CUDD for Binary Decision Diagrams

We developed a tool, named simpleCUDD, to unify and simplify the use of Binary Decision Diagrams (BDDs). The Colorado University Decision Diagrams library (CUDD) is an efficient library for BDDs. However it is difficult to use CUDD from external applications. To overcome this difficulty we developed a unified and simple interface (simpleCUDD) for applications using BDDs. SimpleCUDD makes the use of CUDD much easier while it retains all of the efficiency and the power of CUDD. It can be seen/used as an equivalent of the C++ interface of CUDD, but simpleCUDD was developed completely in C to keep it fast and as portable as possible. Both the generation of the BDD and the traversal of BDDs have been simplified. The user does not need any longer to write code to specify the underlying function of the BDD, instead the user describes the function more naturally as a script. Traversing the resulting BDD has been simplified with the use of node history and dedicated functions. Furthermore, simpleCUDD adds extra functionality like BDD description files, named variables, traversed node history, node value assignment and finally an efficient saving method for the generated BDDs. The new saving algorithm has been compared with the existing methods (DDDMP) for result validity and performance. SimpleCUDD performs equally well in most cases and for some functions, especially more complex ones, better than DDDMP. SimpleCUDD has been used successfully in applications like: ProbLog (Luc De Raedt et al.) and LeProbLog (Bernd Gutmann et al.). Future work consists in improving further the BDD saving algorithm, and develop a Prolog library for using simpleCUDD directly from Prolog.status: publishe