Inference and Learning for Directed Probabilistic Logic Models (Inferentie en leren voor gerichte probabilistische logische modellen)

We are confronted with a growing amount of available data which are not only noisy but also have an increasingly complex structure. The field of machine learning, a subfield of artificial intelligence, focuses on algorithms that deduce useful knowledge from data. Our goal is to represent knowledge using probabilistic logic models and to reason with these models in an automated and efficient manner. Such models bring the expressive power of first-order logic to probabilistic models, enabling them to capture both the relational structure and the uncertainty present in such data.In this dissertation we focus on directed probabilistic logic models and more specifically on CP-logic. The aim of CP-logic is to model causal knowledge that explicitly incorporates dynamic concepts such as events and processes. The fundamental building block is the knowledge why events occur and what the effects of these events are. Efficient inference, however, is a bottleneck in CP-logic and in probabilistic logic models in general, affecting also the cost of learning. We have contributed two methods to improve the efficiency of inference and one method for learning.The first method, first-order Bayes ball, extracts the minimal requisite subtheory of a CP-theory necessary to answer a particular query given evidence. Inference becomes more efficient by restricting computations to the minimal requisite subtheory. Contrary to Bayes ball for Bayesian networks, first-order Bayes ball reasons on the first-order level and it returns the requisite part as a first-order CP-theory. The advantages of working on the first-order level are twofold; first, it is more efficient to find the ground requisite network compared to current methods. Second, the resulting requisite network is first-order, permitting it to be used as input for lifted inference methods which exploit the symmetries present in probabilistic logic models to improve the efficiency of inference with several orders of magnitude. Experiments show that first-order Bayes ball improves existing lifted inference methods by reducing the size of the theory that needs to be analyzed and processed.The second method to improve the efficiency of inference is contextual variable elimination with overlapping contexts which capitalizes on deterministic dependencies present in probabilistic logic models. Two special cases of combining deterministic and probabilistic relations are contextual and causal independencies, both commonly used structures in probabilistic models. The original contextual variable elimination technique compactly represents contextual independence by representing the probabilistic model in terms of confactors but cannot handle causal independence because of some restrictions in these confactors. We lift these restrictions and propose a new algorithm to deal with more general confactors. This allows for a more efficient encoding of confactors and a reduction of the computational cost. Experiments show that our algorithm outperforms contextual variable elimination and variable elimination on multiple problems.Lastly, we propose SEM-CP-logic, an algorithm for learning ground CP-logic from data by leveraging Bayesian network learning techniques. To this end, certain modifications are required to parameter and structure learning for Bayesian networks. Most importantly, the refinement operator used by the search must take into account the fine granularity of CP-logic. Experiments in a controlled artificial domain show that learning CP-theories with SEM-CP-logic requires fewer training data than Bayesian network learning.Contents xi List of Figures xv List of Algorithms xix Introduction 1 Context 1 Probability theory 2 Logic 3 Machine Learning 3 Probabilistic Logic Learning 4 Motivation, Goal and Contributions 5 Motivation and Goal 5 Application 5 Contributions 6 Structure of the Text 8 Implementation 9 Background 11 Probability Theory 11 Sample Space and Random Variables 12 Joint Probability Distribution 16 Marginal Probability Distribution 16 Conditional Probability 17 Bayesian network 20 Factors and variable elimination 23 Barren nodes and D-separation 25 Logic 26 Propositional logic 26 First order logic 29 Logic programming 30 Summary 32 CP-Logic 35 Introduction 35 Bibliographical note 35 Structure of this chapter 36 A causal probabilistic logic 36 Syntax 38 Process semantics 42 LPAD semantics 43 CP-logic subclasses 46 Transforming CP-theories to 1-compliant CP-theories 48 Relating CP-logic to Bayesian networks 51 Relating CP-logic to logic programming 57 Time in CP-logic 58 Probabilistic loops 59 Implementations 63 Conclusions 63 Related formalisms 65 Introduction 65 Bibliographical note 66 Structure of this chapter 66 Bayesian networks 66 CP-Theories Representing Noisy-OR 67 CP-Theories with Multiple Literals in the Rule Bodies 71 CP-Theories with Multiple Atoms in the Rule Heads 71 Multiple literals in head and shared across events 73 Summary 74 Knowledge Based Model Construction 76 ICL 78 PRISM 79 CHRiSM 79 ProbLog 82 Bayesian Logic Programs 84 Relational Bayesian networks 86 Other 87 Conclusions 87 First-order Bayes-ball 91 Introduction 91 Bibliographical note 93 Structure of this chapter 94 Preliminaries 94 Bayes-Ball 94 CP-logic with types and constraints 95 Parameterized Bayesian networks 96 Equivalent Bayesian Network 97 First-Order Bayes-Ball 98 Overview 100 The Algorithm 102 Extensions 106 Extension for Implicit Domains 107 The closed world assumption as evidence 107 Propagating evidence 108 Shattering 108 Experiments 111 Conclusions 113 Contextual Variable Elimination with Overlapping Contexts 117 Introduction 117 Bibliographical Note 119 Structure of this Chapter 119 Existing inference techniques for CP-logic 120 BDD-based approaches 120 Variable elimination 121 Contextual variable elimination (CVE) 121 Results 125 Multiplicative factorization of noisy-max 129 CVE with overlapping contexts 130 The CVE-OC algorithm 131 Experiments 134 Conclusions 136 Learning 139 Introduction 139 Bibliographical note 140 Structure of this chapter 141 Parameter learning 141 Complexity 142 Structure learning 145 Structural EM and the BIC Score 145 Structure Search for CP-Theories 147 Refinement Operator 148 Complexity 153 Experiments 154 Experiments in an Artificial Domain 154 Discovering Causal Relations Between HIV Mutations 159 Conclusions 161 Conclusion 163 Summary of contributions and conclusions 163 Future work 166 Probabilistic loops 166 Lifted inference 167 Learning 167 Continuous variables 168 Inhibitory events 168 Bibliography 171nrpages: 188 + xixstatus: publishe