Probabilistic abductive logic programming using Dirichlet priors

Probabilistic programming is an area of research that aims to develop general inference algorithms for probabilistic models expressed as probabilistic programs whose execution corresponds to inferring the parameters of those models. In this paper, we introduce a probabilistic programming language (PPL) based on abductive logic programming for performing inference in probabilistic models involving categorical distributions with Dirichlet priors. We encode these models as abductive logic programs enriched with probabilistic definitions and queries, and show how to execute and compile them to boolean formulas. Using the latter, we perform generalized inference using one of two proposed Markov Chain Monte Carlo (MCMC) sampling algorithms: an adaptation of uncollapsed Gibbs sampling from related work and a novel collapsed Gibbs sampling (CGS). We show that CGS converges faster than the uncollapsed version on a latent Dirichlet allocation (LDA) task using synthetic data. On similar data, we compare our PPL with LDA-specific algorithms and other PPLs. We find that all methods, except one, perform similarly and that the more expressive the PPL, the slower it is. We illustrate applications of our PPL on real data in two variants of LDA models (Seed and Cluster LDA), and in the repeated insertion model (RIM). In the latter, our PPL yields similar conclusions to inference with EM for Mallows models

Probabilistic abductive logic programming using Dirichlet priors

Probabilistic programming is an area of research that aims to develop general inference algorithms for probabilistic models expressed as probabilistic programs whose execution corresponds to inferring the parameters of those models. In this paper, we introduce a probabilistic programming language (PPL) based on abductive logic programming for performing inference in probabilistic models involving categorical distributions with Dirichlet priors. We encode these models as abductive logic programs enriched with probabilistic definitions and queries, and show how to execute and compile them to boolean formulas. Using the latter, we perform generalized inference using one of two proposed Markov Chain Monte Carlo (MCMC) sampling algorithms: an adaptation of uncollapsed Gibbs sampling from related work and a novel collapsed Gibbs sampling (CGS). We show that CGS converges faster than the uncollapsed version on a latent Dirichlet allocation (LDA) task using synthetic data. On similar data, we compare our PPL with LDA-specific algorithms and other PPLs. We find that all methods, except one, perform similarly and that the more expressive the PPL, the slower it is. We illustrate applications of our PPL on real data in two variants of LDA models (Seed and Cluster LDA), and in the repeated insertion model (RIM). In the latter, our PPL yields similar conclusions to inference with EM for Mallows models

On the Implementation of the Probabilistic Logic Programming Language ProbLog

The past few years have seen a surge of interest in the field of probabilistic logic learning and statistical relational learning. In this endeavor, many probabilistic logics have been developed. ProbLog is a recent probabilistic extension of Prolog motivated by the mining of large biological networks. In ProbLog, facts can be labeled with probabilities. These facts are treated as mutually independent random variables that indicate whether these facts belong to a randomly sampled program. Different kinds of queries can be posed to ProbLog programs. We introduce algorithms that allow the efficient execution of these queries, discuss their implementation on top of the YAP-Prolog system, and evaluate their performance in the context of large networks of biological entities.Comment: 28 pages; To appear in Theory and Practice of Logic Programming (TPLP

Causality without Estimands: from Causal Estimation to Black-Box Introspection

[eng] The notion of cause and effect is fundamental to our understanding of the real world; ice cream sales correlate with jellyfish stings (both increase during summer), but a ban on ice cream could hardly stop jellyfishes. This discrepancy between the patterns that we observe and the results of our actions is essential: without causal knowledge we are mere spectators of the world, unable to understand its inner workings, enact effective change, explain which factors were responsible for a specific outcome or imagine potential scenarios resulting from alternative decisions. The field of statistics has traditionally stayed in the realm of observations, powerless in the measurement of causal effects unless by performing randomized experiments. These consist of dividing a set of individuals in two groups at random and assigning a certain action/treatment to each subgroup, to then compare the outcomes of both. This could be applied, for instance, to measure the impact of large-scale advertisement campaigns on sales, test the effects of smoking on the development of lung cancer, or determine the influence of new pedagogical strategies on eventual career success. However, randomized experiments are not always feasible, as is the case in these examples, due to economic, ethical or timing concerns. Causal Inference is the field that studies how to circumvent this problem: only using observational data, not subject to randomization, it allows us to measure causal effects. Even so, the standard approach for Causal Estimation (CE), estimand-based methods, results in ad hoc models that cannot extrapolate to other datasets with different causal relationships, and often require training a new model every time we want to answer a different query on the same dataset. Contrary to this perspective, estimand-agnostic approaches train a model of the observational distribution that acts as a proxy of the underlying mechanism that generated the data; this model needs to be trained only once and can answer any identifiable queries reliably. However, this latter approach has seldom been studied, primarily because of the difficulty of defining a good model of the target distribution satisfying every causal requirement while still flexible enough to answer the desired causal queries. This dissertation is focused on the definition of a general estimand-agnostic CE framework, Deep Causal Graphs, that can leverage the expressive modelling capabilities of Neural Networks and Normalizing Flows while still providing a flexible and comprehensive estimation toolkit for all kinds of causal queries. We will contrast its capabilities against other estimand-agnostic approaches and measure its performance in comparison with the state of the art in Causal Query Estimation. Finally, we will also illustrate the connection between CE and Machine Learning Interpretability, Explainability and Fairness: since the examination of black-boxes often requires to answer many causal queries (e.g., what is the effect of each input variable on the outcome, or how would the outcome have changed had we intervened on a certain input), estimand-based techniques would force us to train as many different models; in contrast, estimand-agnostic frameworks allow us to ask as many questions as needed with just a single trained model, and therefore are essential for this kind of application

On being a good Bayesian

Bayesianism is fast becoming the dominant paradigm in archaeological chronology construction. This paradigm shift has been brought about in large part by widespread access to tailored computer software which provides users with powerful tools for complex statistical inference with little need to learn about statistical modelling or computer programming. As a result, we run the risk that such software will be reduced to the status of black boxes. This would be a dangerous position for our community since good, principled use of Bayesian methods requires mindfulness when selecting the initial model, defining prior information, checking the reliability and sensitivity of the software runs and interpreting the results obtained. In this article, we provide users with a brief review of the nature of the care required and offer some comments and suggestions to help ensure that our community continues to be respected for its philosophically rigorous scientific approach

Probabilistic Programming Concepts

A multitude of different probabilistic programming languages exists today, all extending a traditional programming language with primitives to support modeling of complex, structured probability distributions. Each of these languages employs its own probabilistic primitives, and comes with a particular syntax, semantics and inference procedure. This makes it hard to understand the underlying programming concepts and appreciate the differences between the different languages. To obtain a better understanding of probabilistic programming, we identify a number of core programming concepts underlying the primitives used by various probabilistic languages, discuss the execution mechanisms that they require and use these to position state-of-the-art probabilistic languages and their implementation. While doing so, we focus on probabilistic extensions of logic programming languages such as Prolog, which have been developed since more than 20 years

Distributional logic programming for Bayesian knowledge representation

We present a formalism for combining logic programming and its flavour of nondeterminism with probabilistic reasoning. In particular, we focus on representing prior knowledge for Bayesian inference. Distributional logic programming (Dlp), is considered in the context of a class of generative probabilistic languages. A characterisation based on probabilistic paths which can play a central role in clausal probabilistic reasoning is presented. We illustrate how the characterisation can be utilised to clarify derived distributions with regards to mixing the logical and probabilistic constituents of generative languages. We use this operational characterisation to define a class of programs that exhibit probabilistic determinism. We show how Dlp can be used to define generative priors over statistical model spaces. For example, a single program can generate all possible Bayesian networks having N nodes while at the same time it defines a prior that penalises networks with large families. Two classes of statistical models are considered: Bayesian networks and classification and regression trees. Finally we discuss: (1) a Metropolis–Hastings algorithm that can take advantage of the defined priors and the probabilistic choice points in the prior programs and (2) its application to real-world machine learning tasks