Learning Score-Optimal Chordal Markov Networks via Branch and Bound

Graphical models are commonly used to encode conditional independence assumptions between random variables. Here we focus on undirected graphical models called chordal Markov networks. Specifically, we will consider the chordal Markov network structure learning problem (CMSL), where the aim is to find (or "learn") a graph structure that best fits the given data with respect to a given decomposable scoring function. We introduce a branch and bound search algorithm for CMSL which represents chordal Markov network structures as decomposable DAGs. We show how revisiting equivalent solution candidates can be avoided in the search by detecting symmetries among graph structures. For the symmetry breaking we apply specific rules by van Beek and Hoffman (CP 2015), and also propose a new rule that takes advantage of the special nature of decomposable DAGs. In addition, we show how we can achieve on-the-fly score pruning for CMSL. We also propose methods for obtaining strong upper bounds for CMSL that help us close branches in the search tree. We implement a dynamic programming algorithm to find the optimal Bayesian network structures and then use the scores of those graphs as upper bounds. We also show how we can relax the requirement for decomposability in decomposable DAGs in order to achieve even stronger upper bounds. Furthermore, we propose a method for obtaining an initial lower bound in CMSL by turning a Bayesian network structure into a chordal Markov network structure. Empirically we show that our approach is competitive with the recently proposed CMSL algorithms by being able to sometimes scale up to 20 variables within 24 hours with unbounded treewidth. We also report that our branch and bound requires considerably less memory than the fastest of the recently proposed algorithms for CMSL

The generalised distribution semantics and projective families of distributions

We generalise the distribution semantics underpinning probabilistic logic programming by distilling its essential concept, the separation of a free random component and a deterministic part. This abstracts the core ideas beyond logic programming as such to encompass frameworks from probabilistic databases, probabilistic finite model theory and discrete lifted Bayesian networks. To demonstrate the usefulness of such a general approach, we completely characterise the projective families of distributions representable in the generalised distribution semantics and we demonstrate both that large classes of interesting projective families cannot be represented in a generalised distribution semantics and that already a very limited fragment of logic programming (acyclic determinate logic programs) in the deterministic part suffices to represent all those projective families that are representable in the generalised distribution semantics at all

Generalising weighted model counting

Given a formula in propositional or (finite-domain) first-order logic and some non-negative weights, weighted model counting (WMC) is a function problem that asks to compute the sum of the weights of the models of the formula. Originally used as a flexible way of performing probabilistic inference on graphical models, WMC has found many applications across artificial intelligence (AI), machine learning, and other domains. Areas of AI that rely on WMC include explainable AI, neural-symbolic AI, probabilistic programming, and statistical relational AI. WMC also has applications in bioinformatics, data mining, natural language processing, prognostics, and robotics. In this work, we are interested in revisiting the foundations of WMC and considering generalisations of some of the key definitions in the interest of conceptual clarity and practical efficiency. We begin by developing a measure-theoretic perspective on WMC, which suggests a new and more general way of defining the weights of an instance. This new representation can be as succinct as standard WMC but can also expand as needed to represent less-structured probability distributions. We demonstrate the performance benefits of the new format by developing a novel WMC encoding for Bayesian networks. We then show how existing WMC encodings for Bayesian networks can be transformed into this more general format and what conditions ensure that the transformation is correct (i.e., preserves the answer). Combining the strengths of the more flexible representation with the tricks used in existing encodings yields further efficiency improvements in Bayesian network probabilistic inference. Next, we turn our attention to the first-order setting. Here, we argue that the capabilities of practical model counting algorithms are severely limited by their inability to perform arbitrary recursive computations. To enable arbitrary recursion, we relax the restrictions that typically accompany domain recursion and generalise circuits (used to express a solution to a model counting problem) to graphs that are allowed to have cycles. These improvements enable us to find efficient solutions to counting fundamental structures such as injections and bijections that were previously unsolvable by any available algorithm. The second strand of this work is concerned with synthetic data generation. Testing algorithms across a wide range of problem instances is crucial to ensure the validity of any claim about one algorithm’s superiority over another. However, benchmarks are often limited and fail to reveal differences among the algorithms. First, we show how random instances of probabilistic logic programs (that typically use WMC algorithms for inference) can be generated using constraint programming. We also introduce a new constraint to control the independence structure of the underlying probability distribution and provide a combinatorial argument for the correctness of the constraint model. This model allows us to, for the first time, experimentally investigate inference algorithms on more than just a handful of instances. Second, we introduce a random model for WMC instances with a parameter that influences primal treewidth—the parameter most commonly used to characterise the difficulty of an instance. We show that the easy-hard-easy pattern with respect to clause density is different for algorithms based on dynamic programming and algebraic decision diagrams than for all other solvers. We also demonstrate that all WMC algorithms scale exponentially with respect to primal treewidth, although at differing rates

Logical limits of abstract argumentation frameworks

International audienceDung’s (1995) argumentation framework takes as input two abstract entities: a set of arguments and a binary relation encoding attacks between these arguments. It returns acceptable sets of arguments, called extensions, w.r.t. a given semantics. While the abstract nature of this setting is seen as a great advantage, it induces a big gap with the application that it is used to. This raises some questions about the compatibility of the setting with a logical formalism (i.e., whether it is possible to instantiate it properly from a logical knowledge base), and about the significance of the various semantics in the application context. In this paper we tackle the above questions. We first propose to fill in the previous gap by extending Dung’s (1995) framework. The idea is to consider all the ingredients involved in an argumentation process. We start with the notion of an abstract monotonic logic which consists of a language (defining the formulas) and a consequence operator. We show how to build, in a systematic way, arguments from a knowledge base formalised in such a logic. We then recall some basic postulates that any instantiation should satisfy. We study how to choose an attack relation so that the instantiation satisfies the postulates. We show that symmetric attack relations are generally not suitable. However, we identify at least one ‘appropriate’ attack relation. Next, we investigate under stable, semi-stable, preferred, grounded and ideal semantics the outputs of logic-based instantiations that satisfy the postulates. For each semantics, we delimit the number of extensions an argumentation system may have, characterise the extensions in terms of subsets of the knowledge base, and finally characterise the set of conclusions that are drawn from the knowledge base. The study reveals that stable, semi-stable and preferred semantics either lead to counter-intuitive results or provide no added value w.r.t. naive semantics. Besides, naive semantics either leads to arbitrary results or generalises the coherence-based approach initially developed by Rescher and Manor (1970). Ideal and grounded semantics either coincide and generalise the free consequence relation developed by Benferhat, Dubois, and Prade (1997), or return arbitrary results. Consequently, Dung’s (1995) framework seems problematic when applied over deductive logical formalisms

Bayesian belief networks for dementia diagnosis and other applications: a comparison of hand-crafting and construction using a novel data driven technique

The Bayesian network (BN) formalism is a powerful representation for encoding domains characterised by uncertainty. However, before it can be used it must first be constructed, which is a major challenge for any real-life problem. There are two broad approaches, namely the hand-crafted approach, which relies on a human expert, and the data-driven approach, which relies on data. The former approach is useful, however issues such as human bias can introduce errors into the model. We have conducted a literature review of the expert-driven approach, and we have cherry-picked a number of common methods, and engineered a framework to assist non-BN experts with expert-driven construction of BNs. The latter construction approach uses algorithms to construct the model from a data set. However, construction from data is provably NP-hard. To solve this problem, approximate, heuristic algorithms have been proposed; in particular, algorithms that assume an order between the nodes, therefore reducing the search space. However, traditionally, this approach relies on an expert providing the order among the variables --- an expert may not always be available, or may be unable to provide the order. Nevertheless, if a good order is available, these order-based algorithms have demonstrated good performance. More recent approaches attempt to ``learn'' a good order then use the order-based algorithm to discover the structure. To eliminate the need for order information during construction, we propose a search in the entire space of Bayesian network structures --- we present a novel approach for carrying out this task, and we demonstrate its performance against existing algorithms that search in the entire space and the space of orders. Finally, we employ the hand-crafting framework to construct models for the task of diagnosis in a ``real-life'' medical domain, dementia diagnosis. We collect real dementia data from clinical practice, and we apply the data-driven algorithms developed to assess the concordance between the reference models developed by hand and the models derived from real clinical data

Quality-Aware Data Source Management

Data is becoming a commodity of tremendous value in many domains. The ease of collecting and publishing data has led to an upsurge in the number of available data sources --- sources that are highly heterogeneous in the domains they cover, the quality of data they provide, and the fees they charge for accessing their data. However, most existing data integration approaches, for combining information from a collection of sources, focus on facilitating integration itself but are agnostic to the actual utility or the quality of the integration result. These approaches do not optimize for the trade-off between the utility and the cost of integration to determine which sources are worth integrating. In this dissertation, I introduce a framework for quality-aware data source management. I define a collection of formal quality metrics for different types of data sources, including sources that provide both structured and unstructured data. I develop techniques to efficiently detect the content focus of a large number of diverse sources, to reason about their content changes over time and to formally compute the utility obtained when integrating subsets of them. I also design efficient algorithms with constant factor approximation guarantees for finding a set of sources that maximizes the utility of the integration result given a cost budget. Finally, I develop a prototype quality-aware data source management system and demonstrate the effectiveness of the developed techniques on real-world applications

Eingebettete dynamische Bayessche Netze n-ter Ordnung

Das Ziel dieser Arbeit war die Konzeption, die Realisation und die Anwendung eines Systems, das eingebettete Systeme mit geringer Rechenleistung und wenig Arbeitsspeicher mit der Fähigkeit ausstattet, probabilistische Prozesse verarbeiten zu können. Die Grundlage für dieses System bildet der differentielle Ansatz von Darwiche zur Lösung Bayesscher Netze, der ein Bayessches Netz in ein multivariates Polynom umwandelt und dann auswertet. Diesen Ansatz von Darwiche haben wir so erweitert, dass nun auch die speziellen Bedürfnisse dynamischer Bayesscher Netze berücksichtigt werden. Aufbauend auf dieser theoretischen Ausarbeitung wurde eine Anwendung mit dem Namen JavaDBN entwickelt, die dynamische Bayessche Netze in spezielle Polynome umwandelt und für diese Quellcode generiert. Dieser Quellcode führt die Berechnungen für die Auswertung des Polynoms und das Anhängen neuer Zeitscheiben mit dem gleichzeitigen Rollup bei konstantem Speicherverbrauch durch. Für die Modellierung dynamischer Bayesscher Netze spezifizieren wir neue Modellierungsstrukturen im Zusammenhang mit der Benutzermodellierung und der Sensorverarbeitung und führen damit den Begriff der dynamischen Bayesschen Netze n-ter Ordnung ein.The aim of this work was the conception, realisation and application of a system that enables embedded systems with low computing power andmemory to execute probabilistic processes. The foundation of this system is the differential approach by Darwiche used to solve Bayesian networks, which converts a Bayesian network into a multivariate polynomial and then evaluates it. We extended this approach, such that it also fulfils the specialized requirements of dynamic Bayesian networks. Based on this theoretical elaboration, an application called JavaDBN was developed to convert dynamic Bayesian networks into specific polynomials and generate their networks'; source code. This source code executes the computations for the evaluation of the polynomials and for the addition of new time slices with simultaneous roll-up with constant space requirements. To model dynamic Bayesian networks, we specified new modelling structures for the domains of user modelling and sensor processing and introduce the term of dynamic Bayesian networks of n-th order

Semantic discovery and reuse of business process patterns

Patterns currently play an important role in modern information systems (IS) development and their use has mainly been restricted to the design and implementation phases of the development lifecycle. Given the increasing significance of business modelling in IS development, patterns have the potential of providing a viable solution for promoting reusability of recurrent generalized models in the very early stages of development. As a statement of research-in-progress this paper focuses on business process patterns and proposes an initial methodological framework for the discovery and reuse of business process patterns within the IS development lifecycle. The framework borrows ideas from the domain engineering literature and proposes the use of semantics to drive both the discovery of patterns as well as their reuse