Standard imsets for undirected and chain graphical models

We derive standard imsets for undirected graphical models and chain graphical models. Standard imsets for undirected graphical models are described in terms of minimal triangulations for maximal prime subgraphs of the undirected graphs. For describing standard imsets for chain graphical models, we first define a triangulation of a chain graph. We then use the triangulation to generalize our results for the undirected graphs to chain graphs.Comment: Published at http://dx.doi.org/10.3150/14-BEJ611 in the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm

Generalized Permutohedra from Probabilistic Graphical Models

A graphical model encodes conditional independence relations via the Markov properties. For an undirected graph these conditional independence relations can be represented by a simple polytope known as the graph associahedron, which can be constructed as a Minkowski sum of standard simplices. There is an analogous polytope for conditional independence relations coming from a regular Gaussian model, and it can be defined using multiinformation or relative entropy. For directed acyclic graphical models and also for mixed graphical models containing undirected, directed and bidirected edges, we give a construction of this polytope, up to equivalence of normal fans, as a Minkowski sum of matroid polytopes. Finally, we apply this geometric insight to construct a new ordering-based search algorithm for causal inference via directed acyclic graphical models.Comment: Appendix B is expanded. Final version to appear in SIAM J. Discrete Mat

Causal discovery with ancestral graphs

Graphical models serve as a visual representation that captures the underlying conditional independence relationships within distributions, employing either directed or undirected graphs. In this thesis, we explore maximal ancestral graphs (MAGs), which is an extension to the conventionaldirected acyclic graphs (DAGs). While DAGs excel in illustrating causal relationships, they fail to capture all the conditional independences on the margin in the absence of latent confounders and selection bias. MAGs provide a more comprehensive depiction of complex dependencies by encompassing both direct causal connections and indirect influences stemming from latent variables and selection bias. The scalability and accuracy of MAG learning algorithms have been some problems due to the complexity of the space of Markov equivalence classes (MECs) of MAGs and instability of scoring criteria. We first use the concept of heads, tails and parametrizing sets to characterize Markov equivalent MAGs. Then we study imsets of MAGs to address the above issues. The framework of imsets (Studeny, 2006) is an algebraic approach to represent conditional independences. Given the remarkable success of standard imsets within DAGs, where they efficiently represent MECs and offer reliable scoring criteria, we endeavor to extend this framework to MAGs. Through an exploration of 0-1 imsets defined by parametrizing sets, we show under which conditions does this extended `standard imset' of MAGs define the correct model. Consequently, we refine the ordered local Markov property of MAGs (Richardson, 2003), demonstrating that the newly proposed refined Markov property can be constructed in polynomial time if we bound maximal head size. Finally, we apply the above results to develop novel score-based learning algorithms for MAGs. To efficiently traverse between MECs of MAGs, we identify some important graphical features within MAGs whose independence models are subsets of others. Leveraging the imsets derived from the refined Markov property, we establish a consistent scoring criterion, offering an alternative to BIC by relying solely on estimates of entropy over subsets of variables. Empirical experiments show promising results when compared to state-of-the-art algorithms

On Boundaries of Statistical Models

In the thesis "On Boundaries of Statistical Models" problems related to a description of probability distributions with zeros, lying in the boundary of a statistical model, are treated. The distributions considered are joint distributions of finite collections of finite discrete random variables. Owing to this restriction, statistical models are subsets of finite dimensional real vector spaces. The support set problem for exponential families, the main class of models considered in the thesis, is to characterize the possible supports of distributions in the boundaries of these statistical models. It is shown that this problem is equivalent to a characterization of the face lattice of a convex polytope, called the convex support. The main tool for treating questions related to the boundary are implicit representations. Exponential families are shown to be sets of solutions of binomial equations, connected to an underlying combinatorial structure, called oriented matroid. Under an additional assumption these equations are polynomial and one is placed in the setting of commutative algebra and algebraic geometry. In this case one recovers results from algebraic statistics. The combinatorial theory of exponential families using oriented matroids makes the established connection between an exponential family and its convex support completely natural: Both are derived from the same oriented matroid. The second part of the thesis deals with hierarchical models, which are a special class of exponential families constructed from simplicial complexes. The main technical tool for their treatment in this thesis are so called elementary circuits. After their introduction, they are used to derive properties of the implicit representations of hierarchical models. Each elementary circuit gives an equation holding on the hierarchical model, and these equations are shown to be the "simplest", in the sense that the smallest degree among the equations corresponding to elementary circuits gives a lower bound on the degree of all equations characterizing the model. Translating this result back to polyhedral geometry yields a neighborliness property of marginal polytopes, the convex supports of hierarchical models. Elementary circuits of small support are related to independence statements holding between the random variables whose joint distributions the hierarchical model describes. Models for which the complete set of circuits consists of elementary circuits are shown to be described by totally unimodular matrices. The thesis also contains an analysis of the case of binary random variables. In this special situation, marginal polytopes can be represented as the convex hulls of linear codes. Among the results here is a classification of full-dimensional linear code polytopes in terms of their subgroups. If represented by polynomial equations, exponential families are the varieties of binomial prime ideals. The third part of the thesis describes tools to treat models defined by not necessarily prime binomial ideals. It follows from Eisenbud and Sturmfels'' results on binomial ideals that these models are unions of exponential families, and apart from solving the support set problem for each of these, one is faced with finding the decomposition. The thesis discusses algorithms for specialized treatment of binomial ideals, exploiting their combinatorial nature. The provided software package Binomials.m2 is shown to be able to compute very large primary decompositions, yielding a counterexample to a recent conjecture in algebraic statistics