14 research outputs found
Reversible MCMC on Markov equivalence classes of sparse directed acyclic graphs
Graphical models are popular statistical tools which are used to represent
dependent or causal complex systems. Statistically equivalent causal or
directed graphical models are said to belong to a Markov equivalent class. It
is of great interest to describe and understand the space of such classes.
However, with currently known algorithms, sampling over such classes is only
feasible for graphs with fewer than approximately 20 vertices. In this paper,
we design reversible irreducible Markov chains on the space of Markov
equivalent classes by proposing a perfect set of operators that determine the
transitions of the Markov chain. The stationary distribution of a proposed
Markov chain has a closed form and can be computed easily. Specifically, we
construct a concrete perfect set of operators on sparse Markov equivalence
classes by introducing appropriate conditions on each possible operator.
Algorithms and their accelerated versions are provided to efficiently generate
Markov chains and to explore properties of Markov equivalence classes of sparse
directed acyclic graphs (DAGs) with thousands of vertices. We find
experimentally that in most Markov equivalence classes of sparse DAGs, (1) most
edges are directed, (2) most undirected subgraphs are small and (3) the number
of these undirected subgraphs grows approximately linearly with the number of
vertices. The article contains supplement arXiv:1303.0632,
http://dx.doi.org/10.1214/13-AOS1125SUPPComment: Published in at http://dx.doi.org/10.1214/13-AOS1125 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Counting and Sampling from Markov Equivalent DAGs Using Clique Trees
A directed acyclic graph (DAG) is the most common graphical model for
representing causal relationships among a set of variables. When restricted to
using only observational data, the structure of the ground truth DAG is
identifiable only up to Markov equivalence, based on conditional independence
relations among the variables. Therefore, the number of DAGs equivalent to the
ground truth DAG is an indicator of the causal complexity of the underlying
structure--roughly speaking, it shows how many interventions or how much
additional information is further needed to recover the underlying DAG. In this
paper, we propose a new technique for counting the number of DAGs in a Markov
equivalence class. Our approach is based on the clique tree representation of
chordal graphs. We show that in the case of bounded degree graphs, the proposed
algorithm is polynomial time. We further demonstrate that this technique can be
utilized for uniform sampling from a Markov equivalence class, which provides a
stochastic way to enumerate DAGs in the equivalence class and may be needed for
finding the best DAG or for causal inference given the equivalence class as
input. We also extend our counting and sampling method to the case where prior
knowledge about the underlying DAG is available, and present applications of
this extension in causal experiment design and estimating the causal effect of
joint interventions
Learning Markov Equivalence Classes of Directed Acyclic Graphs: an Objective Bayes Approach
A Markov equivalence class contains all the Directed Acyclic Graphs (DAGs) encoding the same conditional independencies, and is represented by a Completed Partially Directed Acyclic Graph (CPDAG), also named Essential
Graph (EG).We approach the problem of model selection among noncausal sparse Gaussian DAGs by directly scoring EGs, using an objective Bayes method. Specifically, we construct objective priors for model selection based on the Fractional Bayes Factor, leading to a closed form expression for the marginal likelihood of an EG. Next we propose an MCMC strategy to explore the space of EGs using sparsity constraints, and illustrate the performance of our method on simulation studies, as well as on a real dataset. Our method provides a coherent quantication of inferential uncertainty, requires minimal prior specication, and shows to be competitive in learning the structure of the data-generating EG when compared to alternative state-of-the-art algorithms
Equivalence class selection of categorical graphical models
Learning the structure of dependence relations between variables is a
pervasive issue in the statistical literature. A directed acyclic graph (DAG)
can represent a set of conditional independences, but different DAGs may encode
the same set of relations and are indistinguishable using observational data.
Equivalent DAGs can be collected into classes, each represented by a partially
directed graph known as essential graph (EG). Structure learning directly
conducted on the EG space, rather than on the allied space of DAGs, leads to
theoretical and computational benefits. Still, the majority of efforts in the
literature has been dedicated to Gaussian data, with less attention to methods
designed for multivariate categorical data. We then propose a Bayesian
methodology for structure learning of categorical EGs. Combining a constructive
parameter prior elicitation with a graph-driven likelihood decomposition, we
derive a closed-form expression for the marginal likelihood of a categorical EG
model. Asymptotic properties are studied, and an MCMC sampler scheme developed
for approximate posterior inference. We evaluate our methodology on both
simulated scenarios and real data, with appreciable performance in comparison
with state-of-the-art methods
Partition MCMC for inference on acyclic digraphs
Acyclic digraphs are the underlying representation of Bayesian networks, a
widely used class of probabilistic graphical models. Learning the underlying
graph from data is a way of gaining insights about the structural properties of
a domain. Structure learning forms one of the inference challenges of
statistical graphical models.
MCMC methods, notably structure MCMC, to sample graphs from the posterior
distribution given the data are probably the only viable option for Bayesian
model averaging. Score modularity and restrictions on the number of parents of
each node allow the graphs to be grouped into larger collections, which can be
scored as a whole to improve the chain's convergence. Current examples of
algorithms taking advantage of grouping are the biased order MCMC, which acts
on the alternative space of permuted triangular matrices, and non ergodic edge
reversal moves.
Here we propose a novel algorithm, which employs the underlying combinatorial
structure of DAGs to define a new grouping. As a result convergence is improved
compared to structure MCMC, while still retaining the property of producing an
unbiased sample. Finally the method can be combined with edge reversal moves to
improve the sampler further.Comment: Revised version. 34 pages, 16 figures. R code available at
https://github.com/annlia/partitionMCM