12 research outputs found
Advances in Learning Bayesian Networks of Bounded Treewidth
This work presents novel algorithms for learning Bayesian network structures
with bounded treewidth. Both exact and approximate methods are developed. The
exact method combines mixed-integer linear programming formulations for
structure learning and treewidth computation. The approximate method consists
in uniformly sampling -trees (maximal graphs of treewidth ), and
subsequently selecting, exactly or approximately, the best structure whose
moral graph is a subgraph of that -tree. Some properties of these methods
are discussed and proven. The approaches are empirically compared to each other
and to a state-of-the-art method for learning bounded treewidth structures on a
collection of public data sets with up to 100 variables. The experiments show
that our exact algorithm outperforms the state of the art, and that the
approximate approach is fairly accurate.Comment: 23 pages, 2 figures, 3 table
Consistent Second-Order Conic Integer Programming for Learning Bayesian Networks
Bayesian Networks (BNs) represent conditional probability relations among a
set of random variables (nodes) in the form of a directed acyclic graph (DAG),
and have found diverse applications in knowledge discovery. We study the
problem of learning the sparse DAG structure of a BN from continuous
observational data. The central problem can be modeled as a mixed-integer
program with an objective function composed of a convex quadratic loss function
and a regularization penalty subject to linear constraints. The optimal
solution to this mathematical program is known to have desirable statistical
properties under certain conditions. However, the state-of-the-art optimization
solvers are not able to obtain provably optimal solutions to the existing
mathematical formulations for medium-size problems within reasonable
computational times. To address this difficulty, we tackle the problem from
both computational and statistical perspectives. On the one hand, we propose a
concrete early stopping criterion to terminate the branch-and-bound process in
order to obtain a near-optimal solution to the mixed-integer program, and
establish the consistency of this approximate solution. On the other hand, we
improve the existing formulations by replacing the linear "big-" constraints
that represent the relationship between the continuous and binary indicator
variables with second-order conic constraints. Our numerical results
demonstrate the effectiveness of the proposed approaches
Learning Bayesian Networks with the Saiyan algorithm
Some structure learning algorithms have proven to be effective in reconstructing hypothetical Bayesian Network (BN) graphs from synthetic data. However, in their mission to maximise a scoring function, many become conservative and minimise edges discovered. While simplicity is desired, the output is often a graph that consists of multiple independent graphical fragments or variables that do not enable full propagation of evidence. While this is not a problem in theory, it can be a problem in practice. This paper presents a novel unconventional heuristic local-search structure learning algorithm, called Saiyan, which returns a directed acyclic graph that enables full propagation of evidence. Forcing the algorithm to connect all data variables and to direct all of the edges discovered implies that the additional forced arcs are not expected to be correct at the rate of those identified unrestrictedly, and this evidently has a negative impact on the evaluation score of the discovered graph. Still, based on both synthetic and real-world experiments, the Saiyan algorithm demonstrates competitive performance relative to other state-of-the-art constraint-based, score-based, and hybrid structure learning algorithms
Generalized Permutohedra from Probabilistic Graphical Models
A graphical model encodes conditional independence relations via the Markov
properties. For an undirected graph these conditional independence relations
can be represented by a simple polytope known as the graph associahedron, which
can be constructed as a Minkowski sum of standard simplices. There is an
analogous polytope for conditional independence relations coming from a regular
Gaussian model, and it can be defined using multiinformation or relative
entropy. For directed acyclic graphical models and also for mixed graphical
models containing undirected, directed and bidirected edges, we give a
construction of this polytope, up to equivalence of normal fans, as a Minkowski
sum of matroid polytopes. Finally, we apply this geometric insight to construct
a new ordering-based search algorithm for causal inference via directed acyclic
graphical models.Comment: Appendix B is expanded. Final version to appear in SIAM J. Discrete
Mat
Towards using the chordal graph polytope in learning decomposable models
The motivation for this paper is the integer linear programming approach to learning the structure of a decomposable graphical model. We have chosen to represent decomposable models by means of special zero–one vectors, named characteristic imsets. Our approach leads to the study of a special polytope, defined as the convex hull of all characteristic imsets for chordal graphs, named the chordal graph polytope. In this theoretical paper, we introduce a class of clutter inequalities (valid for the vectors in the polytope) and show that all of them are facet-defining for the polytope. We dare to conjecture that they lead to a complete polyhedral description of the polytope. Finally, we propose a linear programming method to solve the separation problem with these inequalities for the use in a cutting plane approach