36 research outputs found
Complexity of Inference in Graphical Models
Graphical models provide a convenient representation for a broad class of probability distributions.
Due to their powerful and sophisticated modeling capabilities, such models have
found numerous applications in machine learning and other areas. In this paper we consider the
complexity of commonly encountered tasks involving graphical models such as the computation
of the mode of a posterior probability distribution (i.e., MAP estimation), and the computation
of marginal probabilities or the partition function. It is well-known that such inference problems
are hard in the worst case, but are tractable for models with bounded treewidth. We ask
whether treewidth is the only structural criterion of the underlying graph that enables tractable
inference. In other words, is there some class of structures with unbounded treewidth in which
inference is tractable? Subject to a combinatorial hypothesis due to Robertson, Seymour, and
Thomas (1994), we show that low treewidth is indeed the only structural restriction that can
ensure tractability. More precisely we show that for every growing family of graphs indexed
by tree-width, there exists a choice of potential functions such that the corresponding inference
problem is intractable. Thus even for the "best case" graph structures of high treewidth, there is
no polynomial-time inference algorithm. Our analysis employs various concepts from complexity theory and graph theory, with graph minors playing a prominent role
Chordal Decomposition in Rank Minimized Semidefinite Programs with Applications to Subspace Clustering
Semidefinite programs (SDPs) often arise in relaxations of some NP-hard
problems, and if the solution of the SDP obeys certain rank constraints, the
relaxation will be tight. Decomposition methods based on chordal sparsity have
already been applied to speed up the solution of sparse SDPs, but methods for
dealing with rank constraints are underdeveloped. This paper leverages a
minimum rank completion result to decompose the rank constraint on a single
large matrix into multiple rank constraints on a set of smaller matrices. The
re-weighted heuristic is used as a proxy for rank, and the specific form of the
heuristic preserves the sparsity pattern between iterations. Implementations of
rank-minimized SDPs through interior-point and first-order algorithms are
discussed. The problem of subspace clustering is used to demonstrate the
computational improvement of the proposed method.Comment: 6 pages, 6 figure
Minimum Weight Perfect Matching via Blossom Belief Propagation
Max-product Belief Propagation (BP) is a popular message-passing algorithm
for computing a Maximum-A-Posteriori (MAP) assignment over a distribution
represented by a Graphical Model (GM). It has been shown that BP can solve a
number of combinatorial optimization problems including minimum weight
matching, shortest path, network flow and vertex cover under the following
common assumption: the respective Linear Programming (LP) relaxation is tight,
i.e., no integrality gap is present. However, when LP shows an integrality gap,
no model has been known which can be solved systematically via sequential
applications of BP. In this paper, we develop the first such algorithm, coined
Blossom-BP, for solving the minimum weight matching problem over arbitrary
graphs. Each step of the sequential algorithm requires applying BP over a
modified graph constructed by contractions and expansions of blossoms, i.e.,
odd sets of vertices. Our scheme guarantees termination in O(n^2) of BP runs,
where n is the number of vertices in the original graph. In essence, the
Blossom-BP offers a distributed version of the celebrated Edmonds' Blossom
algorithm by jumping at once over many sub-steps with a single BP. Moreover,
our result provides an interpretation of the Edmonds' algorithm as a sequence
of LPs