Scalable Semidefinite Relaxation for Maximum A Posterior Estimation

Maximum a posteriori (MAP) inference over discrete Markov random fields is a fundamental task spanning a wide spectrum of real-world applications, which is known to be NP-hard for general graphs. In this paper, we propose a novel semidefinite relaxation formulation (referred to as SDR) to estimate the MAP assignment. Algorithmically, we develop an accelerated variant of the alternating direction method of multipliers (referred to as SDPAD-LR) that can effectively exploit the special structure of the new relaxation. Encouragingly, the proposed procedure allows solving SDR for large-scale problems, e.g., problems on a grid graph comprising hundreds of thousands of variables with multiple states per node. Compared with prior SDP solvers, SDPAD-LR is capable of attaining comparable accuracy while exhibiting remarkably improved scalability, in contrast to the commonly held belief that semidefinite relaxation can only been applied on small-scale MRF problems. We have evaluated the performance of SDR on various benchmark datasets including OPENGM2 and PIC in terms of both the quality of the solutions and computation time. Experimental results demonstrate that for a broad class of problems, SDPAD-LR outperforms state-of-the-art algorithms in producing better MAP assignment in an efficient manner.Comment: accepted to International Conference on Machine Learning (ICML 2014

The Power of Linear Programming for Valued CSPs

A class of valued constraint satisfaction problems (VCSPs) is characterised by a valued constraint language, a fixed set of cost functions on a finite domain. An instance of the problem is specified by a sum of cost functions from the language with the goal to minimise the sum. This framework includes and generalises well-studied constraint satisfaction problems (CSPs) and maximum constraint satisfaction problems (Max-CSPs). Our main result is a precise algebraic characterisation of valued constraint languages whose instances can be solved exactly by the basic linear programming relaxation. Using this result, we obtain tractability of several novel and previously widely-open classes of VCSPs, including problems over valued constraint languages that are: (1) submodular on arbitrary lattices; (2) bisubmodular (also known as k-submodular) on arbitrary finite domains; (3) weakly (and hence strongly) tree-submodular on arbitrary trees.Comment: Corrected a few typo

Maximum Persistency via Iterative Relaxed Inference with Graphical Models

We consider the NP-hard problem of MAP-inference for undirected discrete graphical models. We propose a polynomial time and practically efficient algorithm for finding a part of its optimal solution. Specifically, our algorithm marks some labels of the considered graphical model either as (i) optimal, meaning that they belong to all optimal solutions of the inference problem; (ii) non-optimal if they provably do not belong to any solution. With access to an exact solver of a linear programming relaxation to the MAP-inference problem, our algorithm marks the maximal possible (in a specified sense) number of labels. We also present a version of the algorithm, which has access to a suboptimal dual solver only and still can ensure the (non-)optimality for the marked labels, although the overall number of the marked labels may decrease. We propose an efficient implementation, which runs in time comparable to a single run of a suboptimal dual solver. Our method is well-scalable and shows state-of-the-art results on computational benchmarks from machine learning and computer vision.Comment: Reworked version, submitted to PAM

Speeding up weighted constraint satisfaction using redundant modeling.

Woo Hiu Chun.Thesis (M.Phil.)--Chinese University of Hong Kong, 2006.Includes bibliographical references (leaves 91-99).Abstracts in English and Chinese.Chapter 1 --- Introduction --- p.1Chapter 1.1 --- Constraint Satisfaction Problems --- p.1Chapter 1.2 --- Weighted Constraint Satisfaction Problems --- p.3Chapter 1.3 --- Redundant Modeling --- p.4Chapter 1.4 --- Motivations and Goals --- p.5Chapter 1.5 --- Outline of the Thesis --- p.6Chapter 2 --- Background --- p.8Chapter 2.1 --- Constraint Satisfaction Problems --- p.8Chapter 2.1.1 --- Backtracking Tree Search --- p.9Chapter 2.1.2 --- Local Consistencies --- p.12Chapter 2.1.3 --- Local Consistencies in Backtracking Search --- p.17Chapter 2.1.4 --- Permutation CSPs --- p.19Chapter 2.2 --- Weighted Constraint Satisfaction Problems --- p.20Chapter 2.2.1 --- Branch and Bound Search --- p.23Chapter 2.2.2 --- Local Consistencies --- p.26Chapter 2.2.3 --- Local Consistencies in Branch and Bound Search --- p.32Chapter 2.3 --- Redundant Modeling --- p.34Chapter 3 --- Generating Redundant WCSP Models --- p.37Chapter 3.1 --- Model Induction for CSPs --- p.38Chapter 3.1.1 --- Stated Constraints --- p.39Chapter 3.1.2 --- No-Double-Assignment Constraints --- p.39Chapter 3.1.3 --- At-Least-One-Assignment Constraints --- p.40Chapter 3.2 --- Generalized Model Induction for WCSPs --- p.43Chapter 4 --- Combining Mutually Redundant WCSPs --- p.47Chapter 4.1 --- Naive Approach --- p.47Chapter 4.2 --- Node Consistency Revisited --- p.51Chapter 4.2.1 --- Refining Node Consistency Definition --- p.52Chapter 4.2.2 --- Enforcing m-NC* c Algorithm --- p.55Chapter 4.3 --- Arc Consistency Revisited --- p.58Chapter 4.3.1 --- Refining Arc Consistency Definition --- p.60Chapter 4.3.2 --- Enforcing m-AC*c Algorithm --- p.62Chapter 5 --- Experiments --- p.67Chapter 5.1 --- Langford's Problem --- p.68Chapter 5.2 --- Latin Square Problem --- p.72Chapter 5.3 --- Discussion --- p.75Chapter 6 --- Related Work --- p.77Chapter 6.1 --- Soft Constraint Satisfaction Problems --- p.77Chapter 6.2 --- Other Local Consistencies in WCSPs --- p.79Chapter 6.2.1 --- Full Arc Consistency --- p.79Chapter 6.2.2 --- Pull Directional Arc Consistency --- p.81Chapter 6.2.3 --- Existential Directional Arc Consistency --- p.82Chapter 6.3 --- Redundant Modeling and Channeling Constraints --- p.83Chapter 7 --- Concluding Remarks --- p.85Chapter 7.1 --- Contributions --- p.85Chapter 7.2 --- Future Work --- p.87List of Symbols --- p.88Bibliograph

The power of linear programming for general-valued CSPs

Let $D$, called the domain, be a fixed finite set and let $\Gamma$, called the valued constraint language, be a fixed set of functions of the form $f:D^m\to\mathbb{Q}\cup\{\infty\}$, where different functions might have different arity $m$. We study the valued constraint satisfaction problem parametrised by $\Gamma$, denoted by VCSP$(\Gamma)$. These are minimisation problems given by $n$ variables and the objective function given by a sum of functions from $\Gamma$, each depending on a subset of the $n$ variables. Finite-valued constraint languages contain functions that take on only rational values and not infinite values. Our main result is a precise algebraic characterisation of valued constraint languages whose instances can be solved exactly by the basic linear programming relaxation (BLP). For a valued constraint language $\Gamma$, BLP is a decision procedure for $\Gamma$ if and only if $\Gamma$ admits a symmetric fractional polymorphism of every arity. For a finite-valued constraint language $\Gamma$, BLP is a decision procedure if and only if $\Gamma$ admits a symmetric fractional polymorphism of some arity, or equivalently, if $\Gamma$ admits a symmetric fractional polymorphism of arity 2. Using these results, we obtain tractability of several novel classes of problems, including problems over valued constraint languages that are: (1) submodular on arbitrary lattices; (2) $k$-submodular on arbitrary finite domains; (3) weakly (and hence strongly) tree-submodular on arbitrary trees.Comment: A full version of a FOCS'12 paper by the last two authors (arXiv:1204.1079) and an ICALP'13 paper by the first author (arXiv:1207.7213) to appear in SIAM Journal on Computing (SICOMP

The complexity of finite-valued CSPs

We study the computational complexity of exact minimisation of rational-valued discrete functions. Let $\Gamma$ be a set of rational-valued functions on a fixed finite domain; such a set is called a finite-valued constraint language. The valued constraint satisfaction problem, $\operatorname{VCSP}(\Gamma)$, is the problem of minimising a function given as a sum of functions from $\Gamma$. We establish a dichotomy theorem with respect to exact solvability for all finite-valued constraint languages defined on domains of arbitrary finite size. We show that every constraint language $\Gamma$ either admits a binary symmetric fractional polymorphism in which case the basic linear programming relaxation solves any instance of $\operatorname{VCSP}(\Gamma)$ exactly, or $\Gamma$ satisfies a simple hardness condition that allows for a polynomial-time reduction from Max-Cut to $\operatorname{VCSP}(\Gamma)$