13 research outputs found
Lagrangian Relaxation for MAP Estimation in Graphical Models
We develop a general framework for MAP estimation in discrete and Gaussian
graphical models using Lagrangian relaxation techniques. The key idea is to
reformulate an intractable estimation problem as one defined on a more
tractable graph, but subject to additional constraints. Relaxing these
constraints gives a tractable dual problem, one defined by a thin graph, which
is then optimized by an iterative procedure. When this iterative optimization
leads to a consistent estimate, one which also satisfies the constraints, then
it corresponds to an optimal MAP estimate of the original model. Otherwise
there is a ``duality gap'', and we obtain a bound on the optimal solution.
Thus, our approach combines convex optimization with dynamic programming
techniques applicable for thin graphs. The popular tree-reweighted max-product
(TRMP) method may be seen as solving a particular class of such relaxations,
where the intractable graph is relaxed to a set of spanning trees. We also
consider relaxations to a set of small induced subgraphs, thin subgraphs (e.g.
loops), and a connected tree obtained by ``unwinding'' cycles. In addition, we
propose a new class of multiscale relaxations that introduce ``summary''
variables. The potential benefits of such generalizations include: reducing or
eliminating the ``duality gap'' in hard problems, reducing the number or
Lagrange multipliers in the dual problem, and accelerating convergence of the
iterative optimization procedure.Comment: 10 pages, presented at 45th Allerton conference on communication,
control and computing, to appear in proceeding
MAP inference via Block-Coordinate Frank-Wolfe Algorithm
We present a new proximal bundle method for Maximum-A-Posteriori (MAP)
inference in structured energy minimization problems. The method optimizes a
Lagrangean relaxation of the original energy minimization problem using a multi
plane block-coordinate Frank-Wolfe method that takes advantage of the specific
structure of the Lagrangean decomposition. We show empirically that our method
outperforms state-of-the-art Lagrangean decomposition based algorithms on some
challenging Markov Random Field, multi-label discrete tomography and graph
matching problems
Getting Feasible Variable Estimates From Infeasible Ones: MRF Local Polytope Study
This paper proposes a method for construction of approximate feasible primal
solutions from dual ones for large-scale optimization problems possessing
certain separability properties. Whereas infeasible primal estimates can
typically be produced from (sub-)gradients of the dual function, it is often
not easy to project them to the primal feasible set, since the projection
itself has a complexity comparable to the complexity of the initial problem. We
propose an alternative efficient method to obtain feasibility and show that its
properties influencing the convergence to the optimum are similar to the
properties of the Euclidean projection. We apply our method to the local
polytope relaxation of inference problems for Markov Random Fields and
demonstrate its superiority over existing methods.Comment: 20 page, 4 figure
Blending Learning and Inference in Structured Prediction
In this paper we derive an efficient algorithm to learn the parameters of
structured predictors in general graphical models. This algorithm blends the
learning and inference tasks, which results in a significant speedup over
traditional approaches, such as conditional random fields and structured
support vector machines. For this purpose we utilize the structures of the
predictors to describe a low dimensional structured prediction task which
encourages local consistencies within the different structures while learning
the parameters of the model. Convexity of the learning task provides the means
to enforce the consistencies between the different parts. The
inference-learning blending algorithm that we propose is guaranteed to converge
to the optimum of the low dimensional primal and dual programs. Unlike many of
the existing approaches, the inference-learning blending allows us to learn
efficiently high-order graphical models, over regions of any size, and very
large number of parameters. We demonstrate the effectiveness of our approach,
while presenting state-of-the-art results in stereo estimation, semantic
segmentation, shape reconstruction, and indoor scene understanding
The power of linear programming for general-valued CSPs
Let , called the domain, be a fixed finite set and let , called
the valued constraint language, be a fixed set of functions of the form
, where different functions might have
different arity . We study the valued constraint satisfaction problem
parametrised by , denoted by VCSP. These are minimisation
problems given by variables and the objective function given by a sum of
functions from , each depending on a subset of the variables.
Finite-valued constraint languages contain functions that take on only rational
values and not infinite values.
Our main result is a precise algebraic characterisation of valued constraint
languages whose instances can be solved exactly by the basic linear programming
relaxation (BLP). For a valued constraint language , BLP is a decision
procedure for if and only if admits a symmetric fractional
polymorphism of every arity. For a finite-valued constraint language ,
BLP is a decision procedure if and only if admits a symmetric
fractional polymorphism of some arity, or equivalently, if admits a
symmetric fractional polymorphism of arity 2.
Using these results, we obtain tractability of several novel classes of
problems, including problems over valued constraint languages that are: (1)
submodular on arbitrary lattices; (2) -submodular on arbitrary finite
domains; (3) weakly (and hence strongly) tree-submodular on arbitrary trees.Comment: A full version of a FOCS'12 paper by the last two authors
(arXiv:1204.1079) and an ICALP'13 paper by the first author (arXiv:1207.7213)
to appear in SIAM Journal on Computing (SICOMP