1,674 research outputs found
Lagrangian Relaxation for MAP Estimation in Graphical Models
We develop a general framework for MAP estimation in discrete and Gaussian
graphical models using Lagrangian relaxation techniques. The key idea is to
reformulate an intractable estimation problem as one defined on a more
tractable graph, but subject to additional constraints. Relaxing these
constraints gives a tractable dual problem, one defined by a thin graph, which
is then optimized by an iterative procedure. When this iterative optimization
leads to a consistent estimate, one which also satisfies the constraints, then
it corresponds to an optimal MAP estimate of the original model. Otherwise
there is a ``duality gap'', and we obtain a bound on the optimal solution.
Thus, our approach combines convex optimization with dynamic programming
techniques applicable for thin graphs. The popular tree-reweighted max-product
(TRMP) method may be seen as solving a particular class of such relaxations,
where the intractable graph is relaxed to a set of spanning trees. We also
consider relaxations to a set of small induced subgraphs, thin subgraphs (e.g.
loops), and a connected tree obtained by ``unwinding'' cycles. In addition, we
propose a new class of multiscale relaxations that introduce ``summary''
variables. The potential benefits of such generalizations include: reducing or
eliminating the ``duality gap'' in hard problems, reducing the number or
Lagrange multipliers in the dual problem, and accelerating convergence of the
iterative optimization procedure.Comment: 10 pages, presented at 45th Allerton conference on communication,
control and computing, to appear in proceeding
Inference by Minimizing Size, Divergence, or their Sum
We speed up marginal inference by ignoring factors that do not significantly
contribute to overall accuracy. In order to pick a suitable subset of factors
to ignore, we propose three schemes: minimizing the number of model factors
under a bound on the KL divergence between pruned and full models; minimizing
the KL divergence under a bound on factor count; and minimizing the weighted
sum of KL divergence and factor count. All three problems are solved using an
approximation of the KL divergence than can be calculated in terms of marginals
computed on a simple seed graph. Applied to synthetic image denoising and to
three different types of NLP parsing models, this technique performs marginal
inference up to 11 times faster than loopy BP, with graph sizes reduced up to
98%-at comparable error in marginals and parsing accuracy. We also show that
minimizing the weighted sum of divergence and size is substantially faster than
minimizing either of the other objectives based on the approximation to
divergence presented here.Comment: Appears in Proceedings of the Twenty-Sixth Conference on Uncertainty
in Artificial Intelligence (UAI2010
- …