3,674 research outputs found
Lagrangian Relaxation for MAP Estimation in Graphical Models
We develop a general framework for MAP estimation in discrete and Gaussian
graphical models using Lagrangian relaxation techniques. The key idea is to
reformulate an intractable estimation problem as one defined on a more
tractable graph, but subject to additional constraints. Relaxing these
constraints gives a tractable dual problem, one defined by a thin graph, which
is then optimized by an iterative procedure. When this iterative optimization
leads to a consistent estimate, one which also satisfies the constraints, then
it corresponds to an optimal MAP estimate of the original model. Otherwise
there is a ``duality gap'', and we obtain a bound on the optimal solution.
Thus, our approach combines convex optimization with dynamic programming
techniques applicable for thin graphs. The popular tree-reweighted max-product
(TRMP) method may be seen as solving a particular class of such relaxations,
where the intractable graph is relaxed to a set of spanning trees. We also
consider relaxations to a set of small induced subgraphs, thin subgraphs (e.g.
loops), and a connected tree obtained by ``unwinding'' cycles. In addition, we
propose a new class of multiscale relaxations that introduce ``summary''
variables. The potential benefits of such generalizations include: reducing or
eliminating the ``duality gap'' in hard problems, reducing the number or
Lagrange multipliers in the dual problem, and accelerating convergence of the
iterative optimization procedure.Comment: 10 pages, presented at 45th Allerton conference on communication,
control and computing, to appear in proceeding
Discrete-Continuous ADMM for Transductive Inference in Higher-Order MRFs
This paper introduces a novel algorithm for transductive inference in
higher-order MRFs, where the unary energies are parameterized by a variable
classifier. The considered task is posed as a joint optimization problem in the
continuous classifier parameters and the discrete label variables. In contrast
to prior approaches such as convex relaxations, we propose an advantageous
decoupling of the objective function into discrete and continuous subproblems
and a novel, efficient optimization method related to ADMM. This approach
preserves integrality of the discrete label variables and guarantees global
convergence to a critical point. We demonstrate the advantages of our approach
in several experiments including video object segmentation on the DAVIS data
set and interactive image segmentation
Asset Allocation under the Basel Accord Risk Measures
Financial institutions are currently required to meet more stringent capital
requirements than they were before the recent financial crisis; in particular,
the capital requirement for a large bank's trading book under the Basel 2.5
Accord more than doubles that under the Basel II Accord. The significant
increase in capital requirements renders it necessary for banks to take into
account the constraint of capital requirement when they make asset allocation
decisions. In this paper, we propose a new asset allocation model that
incorporates the regulatory capital requirements under both the Basel 2.5
Accord, which is currently in effect, and the Basel III Accord, which was
recently proposed and is currently under discussion. We propose an unified
algorithm based on the alternating direction augmented Lagrangian method to
solve the model; we also establish the first-order optimality of the limit
points of the sequence generated by the algorithm under some mild conditions.
The algorithm is simple and easy to implement; each step of the algorithm
consists of solving convex quadratic programming or one-dimensional
subproblems. Numerical experiments on simulated and real market data show that
the algorithm compares favorably with other existing methods, especially in
cases in which the model is non-convex
An Extragradient-Based Alternating Direction Method for Convex Minimization
In this paper, we consider the problem of minimizing the sum of two convex
functions subject to linear linking constraints. The classical alternating
direction type methods usually assume that the two convex functions have
relatively easy proximal mappings. However, many problems arising from
statistics, image processing and other fields have the structure that while one
of the two functions has easy proximal mapping, the other function is smoothly
convex but does not have an easy proximal mapping. Therefore, the classical
alternating direction methods cannot be applied. To deal with the difficulty,
we propose in this paper an alternating direction method based on
extragradients. Under the assumption that the smooth function has a Lipschitz
continuous gradient, we prove that the proposed method returns an
-optimal solution within iterations. We apply the
proposed method to solve a new statistical model called fused logistic
regression. Our numerical experiments show that the proposed method performs
very well when solving the test problems. We also test the performance of the
proposed method through solving the lasso problem arising from statistics and
compare the result with several existing efficient solvers for this problem;
the results are very encouraging indeed
Large-scale Binary Quadratic Optimization Using Semidefinite Relaxation and Applications
In computer vision, many problems such as image segmentation, pixel
labelling, and scene parsing can be formulated as binary quadratic programs
(BQPs). For submodular problems, cuts based methods can be employed to
efficiently solve large-scale problems. However, general nonsubmodular problems
are significantly more challenging to solve. Finding a solution when the
problem is of large size to be of practical interest, however, typically
requires relaxation. Two standard relaxation methods are widely used for
solving general BQPs--spectral methods and semidefinite programming (SDP), each
with their own advantages and disadvantages. Spectral relaxation is simple and
easy to implement, but its bound is loose. Semidefinite relaxation has a
tighter bound, but its computational complexity is high, especially for large
scale problems. In this work, we present a new SDP formulation for BQPs, with
two desirable properties. First, it has a similar relaxation bound to
conventional SDP formulations. Second, compared with conventional SDP methods,
the new SDP formulation leads to a significantly more efficient and scalable
dual optimization approach, which has the same degree of complexity as spectral
methods. We then propose two solvers, namely, quasi-Newton and smoothing Newton
methods, for the dual problem. Both of them are significantly more efficiently
than standard interior-point methods. In practice, the smoothing Newton solver
is faster than the quasi-Newton solver for dense or medium-sized problems,
while the quasi-Newton solver is preferable for large sparse/structured
problems. Our experiments on a few computer vision applications including
clustering, image segmentation, co-segmentation and registration show the
potential of our SDP formulation for solving large-scale BQPs.Comment: Fixed some typos. 18 pages. Accepted to IEEE Transactions on Pattern
Analysis and Machine Intelligenc
Projection methods in conic optimization
There exist efficient algorithms to project a point onto the intersection of
a convex cone and an affine subspace. Those conic projections are in turn the
work-horse of a range of algorithms in conic optimization, having a variety of
applications in science, finance and engineering. This chapter reviews some of
these algorithms, emphasizing the so-called regularization algorithms for
linear conic optimization, and applications in polynomial optimization. This is
a presentation of the material of several recent research articles; we aim here
at clarifying the ideas, presenting them in a general framework, and pointing
out important techniques
- …