3,881 research outputs found
Solving Multiple-Block Separable Convex Minimization Problems Using Two-Block Alternating Direction Method of Multipliers
In this paper, we consider solving multiple-block separable convex
minimization problems using alternating direction method of multipliers (ADMM).
Motivated by the fact that the existing convergence theory for ADMM is mostly
limited to the two-block case, we analyze in this paper, both theoretically and
numerically, a new strategy that first transforms a multi-block problem into an
equivalent two-block problem (either in the primal domain or in the dual
domain) and then solves it using the standard two-block ADMM. In particular, we
derive convergence results for this two-block ADMM approach to solve
multi-block separable convex minimization problems, including an improved
O(1/\epsilon) iteration complexity result. Moreover, we compare the numerical
efficiency of this approach with the standard multi-block ADMM on several
separable convex minimization problems which include basis pursuit, robust
principal component analysis and latent variable Gaussian graphical model
selection. The numerical results show that the multiple-block ADMM, although
lacks theoretical convergence guarantees, typically outperforms two-block
ADMMs
On the Global Linear Convergence of the ADMM with Multi-Block Variables
The alternating direction method of multipliers (ADMM) has been widely used
for solving structured convex optimization problems. In particular, the ADMM
can solve convex programs that minimize the sum of convex functions with
-block variables linked by some linear constraints. While the convergence of
the ADMM for was well established in the literature, it remained an open
problem for a long time whether or not the ADMM for is still
convergent. Recently, it was shown in [3] that without further conditions the
ADMM for may actually fail to converge. In this paper, we show that
under some easily verifiable and reasonable conditions the global linear
convergence of the ADMM when can still be assured, which is important
since the ADMM is a popular method for solving large scale multi-block
optimization models and is known to perform very well in practice even when
. Our study aims to offer an explanation for this phenomenon
A Smooth Primal-Dual Optimization Framework for Nonsmooth Composite Convex Minimization
We propose a new first-order primal-dual optimization framework for a convex
optimization template with broad applications. Our optimization algorithms
feature optimal convergence guarantees under a variety of common structure
assumptions on the problem template. Our analysis relies on a novel combination
of three classic ideas applied to the primal-dual gap function: smoothing,
acceleration, and homotopy. The algorithms due to the new approach achieve the
best known convergence rate results, in particular when the template consists
of only non-smooth functions. We also outline a restart strategy for the
acceleration to significantly enhance the practical performance. We demonstrate
relations with the augmented Lagrangian method and show how to exploit the
strongly convex objectives with rigorous convergence rate guarantees. We
provide numerical evidence with two examples and illustrate that the new
methods can outperform the state-of-the-art, including Chambolle-Pock, and the
alternating direction method-of-multipliers algorithms.Comment: 35 pages, accepted for publication on SIAM J. Optimization. Tech.
Report, Oct. 2015 (last update Sept. 2016
Iteration Complexity Analysis of Multi-Block ADMM for a Family of Convex Minimization without Strong Convexity
The alternating direction method of multipliers (ADMM) is widely used in
solving structured convex optimization problems due to its superior practical
performance. On the theoretical side however, a counterexample was shown in [7]
indicating that the multi-block ADMM for minimizing the sum of
convex functions with block variables linked by linear constraints may
diverge. It is therefore of great interest to investigate further sufficient
conditions on the input side which can guarantee convergence for the
multi-block ADMM. The existing results typically require the strong convexity
on parts of the objective. In this paper, we present convergence and
convergence rate results for the multi-block ADMM applied to solve certain
-block convex minimization problems without requiring strong
convexity. Specifically, we prove the following two results: (1) the
multi-block ADMM returns an -optimal solution within
iterations by solving an associated perturbation to the
original problem; (2) the multi-block ADMM returns an -optimal
solution within iterations when it is applied to solve a
certain sharing problem, under the condition that the augmented Lagrangian
function satisfies the Kurdyka-Lojasiewicz property, which essentially covers
most convex optimization models except for some pathological cases.Comment: arXiv admin note: text overlap with arXiv:1408.426
- …