2,993 research outputs found
On the Global Linear Convergence of the ADMM with Multi-Block Variables
The alternating direction method of multipliers (ADMM) has been widely used
for solving structured convex optimization problems. In particular, the ADMM
can solve convex programs that minimize the sum of convex functions with
-block variables linked by some linear constraints. While the convergence of
the ADMM for was well established in the literature, it remained an open
problem for a long time whether or not the ADMM for is still
convergent. Recently, it was shown in [3] that without further conditions the
ADMM for may actually fail to converge. In this paper, we show that
under some easily verifiable and reasonable conditions the global linear
convergence of the ADMM when can still be assured, which is important
since the ADMM is a popular method for solving large scale multi-block
optimization models and is known to perform very well in practice even when
. Our study aims to offer an explanation for this phenomenon
Adaptive Relaxed ADMM: Convergence Theory and Practical Implementation
Many modern computer vision and machine learning applications rely on solving
difficult optimization problems that involve non-differentiable objective
functions and constraints. The alternating direction method of multipliers
(ADMM) is a widely used approach to solve such problems. Relaxed ADMM is a
generalization of ADMM that often achieves better performance, but its
efficiency depends strongly on algorithm parameters that must be chosen by an
expert user. We propose an adaptive method that automatically tunes the key
algorithm parameters to achieve optimal performance without user oversight.
Inspired by recent work on adaptivity, the proposed adaptive relaxed ADMM
(ARADMM) is derived by assuming a Barzilai-Borwein style linear gradient. A
detailed convergence analysis of ARADMM is provided, and numerical results on
several applications demonstrate fast practical convergence.Comment: CVPR 201
Douglas-Rachford Splitting: Complexity Estimates and Accelerated Variants
We propose a new approach for analyzing convergence of the Douglas-Rachford
splitting method for solving convex composite optimization problems. The
approach is based on a continuously differentiable function, the
Douglas-Rachford Envelope (DRE), whose stationary points correspond to the
solutions of the original (possibly nonsmooth) problem. By proving the
equivalence between the Douglas-Rachford splitting method and a scaled gradient
method applied to the DRE, results from smooth unconstrained optimization are
employed to analyze convergence properties of DRS, to tune the method and to
derive an accelerated version of it
- …