1,622 research outputs found
On the Optimal Linear Convergence Rate of a Generalized Proximal Point Algorithm
The proximal point algorithm (PPA) has been well studied in the literature.
In particular, its linear convergence rate has been studied by Rockafellar in
1976 under certain condition. We consider a generalized PPA in the generic
setting of finding a zero point of a maximal monotone operator, and show that
the condition proposed by Rockafellar can also sufficiently ensure the linear
convergence rate for this generalized PPA. Indeed we show that these linear
convergence rates are optimal. Both the exact and inexact versions of this
generalized PPA are discussed. The motivation to consider this generalized PPA
is that it includes as special cases the relaxed versions of some splitting
methods that are originated from PPA. Thus, linear convergence results of this
generalized PPA can be used to better understand the convergence of some widely
used algorithms in the literature. We focus on the particular convex
minimization context and specify Rockafellar's condition to see how to ensure
the linear convergence rate for some efficient numerical schemes, including the
classical augmented Lagrangian method proposed by Hensen and Powell in 1969 and
its relaxed version, the original alternating direction method of multipliers
(ADMM) by Glowinski and Marrocco in 1975 and its relaxed version (i.e., the
generalized ADMM by Eckstein and Bertsekas in 1992). Some refined conditions
weaker than existing ones are proposed in these particular contexts.Comment: 22 pages, 1 figur
Soft switching techniques for multilevel inverters
Tese (doutorado) - Universidade Federal de Santa Catarina, Centro Tecnológico. Programa de Pós-Graduação em Engenharia Elétrica
Bregman distances and Chebyshev sets
A closed set of a Euclidean space is said to be Chebyshev if every point in
the space has one and only one closest point in the set. Although the situation
is not settled in infinite-dimensional Hilbert spaces, in 1932 Bunt showed that
in Euclidean spaces a closed set is Chebyshev if and only if the set is convex.
In this paper, from the more general perspective of Bregman distances, we show
that if every point in the space has a unique nearest point in a closed set,
then the set is convex. We provide two approaches: one is by nonsmooth
analysis; the other by maximal monotone operator theory. Subdifferentiability
properties of Bregman nearest distance functions are also given
Adaptive Relaxed ADMM: Convergence Theory and Practical Implementation
Many modern computer vision and machine learning applications rely on solving
difficult optimization problems that involve non-differentiable objective
functions and constraints. The alternating direction method of multipliers
(ADMM) is a widely used approach to solve such problems. Relaxed ADMM is a
generalization of ADMM that often achieves better performance, but its
efficiency depends strongly on algorithm parameters that must be chosen by an
expert user. We propose an adaptive method that automatically tunes the key
algorithm parameters to achieve optimal performance without user oversight.
Inspired by recent work on adaptivity, the proposed adaptive relaxed ADMM
(ARADMM) is derived by assuming a Barzilai-Borwein style linear gradient. A
detailed convergence analysis of ARADMM is provided, and numerical results on
several applications demonstrate fast practical convergence.Comment: CVPR 201
- …