33,457 research outputs found
An abstract proximal point algorithm
The proximal point algorithm is a widely used tool for solving a variety of
convex optimization problems such as finding zeros of maximally monotone
operators, fixed points of nonexpansive mappings, as well as minimizing convex
functions. The algorithm works by applying successively so-called "resolvent"
mappings associated to the original object that one aims to optimize. In this
paper we abstract from the corresponding resolvents employed in these problems
the natural notion of jointly firmly nonexpansive families of mappings. This
leads to a streamlined method of proving weak convergence of this class of
algorithms in the context of complete CAT(0) spaces (and hence also in Hilbert
spaces). In addition, we consider the notion of uniform firm nonexpansivity in
order to similarly provide a unified presentation of a case where the algorithm
converges strongly. Methods which stem from proof mining, an applied subfield
of logic, yield in this situation computable and low-complexity rates of
convergence
Non-smooth Non-convex Bregman Minimization: Unification and new Algorithms
We propose a unifying algorithm for non-smooth non-convex optimization. The
algorithm approximates the objective function by a convex model function and
finds an approximate (Bregman) proximal point of the convex model. This
approximate minimizer of the model function yields a descent direction, along
which the next iterate is found. Complemented with an Armijo-like line search
strategy, we obtain a flexible algorithm for which we prove (subsequential)
convergence to a stationary point under weak assumptions on the growth of the
model function error. Special instances of the algorithm with a Euclidean
distance function are, for example, Gradient Descent, Forward--Backward
Splitting, ProxDescent, without the common requirement of a "Lipschitz
continuous gradient". In addition, we consider a broad class of Bregman
distance functions (generated by Legendre functions) replacing the Euclidean
distance. The algorithm has a wide range of applications including many linear
and non-linear inverse problems in signal/image processing and machine
learning
- …