1,600 research outputs found
Non-smooth Non-convex Bregman Minimization: Unification and new Algorithms
We propose a unifying algorithm for non-smooth non-convex optimization. The
algorithm approximates the objective function by a convex model function and
finds an approximate (Bregman) proximal point of the convex model. This
approximate minimizer of the model function yields a descent direction, along
which the next iterate is found. Complemented with an Armijo-like line search
strategy, we obtain a flexible algorithm for which we prove (subsequential)
convergence to a stationary point under weak assumptions on the growth of the
model function error. Special instances of the algorithm with a Euclidean
distance function are, for example, Gradient Descent, Forward--Backward
Splitting, ProxDescent, without the common requirement of a "Lipschitz
continuous gradient". In addition, we consider a broad class of Bregman
distance functions (generated by Legendre functions) replacing the Euclidean
distance. The algorithm has a wide range of applications including many linear
and non-linear inverse problems in signal/image processing and machine
learning
Bregman Proximal Gradient Algorithm with Extrapolation for a class of Nonconvex Nonsmooth Minimization Problems
In this paper, we consider an accelerated method for solving nonconvex and
nonsmooth minimization problems. We propose a Bregman Proximal Gradient
algorithm with extrapolation(BPGe). This algorithm extends and accelerates the
Bregman Proximal Gradient algorithm (BPG), which circumvents the restrictive
global Lipschitz gradient continuity assumption needed in Proximal Gradient
algorithms (PG). The BPGe algorithm has higher generality than the recently
introduced Proximal Gradient algorithm with extrapolation(PGe), and besides,
due to the extrapolation step, BPGe converges faster than BPG algorithm.
Analyzing the convergence, we prove that any limit point of the sequence
generated by BPGe is a stationary point of the problem by choosing parameters
properly. Besides, assuming Kurdyka-{\'L}ojasiewicz property, we prove the
whole sequences generated by BPGe converges to a stationary point. Finally, to
illustrate the potential of the new method BPGe, we apply it to two important
practical problems that arise in many fundamental applications (and that not
satisfy global Lipschitz gradient continuity assumption): Poisson linear
inverse problems and quadratic inverse problems. In the tests the accelerated
BPGe algorithm shows faster convergence results, giving an interesting new
algorithm.Comment: Preprint submitted for publication, February 14, 201
First order algorithms in variational image processing
Variational methods in imaging are nowadays developing towards a quite
universal and flexible tool, allowing for highly successful approaches on tasks
like denoising, deblurring, inpainting, segmentation, super-resolution,
disparity, and optical flow estimation. The overall structure of such
approaches is of the form ; where the functional is a data fidelity term also
depending on some input data and measuring the deviation of from such
and is a regularization functional. Moreover is a (often linear)
forward operator modeling the dependence of data on an underlying image, and
is a positive regularization parameter. While is often
smooth and (strictly) convex, the current practice almost exclusively uses
nonsmooth regularization functionals. The majority of successful techniques is
using nonsmooth and convex functionals like the total variation and
generalizations thereof or -norms of coefficients arising from scalar
products with some frame system. The efficient solution of such variational
problems in imaging demands for appropriate algorithms. Taking into account the
specific structure as a sum of two very different terms to be minimized,
splitting algorithms are a quite canonical choice. Consequently this field has
revived the interest in techniques like operator splittings or augmented
Lagrangians. Here we shall provide an overview of methods currently developed
and recent results as well as some computational studies providing a comparison
of different methods and also illustrating their success in applications.Comment: 60 pages, 33 figure
- âŠ