A forward-backward view of some primal-dual optimization methods in image recovery

A wide array of image recovery problems can be abstracted into the problem of minimizing a sum of composite convex functions in a Hilbert space. To solve such problems, primal-dual proximal approaches have been developed which provide efficient solutions to large-scale optimization problems. The objective of this paper is to show that a number of existing algorithms can be derived from a general form of the forward-backward algorithm applied in a suitable product space. Our approach also allows us to develop useful extensions of existing algorithms by introducing a variable metric. An illustration to image restoration is provided

Stochastic forward-backward and primal-dual approximation algorithms with application to online image restoration

Stochastic approximation techniques have been used in various contexts in data science. We propose a stochastic version of the forward-backward algorithm for minimizing the sum of two convex functions, one of which is not necessarily smooth. Our framework can handle stochastic approximations of the gradient of the smooth function and allows for stochastic errors in the evaluation of the proximity operator of the nonsmooth function. The almost sure convergence of the iterates generated by the algorithm to a minimizer is established under relatively mild assumptions. We also propose a stochastic version of a popular primal-dual proximal splitting algorithm, establish its convergence, and apply it to an online image restoration problem.Comment: 5 Figure

Iteration-Complexity of a Generalized Forward Backward Splitting Algorithm

In this paper, we analyze the iteration-complexity of Generalized Forward--Backward (GFB) splitting algorithm, as proposed in \cite{gfb2011}, for minimizing a large class of composite objectives $f + \sum_{i=1}^n h_i$ on a Hilbert space, where $f$ has a Lipschitz-continuous gradient and the $h_i$'s are simple (\ie their proximity operators are easy to compute). We derive iteration-complexity bounds (pointwise and ergodic) for the inexact version of GFB to obtain an approximate solution based on an easily verifiable termination criterion. Along the way, we prove complexity bounds for relaxed and inexact fixed point iterations built from composition of nonexpansive averaged operators. These results apply more generally to GFB when used to find a zero of a sum of $n > 0$ maximal monotone operators and a co-coercive operator on a Hilbert space. The theoretical findings are exemplified with experiments on video processing.Comment: 5 pages, 2 figure

Forward-Half-Reflected-Partial inverse-Backward Splitting Algorithm for Solving Monotone Inclusions

In this article, we proposed a method for numerically solving monotone inclusions in real Hilbert spaces that involve the sum of a maximally monotone operator, a monotone-Lipschitzian operator, a cocoercive operator, and a normal cone to a vector subspace. Our algorithm splits and exploits the intrinsic properties of each operator involved in the inclusion. The proposed method is derived by combining partial inverse techniques and the {\it forward-half-reflected-backward} (FHRB) splitting method proposed by Malitsky and Tam (2020). Our method inherits the advantages of FHRB, equiring only one activation of the Lipschitzian operator, one activation of the cocoercive operator, two projections onto the closed vector subspace, and one calculation of the resolvent of the maximally monotone operator. Furthermore, we develop a method for solving primal-dual inclusions involving a mixture of sums, linear compositions, parallel sums, Lipschitzian operators, cocoercive operators, and normal cones. We apply our method to constrained composite convex optimization problems as a specific example. Finally, in order to compare our proposed method with existing methods in the literature, we provide numerical experiments on constrained total variation least-squares optimization problems. The numerical results are promising

Solving monotone inclusions involving parallel sums of linearly composed maximally monotone operators

The aim of this article is to present two different primal-dual methods for solving structured monotone inclusions involving parallel sums of compositions of maximally monotone operators with linear bounded operators. By employing some elaborated splitting techniques, all of the operators occurring in the problem formulation are processed individually via forward or backward steps. The treatment of parallel sums of linearly composed maximally monotone operators is motivated by applications in imaging which involve first- and second-order total variation functionals, to which a special attention is given.Comment: 25 page

Convergence Rates with Inexact Non-expansive Operators

In this paper, we present a convergence rate analysis for the inexact Krasnosel'skii-Mann iteration built from nonexpansive operators. Our results include two main parts: we first establish global pointwise and ergodic iteration-complexity bounds, and then, under a metric subregularity assumption, we establish local linear convergence for the distance of the iterates to the set of fixed points. The obtained iteration-complexity result can be applied to analyze the convergence rate of various monotone operator splitting methods in the literature, including the Forward-Backward, the Generalized Forward-Backward, Douglas-Rachford, alternating direction method of multipliers (ADMM) and Primal-Dual splitting methods. For these methods, we also develop easily verifiable termination criteria for finding an approximate solution, which can be seen as a generalization of the termination criterion for the classical gradient descent method. We finally develop a parallel analysis for the non-stationary Krasnosel'skii-Mann iteration. The usefulness of our results is illustrated by applying them to a large class of structured monotone inclusion and convex optimization problems. Experiments on some large scale inverse problems in signal and image processing problems are shown.Comment: This is an extended version of the work presented in http://arxiv.org/abs/1310.6636, and is accepted by the Mathematical Programmin