503 research outputs found
Stochastic forward-backward and primal-dual approximation algorithms with application to online image restoration
Stochastic approximation techniques have been used in various contexts in
data science. We propose a stochastic version of the forward-backward algorithm
for minimizing the sum of two convex functions, one of which is not necessarily
smooth. Our framework can handle stochastic approximations of the gradient of
the smooth function and allows for stochastic errors in the evaluation of the
proximity operator of the nonsmooth function. The almost sure convergence of
the iterates generated by the algorithm to a minimizer is established under
relatively mild assumptions. We also propose a stochastic version of a popular
primal-dual proximal splitting algorithm, establish its convergence, and apply
it to an online image restoration problem.Comment: 5 Figure
Quasinonexpansive Iterations on the Affine Hull of Orbits: From Mann's Mean Value Algorithm to Inertial Methods
Fixed point iterations play a central role in the design and the analysis of
a large number of optimization algorithms. We study a new iterative scheme in
which the update is obtained by applying a composition of quasinonexpansive
operators to a point in the affine hull of the orbit generated up to the
current iterate. This investigation unifies several algorithmic constructs,
including Mann's mean value method, inertial methods, and multi-layer
memoryless methods. It also provides a framework for the development of new
algorithms, such as those we propose for solving monotone inclusion and
minimization problems
Variable Metric Forward-Backward Splitting with Applications to Monotone Inclusions in Duality
We propose a variable metric forward-backward splitting algorithm and prove
its convergence in real Hilbert spaces. We then use this framework to derive
primal-dual splitting algorithms for solving various classes of monotone
inclusions in duality. Some of these algorithms are new even when specialized
to the fixed metric case. Various applications are discussed
Stochastic Approximations and Perturbations in Forward-Backward Splitting for Monotone Operators
We investigate the asymptotic behavior of a stochastic version of the
forward-backward splitting algorithm for finding a zero of the sum of a
maximally monotone set-valued operator and a cocoercive operator in Hilbert
spaces. Our general setting features stochastic approximations of the
cocoercive operator and stochastic perturbations in the evaluation of the
resolvents of the set-valued operator. In addition, relaxations and not
necessarily vanishing proximal parameters are allowed. Weak and strong almost
sure convergence properties of the iterates is established under mild
conditions on the underlying stochastic processes. Leveraging these results, we
also establish the almost sure convergence of the iterates of a stochastic
variant of a primal-dual proximal splitting method for composite minimization
problems
- …