444 research outputs found
Stochastic forward-backward and primal-dual approximation algorithms with application to online image restoration
Stochastic approximation techniques have been used in various contexts in
data science. We propose a stochastic version of the forward-backward algorithm
for minimizing the sum of two convex functions, one of which is not necessarily
smooth. Our framework can handle stochastic approximations of the gradient of
the smooth function and allows for stochastic errors in the evaluation of the
proximity operator of the nonsmooth function. The almost sure convergence of
the iterates generated by the algorithm to a minimizer is established under
relatively mild assumptions. We also propose a stochastic version of a popular
primal-dual proximal splitting algorithm, establish its convergence, and apply
it to an online image restoration problem.Comment: 5 Figure
Stochastic Approximations and Perturbations in Forward-Backward Splitting for Monotone Operators
We investigate the asymptotic behavior of a stochastic version of the
forward-backward splitting algorithm for finding a zero of the sum of a
maximally monotone set-valued operator and a cocoercive operator in Hilbert
spaces. Our general setting features stochastic approximations of the
cocoercive operator and stochastic perturbations in the evaluation of the
resolvents of the set-valued operator. In addition, relaxations and not
necessarily vanishing proximal parameters are allowed. Weak and strong almost
sure convergence properties of the iterates is established under mild
conditions on the underlying stochastic processes. Leveraging these results, we
also establish the almost sure convergence of the iterates of a stochastic
variant of a primal-dual proximal splitting method for composite minimization
problems
A first-order stochastic primal-dual algorithm with correction step
We investigate the convergence properties of a stochastic primal-dual
splitting algorithm for solving structured monotone inclusions involving the
sum of a cocoercive operator and a composite monotone operator. The proposed
method is the stochastic extension to monotone inclusions of a proximal method
studied in {\em Y. Drori, S. Sabach, and M. Teboulle, A simple algorithm for a
class of nonsmooth convex-concave saddle-point problems, 2015} and {\em I.
Loris and C. Verhoeven, On a generalization of the iterative soft-thresholding
algorithm for the case of non-separable penalty, 2011} for saddle point
problems. It consists in a forward step determined by the stochastic evaluation
of the cocoercive operator, a backward step in the dual variables involving the
resolvent of the monotone operator, and an additional forward step using the
stochastic evaluation of the cocoercive introduced in the first step. We prove
weak almost sure convergence of the iterates by showing that the primal-dual
sequence generated by the method is stochastic quasi Fej\'er-monotone with
respect to the set of zeros of the considered primal and dual inclusions.
Additional results on ergodic convergence in expectation are considered for the
special case of saddle point models
A stochastic inertial forward-backward splitting algorithm for multivariate monotone inclusions
We propose an inertial forward-backward splitting algorithm to compute the
zero of a sum of two monotone operators allowing for stochastic errors in the
computation of the operators. More precisely, we establish almost sure
convergence in real Hilbert spaces of the sequence of iterates to an optimal
solution. Then, based on this analysis, we introduce two new classes of
stochastic inertial primal-dual splitting methods for solving structured
systems of composite monotone inclusions and prove their convergence. Our
results extend to the stochastic and inertial setting various types of
structured monotone inclusion problems and corresponding algorithmic solutions.
Application to minimization problems is discussed
Stochastic Quasi-Fej\'er Block-Coordinate Fixed Point Iterations with Random Sweeping
This work proposes block-coordinate fixed point algorithms with applications
to nonlinear analysis and optimization in Hilbert spaces. The asymptotic
analysis relies on a notion of stochastic quasi-Fej\'er monotonicity, which is
thoroughly investigated. The iterative methods under consideration feature
random sweeping rules to select arbitrarily the blocks of variables that are
activated over the course of the iterations and they allow for stochastic
errors in the evaluation of the operators. Algorithms using quasinonexpansive
operators or compositions of averaged nonexpansive operators are constructed,
and weak and strong convergence results are established for the sequences they
generate. As a by-product, novel block-coordinate operator splitting methods
are obtained for solving structured monotone inclusion and convex minimization
problems. In particular, the proposed framework leads to random
block-coordinate versions of the Douglas-Rachford and forward-backward
algorithms and of some of their variants. In the standard case of block,
our results remain new as they incorporate stochastic perturbations
A Review on Deep Learning in Medical Image Reconstruction
Medical imaging is crucial in modern clinics to guide the diagnosis and
treatment of diseases. Medical image reconstruction is one of the most
fundamental and important components of medical imaging, whose major objective
is to acquire high-quality medical images for clinical usage at the minimal
cost and risk to the patients. Mathematical models in medical image
reconstruction or, more generally, image restoration in computer vision, have
been playing a prominent role. Earlier mathematical models are mostly designed
by human knowledge or hypothesis on the image to be reconstructed, and we shall
call these models handcrafted models. Later, handcrafted plus data-driven
modeling started to emerge which still mostly relies on human designs, while
part of the model is learned from the observed data. More recently, as more
data and computation resources are made available, deep learning based models
(or deep models) pushed the data-driven modeling to the extreme where the
models are mostly based on learning with minimal human designs. Both
handcrafted and data-driven modeling have their own advantages and
disadvantages. One of the major research trends in medical imaging is to
combine handcrafted modeling with deep modeling so that we can enjoy benefits
from both approaches. The major part of this article is to provide a conceptual
review of some recent works on deep modeling from the unrolling dynamics
viewpoint. This viewpoint stimulates new designs of neural network
architectures with inspirations from optimization algorithms and numerical
differential equations. Given the popularity of deep modeling, there are still
vast remaining challenges in the field, as well as opportunities which we shall
discuss at the end of this article.Comment: 31 pages, 6 figures. Survey pape
- âŠ