1,568 research outputs found
Forward-backward truncated Newton methods for convex composite optimization
This paper proposes two proximal Newton-CG methods for convex nonsmooth
optimization problems in composite form. The algorithms are based on a a
reformulation of the original nonsmooth problem as the unconstrained
minimization of a continuously differentiable function, namely the
forward-backward envelope (FBE). The first algorithm is based on a standard
line search strategy, whereas the second one combines the global efficiency
estimates of the corresponding first-order methods, while achieving fast
asymptotic convergence rates. Furthermore, they are computationally attractive
since each Newton iteration requires the approximate solution of a linear
system of usually small dimension
Maximum-a-posteriori estimation with Bayesian confidence regions
Solutions to inverse problems that are ill-conditioned or ill-posed may have
significant intrinsic uncertainty. Unfortunately, analysing and quantifying
this uncertainty is very challenging, particularly in high-dimensional
problems. As a result, while most modern mathematical imaging methods produce
impressive point estimation results, they are generally unable to quantify the
uncertainty in the solutions delivered. This paper presents a new general
methodology for approximating Bayesian high-posterior-density credibility
regions in inverse problems that are convex and potentially very
high-dimensional. The approximations are derived by using recent concentration
of measure results related to information theory for log-concave random
vectors. A remarkable property of the approximations is that they can be
computed very efficiently, even in large-scale problems, by using standard
convex optimisation techniques. In particular, they are available as a
by-product in problems solved by maximum-a-posteriori estimation. The
approximations also have favourable theoretical properties, namely they
outer-bound the true high-posterior-density credibility regions, and they are
stable with respect to model dimension. The proposed methodology is illustrated
on two high-dimensional imaging inverse problems related to tomographic
reconstruction and sparse deconvolution, where the approximations are used to
perform Bayesian hypothesis tests and explore the uncertainty about the
solutions, and where proximal Markov chain Monte Carlo algorithms are used as
benchmark to compute exact credible regions and measure the approximation
error
Dynamical systems and forward-backward algorithms associated with the sum of a convex subdifferential and a monotone cocoercive operator
In a Hilbert framework, we introduce continuous and discrete dynamical
systems which aim at solving inclusions governed by structured monotone
operators , where is the subdifferential of a
convex lower semicontinuous function , and is a monotone cocoercive
operator. We first consider the extension to this setting of the regularized
Newton dynamic with two potentials. Then, we revisit some related dynamical
systems, namely the semigroup of contractions generated by , and the
continuous gradient projection dynamic. By a Lyapunov analysis, we show the
convergence properties of the orbits of these systems.
The time discretization of these dynamics gives various forward-backward
splitting methods (some new) for solving structured monotone inclusions
involving non-potential terms. The convergence of these algorithms is obtained
under classical step size limitation. Perspectives are given in the field of
numerical splitting methods for optimization, and multi-criteria decision
processes.Comment: 25 page
The Forward-Backward-Forward Method from continuous and discrete perspective for pseudo-monotone variational inequalities in Hilbert spaces
Tseng's forward-backward-forward algorithm is a valuable alternative for
Korpelevich's extragradient method when solving variational inequalities over a
convex and closed set governed by monotone and Lipschitz continuous operators,
as it requires in every step only one projection operation. However, it is
well-known that Korpelevich's method converges and can therefore be used also
for solving variational inequalities governed by pseudo-monotone and Lipschitz
continuous operators. In this paper, we first associate to a pseudo-monotone
variational inequality a forward-backward-forward dynamical system and carry
out an asymptotic analysis for the generated trajectories. The explicit time
discretization of this system results into Tseng's forward-backward-forward
algorithm with relaxation parameters, which we prove to converge also when it
is applied to pseudo-monotone variational inequalities. In addition, we show
that linear convergence is guaranteed under strong pseudo-monotonicity.
Numerical experiments are carried out for pseudo-monotone variational
inequalities over polyhedral sets and fractional programming problems
- …