10,706 research outputs found
Optimization Methods for Inverse Problems
Optimization plays an important role in solving many inverse problems.
Indeed, the task of inversion often either involves or is fully cast as a
solution of an optimization problem. In this light, the mere non-linear,
non-convex, and large-scale nature of many of these inversions gives rise to
some very challenging optimization problems. The inverse problem community has
long been developing various techniques for solving such optimization tasks.
However, other, seemingly disjoint communities, such as that of machine
learning, have developed, almost in parallel, interesting alternative methods
which might have stayed under the radar of the inverse problem community. In
this survey, we aim to change that. In doing so, we first discuss current
state-of-the-art optimization methods widely used in inverse problems. We then
survey recent related advances in addressing similar challenges in problems
faced by the machine learning community, and discuss their potential advantages
for solving inverse problems. By highlighting the similarities among the
optimization challenges faced by the inverse problem and the machine learning
communities, we hope that this survey can serve as a bridge in bringing
together these two communities and encourage cross fertilization of ideas.Comment: 13 page
An optimal subgradient algorithm for large-scale convex optimization in simple domains
This paper shows that the optimal subgradient algorithm, OSGA, proposed in
\cite{NeuO} can be used for solving structured large-scale convex constrained
optimization problems. Only first-order information is required, and the
optimal complexity bounds for both smooth and nonsmooth problems are attained.
More specifically, we consider two classes of problems: (i) a convex objective
with a simple closed convex domain, where the orthogonal projection on this
feasible domain is efficiently available; (ii) a convex objective with a simple
convex functional constraint. If we equip OSGA with an appropriate
prox-function, the OSGA subproblem can be solved either in a closed form or by
a simple iterative scheme, which is especially important for large-scale
problems. We report numerical results for some applications to show the
efficiency of the proposed scheme. A software package implementing OSGA for
above domains is available
On the filtering effect of iterative regularization algorithms for linear least-squares problems
Many real-world applications are addressed through a linear least-squares
problem formulation, whose solution is calculated by means of an iterative
approach. A huge amount of studies has been carried out in the optimization
field to provide the fastest methods for the reconstruction of the solution,
involving choices of adaptive parameters and scaling matrices. However, in
presence of an ill-conditioned model and real data, the need of a regularized
solution instead of the least-squares one changed the point of view in favour
of iterative algorithms able to combine a fast execution with a stable
behaviour with respect to the restoration error. In this paper we want to
analyze some classical and recent gradient approaches for the linear
least-squares problem by looking at their way of filtering the singular values,
showing in particular the effects of scaling matrices and non-negative
constraints in recovering the correct filters of the solution
Convergence analysis of a proximal Gauss-Newton method
An extension of the Gauss-Newton algorithm is proposed to find local
minimizers of penalized nonlinear least squares problems, under generalized
Lipschitz assumptions. Convergence results of local type are obtained, as well
as an estimate of the radius of the convergence ball. Some applications for
solving constrained nonlinear equations are discussed and the numerical
performance of the method is assessed on some significant test problems
- …