2 research outputs found
Nonlinear Residual Minimization by Iteratively Reweighted Least Squares
We address the numerical solution of minimal norm residuals of {\it
nonlinear} equations in finite dimensions. We take inspiration from the problem
of finding a sparse vector solution by using greedy algorithms based on
iterative residual minimizations in the -norm, for .
Due to the mild smoothness of the problem, especially for , we develop
and analyze a generalized version of Iteratively Reweighted Least Squares
(IRLS). This simple and efficient algorithm performs the solution of
optimization problems involving non-quadratic possibly non-convex and
non-smooth cost functions, which can be transformed into a sequence of common
least squares problems, which can be tackled more efficiently.While its
analysis has been developed in many contexts when the model equation is {\it
linear}, no results are provided in the {\it nonlinear} case. We address the
convergence and the rate of error decay of IRLS for nonlinear problems. The
convergence analysis is based on its reformulation as an alternating
minimization of an energy functional, whose variables are the competitors to
solutions of the intermediate reweighted least squares problems. Under specific
conditions of coercivity and local convexity, we are able to show convergence
of IRLS to minimizers of the nonlinear residual problem. For the case where we
are lacking local convexity, we propose an appropriate convexification.. To
illustrate the theoretical results we conclude the paper with several numerical
experiments. We compare IRLS with standard Matlab functions for an easily
presentable example and numerically validate our theoretical results in the
more complicated framework of phase retrieval problems. Finally we examine the
recovery capability of the algorithm in the context of data corrupted by
impulsive noise where the sparsification of the residual is desired.Comment: 37 pages. arXiv admin note: text overlap with arXiv:0807.0575 by
other author
Inexact Alternating Optimization for Phase Retrieval In the Presence of Outliers
Phase retrieval has been mainly considered in the presence of Gaussian noise.
However, the performance of the algorithms proposed under the Gaussian noise
model severely degrades when grossly corrupted data, i.e., outliers, exist.
This paper investigates techniques for phase retrieval in the presence of
heavy-tailed noise -- which is considered a better model for situations where
outliers exist. An -norm () based estimator is proposed for
fending against such noise, and two-block inexact alternating optimization is
proposed as the algorithmic framework to tackle the resulting optimization
problem. Two specific algorithms are devised by exploring different local
approximations within this framework. Interestingly, the core conditional
minimization steps can be interpreted as iteratively reweighted least squares
and gradient descent. Convergence properties of the algorithms are discussed,
and the Cram\'er-Rao bound (CRB) is derived. Simulations demonstrate that the
proposed algorithms approach the CRB and outperform state-of-the-art algorithms
in heavy-tailed noise.Comment: 23 pages, 16 figure