122 research outputs found
Newton acceleration on manifolds identified by proximal-gradient methods
Proximal methods are known to identify the underlying substructure of
nonsmooth optimization problems. Even more, in many interesting situations, the
output of a proximity operator comes with its structure at no additional cost,
and convergence is improved once it matches the structure of a minimizer.
However, it is impossible in general to know whether the current structure is
final or not; such highly valuable information has to be exploited adaptively.
To do so, we place ourselves in the case where a proximal gradient method can
identify manifolds of differentiability of the nonsmooth objective. Leveraging
this manifold identification, we show that Riemannian Newton-like methods can
be intertwined with the proximal gradient steps to drastically boost the
convergence. We prove the superlinear convergence of the algorithm when solving
some nondegenerated nonsmooth nonconvex optimization problems. We provide
numerical illustrations on optimization problems regularized by -norm
or trace-norm
A Riemannian View on Shape Optimization
Shape optimization based on the shape calculus is numerically mostly
performed by means of steepest descent methods. This paper provides a novel
framework to analyze shape-Newton optimization methods by exploiting a
Riemannian perspective. A Riemannian shape Hessian is defined yielding often
sought properties like symmetry and quadratic convergence for Newton
optimization methods.Comment: 15 pages, 1 figure, 1 table. Forschungsbericht / Universit\"at Trier,
Mathematik, Informatik 2012,
Recursive Importance Sketching for Rank Constrained Least Squares: Algorithms and High-order Convergence
In this paper, we propose a new {\it \underline{R}ecursive} {\it
\underline{I}mportance} {\it \underline{S}ketching} algorithm for {\it
\underline{R}ank} constrained least squares {\it \underline{O}ptimization}
(RISRO). As its name suggests, the algorithm is based on a new sketching
framework, recursive importance sketching. Several existing algorithms in the
literature can be reinterpreted under the new sketching framework and RISRO
offers clear advantages over them. RISRO is easy to implement and
computationally efficient, where the core procedure in each iteration is only
solving a dimension reduced least squares problem. Different from numerous
existing algorithms with locally geometric convergence rate, we establish the
local quadratic-linear and quadratic rate of convergence for RISRO under some
mild conditions. In addition, we discover a deep connection of RISRO to
Riemannian manifold optimization on fixed rank matrices. The effectiveness of
RISRO is demonstrated in two applications in machine learning and statistics:
low-rank matrix trace regression and phase retrieval. Simulation studies
demonstrate the superior numerical performance of RISRO
- …