Global rates of convergence for nonconvex optimization on manifolds

We consider the minimization of a cost function $f$ on a manifold $M$ using Riemannian gradient descent and Riemannian trust regions (RTR). We focus on satisfying necessary optimality conditions within a tolerance $\varepsilon$. Specifically, we show that, under Lipschitz-type assumptions on the pullbacks of $f$ to the tangent spaces of $M$, both of these algorithms produce points with Riemannian gradient smaller than $\varepsilon$ in $O(1/\varepsilon^2)$ iterations. Furthermore, RTR returns a point where also the Riemannian Hessian's least eigenvalue is larger than $-\varepsilon$ in $O(1/\varepsilon^3)$ iterations. There are no assumptions on initialization. The rates match their (sharp) unconstrained counterparts as a function of the accuracy $\varepsilon$ (up to constants) and hence are sharp in that sense. These are the first deterministic results for global rates of convergence to approximate first- and second-order Karush-Kuhn-Tucker points on manifolds. They apply in particular for optimization constrained to compact submanifolds of $\mathbb{R}^n$, under simpler assumptions.Comment: 33 pages, IMA Journal of Numerical Analysis, 201

A Riemannian Trust Region Method for the Canonical Tensor Rank Approximation Problem

The canonical tensor rank approximation problem (TAP) consists of approximating a real-valued tensor by one of low canonical rank, which is a challenging non-linear, non-convex, constrained optimization problem, where the constraint set forms a non-smooth semi-algebraic set. We introduce a Riemannian Gauss-Newton method with trust region for solving small-scale, dense TAPs. The novelty of our approach is threefold. First, we parametrize the constraint set as the Cartesian product of Segre manifolds, hereby formulating the TAP as a Riemannian optimization problem, and we argue why this parametrization is among the theoretically best possible. Second, an original ST-HOSVD-based retraction operator is proposed. Third, we introduce a hot restart mechanism that efficiently detects when the optimization process is tending to an ill-conditioned tensor rank decomposition and which often yields a quick escape path from such spurious decompositions. Numerical experiments show improvements of up to three orders of magnitude in terms of the expected time to compute a successful solution over existing state-of-the-art methods

International Conference on Continuous Optimization (ICCOPT) 2019 Conference Book

The Sixth International Conference on Continuous Optimization took place on the campus of the Technical University of Berlin, August 3-8, 2019. The ICCOPT is a flagship conference of the Mathematical Optimization Society (MOS), organized every three years. ICCOPT 2019 was hosted by the Weierstrass Institute for Applied Analysis and Stochastics (WIAS) Berlin. It included a Summer School and a Conference with a series of plenary and semi-plenary talks, organized and contributed sessions, and poster sessions. This book comprises the full conference program. It contains, in particular, the scientific program in survey style as well as with all details, and information on the social program, the venue, special meetings, and more

On solving trust-region and other regularised subproblems in optimization

The solution of trust-region and regularisation subproblems which arise in unconstrained optimization is considered. Building on the pioneering work of Gay, Mor´e and Sorensen, methods which obtain the solution of a sequence of parametrized linear systems by factorization are used. Enhancements using high-order polynomial approximation and inverse iteration ensure that the resulting method is both globally and asymptotically at least superlinearly convergent in all cases, including in the notorious hard case. Numerical experiments validate the effectiveness of our approach. The resulting software is available as packages TRS and RQS as part of the GALAHAD optimization library, and is especially designed for large-scale problems

Variational Methods in Shape Space

This dissertation deals with the application of variational methods in spaces of geometric shapes. In particular, the treated topics include shape averaging, principal component analysis in shape space, computation of geodesic paths in shape space, as well as shape optimisation. Chapter 1 provides a brief overview over the employed models of shape space. Geometric shapes are identified with two- or three-dimensional, deformable objects. Deformations will be described via physical models; in particular, the objects will be interpreted as consisting of either a hyperelastic solid or a viscous liquid material. Furthermore, the description of shapes via phase fields or level sets is briefly introduced. Chapter 2 reviews different and related approaches to shape space modelling. References to related topics in image segmentation and registration are also provided. Finally, the relevant shape optimisation literature is introduced. Chapter 3 recapitulates the employed concepts from continuum mechanics and phase field modelling and states basic theoretical results needed for the later analysis. Chapter 4 addresses the computation of shape averages, based on a hyperelastic notion of shape dissimilarity: The dissimilarity between two shapes is measured as the minimum deformation energy required to deform the first into the second shape. A corresponding phase-field model is introduced, analysed, and finally implemented numerically via finite elements. A principal component analysis of shapes, which is consistent with the previously introduced average, is considered in Chapter 5. Elastic boundary stresses on the average shape are used as representatives of the input shapes in a linear vector space. On these linear representatives, a standard principal component analysis can be performed, where the employed covariance metric should be properly chosen to depend on the input shapes. Chapter 6 interprets shapes as belonging to objects made of a viscous liquid and correspondingly defines geodesic paths between shapes. The energy of a path is given as the total physical dissipation during the deformation of an object along the path. A rigid body motion invariant time discretisation is achieved by approximating the dissipation along a path segment by the deformation energy of a small solid deformation. The numerical implementation is based on level sets. Chapter 7 is concerned with the optimisation of the geometry and topology of solid structures that are subject to a mechanical load. Given the load configuration, the structure rigidity, its volume, and its surface area shall be optimally balanced. A phase field model is devised and analysed for this purpose. In this context, the use of nonlinear elasticity allows to detect buckling phenomena which would be ignored in linearised elasticity