Geometric methods on low-rank matrix and tensor manifolds

In this chapter we present numerical methods for low-rank matrix and tensor problems that explicitly make use of the geometry of rank constrained matrix and tensor spaces. We focus on two types of problems: The first are optimization problems, like matrix and tensor completion, solving linear systems and eigenvalue problems. Such problems can be solved by numerical optimization for manifolds, called Riemannian optimization methods. We will explain the basic elements of differential geometry in order to apply such methods efficiently to rank constrained matrix and tensor spaces. The second type of problem is ordinary differential equations, defined on matrix and tensor spaces. We show how their solution can be approximated by the dynamical low-rank principle, and discuss several numerical integrators that rely in an essential way on geometric properties that are characteristic to sets of low rank matrices and tensors

The Extrinsic Geometry of Dynamical Systems Tracking Nonlinear Matrix Projections

A generalization of the concepts of extrinsic curvature and Weingarten endomorphism is introduced to study a class of nonlinear maps over embedded matrix manifolds. These (nonlinear) oblique projections generalize (nonlinear) orthogonal projections, i.e., applications mapping a point to its closest neighbor on a matrix manifold. Examples of such maps include the truncated SVD, the polar decomposition, and functions mapping symmetric and nonsymmetric matrices to their linear eigenprojectors. This paper specifically investigates how oblique projections provide their image manifolds with a canonical extrinsic differential structure, over which a generalization of the Weingarten identity is available. By diagonalization of the corresponding Weingarten endomorphism, the manifold principal curvatures are explicitly characterized, which then enables us to (i) derive explicit formulas for the differential of oblique projections and (ii) study the global stability of a governing generic ordinary differential equation (ODE) computing their values. This methodology, exploited for the truncated SVD in [Feppon and Lermusiaux, SIAM J. Matrix Anal. Appl., 39 (2018), pp. 510–538], is generalized to non-Euclidean settings and applied to the four other maps mentioned above and their image manifolds: respectively, the Stiefel, the isospectral, and the Grassmann manifolds and the manifold of fixed rank (nonorthogonal) linear projectors. In all cases studied, the oblique projection of a target matrix is surprisingly the unique stable equilibrium point of the above gradient flow. Three numerical applications concerned with ODEs tracking dominant eigenspaces involving possibly multiple eigenvalues finally showcase the results.United States. Office of Naval Research (grants N00014-14-1-0476 (Science of Autonomy–LEARNS))United States. Office of Naval Research (grants N00014-14-1-0725 (Bays-DA)