A globally convergent flow for computing the best low rank approximation of a matrix

We work in the space of $m$-by-$n$ real matrices with the Frobenius inner product. Consider the following Problem: Given an m-by-n real matrix A and a positive integer k, find the m-by-n matrix with rank k that is closest to A. I discuss a rank-preserving differential equation (d.e.) which solves this problem. If X(t) is a solution of this d.e., then the distance between $X(t)$ and $A$ decreases as t increases; this distance function is a Lyapunov function for the d.e. If $A$ has distinct positive singular values (which is a generic condition) then this d.e. has only one stable equilibrium point. The other equilibrium points are finite in number and unstable. In other words, the basin of attraction of the stable equilibrium point on the manifold of matrices with rank $k$ consists of almost all matrices. This special equilibrium point is the solution of the given problem. Usually constrained optimization problems have many local minimums (most of which are undesirable). So the constrained optimization problem considered here is very special

Zero-Preserving Iso-spectral Flows Based on Parallel Sums

Driessel ["Computing canonical forms using flows", Linear Algebra and Its Applications 2004] introduced the notion of quasi-projection onto the range of a linear transformation from one inner product space into another inner product space. Here we introduce the notion of quasi-projection onto the intersection of the ranges of two linear transformations from two inner product spaces into a third inner product space. As an application, we design a new family of iso-spectral flows on the space of symmetric matrices that preserves zero patterns. We discuss the equilibrium points of these flows. We conjecture that these flows generically converge to diagonal matrices. We perform some numerical experiments with these flows which support this conjecture. We also compare our zero preserving flows with the Toda flow

On computing canonical forms using flows

AbstractLet G be a Lie group acting on a vector space V. I say that a vector field u:V→V is orbital if for all p in V, u(p) is tangent to the orbit of p at p. Now let the vector space V and the vector space tangent to G at the identity have inner products. In this setting I define a simple map (which I call quasi-projection) which transforms any vector field on V into an orbital one. I use the quasi-projection map to define flows which compute canonical forms

The projected gradient method for least squares matrix approximations with spectral constraints

Abstract. The problems of computing least squares approximations for various types of real and symmetric matrices subject to spectral constraints share a common structure. This paper describes a general procedure in using the projected gradient method. It is shown that the projected gradient of the objective functionon the manifold ofconstraints usuallycanbe formulated explicitly. This gives rise to the construction of a descent flow that can be followed numerically. The explicit form also facilitates the computation of the second-order optimality conditions. Examples of applications are discussed. With slight modifications, the procedure can be extended to solve least squares problems for general matrices subject to singular-value constraints. Key words, least squares approximation, projected gradient, spectral constraints, singular-value constraints AMS(MOS) subject classifications. 65F15, 49D10 1. Introduction. Let S(n) denote the subspace of all symmetric matrices in R"x". Given a matrix A S(n), we define an isospectral surface M(A) ofA by (1) M(A):={X R" " [X Q’AQ, Q O(n)} where O(n) is the collection of all orthogonal matrices in R"". Let represent eithe