Relative perturbation theory: IV. sin 2θ theorems

AbstractThe double angle theorems of Davis and Kahan bound the change in an invariant subspace when a Hermitian matrix A is subject to an additive perturbation A→Ã=A+ΔA. This paper supplies analogous results when A is subject to a congruential, or multiplicative, perturbation A→Ã=D*AD. The relative gaps that appear in the bounds involve the spectrum of only one matrix, either A or Ã, in contrast to the gaps that appear in the single angle bounds.The double angle theorems do not directly bound the difference between the old invariant subspace S and the new one S̃ but instead bound the difference between S̃ and its reflection JS̃ where the mirror is S and J reverses S⊥, the orthogonal complement of S. The double angle bounds are proportional to the departure from the identity and from orthogonality of the matrix D̃=defD−1JDJ. Note that D̃ is invariant under the transformation D→D/αforα≠0, whereas the single angle theorems give bounds proportional to D's departure from the identity and from orthogonality.The corresponding results for the singular value problem when a (nonsquare) matrix B is perturbed to B̃=D*1BD2 are also presented

Nearly Optimal Stochastic Approximation for Online Principal Subspace Estimation

Processing streaming data as they arrive is often necessary for high dimensional data analysis. In this paper, we analyse the convergence of a subspace online PCA iteration, as a followup of the recent work of Li, Wang, Liu, and Zhang [Math. Program., Ser. B, DOI 10.1007/s10107-017-1182-z] who considered the case for the most significant principal component only, i.e., a single vector. Under the sub-Gaussian assumption, we obtain a finite-sample error bound that closely matches the minimax information lower bound of Vu and Lei [Ann. Statist. 41:6 (2013), 2905-2947].Comment: 37 page