Relative perturbation theory: IV. sin 2θ theorems

AbstractThe double angle theorems of Davis and Kahan bound the change in an invariant subspace when a Hermitian matrix A is subject to an additive perturbation A→Ã=A+ΔA. This paper supplies analogous results when A is subject to a congruential, or multiplicative, perturbation A→Ã=D*AD. The relative gaps that appear in the bounds involve the spectrum of only one matrix, either A or Ã, in contrast to the gaps that appear in the single angle bounds.The double angle theorems do not directly bound the difference between the old invariant subspace S and the new one S̃ but instead bound the difference between S̃ and its reflection JS̃ where the mirror is S and J reverses S⊥, the orthogonal complement of S. The double angle bounds are proportional to the departure from the identity and from orthogonality of the matrix D̃=defD−1JDJ. Note that D̃ is invariant under the transformation D→D/αforα≠0, whereas the single angle theorems give bounds proportional to D's departure from the identity and from orthogonality.The corresponding results for the singular value problem when a (nonsquare) matrix B is perturbed to B̃=D*1BD2 are also presented

Relative perturbation theory for diagonally dominant matrices

In this paper, strong relative perturbation bounds are developed for a number of linear algebra problems involving diagonally dominant matrices. The key point is to parameterize diagonally dominant matrices using their off-diagonal entries and diagonally dominant parts and to consider small relative componentwise perturbations of these parameters. This allows us to obtain new relative perturbation bounds for the inverse, the solution to linear systems, the symmetric indefinite eigenvalue problem, the singular value problem, and the nonsymmetric eigenvalue problem. These bounds are much stronger than traditional perturbation results, since they are independent of either the standard condition number or the magnitude of eigenvalues/singular values. Together with previously derived perturbation bounds for the LDU factorization and the symmetric positive definite eigenvalue problem, this paper presents a complete and detailed account of relative structured perturbation theory for diagonally dominant matrices.This research was partially supported by the Ministerio de Economía y Competitividad of Spain under grant MTM2012-32542.Publicad

A Method of Moments for Mixture Models and Hidden Markov Models

Mixture models are a fundamental tool in applied statistics and machine learning for treating data taken from multiple subpopulations. The current practice for estimating the parameters of such models relies on local search heuristics (e.g., the EM algorithm) which are prone to failure, and existing consistent methods are unfavorable due to their high computational and sample complexity which typically scale exponentially with the number of mixture components. This work develops an efficient method of moments approach to parameter estimation for a broad class of high-dimensional mixture models with many components, including multi-view mixtures of Gaussians (such as mixtures of axis-aligned Gaussians) and hidden Markov models. The new method leads to rigorous unsupervised learning results for mixture models that were not achieved by previous works; and, because of its simplicity, it offers a viable alternative to EM for practical deployment

Fourier PCA and Robust Tensor Decomposition

Fourier PCA is Principal Component Analysis of a matrix obtained from higher order derivatives of the logarithm of the Fourier transform of a distribution.We make this method algorithmic by developing a tensor decomposition method for a pair of tensors sharing the same vectors in rank-$1$ decompositions. Our main application is the first provably polynomial-time algorithm for underdetermined ICA, i.e., learning an $n \times m$ matrix $A$ from observations $y=Ax$ where $x$ is drawn from an unknown product distribution with arbitrary non-Gaussian components. The number of component distributions $m$ can be arbitrarily higher than the dimension $n$ and the columns of $A$ only need to satisfy a natural and efficiently verifiable nondegeneracy condition. As a second application, we give an alternative algorithm for learning mixtures of spherical Gaussians with linearly independent means. These results also hold in the presence of Gaussian noise.Comment: Extensively revised; details added; minor errors corrected; exposition improve

RELATIVE PERTURBATION THEORY FOR DIAGONALLY DOMINANT MATRICES

Diagonally dominant matrices arise in many applications. In this work, we exploit the structure of diagonally dominant matrices to provide sharp entrywise relative perturbation bounds. We first generalize the results of Dopico and Koev to provide relative perturbation bounds for the LDU factorization with a well conditioned L factor. We then establish relative perturbation bounds for the inverse that are entrywise and independent of the condition number. This allows us to also present relative perturbation bounds for the linear system Ax=b that are independent of the condition number. Lastly, we continue the work of Ye to provide relative perturbation bounds for the eigenvalues of symmetric indefinite matrices and non-symmetric matrices

Moment instabilities in multidimensional systems with noise

We present a systematic study of moment evolution in multidimensional stochastic difference systems, focusing on characterizing systems whose low-order moments diverge in the neighborhood of a stable fixed point. We consider systems with a simple, dominant eigenvalue and stationary, white noise. When the noise is small, we obtain general expressions for the approximate asymptotic distribution and moment Lyapunov exponents. In the case of larger noise, the second moment is calculated using a different approach, which gives an exact result for some types of noise. We analyze the dependence of the moments on the system's dimension, relevant system properties, the form of the noise, and the magnitude of the noise. We determine a critical value for noise strength, as a function of the unperturbed system's convergence rate, above which the second moment diverges and large fluctuations are likely. Analytical results are validated by numerical simulations. We show that our results cannot be extended to the continuous time limit except in certain special cases.Comment: 21 pages, 15 figure