3,520 research outputs found
Small-Deviation Inequalities for Sums of Random Matrices
Random matrices have played an important role in many fields including
machine learning, quantum information theory and optimization. One of the main
research focuses is on the deviation inequalities for eigenvalues of random
matrices. Although there are intensive studies on the large-deviation
inequalities for random matrices, only a few of works discuss the
small-deviation behavior of random matrices. In this paper, we present the
small-deviation inequalities for the largest eigenvalues of sums of random
matrices. Since the resulting inequalities are independent of the matrix
dimension, they are applicable to the high-dimensional and even the
infinite-dimensional cases
Dimension-free tail inequalities for sums of random matrices
We derive exponential tail inequalities for sums of random matrices with no
dependence on the explicit matrix dimensions. These are similar to the matrix
versions of the Chernoff bound and Bernstein inequality except with the
explicit matrix dimensions replaced by a trace quantity that can be small even
when the dimension is large or infinite. Some applications to principal
component analysis and approximate matrix multiplication are given to
illustrate the utility of the new bounds
Tail bounds for all eigenvalues of a sum of random matrices
This work introduces the minimax Laplace transform method, a modification of
the cumulant-based matrix Laplace transform method developed in "User-friendly
tail bounds for sums of random matrices" (arXiv:1004.4389v6) that yields both
upper and lower bounds on each eigenvalue of a sum of random self-adjoint
matrices. This machinery is used to derive eigenvalue analogues of the
classical Chernoff, Bennett, and Bernstein bounds.
Two examples demonstrate the efficacy of the minimax Laplace transform. The
first concerns the effects of column sparsification on the spectrum of a matrix
with orthonormal rows. Here, the behavior of the singular values can be
described in terms of coherence-like quantities. The second example addresses
the question of relative accuracy in the estimation of eigenvalues of the
covariance matrix of a random process. Standard results on the convergence of
sample covariance matrices provide bounds on the number of samples needed to
obtain relative accuracy in the spectral norm, but these results only guarantee
relative accuracy in the estimate of the maximum eigenvalue. The minimax
Laplace transform argument establishes that if the lowest eigenvalues decay
sufficiently fast, on the order of (K^2*r*log(p))/eps^2 samples, where K is the
condition number of an optimal rank-r approximation to C, are sufficient to
ensure that the dominant r eigenvalues of the covariance matrix of a N(0, C)
random vector are estimated to within a factor of 1+-eps with high probability.Comment: 20 pages, 1 figure, see also arXiv:1004.4389v
Rates of convergence for empirical spectral measures: a soft approach
Understanding the limiting behavior of eigenvalues of random matrices is the
central problem of random matrix theory. Classical limit results are known for
many models, and there has been significant recent progress in obtaining more
quantitative, non-asymptotic results. In this paper, we describe a systematic
approach to bounding rates of convergence and proving tail inequalities for the
empirical spectral measures of a wide variety of random matrix ensembles. We
illustrate the approach by proving asymptotically almost sure rates of
convergence of the empirical spectral measure in the following ensembles:
Wigner matrices, Wishart matrices, Haar-distributed matrices from the compact
classical groups, powers of Haar matrices, randomized sums and random
compressions of Hermitian matrices, a random matrix model for the Hamiltonians
of quantum spin glasses, and finally the complex Ginibre ensemble. Many of the
results appeared previously and are being collected and described here as
illustrations of the general method; however, some details (particularly in the
Wigner and Wishart cases) are new.
Our approach makes use of techniques from probability in Banach spaces, in
particular concentration of measure and bounds for suprema of stochastic
processes, in combination with more classical tools from matrix analysis,
approximation theory, and Fourier analysis. It is highly flexible, as evidenced
by the broad list of examples. It is moreover based largely on "soft" methods,
and involves little hard analysis
Moving Beyond Sub-Gaussianity in High-Dimensional Statistics: Applications in Covariance Estimation and Linear Regression
Concentration inequalities form an essential toolkit in the study of high
dimensional (HD) statistical methods. Most of the relevant statistics
literature in this regard is based on sub-Gaussian or sub-exponential tail
assumptions. In this paper, we first bring together various probabilistic
inequalities for sums of independent random variables under much weaker
exponential type (namely sub-Weibull) tail assumptions. These results extract a
part sub-Gaussian tail behavior in finite samples, matching the asymptotics
governed by the central limit theorem, and are compactly represented in terms
of a new Orlicz quasi-norm - the Generalized Bernstein-Orlicz norm - that
typifies such tail behaviors.
We illustrate the usefulness of these inequalities through the analysis of
four fundamental problems in HD statistics. In the first two problems, we study
the rate of convergence of the sample covariance matrix in terms of the maximum
elementwise norm and the maximum k-sub-matrix operator norm which are key
quantities of interest in bootstrap, HD covariance matrix estimation and HD
inference. The third example concerns the restricted eigenvalue condition,
required in HD linear regression, which we verify for all sub-Weibull random
vectors through a unified analysis, and also prove a more general result
related to restricted strong convexity in the process. In the final example, we
consider the Lasso estimator for linear regression and establish its rate of
convergence under much weaker than usual tail assumptions (on the errors as
well as the covariates), while also allowing for misspecified models and both
fixed and random design. To our knowledge, these are the first such results for
Lasso obtained in this generality. The common feature in all our results over
all the examples is that the convergence rates under most exponential tails
match the usual ones under sub-Gaussian assumptions.Comment: 64 pages; Revised version (discussions added and some results
modified in Section 4, minor changes made throughout
- …