32,038 research outputs found
The Sparsity Gap: Uncertainty Principles Proportional to Dimension
In an incoherent dictionary, most signals that admit a sparse representation
admit a unique sparse representation. In other words, there is no way to
express the signal without using strictly more atoms. This work demonstrates
that sparse signals typically enjoy a higher privilege: each nonoptimal
representation of the signal requires far more atoms than the sparsest
representation-unless it contains many of the same atoms as the sparsest
representation. One impact of this finding is to confer a certain degree of
legitimacy on the particular atoms that appear in a sparse representation. This
result can also be viewed as an uncertainty principle for random sparse signals
over an incoherent dictionary.Comment: 6 pages. To appear in the Proceedings of the 44th Ann. IEEE Conf. on
Information Sciences and System
The random paving property for uniformly bounded matrices
This note presents a new proof of an important result due to Bourgain and
Tzafriri that provides a partial solution to the Kadison--Singer problem. The
result shows that every unit-norm matrix whose entries are relatively small in
comparison with its dimension can be paved by a partition of constant size.
That is, the coordinates can be partitioned into a constant number of blocks so
that the restriction of the matrix to each block of coordinates has norm less
than one half. The original proof of Bourgain and Tzafriri involves a long,
delicate calculation. The new proof relies on the systematic use of
symmetrization and (noncommutative) Khintchine inequalities to estimate the
norms of some random matrices.Comment: 12 pages; v2 with cosmetic changes; v3 with corrections to Prop. 4;
v4 with minor changes to text; v5 with correction to discussion of
noncommutative Khintchine inequality; v6 with slight improvement to main
theore
The Expected Norm of a Sum of Independent Random Matrices: An Elementary Approach
In contemporary applied and computational mathematics, a frequent challenge
is to bound the expectation of the spectral norm of a sum of independent random
matrices. This quantity is controlled by the norm of the expected square of the
random matrix and the expectation of the maximum squared norm achieved by one
of the summands; there is also a weak dependence on the dimension of the random
matrix. The purpose of this paper is to give a complete, elementary proof of
this important, but underappreciated, inequality.Comment: 20 page
Freedman’s Inequality for Matrix Martingales
Freedman's inequality is a martingale counterpart to Bernstein's inequality. This result shows that the large-deviation behavior of a martingale is controlled by the predictable quadratic variation and a uniform upper bound for the martingale difference sequence. Oliveira has recently established a natural extension of Freedman's inequality that provides tail bounds for the maximum singular value of a matrix-valued martingale. This note describes a different proof of the matrix Freedman inequality that depends on a deep theorem of Lieb from matrix analysis. This argument delivers sharp constants in the matrix Freedman inequality, and it also yields tail bounds for other types of matrix martingales. The new techniques are adapted from recent work by the present author
Greed is good: algorithmic results for sparse approximation
This article presents new results on using a greedy algorithm, orthogonal matching pursuit (OMP), to solve the sparse approximation problem over redundant dictionaries. It provides a sufficient condition under which both OMP and Donoho's basis pursuit (BP) paradigm can recover the optimal representation of an exactly sparse signal. It leverages this theory to show that both OMP and BP succeed for every sparse input signal from a wide class of dictionaries. These quasi-incoherent dictionaries offer a natural generalization of incoherent dictionaries, and the cumulative coherence function is introduced to quantify the level of incoherence. This analysis unifies all the recent results on BP and extends them to OMP. Furthermore, the paper develops a sufficient condition under which OMP can identify atoms from an optimal approximation of a nonsparse signal. From there, it argues that OMP is an approximation algorithm for the sparse problem over a quasi-incoherent dictionary. That is, for every input signal, OMP calculates a sparse approximant whose error is only a small factor worse than the minimal error that can be attained with the same number of terms
- …