94 research outputs found
Observed Universality of Phase Transitions in High-Dimensional Geometry, with Implications for Modern Data Analysis and Signal Processing
We review connections between phase transitions in high-dimensional
combinatorial geometry and phase transitions occurring in modern
high-dimensional data analysis and signal processing. In data analysis, such
transitions arise as abrupt breakdown of linear model selection, robust data
fitting or compressed sensing reconstructions, when the complexity of the model
or the number of outliers increases beyond a threshold. In combinatorial
geometry these transitions appear as abrupt changes in the properties of face
counts of convex polytopes when the dimensions are varied. The thresholds in
these very different problems appear in the same critical locations after
appropriate calibration of variables.
These thresholds are important in each subject area: for linear modelling,
they place hard limits on the degree to which the now-ubiquitous
high-throughput data analysis can be successful; for robustness, they place
hard limits on the degree to which standard robust fitting methods can tolerate
outliers before breaking down; for compressed sensing, they define the sharp
boundary of the undersampling/sparsity tradeoff in undersampling theorems.
Existing derivations of phase transitions in combinatorial geometry assume
the underlying matrices have independent and identically distributed (iid)
Gaussian elements. In applications, however, it often seems that Gaussianity is
not required. We conducted an extensive computational experiment and formal
inferential analysis to test the hypothesis that these phase transitions are
{\it universal} across a range of underlying matrix ensembles. The experimental
results are consistent with an asymptotic large- universality across matrix
ensembles; finite-sample universality can be rejected.Comment: 47 pages, 24 figures, 10 table
Orthonormal Expansion l1-Minimization Algorithms for Compressed Sensing
Compressed sensing aims at reconstructing sparse signals from significantly
reduced number of samples, and a popular reconstruction approach is
-norm minimization. In this correspondence, a method called orthonormal
expansion is presented to reformulate the basis pursuit problem for noiseless
compressed sensing. Two algorithms are proposed based on convex optimization:
one exactly solves the problem and the other is a relaxed version of the first
one. The latter can be considered as a modified iterative soft thresholding
algorithm and is easy to implement. Numerical simulation shows that, in dealing
with noise-free measurements of sparse signals, the relaxed version is
accurate, fast and competitive to the recent state-of-the-art algorithms. Its
practical application is demonstrated in a more general case where signals of
interest are approximately sparse and measurements are contaminated with noise.Comment: 7 pages, 2 figures, 1 tabl
On Phase Transition of Compressed Sensing in the Complex Domain
The phase transition is a performance measure of the sparsity-undersampling
tradeoff in compressed sensing (CS). This letter reports our first observation
and evaluation of an empirical phase transition of the minimization
approach to the complex valued CS (CVCS), which is positioned well above the
known phase transition of the real valued CS in the phase plane. This result
can be considered as an extension of the existing phase transition theory of
the block-sparse CS (BSCS) based on the universality argument, since the CVCS
problem does not meet the condition required by the phase transition theory of
BSCS but its observed phase transition coincides with that of BSCS. Our result
is obtained by applying the recently developed ONE-L1 algorithms to the
empirical evaluation of the phase transition of CVCS.Comment: 4 pages, 3 figure
- β¦