3,458 research outputs found
Observed Universality of Phase Transitions in High-Dimensional Geometry, with Implications for Modern Data Analysis and Signal Processing
We review connections between phase transitions in high-dimensional
combinatorial geometry and phase transitions occurring in modern
high-dimensional data analysis and signal processing. In data analysis, such
transitions arise as abrupt breakdown of linear model selection, robust data
fitting or compressed sensing reconstructions, when the complexity of the model
or the number of outliers increases beyond a threshold. In combinatorial
geometry these transitions appear as abrupt changes in the properties of face
counts of convex polytopes when the dimensions are varied. The thresholds in
these very different problems appear in the same critical locations after
appropriate calibration of variables.
These thresholds are important in each subject area: for linear modelling,
they place hard limits on the degree to which the now-ubiquitous
high-throughput data analysis can be successful; for robustness, they place
hard limits on the degree to which standard robust fitting methods can tolerate
outliers before breaking down; for compressed sensing, they define the sharp
boundary of the undersampling/sparsity tradeoff in undersampling theorems.
Existing derivations of phase transitions in combinatorial geometry assume
the underlying matrices have independent and identically distributed (iid)
Gaussian elements. In applications, however, it often seems that Gaussianity is
not required. We conducted an extensive computational experiment and formal
inferential analysis to test the hypothesis that these phase transitions are
{\it universal} across a range of underlying matrix ensembles. The experimental
results are consistent with an asymptotic large- universality across matrix
ensembles; finite-sample universality can be rejected.Comment: 47 pages, 24 figures, 10 table
Fractals in the Nervous System: conceptual Implications for Theoretical Neuroscience
This essay is presented with two principal objectives in mind: first, to
document the prevalence of fractals at all levels of the nervous system, giving
credence to the notion of their functional relevance; and second, to draw
attention to the as yet still unresolved issues of the detailed relationships
among power law scaling, self-similarity, and self-organized criticality. As
regards criticality, I will document that it has become a pivotal reference
point in Neurodynamics. Furthermore, I will emphasize the not yet fully
appreciated significance of allometric control processes. For dynamic fractals,
I will assemble reasons for attributing to them the capacity to adapt task
execution to contextual changes across a range of scales. The final Section
consists of general reflections on the implications of the reviewed data, and
identifies what appear to be issues of fundamental importance for future
research in the rapidly evolving topic of this review
The generalized Lasso with non-linear observations
We study the problem of signal estimation from non-linear observations when
the signal belongs to a low-dimensional set buried in a high-dimensional space.
A rough heuristic often used in practice postulates that non-linear
observations may be treated as noisy linear observations, and thus the signal
may be estimated using the generalized Lasso. This is appealing because of the
abundance of efficient, specialized solvers for this program. Just as noise may
be diminished by projecting onto the lower dimensional space, the error from
modeling non-linear observations with linear observations will be greatly
reduced when using the signal structure in the reconstruction. We allow general
signal structure, only assuming that the signal belongs to some set K in R^n.
We consider the single-index model of non-linearity. Our theory allows the
non-linearity to be discontinuous, not one-to-one and even unknown. We assume a
random Gaussian model for the measurement matrix, but allow the rows to have an
unknown covariance matrix. As special cases of our results, we recover
near-optimal theory for noisy linear observations, and also give the first
theoretical accuracy guarantee for 1-bit compressed sensing with unknown
covariance matrix of the measurement vectors.Comment: 21 page
Determination of Nonlinear Genetic Architecture using Compressed Sensing
We introduce a statistical method that can reconstruct nonlinear genetic
models (i.e., including epistasis, or gene-gene interactions) from
phenotype-genotype (GWAS) data. The computational and data resource
requirements are similar to those necessary for reconstruction of linear
genetic models (or identification of gene-trait associations), assuming a
condition of generalized sparsity, which limits the total number of gene-gene
interactions. An example of a sparse nonlinear model is one in which a typical
locus interacts with several or even many others, but only a small subset of
all possible interactions exist. It seems plausible that most genetic
architectures fall in this category. Our method uses a generalization of
compressed sensing (L1-penalized regression) applied to nonlinear functions of
the sensing matrix. We give theoretical arguments suggesting that the method is
nearly optimal in performance, and demonstrate its effectiveness on broad
classes of nonlinear genetic models using both real and simulated human
genomes.Comment: 20 pages, 8 figures. arXiv admin note: text overlap with
arXiv:1408.342
Convexity in source separation: Models, geometry, and algorithms
Source separation or demixing is the process of extracting multiple
components entangled within a signal. Contemporary signal processing presents a
host of difficult source separation problems, from interference cancellation to
background subtraction, blind deconvolution, and even dictionary learning.
Despite the recent progress in each of these applications, advances in
high-throughput sensor technology place demixing algorithms under pressure to
accommodate extremely high-dimensional signals, separate an ever larger number
of sources, and cope with more sophisticated signal and mixing models. These
difficulties are exacerbated by the need for real-time action in automated
decision-making systems.
Recent advances in convex optimization provide a simple framework for
efficiently solving numerous difficult demixing problems. This article provides
an overview of the emerging field, explains the theory that governs the
underlying procedures, and surveys algorithms that solve them efficiently. We
aim to equip practitioners with a toolkit for constructing their own demixing
algorithms that work, as well as concrete intuition for why they work
- …