620 research outputs found
Minimax Optimal Sparse Signal Recovery with Poisson Statistics
We are motivated by problems that arise in a number of applications such as
Online Marketing and Explosives detection, where the observations are usually
modeled using Poisson statistics. We model each observation as a Poisson random
variable whose mean is a sparse linear superposition of known patterns. Unlike
many conventional problems observations here are not identically distributed
since they are associated with different sensing modalities. We analyze the
performance of a Maximum Likelihood (ML) decoder, which for our Poisson setting
involves a non-linear optimization but yet is computationally tractable. We
derive fundamental sample complexity bounds for sparse recovery when the
measurements are contaminated with Poisson noise. In contrast to the
least-squares linear regression setting with Gaussian noise, we observe that in
addition to sparsity, the scale of the parameters also fundamentally impacts
error in the Poisson setting. We show tightness of our upper bounds
both theoretically and experimentally. In particular, we derive a minimax
matching lower bound on the mean-squared error and show that our constrained ML
decoder is minimax optimal for this regime.Comment: Submitted to IEEE Trans. on Signal Processing. arXiv admin note:
substantial text overlap with arXiv:1307.466
Minimax Lower Bounds for Noisy Matrix Completion Under Sparse Factor Models
This paper examines fundamental error characteristics for a general class of
matrix completion problems, where the matrix of interest is a product of two a
priori unknown matrices, one of which is sparse, and the observations are
noisy. Our main contributions come in the form of minimax lower bounds for the
expected per-element squared error for this problem under under several common
noise models. Specifically, we analyze scenarios where the corruptions are
characterized by additive Gaussian noise or additive heavier-tailed (Laplace)
noise, Poisson-distributed observations, and highly-quantized (e.g., one-bit)
observations, as instances of our general result. Our results establish that
the error bounds derived in (Soni et al., 2016) for complexity-regularized
maximum likelihood estimators achieve, up to multiplicative constants and
logarithmic factors, the minimax error rates in each of these noise scenarios,
provided that the nominal number of observations is large enough, and the
sparse factor has (on an average) at least one non-zero per column.Comment: 21 page
Minimax Optimal Rates for Poisson Inverse Problems with Physical Constraints
This paper considers fundamental limits for solving sparse inverse problems
in the presence of Poisson noise with physical constraints. Such problems arise
in a variety of applications, including photon-limited imaging systems based on
compressed sensing. Most prior theoretical results in compressed sensing and
related inverse problems apply to idealized settings where the noise is i.i.d.,
and do not account for signal-dependent noise and physical sensing constraints.
Prior results on Poisson compressed sensing with signal-dependent noise and
physical constraints provided upper bounds on mean squared error performance
for a specific class of estimators. However, it was unknown whether those
bounds were tight or if other estimators could achieve significantly better
performance. This work provides minimax lower bounds on mean-squared error for
sparse Poisson inverse problems under physical constraints. Our lower bounds
are complemented by minimax upper bounds. Our upper and lower bounds reveal
that due to the interplay between the Poisson noise model, the sparsity
constraint and the physical constraints: (i) the mean-squared error does not
depend on the sample size other than to ensure the sensing matrix satisfies
RIP-like conditions and the intensity of the input signal plays a critical
role; and (ii) the mean-squared error has two distinct regimes, a low-intensity
and a high-intensity regime and the transition point from the low-intensity to
high-intensity regime depends on the input signal . In the low-intensity
regime the mean-squared error is independent of while in the high-intensity
regime, the mean-squared error scales as , where is the
sparsity level, is the number of pixels or parameters and is the signal
intensity.Comment: 30 pages, 5 figure
A data-dependent weighted LASSO under Poisson noise
Sparse linear inverse problems appear in a variety of settings, but often the
noise contaminating observations cannot accurately be described as bounded by
or arising from a Gaussian distribution. Poisson observations in particular are
a feature of several real-world applications. Previous work on sparse Poisson
inverse problems encountered several limiting technical hurdles. This paper
describes a novel alternative analysis approach for sparse Poisson inverse
problems that (a) sidesteps the technical challenges in previous work, (b)
admits estimators that can readily be computed using off-the-shelf LASSO
algorithms, and (c) hints at a general framework for broad classes of noise in
sparse linear inverse problems. At the heart of this new approach lies a
weighted LASSO estimator for which data-dependent weights are based on Poisson
concentration inequalities. Unlike previous analyses of the weighted LASSO, the
proposed analysis depends on conditions which can be checked or shown to hold
in general settings with high probability.Comment: 25 pages (48 pages with appendix), 3 figure
Reconstruction Error Bounds for Compressed Sensing under Poisson Noise using the Square Root of the Jensen-Shannon Divergence
Reconstruction error bounds in compressed sensing under Gaussian or uniform
bounded noise do not translate easily to the case of Poisson noise. Reasons for
this include the signal dependent nature of Poisson noise, and also the fact
that the negative log likelihood (NLL) in case of a Poisson distribution (which
is related to the generalized Kullback-Leibler divergence (GKLD)) is not a
metric and does not obey the triangle inequality. There exist prior theoretical
results in the form of provable error bounds for computationally tractable
estimators for compressed sensing problems under Poisson noise. However, these
results do not apply to realistic compressive systems, which must obey some
crucial constraints such as non-negativity and flux preservation. On the other
hand, there exist provable error bounds for such realistic systems in the
published literature, but they are for estimators that are computationally
intractable. In this paper, we develop error bounds for a computationally
tractable estimator which also applies to realistic compressive systems obeying
the required constraints. Our technique replaces the GKLD, with an information
theoretic metric - namely the square root of the Jensen-Shannon divergence
(JSD), which is related to an approximate, symmetrized version of the Poisson
NLL. We show that this allows for simple proofs of the error bounds. We propose
and prove interesting statistical properties of the square root of JSD and
exploit other known ones. Numerical experiments are performed showing the use
of the technique in signal and image reconstruction from compressed
measurements under Poisson noise. Our technique applies to sparse/ compressible
signals in any orthonormal basis, works with high probability for any randomly
generated non-negative and flux-preserving sensing matrix and is proposes an
estimator whose parameters are purely statistically motivated.Comment: We are submitting a newer version of the article, with statistical
and experimental analysis of the statistical properties (distribution) of the
quantity , i.e. the square root of the Jensen Shannon
divergence between the Poisson-noisy measurement vector and the unknown
measurement vector $\Phi x
Statistical and Computational Limits for Sparse Matrix Detection
This paper investigates the fundamental limits for detecting a
high-dimensional sparse matrix contaminated by white Gaussian noise from both
the statistical and computational perspectives. We consider
matrices whose rows and columns are individually -sparse. We provide a tight
characterization of the statistical and computational limits for sparse matrix
detection, which precisely describe when achieving optimal detection is easy,
hard, or impossible, respectively. Although the sparse matrices considered in
this paper have no apparent submatrix structure and the corresponding
estimation problem has no computational issue at all, the detection problem has
a surprising computational barrier when the sparsity level exceeds the
cubic root of the matrix size : attaining the optimal detection boundary is
computationally at least as hard as solving the planted clique problem.
The same statistical and computational limits also hold in the sparse
covariance matrix model, where each variable is correlated with at most
others. A key step in the construction of the statistically optimal test is a
structural property for sparse matrices, which can be of independent interest
High-dimensional Log-Error-in-Variable Regression with Applications to Microbial Compositional Data Analysis
In microbiome and genomic study, the regression of compositional data has
been a crucial tool for identifying microbial taxa or genes that are associated
with clinical phenotypes. To account for the variation in sequencing depth, the
classic log-contrast model is often used where read counts are normalized into
compositions. However, zero read counts and the randomness in covariates remain
critical issues.
In this article, we introduce a surprisingly simple, interpretable, and
efficient method for the estimation of compositional data regression through
the lens of a novel high-dimensional log-error-in-variable regression model.
The proposed method provides both corrections on sequencing data with possible
overdispersion and simultaneously avoids any subjective imputation of zero read
counts. We provide theoretical justifications with matching upper and lower
bounds for the estimation error. We also consider a general
log-error-in-variable regression model with corresponding estimation method to
accommodate broader situations. The merit of the procedure is illustrated
through real data analysis and simulation studies
Reconstruction Error Bounds for Compressed Sensing under Poisson or Poisson-Gaussian Noise Using Variance Stabilization Transforms
Most existing bounds for signal reconstruction from compressive measurements
make the assumption of additive signal-independent noise. However in many
compressive imaging systems, the noise statistics are more accurately
represented by Poisson or Poisson-Gaussian noise models. In this paper, we
derive upper bounds for signal reconstruction error from compressive
measurements which are corrupted by Poisson or Poisson-Gaussian noise. The
features of our bounds are as follows: (1) The bounds are derived for a
probabilistically motivated, computationally tractable convex estimator with
principled parameter selection. The estimator penalizes signal sparsity subject
to a constraint that imposes an upper bound on a term based on variance
stabilization transforms to approximate the Poisson or Poisson-Gaussian
negative log-likelihoods. (2) They are applicable to signals that are sparse as
well as compressible in any orthonormal basis, and are derived for compressive
systems obeying realistic constraints such as non-negativity and
flux-preservation. We present extensive numerical results for signal
reconstruction under varying number of measurements and varying signal
intensity levels.Comment: Revised version with more elaborate statistical analysis of the term
in case of Poisson as well as Poisson-Gaussian nois
Multi-sample Estimation of Bacterial Composition Matrix in Metagenomics Data
Metagenomics sequencing is routinely applied to quantify bacterial abundances
in microbiome studies, where the bacterial composition is estimated based on
the sequencing read counts. Due to limited sequencing depth and DNA dropouts,
many rare bacterial taxa might not be captured in the final sequencing reads,
which results in many zero counts. Naive composition estimation using count
normalization leads to many zero proportions, which tend to result in
inaccurate estimates of bacterial abundance and diversity. This paper takes a
multi-sample approach to the estimation of bacterial abundances in order to
borrow information across samples and across species. Empirical results from
real data sets suggest that the composition matrix over multiple samples is
approximately low rank, which motivates a regularized maximum likelihood
estimation with a nuclear norm penalty. An efficient optimization algorithm
using the generalized accelerated proximal gradient and Euclidean projection
onto simplex space is developed. The theoretical upper bounds and the minimax
lower bounds of the estimation errors, measured by the Kullback-Leibler
divergence and the Frobenius norm, are established. Simulation studies
demonstrate that the proposed estimator outperforms the naive estimators. The
method is applied to an analysis of a human gut microbiome dataset
Rare and Weak effects in Large-Scale Inference: methods and phase diagrams
Often when we deal with `Big Data', the true effects we are interested in are
Rare and Weak (RW). Researchers measure a large number of features, hoping to
find perhaps only a small fraction of them to be relevant to the research in
question; the effect sizes of the relevant features are individually small so
the true effects are not strong enough to stand out for themselves.
Higher Criticism (HC) and Graphlet Screening (GS) are two classes of methods
that are specifically designed for the Rare/Weak settings. HC was introduced to
determine whether there are any relevant effects in all the measured features.
More recently, HC was applied to classification, where it provides a method for
selecting useful predictive features for trained classification rules. GS was
introduced as a graph-guided multivariate screening procedure, and was used for
variable selection.
We develop a theoretic framework where we use an Asymptotic Rare and Weak
(ARW) model simultaneously controlling the size and prevalence of
useful/significant features among the useless/null bulk. At the heart of the
ARW model is the so-called phase diagram, which is a way to visualize clearly
the class of ARW settings where the relevant effects are so rare or weak that
desired goals (signal detection, variable selection, etc.) are simply
impossible to achieve. We show that HC and GS have important advantages over
better known procedures and achieve the optimal phase diagrams in a variety of
ARW settings.
HC and GS are flexible ideas that adapt easily to many interesting
situations. We review the basics of these ideas and some of the recent
extensions, discuss their connections to existing literature, and suggest some
new applications of these ideas.Comment: 31 pages, 4 figure
- β¦