80 research outputs found
Reconstruction from anisotropic random measurements
Random matrices are widely used in sparse recovery problems, and the relevant
properties of matrices with i.i.d. entries are well understood. The current
paper discusses the recently introduced Restricted Eigenvalue (RE) condition,
which is among the most general assumptions on the matrix, guaranteeing
recovery. We prove a reduction principle showing that the RE condition can be
guaranteed by checking the restricted isometry on a certain family of
low-dimensional subspaces. This principle allows us to establish the RE
condition for several broad classes of random matrices with dependent entries,
including random matrices with subgaussian rows and non-trivial covariance
structure, as well as matrices with independent rows, and uniformly bounded
entries.Comment: 30 Page
RIPless compressed sensing from anisotropic measurements
Compressed sensing is the art of reconstructing a sparse vector from its
inner products with respect to a small set of randomly chosen measurement
vectors. It is usually assumed that the ensemble of measurement vectors is in
isotropic position in the sense that the associated covariance matrix is
proportional to the identity matrix. In this paper, we establish bounds on the
number of required measurements in the anisotropic case, where the ensemble of
measurement vectors possesses a non-trivial covariance matrix. Essentially, we
find that the required sampling rate grows proportionally to the condition
number of the covariance matrix. In contrast to other recent contributions to
this problem, our arguments do not rely on any restricted isometry properties
(RIP's), but rather on ideas from convex geometry which have been
systematically studied in the theory of low-rank matrix recovery. This allows
for a simple argument and slightly improved bounds, but may lead to a worse
dependency on noise (which we do not consider in the present paper).Comment: 19 pages. To appear in Linear Algebra and its Applications, Special
Issue on Sparse Approximate Solution of Linear System
Post-Selection Inference for Generalized Linear Models with Many Controls
This paper considers generalized linear models in the presence of many
controls. We lay out a general methodology to estimate an effect of interest
based on the construction of an instrument that immunize against model
selection mistakes and apply it to the case of logistic binary choice model.
More specifically we propose new methods for estimating and constructing
confidence regions for a regression parameter of primary interest , a
parameter in front of the regressor of interest, such as the treatment variable
or a policy variable. These methods allow to estimate at the
root- rate when the total number of other regressors, called controls,
potentially exceed the sample size using sparsity assumptions. The sparsity
assumption means that there is a subset of controls which suffices to
accurately approximate the nuisance part of the regression function.
Importantly, the estimators and these resulting confidence regions are valid
uniformly over -sparse models satisfying and other
technical conditions. These procedures do not rely on traditional consistent
model selection arguments for their validity. In fact, they are robust with
respect to moderate model selection mistakes in variable selection. Under
suitable conditions, the estimators are semi-parametrically efficient in the
sense of attaining the semi-parametric efficiency bounds for the class of
models in this paper
Trimmed Density Ratio Estimation
Density ratio estimation is a vital tool in both machine learning and
statistical community. However, due to the unbounded nature of density ratio,
the estimation procedure can be vulnerable to corrupted data points, which
often pushes the estimated ratio toward infinity. In this paper, we present a
robust estimator which automatically identifies and trims outliers. The
proposed estimator has a convex formulation, and the global optimum can be
obtained via subgradient descent. We analyze the parameter estimation error of
this estimator under high-dimensional settings. Experiments are conducted to
verify the effectiveness of the estimator.Comment: Made minor revisions. Restructured the introductory section
Sparse Signal Recovery under Poisson Statistics
We are motivated by problems that arise in a number of applications such as
Online Marketing and explosives detection, where the observations are usually
modeled using Poisson statistics. We model each observation as a Poisson random
variable whose mean is a sparse linear superposition of known patterns. Unlike
many conventional problems observations here are not identically distributed
since they are associated with different sensing modalities. We analyze the
performance of a Maximum Likelihood (ML) decoder, which for our Poisson setting
involves a non-linear optimization but yet is computationally tractable. We
derive fundamental sample complexity bounds for sparse recovery when the
measurements are contaminated with Poisson noise. In contrast to the
least-squares linear regression setting with Gaussian noise, we observe that in
addition to sparsity, the scale of the parameters also fundamentally impacts
sample complexity. We introduce a novel notion of Restricted Likelihood
Perturbation (RLP), to jointly account for scale and sparsity. We derive sample
complexity bounds for regularized ML estimators in terms of RLP and
further specialize these results for deterministic and random sensing matrix
designs.Comment: 13 pages, 11 figures, 2 tables, submitted to IEEE Transactions on
Signal Processin
- …