2,584 research outputs found
High-dimensional estimation with geometric constraints
Consider measuring an n-dimensional vector x through the inner product with
several measurement vectors, a_1, a_2, ..., a_m. It is common in both signal
processing and statistics to assume the linear response model y_i = +
e_i, where e_i is a noise term. However, in practice the precise relationship
between the signal x and the observations y_i may not follow the linear model,
and in some cases it may not even be known. To address this challenge, in this
paper we propose a general model where it is only assumed that each observation
y_i may depend on a_i only through . We do not assume that the
dependence is known. This is a form of the semiparametric single index model,
and it includes the linear model as well as many forms of the generalized
linear model as special cases. We further assume that the signal x has some
structure, and we formulate this as a general assumption that x belongs to some
known (but arbitrary) feasible set K. We carefully detail the benefit of using
the signal structure to improve estimation. The theory is based on the mean
width of K, a geometric parameter which can be used to understand its effective
dimension in estimation problems. We determine a simple, efficient two-step
procedure for estimating the signal based on this model -- a linear estimation
followed by metric projection onto K. We give general conditions under which
the estimator is minimax optimal up to a constant. This leads to the intriguing
conclusion that in the high noise regime, an unknown non-linearity in the
observations does not significantly reduce one's ability to determine the
signal, even when the non-linearity may be non-invertible. Our results may be
specialized to understand the effect of non-linearities in compressed sensing.Comment: This version incorporates minor revisions suggested by referee
Matrix Completion With Noise
On the heels of compressed sensing, a remarkable new field has very recently
emerged. This field addresses a broad range of problems of significant
practical interest, namely, the recovery of a data matrix from what appears to
be incomplete, and perhaps even corrupted, information. In its simplest form,
the problem is to recover a matrix from a small sample of its entries, and
comes up in many areas of science and engineering including collaborative
filtering, machine learning, control, remote sensing, and computer vision to
name a few.
This paper surveys the novel literature on matrix completion, which shows
that under some suitable conditions, one can recover an unknown low-rank matrix
from a nearly minimal set of entries by solving a simple convex optimization
problem, namely, nuclear-norm minimization subject to data constraints.
Further, this paper introduces novel results showing that matrix completion is
provably accurate even when the few observed entries are corrupted with a small
amount of noise. A typical result is that one can recover an unknown n x n
matrix of low rank r from just about nr log^2 n noisy samples with an error
which is proportional to the noise level. We present numerical results which
complement our quantitative analysis and show that, in practice, nuclear norm
minimization accurately fills in the many missing entries of large low-rank
matrices from just a few noisy samples. Some analogies between matrix
completion and compressed sensing are discussed throughout.Comment: 11 pages, 4 figures, 1 tabl
- …