14,611 research outputs found
Noisy Tensor Completion for Tensors with a Sparse Canonical Polyadic Factor
In this paper we study the problem of noisy tensor completion for tensors
that admit a canonical polyadic or CANDECOMP/PARAFAC (CP) decomposition with
one of the factors being sparse. We present general theoretical error bounds
for an estimate obtained by using a complexity-regularized maximum likelihood
principle and then instantiate these bounds for the case of additive white
Gaussian noise. We also provide an ADMM-type algorithm for solving the
complexity-regularized maximum likelihood problem and validate the theoretical
finding via experiments on synthetic data set
Rank Minimization over Finite Fields: Fundamental Limits and Coding-Theoretic Interpretations
This paper establishes information-theoretic limits in estimating a finite
field low-rank matrix given random linear measurements of it. These linear
measurements are obtained by taking inner products of the low-rank matrix with
random sensing matrices. Necessary and sufficient conditions on the number of
measurements required are provided. It is shown that these conditions are sharp
and the minimum-rank decoder is asymptotically optimal. The reliability
function of this decoder is also derived by appealing to de Caen's lower bound
on the probability of a union. The sufficient condition also holds when the
sensing matrices are sparse - a scenario that may be amenable to efficient
decoding. More precisely, it is shown that if the n\times n-sensing matrices
contain, on average, \Omega(nlog n) entries, the number of measurements
required is the same as that when the sensing matrices are dense and contain
entries drawn uniformly at random from the field. Analogies are drawn between
the above results and rank-metric codes in the coding theory literature. In
fact, we are also strongly motivated by understanding when minimum rank
distance decoding of random rank-metric codes succeeds. To this end, we derive
distance properties of equiprobable and sparse rank-metric codes. These
distance properties provide a precise geometric interpretation of the fact that
the sparse ensemble requires as few measurements as the dense one. Finally, we
provide a non-exhaustive procedure to search for the unknown low-rank matrix.Comment: Accepted to the IEEE Transactions on Information Theory; Presented at
IEEE International Symposium on Information Theory (ISIT) 201
High-dimensional estimation with geometric constraints
Consider measuring an n-dimensional vector x through the inner product with
several measurement vectors, a_1, a_2, ..., a_m. It is common in both signal
processing and statistics to assume the linear response model y_i = +
e_i, where e_i is a noise term. However, in practice the precise relationship
between the signal x and the observations y_i may not follow the linear model,
and in some cases it may not even be known. To address this challenge, in this
paper we propose a general model where it is only assumed that each observation
y_i may depend on a_i only through . We do not assume that the
dependence is known. This is a form of the semiparametric single index model,
and it includes the linear model as well as many forms of the generalized
linear model as special cases. We further assume that the signal x has some
structure, and we formulate this as a general assumption that x belongs to some
known (but arbitrary) feasible set K. We carefully detail the benefit of using
the signal structure to improve estimation. The theory is based on the mean
width of K, a geometric parameter which can be used to understand its effective
dimension in estimation problems. We determine a simple, efficient two-step
procedure for estimating the signal based on this model -- a linear estimation
followed by metric projection onto K. We give general conditions under which
the estimator is minimax optimal up to a constant. This leads to the intriguing
conclusion that in the high noise regime, an unknown non-linearity in the
observations does not significantly reduce one's ability to determine the
signal, even when the non-linearity may be non-invertible. Our results may be
specialized to understand the effect of non-linearities in compressed sensing.Comment: This version incorporates minor revisions suggested by referee
- …