30,173 research outputs found
Simple Bounds for Recovering Low-complexity Models
This note presents a unified analysis of the recovery of simple objects from
random linear measurements. When the linear functionals are Gaussian, we show
that an s-sparse vector in R^n can be efficiently recovered from 2s log n
measurements with high probability and a rank r, n by n matrix can be
efficiently recovered from r(6n-5r) with high probability. For sparse vectors,
this is within an additive factor of the best known nonasymptotic bounds. For
low-rank matrices, this matches the best known bounds. We present a parallel
analysis for block sparse vectors obtaining similarly tight bounds. In the case
of sparse and block sparse signals, we additionally demonstrate that our bounds
are only slightly weakened when the measurement map is a random sign matrix.
Our results are based on analyzing a particular dual point which certifies
optimality conditions of the respective convex programming problem. Our
calculations rely only on standard large deviation inequalities and our
analysis is self-contained
Fast Methods for Recovering Sparse Parameters in Linear Low Rank Models
In this paper, we investigate the recovery of a sparse weight vector
(parameters vector) from a set of noisy linear combinations. However, only
partial information about the matrix representing the linear combinations is
available. Assuming a low-rank structure for the matrix, one natural solution
would be to first apply a matrix completion on the data, and then to solve the
resulting compressed sensing problem. In big data applications such as massive
MIMO and medical data, the matrix completion step imposes a huge computational
burden. Here, we propose to reduce the computational cost of the completion
task by ignoring the columns corresponding to zero elements in the sparse
vector. To this end, we employ a technique to initially approximate the support
of the sparse vector. We further propose to unify the partial matrix completion
and sparse vector recovery into an augmented four-step problem. Simulation
results reveal that the augmented approach achieves the best performance, while
both proposed methods outperform the natural two-step technique with
substantially less computational requirements
Training Input-Output Recurrent Neural Networks through Spectral Methods
We consider the problem of training input-output recurrent neural networks
(RNN) for sequence labeling tasks. We propose a novel spectral approach for
learning the network parameters. It is based on decomposition of the
cross-moment tensor between the output and a non-linear transformation of the
input, based on score functions. We guarantee consistent learning with
polynomial sample and computational complexity under transparent conditions
such as non-degeneracy of model parameters, polynomial activations for the
neurons, and a Markovian evolution of the input sequence. We also extend our
results to Bidirectional RNN which uses both previous and future information to
output the label at each time point, and is employed in many NLP tasks such as
POS tagging
Simultaneously Structured Models with Application to Sparse and Low-rank Matrices
The topic of recovery of a structured model given a small number of linear
observations has been well-studied in recent years. Examples include recovering
sparse or group-sparse vectors, low-rank matrices, and the sum of sparse and
low-rank matrices, among others. In various applications in signal processing
and machine learning, the model of interest is known to be structured in
several ways at the same time, for example, a matrix that is simultaneously
sparse and low-rank.
Often norms that promote each individual structure are known, and allow for
recovery using an order-wise optimal number of measurements (e.g.,
norm for sparsity, nuclear norm for matrix rank). Hence, it is reasonable to
minimize a combination of such norms. We show that, surprisingly, if we use
multi-objective optimization with these norms, then we can do no better,
order-wise, than an algorithm that exploits only one of the present structures.
This result suggests that to fully exploit the multiple structures, we need an
entirely new convex relaxation, i.e. not one that is a function of the convex
relaxations used for each structure. We then specialize our results to the case
of sparse and low-rank matrices. We show that a nonconvex formulation of the
problem can recover the model from very few measurements, which is on the order
of the degrees of freedom of the matrix, whereas the convex problem obtained
from a combination of the and nuclear norms requires many more
measurements. This proves an order-wise gap between the performance of the
convex and nonconvex recovery problems in this case. Our framework applies to
arbitrary structure-inducing norms as well as to a wide range of measurement
ensembles. This allows us to give performance bounds for problems such as
sparse phase retrieval and low-rank tensor completion.Comment: 38 pages, 9 figure
- …