7,201 research outputs found
Compressed sensing reconstruction using Expectation Propagation
Many interesting problems in fields ranging from telecommunications to
computational biology can be formalized in terms of large underdetermined
systems of linear equations with additional constraints or regularizers. One of
the most studied ones, the Compressed Sensing problem (CS), consists in finding
the solution with the smallest number of non-zero components of a given system
of linear equations for known
measurement vector and sensing matrix . Here, we
will address the compressed sensing problem within a Bayesian inference
framework where the sparsity constraint is remapped into a singular prior
distribution (called Spike-and-Slab or Bernoulli-Gauss). Solution to the
problem is attempted through the computation of marginal distributions via
Expectation Propagation (EP), an iterative computational scheme originally
developed in Statistical Physics. We will show that this strategy is
comparatively more accurate than the alternatives in solving instances of CS
generated from statistically correlated measurement matrices. For computational
strategies based on the Bayesian framework such as variants of Belief
Propagation, this is to be expected, as they implicitly rely on the hypothesis
of statistical independence among the entries of the sensing matrix. Perhaps
surprisingly, the method outperforms uniformly also all the other
state-of-the-art methods in our tests.Comment: 20 pages, 6 figure
Inferring Sparsity: Compressed Sensing using Generalized Restricted Boltzmann Machines
In this work, we consider compressed sensing reconstruction from
measurements of -sparse structured signals which do not possess a writable
correlation model. Assuming that a generative statistical model, such as a
Boltzmann machine, can be trained in an unsupervised manner on example signals,
we demonstrate how this signal model can be used within a Bayesian framework of
signal reconstruction. By deriving a message-passing inference for general
distribution restricted Boltzmann machines, we are able to integrate these
inferred signal models into approximate message passing for compressed sensing
reconstruction. Finally, we show for the MNIST dataset that this approach can
be very effective, even for .Comment: IEEE Information Theory Workshop, 201
Sketching for Large-Scale Learning of Mixture Models
Learning parameters from voluminous data can be prohibitive in terms of
memory and computational requirements. We propose a "compressive learning"
framework where we estimate model parameters from a sketch of the training
data. This sketch is a collection of generalized moments of the underlying
probability distribution of the data. It can be computed in a single pass on
the training set, and is easily computable on streams or distributed datasets.
The proposed framework shares similarities with compressive sensing, which aims
at drastically reducing the dimension of high-dimensional signals while
preserving the ability to reconstruct them. To perform the estimation task, we
derive an iterative algorithm analogous to sparse reconstruction algorithms in
the context of linear inverse problems. We exemplify our framework with the
compressive estimation of a Gaussian Mixture Model (GMM), providing heuristics
on the choice of the sketching procedure and theoretical guarantees of
reconstruction. We experimentally show on synthetic data that the proposed
algorithm yields results comparable to the classical Expectation-Maximization
(EM) technique while requiring significantly less memory and fewer computations
when the number of database elements is large. We further demonstrate the
potential of the approach on real large-scale data (over 10 8 training samples)
for the task of model-based speaker verification. Finally, we draw some
connections between the proposed framework and approximate Hilbert space
embedding of probability distributions using random features. We show that the
proposed sketching operator can be seen as an innovative method to design
translation-invariant kernels adapted to the analysis of GMMs. We also use this
theoretical framework to derive information preservation guarantees, in the
spirit of infinite-dimensional compressive sensing
Optimized Compressed Sensing Matrix Design for Noisy Communication Channels
We investigate a power-constrained sensing matrix design problem for a
compressed sensing framework. We adopt a mean square error (MSE) performance
criterion for sparse source reconstruction in a system where the
source-to-sensor channel and the sensor-to-decoder communication channel are
noisy. Our proposed sensing matrix design procedure relies upon minimizing a
lower-bound on the MSE. Under certain conditions, we derive closed-form
solutions to the optimization problem. Through numerical experiments, by
applying practical sparse reconstruction algorithms, we show the strength of
the proposed scheme by comparing it with other relevant methods. We discuss the
computational complexity of our design method, and develop an equivalent
stochastic optimization method to the problem of interest that can be solved
approximately with a significantly less computational burden. We illustrate
that the low-complexity method still outperforms the popular competing methods.Comment: Submitted to IEEE ICC 2015 (EXTENDED VERSION
Exact Performance Analysis of the Oracle Receiver for Compressed Sensing Reconstruction
A sparse or compressible signal can be recovered from a certain number of
noisy random projections, smaller than what dictated by classic Shannon/Nyquist
theory. In this paper, we derive the closed-form expression of the mean square
error performance of the oracle receiver, knowing the sparsity pattern of the
signal. With respect to existing bounds, our result is exact and does not
depend on a particular realization of the sensing matrix. Moreover, our result
holds irrespective of whether the noise affecting the measurements is white or
correlated. Numerical results show a perfect match between equations and
simulations, confirming the validity of the result.Comment: To be published in ICASSP 2014 proceeding
Power-Constrained Sparse Gaussian Linear Dimensionality Reduction over Noisy Channels
In this paper, we investigate power-constrained sensing matrix design in a
sparse Gaussian linear dimensionality reduction framework. Our study is carried
out in a single--terminal setup as well as in a multi--terminal setup
consisting of orthogonal or coherent multiple access channels (MAC). We adopt
the mean square error (MSE) performance criterion for sparse source
reconstruction in a system where source-to-sensor channel(s) and
sensor-to-decoder communication channel(s) are noisy. Our proposed sensing
matrix design procedure relies upon minimizing a lower-bound on the MSE in
single-- and multiple--terminal setups. We propose a three-stage sensing matrix
optimization scheme that combines semi-definite relaxation (SDR) programming, a
low-rank approximation problem and power-rescaling. Under certain conditions,
we derive closed-form solutions to the proposed optimization procedure. Through
numerical experiments, by applying practical sparse reconstruction algorithms,
we show the superiority of the proposed scheme by comparing it with other
relevant methods. This performance improvement is achieved at the price of
higher computational complexity. Hence, in order to address the complexity
burden, we present an equivalent stochastic optimization method to the problem
of interest that can be solved approximately, while still providing a superior
performance over the popular methods.Comment: Accepted for publication in IEEE Transactions on Signal Processing
(16 pages
- âŠ