3,675 research outputs found
Compressed sensing reconstruction using Expectation Propagation
Many interesting problems in fields ranging from telecommunications to
computational biology can be formalized in terms of large underdetermined
systems of linear equations with additional constraints or regularizers. One of
the most studied ones, the Compressed Sensing problem (CS), consists in finding
the solution with the smallest number of non-zero components of a given system
of linear equations for known
measurement vector and sensing matrix . Here, we
will address the compressed sensing problem within a Bayesian inference
framework where the sparsity constraint is remapped into a singular prior
distribution (called Spike-and-Slab or Bernoulli-Gauss). Solution to the
problem is attempted through the computation of marginal distributions via
Expectation Propagation (EP), an iterative computational scheme originally
developed in Statistical Physics. We will show that this strategy is
comparatively more accurate than the alternatives in solving instances of CS
generated from statistically correlated measurement matrices. For computational
strategies based on the Bayesian framework such as variants of Belief
Propagation, this is to be expected, as they implicitly rely on the hypothesis
of statistical independence among the entries of the sensing matrix. Perhaps
surprisingly, the method outperforms uniformly also all the other
state-of-the-art methods in our tests.Comment: 20 pages, 6 figure
Non-Gaussian Component Analysis using Entropy Methods
Non-Gaussian component analysis (NGCA) is a problem in multidimensional data
analysis which, since its formulation in 2006, has attracted considerable
attention in statistics and machine learning. In this problem, we have a random
variable in -dimensional Euclidean space. There is an unknown subspace
of the -dimensional Euclidean space such that the orthogonal
projection of onto is standard multidimensional Gaussian and the
orthogonal projection of onto , the orthogonal complement
of , is non-Gaussian, in the sense that all its one-dimensional
marginals are different from the Gaussian in a certain metric defined in terms
of moments. The NGCA problem is to approximate the non-Gaussian subspace
given samples of .
Vectors in correspond to `interesting' directions, whereas
vectors in correspond to the directions where data is very noisy. The
most interesting applications of the NGCA model is for the case when the
magnitude of the noise is comparable to that of the true signal, a setting in
which traditional noise reduction techniques such as PCA don't apply directly.
NGCA is also related to dimension reduction and to other data analysis problems
such as ICA. NGCA-like problems have been studied in statistics for a long time
using techniques such as projection pursuit.
We give an algorithm that takes polynomial time in the dimension and has
an inverse polynomial dependence on the error parameter measuring the angle
distance between the non-Gaussian subspace and the subspace output by the
algorithm. Our algorithm is based on relative entropy as the contrast function
and fits under the projection pursuit framework. The techniques we develop for
analyzing our algorithm maybe of use for other related problems
Optimized Compressed Sensing Matrix Design for Noisy Communication Channels
We investigate a power-constrained sensing matrix design problem for a
compressed sensing framework. We adopt a mean square error (MSE) performance
criterion for sparse source reconstruction in a system where the
source-to-sensor channel and the sensor-to-decoder communication channel are
noisy. Our proposed sensing matrix design procedure relies upon minimizing a
lower-bound on the MSE. Under certain conditions, we derive closed-form
solutions to the optimization problem. Through numerical experiments, by
applying practical sparse reconstruction algorithms, we show the strength of
the proposed scheme by comparing it with other relevant methods. We discuss the
computational complexity of our design method, and develop an equivalent
stochastic optimization method to the problem of interest that can be solved
approximately with a significantly less computational burden. We illustrate
that the low-complexity method still outperforms the popular competing methods.Comment: Submitted to IEEE ICC 2015 (EXTENDED VERSION
Rectified Gaussian Scale Mixtures and the Sparse Non-Negative Least Squares Problem
In this paper, we develop a Bayesian evidence maximization framework to solve
the sparse non-negative least squares (S-NNLS) problem. We introduce a family
of probability densities referred to as the Rectified Gaussian Scale Mixture
(R- GSM) to model the sparsity enforcing prior distribution for the solution.
The R-GSM prior encompasses a variety of heavy-tailed densities such as the
rectified Laplacian and rectified Student- t distributions with a proper choice
of the mixing density. We utilize the hierarchical representation induced by
the R-GSM prior and develop an evidence maximization framework based on the
Expectation-Maximization (EM) algorithm. Using the EM based method, we estimate
the hyper-parameters and obtain a point estimate for the solution. We refer to
the proposed method as rectified sparse Bayesian learning (R-SBL). We provide
four R- SBL variants that offer a range of options for computational complexity
and the quality of the E-step computation. These methods include the Markov
chain Monte Carlo EM, linear minimum mean-square-error estimation, approximate
message passing and a diagonal approximation. Using numerical experiments, we
show that the proposed R-SBL method outperforms existing S-NNLS solvers in
terms of both signal and support recovery performance, and is also very robust
against the structure of the design matrix.Comment: Under Review by IEEE Transactions on Signal Processin
Partially Linear Estimation with Application to Sparse Signal Recovery From Measurement Pairs
We address the problem of estimating a random vector X from two sets of
measurements Y and Z, such that the estimator is linear in Y. We show that the
partially linear minimum mean squared error (PLMMSE) estimator does not require
knowing the joint distribution of X and Y in full, but rather only its
second-order moments. This renders it of potential interest in various
applications. We further show that the PLMMSE method is minimax-optimal among
all estimators that solely depend on the second-order statistics of X and Y. We
demonstrate our approach in the context of recovering a signal, which is sparse
in a unitary dictionary, from noisy observations of it and of a filtered
version of it. We show that in this setting PLMMSE estimation has a clear
computational advantage, while its performance is comparable to
state-of-the-art algorithms. We apply our approach both in static and dynamic
estimation applications. In the former category, we treat the problem of image
enhancement from blurred/noisy image pairs, where we show that PLMMSE
estimation performs only slightly worse than state-of-the art algorithms, while
running an order of magnitude faster. In the dynamic setting, we provide a
recursive implementation of the estimator and demonstrate its utility in the
context of tracking maneuvering targets from position and acceleration
measurements.Comment: 13 pages, 5 figure
Power-Constrained Sparse Gaussian Linear Dimensionality Reduction over Noisy Channels
In this paper, we investigate power-constrained sensing matrix design in a
sparse Gaussian linear dimensionality reduction framework. Our study is carried
out in a single--terminal setup as well as in a multi--terminal setup
consisting of orthogonal or coherent multiple access channels (MAC). We adopt
the mean square error (MSE) performance criterion for sparse source
reconstruction in a system where source-to-sensor channel(s) and
sensor-to-decoder communication channel(s) are noisy. Our proposed sensing
matrix design procedure relies upon minimizing a lower-bound on the MSE in
single-- and multiple--terminal setups. We propose a three-stage sensing matrix
optimization scheme that combines semi-definite relaxation (SDR) programming, a
low-rank approximation problem and power-rescaling. Under certain conditions,
we derive closed-form solutions to the proposed optimization procedure. Through
numerical experiments, by applying practical sparse reconstruction algorithms,
we show the superiority of the proposed scheme by comparing it with other
relevant methods. This performance improvement is achieved at the price of
higher computational complexity. Hence, in order to address the complexity
burden, we present an equivalent stochastic optimization method to the problem
of interest that can be solved approximately, while still providing a superior
performance over the popular methods.Comment: Accepted for publication in IEEE Transactions on Signal Processing
(16 pages
- …