2,589 research outputs found
Expectation-maximization Gaussian-mixture approximate message passing
Abstract—When recovering a sparse signal from noisy compressive linear measurements, the distribution of the signal’s non-zero coefficients can have a profound effect on recovery mean-squared error (MSE). If this distribution was aprioriknown, then one could use computationally efficient approximate message passing (AMP) techniques for nearly minimum MSE (MMSE) recovery. In practice, however, the distribution is unknown, motivating the use of robust algorithms like LASSO—which is nearly minimax optimal—at the cost of significantly larger MSE for non-least-favorable distributions. As an alternative, we propose an empirical-Bayesian technique that simultaneously learns the signal distribution while MMSE-recovering the signal—according to the learned distribution—using AMP. In particular, we model the non-zero distribution as a Gaussian mixture and learn its parameters through expectation maximization, using AMP to implement the expectation step. Numerical experiments on a wide range of signal classes confirm the state-of-the-art performance of our approach, in both reconstruction error and runtime, in the high-dimensional regime, for most (but not all) sensing operators. Index Terms—Compressed sensing, belief propagation, expectation maximization algorithms, Gaussian mixture model. I
Expectation-Maximization Gaussian-Mixture Approximate Message Passing
When recovering a sparse signal from noisy compressive linear measurements,
the distribution of the signal's non-zero coefficients can have a profound
effect on recovery mean-squared error (MSE). If this distribution was apriori
known, then one could use computationally efficient approximate message passing
(AMP) techniques for nearly minimum MSE (MMSE) recovery. In practice, though,
the distribution is unknown, motivating the use of robust algorithms like
LASSO---which is nearly minimax optimal---at the cost of significantly larger
MSE for non-least-favorable distributions. As an alternative, we propose an
empirical-Bayesian technique that simultaneously learns the signal distribution
while MMSE-recovering the signal---according to the learned
distribution---using AMP. In particular, we model the non-zero distribution as
a Gaussian mixture, and learn its parameters through expectation maximization,
using AMP to implement the expectation step. Numerical experiments on a wide
range of signal classes confirm the state-of-the-art performance of our
approach, in both reconstruction error and runtime, in the high-dimensional
regime, for most (but not all) sensing operators
Rectified Gaussian Scale Mixtures and the Sparse Non-Negative Least Squares Problem
In this paper, we develop a Bayesian evidence maximization framework to solve
the sparse non-negative least squares (S-NNLS) problem. We introduce a family
of probability densities referred to as the Rectified Gaussian Scale Mixture
(R- GSM) to model the sparsity enforcing prior distribution for the solution.
The R-GSM prior encompasses a variety of heavy-tailed densities such as the
rectified Laplacian and rectified Student- t distributions with a proper choice
of the mixing density. We utilize the hierarchical representation induced by
the R-GSM prior and develop an evidence maximization framework based on the
Expectation-Maximization (EM) algorithm. Using the EM based method, we estimate
the hyper-parameters and obtain a point estimate for the solution. We refer to
the proposed method as rectified sparse Bayesian learning (R-SBL). We provide
four R- SBL variants that offer a range of options for computational complexity
and the quality of the E-step computation. These methods include the Markov
chain Monte Carlo EM, linear minimum mean-square-error estimation, approximate
message passing and a diagonal approximation. Using numerical experiments, we
show that the proposed R-SBL method outperforms existing S-NNLS solvers in
terms of both signal and support recovery performance, and is also very robust
against the structure of the design matrix.Comment: Under Review by IEEE Transactions on Signal Processin
Dynamic Compressive Sensing of Time-Varying Signals via Approximate Message Passing
In this work the dynamic compressive sensing (CS) problem of recovering
sparse, correlated, time-varying signals from sub-Nyquist, non-adaptive, linear
measurements is explored from a Bayesian perspective. While there has been a
handful of previously proposed Bayesian dynamic CS algorithms in the
literature, the ability to perform inference on high-dimensional problems in a
computationally efficient manner remains elusive. In response, we propose a
probabilistic dynamic CS signal model that captures both amplitude and support
correlation structure, and describe an approximate message passing algorithm
that performs soft signal estimation and support detection with a computational
complexity that is linear in all problem dimensions. The algorithm, DCS-AMP,
can perform either causal filtering or non-causal smoothing, and is capable of
learning model parameters adaptively from the data through an
expectation-maximization learning procedure. We provide numerical evidence that
DCS-AMP performs within 3 dB of oracle bounds on synthetic data under a variety
of operating conditions. We further describe the result of applying DCS-AMP to
two real dynamic CS datasets, as well as a frequency estimation task, to
bolster our claim that DCS-AMP is capable of offering state-of-the-art
performance and speed on real-world high-dimensional problems.Comment: 32 pages, 7 figure
Message-Passing Algorithms for Channel Estimation and Decoding Using Approximate Inference
We design iterative receiver schemes for a generic wireless communication
system by treating channel estimation and information decoding as an inference
problem in graphical models. We introduce a recently proposed inference
framework that combines belief propagation (BP) and the mean field (MF)
approximation and includes these algorithms as special cases. We also show that
the expectation propagation and expectation maximization algorithms can be
embedded in the BP-MF framework with slight modifications. By applying the
considered inference algorithms to our probabilistic model, we derive four
different message-passing receiver schemes. Our numerical evaluation
demonstrates that the receiver based on the BP-MF framework and its variant
based on BP-EM yield the best compromise between performance, computational
complexity and numerical stability among all candidate algorithms.Comment: Accepted for publication in the Proceedings of 2012 IEEE
International Symposium on Information Theor
Empirical Bayes and Full Bayes for Signal Estimation
We consider signals that follow a parametric distribution where the parameter
values are unknown. To estimate such signals from noisy measurements in scalar
channels, we study the empirical performance of an empirical Bayes (EB)
approach and a full Bayes (FB) approach. We then apply EB and FB to solve
compressed sensing (CS) signal estimation problems by successively denoising a
scalar Gaussian channel within an approximate message passing (AMP) framework.
Our numerical results show that FB achieves better performance than EB in
scalar channel denoising problems when the signal dimension is small. In the CS
setting, the signal dimension must be large enough for AMP to work well; for
large signal dimensions, AMP has similar performance with FB and EB.Comment: This work was presented at the Information Theory and Application
workshop (ITA), San Diego, CA, Feb. 201
- …