194 research outputs found
Bayesian Estimation for Continuous-Time Sparse Stochastic Processes
We consider continuous-time sparse stochastic processes from which we have
only a finite number of noisy/noiseless samples. Our goal is to estimate the
noiseless samples (denoising) and the signal in-between (interpolation
problem).
By relying on tools from the theory of splines, we derive the joint a priori
distribution of the samples and show how this probability density function can
be factorized. The factorization enables us to tractably implement the maximum
a posteriori and minimum mean-square error (MMSE) criteria as two statistical
approaches for estimating the unknowns. We compare the derived statistical
methods with well-known techniques for the recovery of sparse signals, such as
the norm and Log (- relaxation) regularization
methods. The simulation results show that, under certain conditions, the
performance of the regularization techniques can be very close to that of the
MMSE estimator.Comment: To appear in IEEE TS
Rectified Gaussian Scale Mixtures and the Sparse Non-Negative Least Squares Problem
In this paper, we develop a Bayesian evidence maximization framework to solve
the sparse non-negative least squares (S-NNLS) problem. We introduce a family
of probability densities referred to as the Rectified Gaussian Scale Mixture
(R- GSM) to model the sparsity enforcing prior distribution for the solution.
The R-GSM prior encompasses a variety of heavy-tailed densities such as the
rectified Laplacian and rectified Student- t distributions with a proper choice
of the mixing density. We utilize the hierarchical representation induced by
the R-GSM prior and develop an evidence maximization framework based on the
Expectation-Maximization (EM) algorithm. Using the EM based method, we estimate
the hyper-parameters and obtain a point estimate for the solution. We refer to
the proposed method as rectified sparse Bayesian learning (R-SBL). We provide
four R- SBL variants that offer a range of options for computational complexity
and the quality of the E-step computation. These methods include the Markov
chain Monte Carlo EM, linear minimum mean-square-error estimation, approximate
message passing and a diagonal approximation. Using numerical experiments, we
show that the proposed R-SBL method outperforms existing S-NNLS solvers in
terms of both signal and support recovery performance, and is also very robust
against the structure of the design matrix.Comment: Under Review by IEEE Transactions on Signal Processin
Partially Linear Estimation with Application to Sparse Signal Recovery From Measurement Pairs
We address the problem of estimating a random vector X from two sets of
measurements Y and Z, such that the estimator is linear in Y. We show that the
partially linear minimum mean squared error (PLMMSE) estimator does not require
knowing the joint distribution of X and Y in full, but rather only its
second-order moments. This renders it of potential interest in various
applications. We further show that the PLMMSE method is minimax-optimal among
all estimators that solely depend on the second-order statistics of X and Y. We
demonstrate our approach in the context of recovering a signal, which is sparse
in a unitary dictionary, from noisy observations of it and of a filtered
version of it. We show that in this setting PLMMSE estimation has a clear
computational advantage, while its performance is comparable to
state-of-the-art algorithms. We apply our approach both in static and dynamic
estimation applications. In the former category, we treat the problem of image
enhancement from blurred/noisy image pairs, where we show that PLMMSE
estimation performs only slightly worse than state-of-the art algorithms, while
running an order of magnitude faster. In the dynamic setting, we provide a
recursive implementation of the estimator and demonstrate its utility in the
context of tracking maneuvering targets from position and acceleration
measurements.Comment: 13 pages, 5 figure
Vector Approximate Message Passing for the Generalized Linear Model
The generalized linear model (GLM), where a random vector is
observed through a noisy, possibly nonlinear, function of a linear transform
output , arises in a range of applications such
as robust regression, binary classification, quantized compressed sensing,
phase retrieval, photon-limited imaging, and inference from neural spike
trains. When is large and i.i.d. Gaussian, the generalized
approximate message passing (GAMP) algorithm is an efficient means of MAP or
marginal inference, and its performance can be rigorously characterized by a
scalar state evolution. For general , though, GAMP can
misbehave. Damping and sequential-updating help to robustify GAMP, but their
effects are limited. Recently, a "vector AMP" (VAMP) algorithm was proposed for
additive white Gaussian noise channels. VAMP extends AMP's guarantees from
i.i.d. Gaussian to the larger class of rotationally invariant
. In this paper, we show how VAMP can be extended to the GLM.
Numerical experiments show that the proposed GLM-VAMP is much more robust to
ill-conditioning in than damped GAMP
The Sampling Rate-Distortion Tradeoff for Sparsity Pattern Recovery in Compressed Sensing
Recovery of the sparsity pattern (or support) of an unknown sparse vector
from a limited number of noisy linear measurements is an important problem in
compressed sensing. In the high-dimensional setting, it is known that recovery
with a vanishing fraction of errors is impossible if the measurement rate and
the per-sample signal-to-noise ratio (SNR) are finite constants, independent of
the vector length. In this paper, it is shown that recovery with an arbitrarily
small but constant fraction of errors is, however, possible, and that in some
cases computationally simple estimators are near-optimal. Bounds on the
measurement rate needed to attain a desired fraction of errors are given in
terms of the SNR and various key parameters of the unknown vector for several
different recovery algorithms. The tightness of the bounds, in a scaling sense,
as a function of the SNR and the fraction of errors, is established by
comparison with existing information-theoretic necessary bounds. Near
optimality is shown for a wide variety of practically motivated signal models
- …