1,784 research outputs found
Bayesian separation of spectral sources under non-negativity and full additivity constraints
This paper addresses the problem of separating spectral sources which are
linearly mixed with unknown proportions. The main difficulty of the problem is
to ensure the full additivity (sum-to-one) of the mixing coefficients and
non-negativity of sources and mixing coefficients. A Bayesian estimation
approach based on Gamma priors was recently proposed to handle the
non-negativity constraints in a linear mixture model. However, incorporating
the full additivity constraint requires further developments. This paper
studies a new hierarchical Bayesian model appropriate to the non-negativity and
sum-to-one constraints associated to the regressors and regression coefficients
of linear mixtures. The estimation of the unknown parameters of this model is
performed using samples generated using an appropriate Gibbs sampler. The
performance of the proposed algorithm is evaluated through simulation results
conducted on synthetic mixture models. The proposed approach is also applied to
the processing of multicomponent chemical mixtures resulting from Raman
spectroscopy.Comment: v4: minor grammatical changes; Signal Processing, 200
Rectified Gaussian Scale Mixtures and the Sparse Non-Negative Least Squares Problem
In this paper, we develop a Bayesian evidence maximization framework to solve
the sparse non-negative least squares (S-NNLS) problem. We introduce a family
of probability densities referred to as the Rectified Gaussian Scale Mixture
(R- GSM) to model the sparsity enforcing prior distribution for the solution.
The R-GSM prior encompasses a variety of heavy-tailed densities such as the
rectified Laplacian and rectified Student- t distributions with a proper choice
of the mixing density. We utilize the hierarchical representation induced by
the R-GSM prior and develop an evidence maximization framework based on the
Expectation-Maximization (EM) algorithm. Using the EM based method, we estimate
the hyper-parameters and obtain a point estimate for the solution. We refer to
the proposed method as rectified sparse Bayesian learning (R-SBL). We provide
four R- SBL variants that offer a range of options for computational complexity
and the quality of the E-step computation. These methods include the Markov
chain Monte Carlo EM, linear minimum mean-square-error estimation, approximate
message passing and a diagonal approximation. Using numerical experiments, we
show that the proposed R-SBL method outperforms existing S-NNLS solvers in
terms of both signal and support recovery performance, and is also very robust
against the structure of the design matrix.Comment: Under Review by IEEE Transactions on Signal Processin
Efficient implementation of Markov chain Monte Carlo when using an unbiased likelihood estimator
When an unbiased estimator of the likelihood is used within a
Metropolis--Hastings chain, it is necessary to trade off the number of Monte
Carlo samples used to construct this estimator against the asymptotic variances
of averages computed under this chain. Many Monte Carlo samples will typically
result in Metropolis--Hastings averages with lower asymptotic variances than
the corresponding Metropolis--Hastings averages using fewer samples. However,
the computing time required to construct the likelihood estimator increases
with the number of Monte Carlo samples. Under the assumption that the
distribution of the additive noise introduced by the log-likelihood estimator
is Gaussian with variance inversely proportional to the number of Monte Carlo
samples and independent of the parameter value at which it is evaluated, we
provide guidelines on the number of samples to select. We demonstrate our
results by considering a stochastic volatility model applied to stock index
returns.Comment: 34 pages, 9 figures, 3 table
Convergence of Smoothed Empirical Measures with Applications to Entropy Estimation
This paper studies convergence of empirical measures smoothed by a Gaussian
kernel. Specifically, consider approximating , for
, by
, where is the empirical measure,
under different statistical distances. The convergence is examined in terms of
the Wasserstein distance, total variation (TV), Kullback-Leibler (KL)
divergence, and -divergence. We show that the approximation error under
the TV distance and 1-Wasserstein distance () converges at rate
in remarkable contrast to a typical
rate for unsmoothed (and ). For the
KL divergence, squared 2-Wasserstein distance (), and
-divergence, the convergence rate is , but only if
achieves finite input-output mutual information across the additive
white Gaussian noise channel. If the latter condition is not met, the rate
changes to for the KL divergence and , while
the -divergence becomes infinite - a curious dichotomy. As a main
application we consider estimating the differential entropy
in the high-dimensional regime. The distribution
is unknown but i.i.d samples from it are available. We first show that
any good estimator of must have sample complexity
that is exponential in . Using the empirical approximation results we then
show that the absolute-error risk of the plug-in estimator converges at the
parametric rate , thus establishing the minimax
rate-optimality of the plug-in. Numerical results that demonstrate a
significant empirical superiority of the plug-in approach to general-purpose
differential entropy estimators are provided.Comment: arXiv admin note: substantial text overlap with arXiv:1810.1158
The Extended Parameter Filter
The parameters of temporal models, such as dynamic Bayesian networks, may be
modelled in a Bayesian context as static or atemporal variables that influence
transition probabilities at every time step. Particle filters fail for models
that include such variables, while methods that use Gibbs sampling of parameter
variables may incur a per-sample cost that grows linearly with the length of
the observation sequence. Storvik devised a method for incremental computation
of exact sufficient statistics that, for some cases, reduces the per-sample
cost to a constant. In this paper, we demonstrate a connection between
Storvik's filter and a Kalman filter in parameter space and establish more
general conditions under which Storvik's filter works. Drawing on an analogy to
the extended Kalman filter, we develop and analyze, both theoretically and
experimentally, a Taylor approximation to the parameter posterior that allows
Storvik's method to be applied to a broader class of models. Our experiments on
both synthetic examples and real applications show improvement over existing
methods
- …