2,452 research outputs found
Bayesian Robust Tensor Factorization for Incomplete Multiway Data
We propose a generative model for robust tensor factorization in the presence
of both missing data and outliers. The objective is to explicitly infer the
underlying low-CP-rank tensor capturing the global information and a sparse
tensor capturing the local information (also considered as outliers), thus
providing the robust predictive distribution over missing entries. The
low-CP-rank tensor is modeled by multilinear interactions between multiple
latent factors on which the column sparsity is enforced by a hierarchical
prior, while the sparse tensor is modeled by a hierarchical view of Student-
distribution that associates an individual hyperparameter with each element
independently. For model learning, we develop an efficient closed-form
variational inference under a fully Bayesian treatment, which can effectively
prevent the overfitting problem and scales linearly with data size. In contrast
to existing related works, our method can perform model selection automatically
and implicitly without need of tuning parameters. More specifically, it can
discover the groundtruth of CP rank and automatically adapt the sparsity
inducing priors to various types of outliers. In addition, the tradeoff between
the low-rank approximation and the sparse representation can be optimized in
the sense of maximum model evidence. The extensive experiments and comparisons
with many state-of-the-art algorithms on both synthetic and real-world datasets
demonstrate the superiorities of our method from several perspectives.Comment: in IEEE Transactions on Neural Networks and Learning Systems, 201
Rectified Gaussian Scale Mixtures and the Sparse Non-Negative Least Squares Problem
In this paper, we develop a Bayesian evidence maximization framework to solve
the sparse non-negative least squares (S-NNLS) problem. We introduce a family
of probability densities referred to as the Rectified Gaussian Scale Mixture
(R- GSM) to model the sparsity enforcing prior distribution for the solution.
The R-GSM prior encompasses a variety of heavy-tailed densities such as the
rectified Laplacian and rectified Student- t distributions with a proper choice
of the mixing density. We utilize the hierarchical representation induced by
the R-GSM prior and develop an evidence maximization framework based on the
Expectation-Maximization (EM) algorithm. Using the EM based method, we estimate
the hyper-parameters and obtain a point estimate for the solution. We refer to
the proposed method as rectified sparse Bayesian learning (R-SBL). We provide
four R- SBL variants that offer a range of options for computational complexity
and the quality of the E-step computation. These methods include the Markov
chain Monte Carlo EM, linear minimum mean-square-error estimation, approximate
message passing and a diagonal approximation. Using numerical experiments, we
show that the proposed R-SBL method outperforms existing S-NNLS solvers in
terms of both signal and support recovery performance, and is also very robust
against the structure of the design matrix.Comment: Under Review by IEEE Transactions on Signal Processin
Sparse Estimation using Bayesian Hierarchical Prior Modeling for Real and Complex Linear Models
In sparse Bayesian learning (SBL), Gaussian scale mixtures (GSMs) have been
used to model sparsity-inducing priors that realize a class of concave penalty
functions for the regression task in real-valued signal models. Motivated by
the relative scarcity of formal tools for SBL in complex-valued models, this
paper proposes a GSM model - the Bessel K model - that induces concave penalty
functions for the estimation of complex sparse signals. The properties of the
Bessel K model are analyzed when it is applied to Type I and Type II
estimation. This analysis reveals that, by tuning the parameters of the mixing
pdf different penalty functions are invoked depending on the estimation type
used, the value of the noise variance, and whether real or complex signals are
estimated. Using the Bessel K model, we derive a sparse estimator based on a
modification of the expectation-maximization algorithm formulated for Type II
estimation. The estimator includes as a special instance the algorithms
proposed by Tipping and Faul [1] and by Babacan et al. [2]. Numerical results
show the superiority of the proposed estimator over these state-of-the-art
estimators in terms of convergence speed, sparseness, reconstruction error, and
robustness in low and medium signal-to-noise ratio regimes.Comment: The paper provides a new comprehensive analysis of the theoretical
foundations of the proposed estimators. Minor modification of the titl
- …