440 research outputs found
On Tensors, Sparsity, and Nonnegative Factorizations
Tensors have found application in a variety of fields, ranging from
chemometrics to signal processing and beyond. In this paper, we consider the
problem of multilinear modeling of sparse count data. Our goal is to develop a
descriptive tensor factorization model of such data, along with appropriate
algorithms and theory. To do so, we propose that the random variation is best
described via a Poisson distribution, which better describes the zeros observed
in the data as compared to the typical assumption of a Gaussian distribution.
Under a Poisson assumption, we fit a model to observed data using the negative
log-likelihood score. We present a new algorithm for Poisson tensor
factorization called CANDECOMP-PARAFAC Alternating Poisson Regression (CP-APR)
that is based on a majorization-minimization approach. It can be shown that
CP-APR is a generalization of the Lee-Seung multiplicative updates. We show how
to prevent the algorithm from converging to non-KKT points and prove
convergence of CP-APR under mild conditions. We also explain how to implement
CP-APR for large-scale sparse tensors and present results on several data sets,
both real and simulated
Bayesian Conditional Tensor Factorizations for High-Dimensional Classification
In many application areas, data are collected on a categorical response and
high-dimensional categorical predictors, with the goals being to build a
parsimonious model for classification while doing inferences on the important
predictors. In settings such as genomics, there can be complex interactions
among the predictors. By using a carefully-structured Tucker factorization, we
define a model that can characterize any conditional probability, while
facilitating variable selection and modeling of higher-order interactions.
Following a Bayesian approach, we propose a Markov chain Monte Carlo algorithm
for posterior computation accommodating uncertainty in the predictors to be
included. Under near sparsity assumptions, the posterior distribution for the
conditional probability is shown to achieve close to the parametric rate of
contraction even in ultra high-dimensional settings. The methods are
illustrated using simulation examples and biomedical applications
A Tensor-Based Dictionary Learning Approach to Tomographic Image Reconstruction
We consider tomographic reconstruction using priors in the form of a
dictionary learned from training images. The reconstruction has two stages:
first we construct a tensor dictionary prior from our training data, and then
we pose the reconstruction problem in terms of recovering the expansion
coefficients in that dictionary. Our approach differs from past approaches in
that a) we use a third-order tensor representation for our images and b) we
recast the reconstruction problem using the tensor formulation. The dictionary
learning problem is presented as a non-negative tensor factorization problem
with sparsity constraints. The reconstruction problem is formulated in a convex
optimization framework by looking for a solution with a sparse representation
in the tensor dictionary. Numerical results show that our tensor formulation
leads to very sparse representations of both the training images and the
reconstructions due to the ability of representing repeated features compactly
in the dictionary.Comment: 29 page
Descent methods for Nonnegative Matrix Factorization
In this paper, we present several descent methods that can be applied to
nonnegative matrix factorization and we analyze a recently developped fast
block coordinate method called Rank-one Residue Iteration (RRI). We also give a
comparison of these different methods and show that the new block coordinate
method has better properties in terms of approximation error and complexity. By
interpreting this method as a rank-one approximation of the residue matrix, we
prove that it \emph{converges} and also extend it to the nonnegative tensor
factorization and introduce some variants of the method by imposing some
additional controllable constraints such as: sparsity, discreteness and
smoothness.Comment: 47 pages. New convergence proof using damped version of RRI. To
appear in Numerical Linear Algebra in Signals, Systems and Control. Accepted.
Illustrating Matlab code is included in the source bundl
- …