2,437 research outputs found
Tensor decompositions for learning latent variable models
This work considers a computationally and statistically efficient parameter
estimation method for a wide class of latent variable models---including
Gaussian mixture models, hidden Markov models, and latent Dirichlet
allocation---which exploits a certain tensor structure in their low-order
observable moments (typically, of second- and third-order). Specifically,
parameter estimation is reduced to the problem of extracting a certain
(orthogonal) decomposition of a symmetric tensor derived from the moments; this
decomposition can be viewed as a natural generalization of the singular value
decomposition for matrices. Although tensor decompositions are generally
intractable to compute, the decomposition of these specially structured tensors
can be efficiently obtained by a variety of approaches, including power
iterations and maximization approaches (similar to the case of matrices). A
detailed analysis of a robust tensor power method is provided, establishing an
analogue of Wedin's perturbation theorem for the singular vectors of matrices.
This implies a robust and computationally tractable estimation approach for
several popular latent variable models
Efficient Orthogonal Tensor Decomposition, with an Application to Latent Variable Model Learning
Decomposing tensors into orthogonal factors is a well-known task in
statistics, machine learning, and signal processing. We study orthogonal outer
product decompositions where the factors in the summands in the decomposition
are required to be orthogonal across summands, by relating this orthogonal
decomposition to the singular value decompositions of the flattenings. We show
that it is a non-trivial assumption for a tensor to have such an orthogonal
decomposition, and we show that it is unique (up to natural symmetries) in case
it exists, in which case we also demonstrate how it can be efficiently and
reliably obtained by a sequence of singular value decompositions. We
demonstrate how the factoring algorithm can be applied for parameter
identification in latent variable and mixture models
- …