2 research outputs found
Bayesian Estimation of Multidimensional Latent Variables and Its Asymptotic Accuracy
Hierarchical learning models, such as mixture models and Bayesian networks,
are widely employed for unsupervised learning tasks, such as clustering
analysis. They consist of observable and hidden variables, which represent the
given data and their hidden generation process, respectively. It has been
pointed out that conventional statistical analysis is not applicable to these
models, because redundancy of the latent variable produces singularities in the
parameter space. In recent years, a method based on algebraic geometry has
allowed us to analyze the accuracy of predicting observable variables when
using Bayesian estimation. However, how to analyze latent variables has not
been sufficiently studied, even though one of the main issues in unsupervised
learning is to determine how accurately the latent variable is estimated. A
previous study proposed a method that can be used when the range of the latent
variable is redundant compared with the model generating data. The present
paper extends that method to the situation in which the latent variables have
redundant dimensions. We formulate new error functions and derive their
asymptotic forms. Calculation of the error functions is demonstrated in
two-layered Bayesian networks
Asymptotic Accuracy of Bayes Estimation for Latent Variables with Redundancy
Hierarchical parametric models consisting of observable and latent variables
are widely used for unsupervised learning tasks. For example, a mixture model
is a representative hierarchical model for clustering. From the statistical
point of view, the models can be regular or singular due to the distribution of
data. In the regular case, the models have the identifiability; there is
one-to-one relation between a probability density function for the model
expression and the parameter. The Fisher information matrix is positive
definite, and the estimation accuracy of both observable and latent variables
has been studied. In the singular case, on the other hand, the models are not
identifiable and the Fisher matrix is not positive definite. Conventional
statistical analysis based on the inverse Fisher matrix is not applicable.
Recently, an algebraic geometrical analysis has been developed and is used to
elucidate the Bayes estimation of observable variables. The present paper
applies this analysis to latent-variable estimation and determines its
theoretical performance. Our results clarify behavior of the convergence of the
posterior distribution. It is found that the posterior of the
observable-variable estimation can be different from the one in the
latent-variable estimation. Because of the difference, the Markov chain Monte
Carlo method based on the parameter and the latent variable cannot construct
the desired posterior distribution.Comment: 35 pages, 2 figure