4 research outputs found
Equations of States in Statistical Learning for a Nonparametrizable and Regular Case
Many learning machines that have hierarchical structure or hidden variables
are now being used in information science, artificial intelligence, and
bioinformatics. However, several learning machines used in such fields are not
regular but singular statistical models, hence their generalization performance
is still left unknown. To overcome these problems, in the previous papers, we
proved new equations in statistical learning, by which we can estimate the
Bayes generalization loss from the Bayes training loss and the functional
variance, on the condition that the true distribution is a singularity
contained in a learning machine. In this paper, we prove that the same
equations hold even if a true distribution is not contained in a parametric
model. Also we prove that, the proposed equations in a regular case are
asymptotically equivalent to the Takeuchi information criterion. Therefore, the
proposed equations are always applicable without any condition on the unknown
true distribution
Statistical Learning Theory of Quasi-Regular Cases
Many learning machines such as normal mixtures and layered neural networks
are not regular but singular statistical models, because the map from a
parameter to a probability distribution is not one-to-one. The conventional
statistical asymptotic theory can not be applied to such learning machines
because the likelihood function can not be approximated by any normal
distribution. Recently, new statistical theory has been established based on
algebraic geometry and it was clarified that the generalization and training
errors are determined by two birational invariants, the real log canonical
threshold and the singular fluctuation. However, their concrete values are left
unknown. In the present paper, we propose a new concept, a quasi-regular case
in statistical learning theory. A quasi-regular case is not a regular case but
a singular case, however, it has the same property as a regular case. In fact,
we prove that, in a quasi-regular case, two birational invariants are equal to
each other, resulting that the symmetry of the generalization and training
errors holds. Moreover, the concrete values of two birational invariants are
explicitly obtained, the quasi-regular case is useful to study statistical
learning theory