317,356 research outputs found
Statistical Mechanics of Time Domain Ensemble Learning
Conventional ensemble learning combines students in the space domain. On the
other hand, in this paper we combine students in the time domain and call it
time domain ensemble learning. In this paper, we analyze the generalization
performance of time domain ensemble learning in the framework of online
learning using a statistical mechanical method. We treat a model in which both
the teacher and the student are linear perceptrons with noises. Time domain
ensemble learning is twice as effective as conventional space domain ensemble
learning.Comment: 10 pages, 10 figure
Statistical Mechanics of Linear and Nonlinear Time-Domain Ensemble Learning
Conventional ensemble learning combines students in the space domain. In this
paper, however, we combine students in the time domain and call it time-domain
ensemble learning. We analyze, compare, and discuss the generalization
performances regarding time-domain ensemble learning of both a linear model and
a nonlinear model. Analyzing in the framework of online learning using a
statistical mechanical method, we show the qualitatively different behaviors
between the two models. In a linear model, the dynamical behaviors of the
generalization error are monotonic. We analytically show that time-domain
ensemble learning is twice as effective as conventional ensemble learning.
Furthermore, the generalization error of a nonlinear model features
nonmonotonic dynamical behaviors when the learning rate is small. We
numerically show that the generalization performance can be improved remarkably
by using this phenomenon and the divergence of students in the time domain.Comment: 11 pages, 7 figure
Online Learning with Ensembles
Supervised online learning with an ensemble of students randomized by the
choice of initial conditions is analyzed. For the case of the perceptron
learning rule, asymptotically the same improvement in the generalization error
of the ensemble compared to the performance of a single student is found as in
Gibbs learning. For more optimized learning rules, however, using an ensemble
yields no improvement. This is explained by showing that for any learning rule
a transform exists, such that a single student using
has the same generalization behaviour as an ensemble of
-students.Comment: 8 pages, 1 figure. Submitted to J.Phys.
On-line Learning of an Unlearnable True Teacher through Mobile Ensemble Teachers
On-line learning of a hierarchical learning model is studied by a method from
statistical mechanics. In our model a student of a simple perceptron learns
from not a true teacher directly, but ensemble teachers who learn from the true
teacher with a perceptron learning rule. Since the true teacher and the
ensemble teachers are expressed as non-monotonic perceptron and simple ones,
respectively, the ensemble teachers go around the unlearnable true teacher with
the distance between them fixed in an asymptotic steady state. The
generalization performance of the student is shown to exceed that of the
ensemble teachers in a transient state, as was shown in similar
ensemble-teachers models. Further, it is found that moving the ensemble
teachers even in the steady state, in contrast to the fixed ensemble teachers,
is efficient for the performance of the student.Comment: 18 pages, 8 figure
- …