4,352 research outputs found
Statistical Mechanics of Time Domain Ensemble Learning
Conventional ensemble learning combines students in the space domain. On the
other hand, in this paper we combine students in the time domain and call it
time domain ensemble learning. In this paper, we analyze the generalization
performance of time domain ensemble learning in the framework of online
learning using a statistical mechanical method. We treat a model in which both
the teacher and the student are linear perceptrons with noises. Time domain
ensemble learning is twice as effective as conventional space domain ensemble
learning.Comment: 10 pages, 10 figure
Statistical Mechanics of Nonlinear On-line Learning for Ensemble Teachers
We analyze the generalization performance of a student in a model composed of
nonlinear perceptrons: a true teacher, ensemble teachers, and the student. We
calculate the generalization error of the student analytically or numerically
using statistical mechanics in the framework of on-line learning. We treat two
well-known learning rules: Hebbian learning and perceptron learning. As a
result, it is proven that the nonlinear model shows qualitatively different
behaviors from the linear model. Moreover, it is clarified that Hebbian
learning and perceptron learning show qualitatively different behaviors from
each other. In Hebbian learning, we can analytically obtain the solutions. In
this case, the generalization error monotonically decreases. The steady value
of the generalization error is independent of the learning rate. The larger the
number of teachers is and the more variety the ensemble teachers have, the
smaller the generalization error is. In perceptron learning, we have to
numerically obtain the solutions. In this case, the dynamical behaviors of the
generalization error are non-monotonic. The smaller the learning rate is, the
larger the number of teachers is; and the more variety the ensemble teachers
have, the smaller the minimum value of the generalization error is.Comment: 13 pages, 9 figure
On-line Learning of an Unlearnable True Teacher through Mobile Ensemble Teachers
On-line learning of a hierarchical learning model is studied by a method from
statistical mechanics. In our model a student of a simple perceptron learns
from not a true teacher directly, but ensemble teachers who learn from the true
teacher with a perceptron learning rule. Since the true teacher and the
ensemble teachers are expressed as non-monotonic perceptron and simple ones,
respectively, the ensemble teachers go around the unlearnable true teacher with
the distance between them fixed in an asymptotic steady state. The
generalization performance of the student is shown to exceed that of the
ensemble teachers in a transient state, as was shown in similar
ensemble-teachers models. Further, it is found that moving the ensemble
teachers even in the steady state, in contrast to the fixed ensemble teachers,
is efficient for the performance of the student.Comment: 18 pages, 8 figure
Statistical Mechanics of Linear and Nonlinear Time-Domain Ensemble Learning
Conventional ensemble learning combines students in the space domain. In this
paper, however, we combine students in the time domain and call it time-domain
ensemble learning. We analyze, compare, and discuss the generalization
performances regarding time-domain ensemble learning of both a linear model and
a nonlinear model. Analyzing in the framework of online learning using a
statistical mechanical method, we show the qualitatively different behaviors
between the two models. In a linear model, the dynamical behaviors of the
generalization error are monotonic. We analytically show that time-domain
ensemble learning is twice as effective as conventional ensemble learning.
Furthermore, the generalization error of a nonlinear model features
nonmonotonic dynamical behaviors when the learning rate is small. We
numerically show that the generalization performance can be improved remarkably
by using this phenomenon and the divergence of students in the time domain.Comment: 11 pages, 7 figure
Oscillation Phenomena in the disk around the massive black hole Sagittarius A*
We report the detection of radio QPOs with structure changes using the Very
Long Baseline Array (VLBA) at 43 GHz. We found conspicuous patterned changes of
the structure with P = 16.8 +- 1.4, 22.2 +- 1.4, 31.2 +- 1.5, 56.4 +- 6 min,
very roughly in a 3:4:6:10 ratio. The first two periods show a rotating one-arm
structure, while the P = 31.4 min shows a rotating 3-arm structure, as if
viewed edge-on. At the central 50 microasec the P = 56.4 min period shows a
double amplitude variation of those in its surroundings. Spatial distributions
of the oscillation periods suggest that the disk of SgrA* is roughly edge-on,
rotating around an axis with PA = -10 degree. Presumably, the observed VLBI
images of SgrA* at 43 GHz retain several features of the black hole accretion
disk of SgrA* in spite of being obscured and broadened by scattering of
surrounding plasma.Comment: 24 pages, 20 figures, revised version submitted to MN main journal
(2010, Jan., 12th
Average profiles of the solar wind and outer radiation belt during the extreme flux enhancement of relativistic electrons at geosynchronous orbit
We report average profiles of the solar wind and outer radiation belt during the extreme flux enhancement of relativistic electrons at geosynchronous orbit (GEO). It is found that seven of top ten extreme events at GEO during solar cycle 23 are associated with the magnetosphere inflation during the storm recovery phase as caused by the large-scale solar wind structure of very low dynamic pressure (<1.0 nPa) during rapid speed decrease from very high (>650 km/s) to typical (400–500 km/s) in a few days. For the seven events, the solar wind parameters, geomagnetic activity indices, and relativistic electron flux and geomagnetic field at GEO are superposed at the local noon period of GOES satellites to investigate the physical cause. The average profiles support the "double inflation" mechanism that the rarefaction of the solar wind and subsequent magnetosphere inflation are one of the best conditions to produce the extreme flux enhancement at GEO because of the excellent magnetic confinement of relativistic electrons by reducing the drift loss of trapped electrons at dayside magnetopause
Optimization of the Asymptotic Property of Mutual Learning Involving an Integration Mechanism of Ensemble Learning
We propose an optimization method of mutual learning which converges into the
identical state of optimum ensemble learning within the framework of on-line
learning, and have analyzed its asymptotic property through the statistical
mechanics method.The proposed model consists of two learning steps: two
students independently learn from a teacher, and then the students learn from
each other through the mutual learning. In mutual learning, students learn from
each other and the generalization error is improved even if the teacher has not
taken part in the mutual learning. However, in the case of different initial
overlaps(direction cosine) between teacher and students, a student with a
larger initial overlap tends to have a larger generalization error than that of
before the mutual learning. To overcome this problem, our proposed optimization
method of mutual learning optimizes the step sizes of two students to minimize
the asymptotic property of the generalization error. Consequently, the
optimized mutual learning converges to a generalization error identical to that
of the optimal ensemble learning. In addition, we show the relationship between
the optimum step size of the mutual learning and the integration mechanism of
the ensemble learning.Comment: 13 pages, 3 figures, submitted to Journal of Physical Society of
Japa
Ensemble learning of linear perceptron; Online learning theory
Within the framework of on-line learning, we study the generalization error
of an ensemble learning machine learning from a linear teacher perceptron. The
generalization error achieved by an ensemble of linear perceptrons having
homogeneous or inhomogeneous initial weight vectors is precisely calculated at
the thermodynamic limit of a large number of input elements and shows rich
behavior. Our main findings are as follows. For learning with homogeneous
initial weight vectors, the generalization error using an infinite number of
linear student perceptrons is equal to only half that of a single linear
perceptron, and converges with that of the infinite case with O(1/K) for a
finite number of K linear perceptrons. For learning with inhomogeneous initial
weight vectors, it is advantageous to use an approach of weighted averaging
over the output of the linear perceptrons, and we show the conditions under
which the optimal weights are constant during the learning process. The optimal
weights depend on only correlation of the initial weight vectors.Comment: 14 pages, 3 figures, submitted to Physical Review
- …