5,702 research outputs found
Statistical Mechanics of Time Domain Ensemble Learning
Conventional ensemble learning combines students in the space domain. On the
other hand, in this paper we combine students in the time domain and call it
time domain ensemble learning. In this paper, we analyze the generalization
performance of time domain ensemble learning in the framework of online
learning using a statistical mechanical method. We treat a model in which both
the teacher and the student are linear perceptrons with noises. Time domain
ensemble learning is twice as effective as conventional space domain ensemble
learning.Comment: 10 pages, 10 figure
Statistical Mechanics of Nonlinear On-line Learning for Ensemble Teachers
We analyze the generalization performance of a student in a model composed of
nonlinear perceptrons: a true teacher, ensemble teachers, and the student. We
calculate the generalization error of the student analytically or numerically
using statistical mechanics in the framework of on-line learning. We treat two
well-known learning rules: Hebbian learning and perceptron learning. As a
result, it is proven that the nonlinear model shows qualitatively different
behaviors from the linear model. Moreover, it is clarified that Hebbian
learning and perceptron learning show qualitatively different behaviors from
each other. In Hebbian learning, we can analytically obtain the solutions. In
this case, the generalization error monotonically decreases. The steady value
of the generalization error is independent of the learning rate. The larger the
number of teachers is and the more variety the ensemble teachers have, the
smaller the generalization error is. In perceptron learning, we have to
numerically obtain the solutions. In this case, the dynamical behaviors of the
generalization error are non-monotonic. The smaller the learning rate is, the
larger the number of teachers is; and the more variety the ensemble teachers
have, the smaller the minimum value of the generalization error is.Comment: 13 pages, 9 figure
On-line Learning of an Unlearnable True Teacher through Mobile Ensemble Teachers
On-line learning of a hierarchical learning model is studied by a method from
statistical mechanics. In our model a student of a simple perceptron learns
from not a true teacher directly, but ensemble teachers who learn from the true
teacher with a perceptron learning rule. Since the true teacher and the
ensemble teachers are expressed as non-monotonic perceptron and simple ones,
respectively, the ensemble teachers go around the unlearnable true teacher with
the distance between them fixed in an asymptotic steady state. The
generalization performance of the student is shown to exceed that of the
ensemble teachers in a transient state, as was shown in similar
ensemble-teachers models. Further, it is found that moving the ensemble
teachers even in the steady state, in contrast to the fixed ensemble teachers,
is efficient for the performance of the student.Comment: 18 pages, 8 figure
Statistical Mechanics of Linear and Nonlinear Time-Domain Ensemble Learning
Conventional ensemble learning combines students in the space domain. In this
paper, however, we combine students in the time domain and call it time-domain
ensemble learning. We analyze, compare, and discuss the generalization
performances regarding time-domain ensemble learning of both a linear model and
a nonlinear model. Analyzing in the framework of online learning using a
statistical mechanical method, we show the qualitatively different behaviors
between the two models. In a linear model, the dynamical behaviors of the
generalization error are monotonic. We analytically show that time-domain
ensemble learning is twice as effective as conventional ensemble learning.
Furthermore, the generalization error of a nonlinear model features
nonmonotonic dynamical behaviors when the learning rate is small. We
numerically show that the generalization performance can be improved remarkably
by using this phenomenon and the divergence of students in the time domain.Comment: 11 pages, 7 figure
Adhesion, friction, and wear of plasma-deposited thin silicon nitride films at temperatures to 700 C
The adhesion, friction, and wear behavior of silicon nitride films deposited by low- and high-frequency plasmas (30 kHz and 13.56 MHz) at various temperatures to 700 C in vacuum were examined. The results of the investigation indicated that the Si/N ratios were much greater for the films deposited at 13.56 MHz than for those deposited at 30 kHz. Amorphous silicon was present in both low- and high-frequency plasma-deposited silicon nitride films. However, more amorphous silicon occurred in the films deposited at 13.56 MHz than in those deposited at 30 kHz. Temperature significantly influenced adhesion, friction, and wear of the silicon nitride films. Wear occurred in the contact area at high temperature. The wear correlated with the increase in adhesion and friction for the low- and high-frequency plasma-deposited films above 600 and 500 C, respectively. The low- and high-frequency plasma-deposited thin silicon nitride films exhibited a capability for lubrication (low adhesion and friction) in vacuum at temperatures to 500 and 400 C, respectively
Ensemble learning of linear perceptron; Online learning theory
Within the framework of on-line learning, we study the generalization error
of an ensemble learning machine learning from a linear teacher perceptron. The
generalization error achieved by an ensemble of linear perceptrons having
homogeneous or inhomogeneous initial weight vectors is precisely calculated at
the thermodynamic limit of a large number of input elements and shows rich
behavior. Our main findings are as follows. For learning with homogeneous
initial weight vectors, the generalization error using an infinite number of
linear student perceptrons is equal to only half that of a single linear
perceptron, and converges with that of the infinite case with O(1/K) for a
finite number of K linear perceptrons. For learning with inhomogeneous initial
weight vectors, it is advantageous to use an approach of weighted averaging
over the output of the linear perceptrons, and we show the conditions under
which the optimal weights are constant during the learning process. The optimal
weights depend on only correlation of the initial weight vectors.Comment: 14 pages, 3 figures, submitted to Physical Review
Optimization of the Asymptotic Property of Mutual Learning Involving an Integration Mechanism of Ensemble Learning
We propose an optimization method of mutual learning which converges into the
identical state of optimum ensemble learning within the framework of on-line
learning, and have analyzed its asymptotic property through the statistical
mechanics method.The proposed model consists of two learning steps: two
students independently learn from a teacher, and then the students learn from
each other through the mutual learning. In mutual learning, students learn from
each other and the generalization error is improved even if the teacher has not
taken part in the mutual learning. However, in the case of different initial
overlaps(direction cosine) between teacher and students, a student with a
larger initial overlap tends to have a larger generalization error than that of
before the mutual learning. To overcome this problem, our proposed optimization
method of mutual learning optimizes the step sizes of two students to minimize
the asymptotic property of the generalization error. Consequently, the
optimized mutual learning converges to a generalization error identical to that
of the optimal ensemble learning. In addition, we show the relationship between
the optimum step size of the mutual learning and the integration mechanism of
the ensemble learning.Comment: 13 pages, 3 figures, submitted to Journal of Physical Society of
Japa
Adhesion, friction, and deformation of ion-beam-deposited boron nitride films
The tribological properties and mechanical strength of boron nitride films were investigated. The BN films were predominantly amorphous and nonstoichiometric and contained small amounts of oxides and carbides. It was found that the yield pressure at full plasticity, the critical load to fracture, and the shear strength of interfacial adhesive bonds (considered as adhesion) depended on the type of metallic substrate on which the BN was deposited. The harder the substrate, the greater the critical load and the adhesion. The yield pressures of the BN film were 12 GPa for the 440C stainless steel substrate, 4.1 GPa for the 304 stainless steel substrate, and 3.3 GPa for the titanium substrate
Analysis of ensemble learning using simple perceptrons based on online learning theory
Ensemble learning of nonlinear perceptrons, which determine their outputs
by sign functions, is discussed within the framework of online learning and
statistical mechanics. One purpose of statistical learning theory is to
theoretically obtain the generalization error. This paper shows that ensemble
generalization error can be calculated by using two order parameters, that is,
the similarity between a teacher and a student, and the similarity among
students. The differential equations that describe the dynamical behaviors of
these order parameters are derived in the case of general learning rules. The
concrete forms of these differential equations are derived analytically in the
cases of three well-known rules: Hebbian learning, perceptron learning and
AdaTron learning. Ensemble generalization errors of these three rules are
calculated by using the results determined by solving their differential
equations. As a result, these three rules show different characteristics in
their affinity for ensemble learning, that is ``maintaining variety among
students." Results show that AdaTron learning is superior to the other two
rules with respect to that affinity.Comment: 30 pages, 17 figure
- …