5,546 research outputs found
Dynamics of Supervised Learning with Restricted Training Sets
We study the dynamics of supervised learning in layered neural networks, in
the regime where the size of the training set is proportional to the number
of inputs. Here the local fields are no longer described by Gaussian
probability distributions. We show how dynamical replica theory can be used to
predict the evolution of macroscopic observables, including the relevant
performance measures, incorporating the old formalism in the limit
as a special case. For simplicity we restrict ourselves
to single-layer networks and realizable tasks.Comment: 36 pages, latex2e, 12 eps figures (to be publ in: Proc Newton Inst
Workshop on On-Line Learning '97
Superfluidity of Dense He in Vycor
We calculate properties of a model of He in Vycor using the Path Integral
Monte Carlo method. We find that He forms a distinct layered structure with
a highly localized first layer, a disordered second layer with some atoms
delocalized and able to give rise to the observed superfluid response, and
higher layers nearly perfect crystals. The addition of a single He atom was
enough to bring down the total superfluidity by blocking the exchange in the
second layer. Our results are consistent with the persistent liquid layer model
to explain the observations. Such a model may be relevant to the experiments on
bulk solid He, if there is a fine network of grain boundaries in those
systems.Comment: 4 pages, 4 figure
Dynamics of Learning with Restricted Training Sets I: General Theory
We study the dynamics of supervised learning in layered neural networks, in
the regime where the size of the training set is proportional to the number
of inputs. Here the local fields are no longer described by Gaussian
probability distributions and the learning dynamics is of a spin-glass nature,
with the composition of the training set playing the role of quenched disorder.
We show how dynamical replica theory can be used to predict the evolution of
macroscopic observables, including the two relevant performance measures
(training error and generalization error), incorporating the old formalism
developed for complete training sets in the limit as a
special case. For simplicity we restrict ourselves in this paper to
single-layer networks and realizable tasks.Comment: 39 pages, LaTe
Solutions for certain classes of Riccati differential equation
We derive some analytic closed-form solutions for a class of Riccati equation
y'(x)-\lambda_0(x)y(x)\pm y^2(x)=\pm s_0(x), where \lambda_0(x), s_0(x) are
C^{\infty}-functions. We show that if \delta_n=\lambda_n
s_{n-1}-\lambda_{n-1}s_n=0, where \lambda_{n}=
\lambda_{n-1}^\prime+s_{n-1}+\lambda_0\lambda_{n-1} and
s_{n}=s_{n-1}^\prime+s_0\lambda_{k-1}, n=1,2,..., then The Riccati equation has
a solution given by y(x)=\mp s_{n-1}(x)/\lambda_{n-1}(x). Extension to the
generalized Riccati equation y'(x)+P(x)y(x)+Q(x)y^2(x)=R(x) is also
investigated.Comment: 10 page
Effective photon mass and exact translating quantum relativistic structures
Using a variation of the celebrated Volkov solution, the Klein-Gordon
equation for a charged particle is reduced to a set of ordinary differential
equations, exactly solvable in specific cases. The new quantum relativistic
structures can reveal a localization in the radial direction perpendicular to
the wave packet propagation, thanks to a non-vanishing scalar potential. The
external electromagnetic field, the particle current density and the charge
density are determined. The stability analysis of the solutions is performed by
means of numerical simulations. The results are useful for the description of a
charged quantum test particle in the relativistic regime, provided spin effects
are not decisive
Exhibiting cross-diffusion-induced patterns for reaction-diffusion systems on evolving domains and surfaces
The aim of this manuscript is to present for the first time the application of the finite element method for solving reaction-diffusion systems with cross-diffusion on continuously evolving domains and surfaces. Furthermore we present pattern formation generated by the reaction-diffusion systemwith cross-diffusion on evolving domains and surfaces. A two-component reaction-diffusion system with linear cross-diffusion in both u and v is presented. The finite element method is based on the approximation of the domain or surface by a triangulated domain or surface consisting of a union of triangles. For surfaces, the vertices of the triangulation lie on the continuous surface. A finite element space of functions is then defined by taking the continuous functions which are linear affine on each simplex of the triangulated domain or surface. To demonstrate the role of cross-diffusion to the theory of pattern formation, we compute patterns with model kinetic parameter values that belong only to the cross-diffusion parameter space; these do not belong to the standard parameter space for classical reaction-diffusion systems. Numerical results exhibited show the robustness, flexibility, versatility, and generality of our methodology; the methodology can deal with complicated evolution laws of the domain and surface, and these include uniform isotropic and anisotropic growth profiles as well as those profiles driven by chemical concentrations residing in the domain or on the surface
Analysis of dropout learning regarded as ensemble learning
Deep learning is the state-of-the-art in fields such as visual object
recognition and speech recognition. This learning uses a large number of
layers, huge number of units, and connections. Therefore, overfitting is a
serious problem. To avoid this problem, dropout learning is proposed. Dropout
learning neglects some inputs and hidden units in the learning process with a
probability, p, and then, the neglected inputs and hidden units are combined
with the learned network to express the final output. We find that the process
of combining the neglected hidden units with the learned network can be
regarded as ensemble learning, so we analyze dropout learning from this point
of view.Comment: 9 pages, 8 figures, submitted to Conferenc
Optimization of the Asymptotic Property of Mutual Learning Involving an Integration Mechanism of Ensemble Learning
We propose an optimization method of mutual learning which converges into the
identical state of optimum ensemble learning within the framework of on-line
learning, and have analyzed its asymptotic property through the statistical
mechanics method.The proposed model consists of two learning steps: two
students independently learn from a teacher, and then the students learn from
each other through the mutual learning. In mutual learning, students learn from
each other and the generalization error is improved even if the teacher has not
taken part in the mutual learning. However, in the case of different initial
overlaps(direction cosine) between teacher and students, a student with a
larger initial overlap tends to have a larger generalization error than that of
before the mutual learning. To overcome this problem, our proposed optimization
method of mutual learning optimizes the step sizes of two students to minimize
the asymptotic property of the generalization error. Consequently, the
optimized mutual learning converges to a generalization error identical to that
of the optimal ensemble learning. In addition, we show the relationship between
the optimum step size of the mutual learning and the integration mechanism of
the ensemble learning.Comment: 13 pages, 3 figures, submitted to Journal of Physical Society of
Japa
- …