Dynamics of Supervised Learning with Restricted Training Sets

We study the dynamics of supervised learning in layered neural networks, in the regime where the size $p$ of the training set is proportional to the number $N$ of inputs. Here the local fields are no longer described by Gaussian probability distributions. We show how dynamical replica theory can be used to predict the evolution of macroscopic observables, including the relevant performance measures, incorporating the old formalism in the limit $\alpha=p/N\to\infty$ as a special case. For simplicity we restrict ourselves to single-layer networks and realizable tasks.Comment: 36 pages, latex2e, 12 eps figures (to be publ in: Proc Newton Inst Workshop on On-Line Learning '97

Dynamics of Learning with Restricted Training Sets I: General Theory

We study the dynamics of supervised learning in layered neural networks, in the regime where the size $p$ of the training set is proportional to the number $N$ of inputs. Here the local fields are no longer described by Gaussian probability distributions and the learning dynamics is of a spin-glass nature, with the composition of the training set playing the role of quenched disorder. We show how dynamical replica theory can be used to predict the evolution of macroscopic observables, including the two relevant performance measures (training error and generalization error), incorporating the old formalism developed for complete training sets in the limit $\alpha=p/N\to\infty$ as a special case. For simplicity we restrict ourselves in this paper to single-layer networks and realizable tasks.Comment: 39 pages, LaTe

Optimisation of on-line principal component analysis

Different techniques, used to optimise on-line principal component analysis, are investigated by methods of statistical mechanics. These include local and global optimisation of node-dependent learning-rates which are shown to be very efficient in speeding up the learning process. They are investigated further for gaining insight into the learning rates' time-dependence, which is then employed for devising simple practical methods to improve training performance. Simulations demonstrate the benefit gained from using the new methods.Comment: 10 pages, 5 figure

Duality in Off-Shell Electromagnetism

In this paper, we examine the Dirac monopole in the framework of Off-Shell Electromagnetism, the five dimensional U(1) gauge theory associated with Stueckelberg-Schrodinger relativistic quantum theory. After reviewing the Dirac model in four dimensions, we show that the structure of the five dimensional theory prevents a natural generalization of the Dirac monopole, since the theory is not symmetric under duality transformations. It is shown that the duality symmetry can be restored by generalizing the electromagnetic field strength to an element of a Clifford algebra. Nevertheless, the generalized framework does not permit us to recover the phenomenological (or conventional) absence of magnetic monopoles.Comment: 18 page

A numerical study of two-photon ionization of helium using the Pyprop framework

Few-photon induced breakup of helium is studied using a newly developed ab initio numerical framework for solving the six-dimensional time-dependent Schroedinger equation. We present details of the method and calculate (generalized) cross sections for the process of two-photon nonsequential (direct) double ionization at photon energies ranging from 39.4 to 54.4 eV, a process that has been very much debated in recent years and is not yet fully understood. In particular, we have studied the convergence property of the total cross section in the vicinity of the upper threshold (54.4 eV), versus the pulse duration of the applied laser field. We find that the cross section exhibits an increasing trend near the threshold, as has also been observed by others, and show that this rise cannot solely be attributed to an unintended inclusion of the sequential two-photon double ionization process, caused by the bandwidth of the applied field.Comment: 7 pages, 3 figure

Typical kernel size and number of sparse random matrices over Galois fields: a statistical physics approach

Using methods of statistical physics, we study the average number and kernel size of general sparse random matrices over GF(q), with a given connectivity profile, in the thermodynamical limit of large matrices. We introduce a mapping of GF(q) matrices onto spin systems using the representation of the cyclic group of order q as the q-th complex roots of unity. This representation facilitates the derivation of the average kernel size of random matrices using the replica approach, under the replica symmetric ansatz, resulting in saddle point equations for general connectivity distributions. Numerical solutions are then obtained for particular cases by population dynamics. Similar techniques also allow us to obtain an expression for the exact and average number of random matrices for any general connectivity profile. We present numerical results for particular distributions

Solvable Systems of Linear Differential Equations

The asymptotic iteration method (AIM) is an iterative technique used to find exact and approximate solutions to second-order linear differential equations. In this work, we employed AIM to solve systems of two first-order linear differential equations. The termination criteria of AIM will be re-examined and the whole theory is re-worked in order to fit this new application. As a result of our investigation, an interesting connection between the solution of linear systems and the solution of Riccati equations is established. Further, new classes of exactly solvable systems of linear differential equations with variable coefficients are obtained. The method discussed allow to construct many solvable classes through a simple procedure.Comment: 13 page

Spin texture on the Fermi surface of tensile strained HgTe

We present ab initio and k.p calculations of the spin texture on the Fermi surface of tensile strained HgTe, which is obtained by stretching the zincblende lattice along the (111) axis. Tensile strained HgTe is a semimetal with pointlike accidental degeneracies between a mirror symmetry protected twofold degenerate band and two nondegenerate bands near the Fermi level. The Fermi surface consists of two ellipsoids which contact at the point where the Fermi level crosses the twofold degenerate band along the (111) axis. However, the spin texture of occupied states indicates that neither ellipsoid carries a compensating Chern number. Consequently, the spin texture is locked in the plane perpendicular to the (111) axis, exhibits a nonzero winding number in that plane, and changes winding number from one end of the Fermi ellipsoids to the other. The change in the winding of the spin texture suggests the existence of singular points. An ordered alloy of HgTe with ZnTe has the same effect as stretching the zincblende lattice in the (111) direction. We present ab initio calculations of ordered Hg_xZn_1-xTe that confirm the existence of a spin texture locked in a 2D plane on the Fermi surface with different winding numbers on either end.Comment: 8 pages, 8 figure