Dynamics of Learning with Restricted Training Sets I: General Theory

We study the dynamics of supervised learning in layered neural networks, in the regime where the size $p$ of the training set is proportional to the number $N$ of inputs. Here the local fields are no longer described by Gaussian probability distributions and the learning dynamics is of a spin-glass nature, with the composition of the training set playing the role of quenched disorder. We show how dynamical replica theory can be used to predict the evolution of macroscopic observables, including the two relevant performance measures (training error and generalization error), incorporating the old formalism developed for complete training sets in the limit $\alpha=p/N\to\infty$ as a special case. For simplicity we restrict ourselves in this paper to single-layer networks and realizable tasks.Comment: 39 pages, LaTe

Dynamics of Supervised Learning with Restricted Training Sets

We study the dynamics of supervised learning in layered neural networks, in the regime where the size $p$ of the training set is proportional to the number $N$ of inputs. Here the local fields are no longer described by Gaussian probability distributions. We show how dynamical replica theory can be used to predict the evolution of macroscopic observables, including the relevant performance measures, incorporating the old formalism in the limit $\alpha=p/N\to\infty$ as a special case. For simplicity we restrict ourselves to single-layer networks and realizable tasks.Comment: 36 pages, latex2e, 12 eps figures (to be publ in: Proc Newton Inst Workshop on On-Line Learning '97

The diffuse Nitsche method: Dirichlet constraints on phase-field boundaries

We explore diffuse formulations of Nitsche's method for consistently imposing Dirichlet boundary conditions on phase-field approximations of sharp domains. Leveraging the properties of the phase-field gradient, we derive the variational formulation of the diffuse Nitsche method by transferring all integrals associated with the Dirichlet boundary from a geometrically sharp surface format in the standard Nitsche method to a geometrically diffuse volumetric format. We also derive conditions for the stability of the discrete system and formulate a diffuse local eigenvalue problem, from which the stabilization parameter can be estimated automatically in each element. We advertise metastable phase-field solutions of the Allen-Cahn problem for transferring complex imaging data into diffuse geometric models. In particular, we discuss the use of mixed meshes, that is, an adaptively refined mesh for the phase-field in the diffuse boundary region and a uniform mesh for the representation of the physics-based solution fields. We illustrate accuracy and convergence properties of the diffuse Nitsche method and demonstrate its advantages over diffuse penalty-type methods. In the context of imaging based analysis, we show that the diffuse Nitsche method achieves the same accuracy as the standard Nitsche method with sharp surfaces, if the inherent length scales, i.e., the interface width of the phase-field, the voxel spacing and the mesh size, are properly related. We demonstrate the flexibility of the new method by analyzing stresses in a human vertebral body

Non-negativity preserving numerical algorithms for stochastic differential equations

Construction of splitting-step methods and properties of related non-negativity and boundary preserving numerical algorithms for solving stochastic differential equations (SDEs) of Ito-type are discussed. We present convergence proofs for a newly designed splitting-step algorithm and simulation studies for numerous numerical examples ranging from stochastic dynamics occurring in asset pricing theory in mathematical finance (SDEs of CIR and CEV models) to measure-valued diffusion and superBrownian motion (SPDEs) as met in biology and physics.Comment: 23 pages, 7 figures. Figures 6.2 and 6.3 in low resolution due to upload size restrictions. Original resolution at http://gisc.uc3m.es/~moro/profesional.htm

An application of the Maslov complex germ method to the 1D nonlocal Fisher-KPP equation

A semiclassical approximation approach based on the Maslov complex germ method is considered in detail for the 1D nonlocal Fisher-Kolmogorov-Petrovskii-Piskunov equation under the supposition of weak diffusion. In terms of the semiclassical formalism developed, the original nonlinear equation is reduced to an associated linear partial differential equation and some algebraic equations for the coefficients of the linear equation with a given accuracy of the asymptotic parameter. The solutions of the nonlinear equation are constructed from the solutions of both the linear equation and the algebraic equations. The solutions of the linear problem are found with the use of symmetry operators. A countable family of the leading terms of the semiclassical asymptotics is constructed in explicit form. The semiclassical asymptotics are valid by construction in a finite time interval. We construct asymptotics which are different from the semiclassical ones and can describe evolution of the solutions of the Fisher-Kolmogorov-Petrovskii-Piskunov equation at large times. In the example considered, an initial unimodal distribution becomes multimodal, which can be treated as an example of a space structure.Comment: 28 pages, version accepted for publication in Int. J. Geom. Methods Mod. Phy

Computation of three-dimensional nozzle-exhaust flow fields with the GIM code

A methodology is introduced for constructing numerical analogs of the partial differential equations of continuum mechanics. A general formulation is provided which permits classical finite element and many of the finite difference methods to be derived directly. The approach, termed the General Interpolants Method (GIM), can combined the best features of finite element and finite difference methods. A quasi-variational procedure is used to formulate the element equations, to introduce boundary conditions into the method and to provide a natural assembly sequence. A derivation is given in terms of general interpolation functions from this procedure. Example computations for transonic and supersonic flows in two and three dimensions are given to illustrate the utility of GIM. A three-dimensional nozzle-exhaust flow field is solved including interaction with the freestream and a coupled treatment of the shear layer. Potential applications of the GIM code to a variety of computational fluid dynamics problems is then discussed in terms of existing capability or by extension of the methodology

The Minimum-Uncertainty Squeezed States for for Atoms and Photons in a Cavity

We describe a six-parameter family of the minimum-uncertainty squeezed states for the harmonic oscillator in nonrelativistic quantum mechanics. They are derived by the action of corresponding maximal kinematical invariance group on the standard ground state solution. We show that the product of the variances attains the required minimum value 1/4 only at the instances that one variance is a minimum and the other is a maximum, when the squeezing of one of the variances occurs. The generalized coherent states are explicitly constructed and their Wigner function is studied. The overlap coefficients between the squeezed, or generalized harmonic, and the Fock states are explicitly evaluated in terms of hypergeometric functions. The corresponding photons statistics are discussed and some applications to quantum optics, cavity quantum electrodynamics, and superfocusing in channeling scattering are mentioned. Explicit solutions of the Heisenberg equations for radiation field operators with squeezing are found.Comment: 27 pages, no figures, 174 references J. Phys. B: At. Mol. Opt. Phys., Special Issue celebrating the 20th anniversary of quantum state engineering (R. Blatt, A. Lvovsky, and G. Milburn, Guest Editors), May 201