Dynamics of on-line Hebbian learning with structurally unrealizable restricted training sets

We present an exact solution for the dynamics of on-line Hebbian learning in neural networks, with restricted and unrealizable training sets. In contrast to other studies on learning with restricted training sets, unrealizability is here caused by structural mismatch, rather than data noise: the teacher machine is a perceptron with a reversed wedge-type transfer function, while the student machine is a perceptron with a sigmoidal transfer function. We calculate the glassy dynamics of the macroscopic performance measures, training error and generalization error, and the (non-Gaussian) student field distribution. Our results, which find excellent confirmation in numerical simulations, provide a new benchmark test for general formalisms with which to study unrealizable learning processes with restricted training sets.Comment: 7 pages including 3 figures, using IOP latex2e preprint class fil

Multiplpe Choice Minority Game With Different Publicly Known Histories

In the standard Minority Game, players use historical minority choices as the sole public information to pick one out of the two alternatives. However, publishing historical minority choices is not the only way to present global system information to players when more than two alternatives are available. Thus, it is instructive to study the dynamics and cooperative behaviors of this extended game as a function of the global information provided. We numerically find that although the system dynamics depends on the kind of public information given to the players, the degree of cooperation follows the same trend as that of the standard Minority Game. We also explain most of our findings by the crowd-anticrowd theory.Comment: Extensively revised, to appear in New J Phys, 7 pages with 4 figure

Feed-Forward Chains of Recurrent Attractor Neural Networks Near Saturation

We perform a stationary state replica analysis for a layered network of Ising spin neurons, with recurrent Hebbian interactions within each layer, in combination with strictly feed-forward Hebbian interactions between successive layers. This model interpolates between the fully recurrent and symmetric attractor network studied by Amit el al, and the strictly feed-forward attractor network studied by Domany et al. Due to the absence of detailed balance, it is as yet solvable only in the zero temperature limit. The built-in competition between two qualitatively different modes of operation, feed-forward (ergodic within layers) versus recurrent (non- ergodic within layers), is found to induce interesting phase transitions.Comment: 14 pages LaTex with 4 postscript figures submitted to J. Phys.

Closure of Macroscopic Laws in Disordered Spin Systems: A Toy Model

We use a linear system of Langevin spins with disordered interactions as an exactly solvable toy model to investigate a procedure, recently proposed by Coolen and Sherrington, for closing the hierarchy of macroscopic order parameter equations in disordered spin systems. The closure procedure, based on the removal of microscopic memory effects, is shown to reproduce the correct equations for short times and in equilibrium. For intermediate time-scales the procedure does not lead to the exact equations, yet for homogeneous initial conditions succeeds at capturing the main characteristics of the flow in the order parameter plane. The procedure fails in terms of the long-term temporal dependence of the order parameters. For low energy inhomogeneous initial conditions and near criticality (where zero modes appear) deviations in temporal behaviour are most apparent. For homogeneous initial conditions the impact of microscopic memory effects on the evolution of macroscopic order parameters in disordered spin systems appears to be mainly an overall slowing down.Comment: 14 pages, LateX, OUTP-94-24

Dynamical Solution of the On-Line Minority Game

We solve the dynamics of the on-line minority game, with general types of decision noise, using generating functional techniques a la De Dominicis and the temporal regularization procedure of Bedeaux et al. The result is a macroscopic dynamical theory in the form of closed equations for correlation- and response functions defined via an effective continuous-time single-trader process, which are exact in both the ergodic and in the non-ergodic regime of the minority game. Our solution also explains why, although one cannot formally truncate the Kramers-Moyal expansion of the process after the Fokker-Planck term, upon doing so one still finds the correct solution, that the previously proposed diffusion matrices for the Fokker-Planck term are incomplete, and how previously proposed approximations of the market volatility can be traced back to ergodicity assumptions.Comment: 25 pages LaTeX, no figure

Statistical mechanics and stability of a model eco-system

We study a model ecosystem by means of dynamical techniques from disordered systems theory. The model describes a set of species subject to competitive interactions through a background of resources, which they feed upon. Additionally direct competitive or co-operative interaction between species may occur through a random coupling matrix. We compute the order parameters of the system in a fixed point regime, and identify the onset of instability and compute the phase diagram. We focus on the effects of variability of resources, direct interaction between species, co-operation pressure and dilution on the stability and the diversity of the ecosystem. It is shown that resources can be exploited optimally only in absence of co-operation pressure or direct interaction between species.Comment: 23 pages, 13 figures; text of paper modified, discussion extended, references adde

Spin models on random graphs with controlled topologies beyond degree constraints

We study Ising spin models on finitely connected random interaction graphs which are drawn from an ensemble in which not only the degree distribution $p(k)$ can be chosen arbitrarily, but which allows for further fine-tuning of the topology via preferential attachment of edges on the basis of an arbitrary function Q(k,k') of the degrees of the vertices involved. We solve these models using finite connectivity equilibrium replica theory, within the replica symmetric ansatz. In our ensemble of graphs, phase diagrams of the spin system are found to depend no longer only on the chosen degree distribution, but also on the choice made for Q(k,k'). The increased ability to control interaction topology in solvable models beyond prescribing only the degree distribution of the interaction graph enables a more accurate modeling of real-world interacting particle systems by spin systems on suitably defined random graphs.Comment: 21 pages, 4 figures, submitted to J Phys

Dynamical replica theoretic analysis of CDMA detection dynamics

We investigate the detection dynamics of the Gibbs sampler for code-division multiple access (CDMA) multiuser detection. Our approach is based upon dynamical replica theory which allows an analytic approximation to the dynamics. We use this tool to investigate the basins of attraction when phase coexistence occurs and examine its efficacy via comparison with Monte Carlo simulations.Comment: 18 pages, 2 figure

Application of two-parameter dynamical replica theory to retrieval dynamics of associative memory with non-monotonic neurons

The two-parameter dynamical replica theory (2-DRT) is applied to investigate retrieval properties of non-monotonic associative memory, a model which lacks thermodynamic potential functions. 2-DRT reproduces dynamical properties of the model quite well, including the capacity and basin of attraction. Superretrieval state is also discussed in the framework of 2-DRT. The local stability condition of the superretrieval state is given, which provides a better estimate of the region in which superretrieval is observed experimentally than the self-consistent signal-to-noise analysis (SCSNA) does.Comment: 16 pages, 19 postscript figure

Generating Functional Analysis of the Dynamics of the Batch Minority Game with Random External Information

We study the dynamics of the batch minority game, with random external information, using generating functional techniques a la De Dominicis. The relevant control parameter in this model is the ratio $\alpha=p/N$ of the number $p$ of possible values for the external information over the number $N$ of trading agents. In the limit $N\to\infty$ we calculate the location $\alpha_c$ of the phase transition (signaling the onset of anomalous response), and solve the statics for $\alpha>\alpha_c$ exactly. The temporal correlations in global market fluctuations turn out not to decay to zero for infinitely widely separated times. For $\alpha<\alpha_c$ the stationary state is shown to be non-unique. For $\alpha\to 0$ we analyse our equations in leading order in $\alpha$, and find asymptotic solutions with diverging volatility \sigma=\order(\alpha^{-{1/2}}) (as regularly observed in simulations), but also asymptotic solutions with vanishing volatility \sigma=\order(\alpha^{{1/2}}). The former, however, are shown to emerge only if the agents' initial strategy valuations are below a specific critical value.Comment: 15 pages, 6 figures, uses Revtex. Replaced an old version of volatility graph that. Rephrased and updated some reference