Thouless-Anderson-Palmer equation for analog neural network with temporally fluctuating white synaptic noise

Effects of synaptic noise on the retrieval process of associative memory neural networks are studied from the viewpoint of neurobiological and biophysical understanding of information processing in the brain. We investigate the statistical mechanical properties of stochastic analog neural networks with temporally fluctuating synaptic noise, which is assumed to be white noise. Such networks, in general, defy the use of the replica method, since they have no energy concept. The self-consistent signal-to-noise analysis (SCSNA), which is an alternative to the replica method for deriving a set of order parameter equations, requires no energy concept and thus becomes available in studying networks without energy functions. Applying the SCSNA to stochastic network requires the knowledge of the Thouless-Anderson-Palmer (TAP) equation which defines the deterministic networks equivalent to the original stochastic ones. The study of the TAP equation which is of particular interest for the case without energy concept is very few, while it is closely related to the SCSNA in the case with energy concept. This paper aims to derive the TAP equation for networks with synaptic noise together with a set of order parameter equations by a hybrid use of the cavity method and the SCSNA.Comment: 13 pages, 3 figure

Phase transitions driven by L\'evy stable noise: exact solutions and stability analysis of nonlinear fractional Fokker-Planck equations

Phase transitions and effects of external noise on many body systems are one of the main topics in physics. In mean field coupled nonlinear dynamical stochastic systems driven by Brownian noise, various types of phase transitions including nonequilibrium ones may appear. A Brownian motion is a special case of L\'evy motion and the stochastic process based on the latter is an alternative choice for studying cooperative phenomena in various fields. Recently, fractional Fokker-Planck equations associated with L\'evy noise have attracted much attention and behaviors of systems with double-well potential subjected to L\'evy noise have been studied intensively. However, most of such studies have resorted to numerical computation. We construct an {\it analytically solvable model} to study the occurrence of phase transitions driven by L\'evy stable noise.Comment: submitted to EP

Oscillator neural network model with distributed native frequencies

We study associative memory of an oscillator neural network with distributed native frequencies. The model is based on the use of the Hebb learning rule with random patterns ($\xi_i^{\mu}=\pm 1$), and the distribution function of native frequencies is assumed to be symmetric with respect to its average. Although the system with an extensive number of stored patterns is not allowed to get entirely synchronized, long time behaviors of the macroscopic order parameters describing partial synchronization phenomena can be obtained by discarding the contribution from the desynchronized part of the system. The oscillator network is shown to work as associative memory accompanied by synchronized oscillations. A phase diagram representing properties of memory retrieval is presented in terms of the parameters characterizing the native frequency distribution. Our analytical calculations based on the self-consistent signal-to-noise analysis are shown to be in excellent agreement with numerical simulations, confirming the validity of our theoretical treatment.Comment: 9 pages, revtex, 6 postscript figures, to be published in J. Phys.

The Blume-Emery-Griffiths neural network: dynamics for arbitrary temperature

The parallel dynamics of the fully connected Blume-Emery-Griffiths neural network model is studied for arbitrary temperature. By employing a probabilistic signal-to-noise approach, a recursive scheme is found determining the time evolution of the distribution of the local fields and, hence, the evolution of the order parameters. A comparison of this approach is made with the generating functional method, allowing to calculate any physical relevant quantity as a function of time. Explicit analytic formula are given in both methods for the first few time steps of the dynamics. Up to the third time step the results are identical. Some arguments are presented why beyond the third time step the results differ for certain values of the model parameters. Furthermore, fixed-point equations are derived in the stationary limit. Numerical simulations confirm our theoretical findings.Comment: 26 pages in Latex, 8 eps figure

Equivalence among different formalisms in the Tsallis entropy framework

In a recent paper [Phys. Lett. A {\bf335}, 351 (2005)] the authors discussed the equivalence among the various probability distribution functions of a system in equilibrium in the Tsallis entropy framework. In the present letter we extend these results to a system which is out of equilibrium and evolves to a stationary state according to a nonlinear Fokker-Planck equation. By means of time-scale conversion, it is shown that there exists a ``correspondence'' among the self-similar solutions of the nonlinear Fokker-Planck equations associated with the different Tsallis formalisms. The time-scale conversion is related to the corresponding Lyapunov functions of the respective nonlinear Fokker-Planck equations.Comment: 20 pages, 1 figure, Elsart macro style, version accepted on Physica

Acceleration effect of coupled oscillator systems

We have developed a curved isochron clock (CIC) by modifying the radial isochron clock to provide a clean example of the acceleration (deceleration) effect. By analyzing a two-body system of coupled CICs, we determined that an unbalanced mutual interaction caused by curved isochron sets is the minimum mechanism needed for generating the acceleration (deceleration) effect in coupled oscillator systems. From this we can see that the Sakaguchi and Kuramoto (SK) model which is a class of non-frustrated mean feild model has an acceleration (deceleration) effect mechanism. To study frustrated coupled oscillator systems, we extended the SK model to two oscillator associative memory models, one with symmetric and one with asymmetric dilution of coupling, which also have the minimum mechanism of the acceleration (deceleration) effect. We theoretically found that the {\it Onsager reaction term} (ORT), which is unique to frustrated systems, plays an important role in the acceleration (de! celeration) effect. These two models are ideal for evaluating the effect of the ORT because, with the exception of the ORT, they have the same order parameter equations. We found that the two models have identical macroscopic properties, except for the acceleration effect caused by the ORT. By comparing the results of the two models, we can extract the effect of the ORT from only the rotation speeds of the oscillators.Comment: 35 pages, 10 figure

Dynamical Probability Distribution Function of the SK Model at High Temperatures

The microscopic probability distribution function of the Sherrington-Kirkpatrick (SK) model of spin glasses is calculated explicitly as a function of time by a high-temperature expansion. The resulting formula to the third order of the inverse temperature shows that an assumption made by Coolen, Laughton and Sherrington in their recent theory of dynamics is violated. Deviations of their theory from exact results are estimated quantitatively. Our formula also yields explicit expressions of the time dependence of various macroscopic physical quantities when the temperature is suddenly changed within the high-temperature region.Comment: LaTeX, 6 pages, Figures upon request (here revised), To be published in J. Phys. Soc. Jpn. 65 (1996) No.

An associative network with spatially organized connectivity

We investigate the properties of an autoassociative network of threshold-linear units whose synaptic connectivity is spatially structured and asymmetric. Since the methods of equilibrium statistical mechanics cannot be applied to such a network due to the lack of a Hamiltonian, we approach the problem through a signal-to-noise analysis, that we adapt to spatially organized networks. The conditions are analyzed for the appearance of stable, spatially non-uniform profiles of activity with large overlaps with one of the stored patterns. It is also shown, with simulations and analytic results, that the storage capacity does not decrease much when the connectivity of the network becomes short range. In addition, the method used here enables us to calculate exactly the storage capacity of a randomly connected network with arbitrary degree of dilution.Comment: 27 pages, 6 figures; Accepted for publication in JSTA

Associative memory storing an extensive number of patterns based on a network of oscillators with distributed natural frequencies in the presence of external white noise

We study associative memory based on temporal coding in which successful retrieval is realized as an entrainment in a network of simple phase oscillators with distributed natural frequencies under the influence of white noise. The memory patterns are assumed to be given by uniformly distributed random numbers on $[0,2\pi)$ so that the patterns encode the phase differences of the oscillators. To derive the macroscopic order parameter equations for the network with an extensive number of stored patterns, we introduce the effective transfer function by assuming the fixed-point equation of the form of the TAP equation, which describes the time-averaged output as a function of the effective time-averaged local field. Properties of the networks associated with synchronization phenomena for a discrete symmetric natural frequency distribution with three frequency components are studied based on the order parameter equations, and are shown to be in good agreement with the results of numerical simulations. Two types of retrieval states are found to occur with respect to the degree of synchronization, when the size of the width of the natural frequency distribution is changed.Comment: published in Phys. Rev.