Thouless-Anderson-Palmer equation for analog neural network with temporally fluctuating white synaptic noise

Effects of synaptic noise on the retrieval process of associative memory neural networks are studied from the viewpoint of neurobiological and biophysical understanding of information processing in the brain. We investigate the statistical mechanical properties of stochastic analog neural networks with temporally fluctuating synaptic noise, which is assumed to be white noise. Such networks, in general, defy the use of the replica method, since they have no energy concept. The self-consistent signal-to-noise analysis (SCSNA), which is an alternative to the replica method for deriving a set of order parameter equations, requires no energy concept and thus becomes available in studying networks without energy functions. Applying the SCSNA to stochastic network requires the knowledge of the Thouless-Anderson-Palmer (TAP) equation which defines the deterministic networks equivalent to the original stochastic ones. The study of the TAP equation which is of particular interest for the case without energy concept is very few, while it is closely related to the SCSNA in the case with energy concept. This paper aims to derive the TAP equation for networks with synaptic noise together with a set of order parameter equations by a hybrid use of the cavity method and the SCSNA.Comment: 13 pages, 3 figure

Pattern-recalling processes in quantum Hopfield networks far from saturation

As a mathematical model of associative memories, the Hopfield model was now well-established and a lot of studies to reveal the pattern-recalling process have been done from various different approaches. As well-known, a single neuron is itself an uncertain, noisy unit with a finite unnegligible error in the input-output relation. To model the situation artificially, a kind of 'heat bath' that surrounds neurons is introduced. The heat bath, which is a source of noise, is specified by the 'temperature'. Several studies concerning the pattern-recalling processes of the Hopfield model governed by the Glauber-dynamics at finite temperature were already reported. However, we might extend the 'thermal noise' to the quantum-mechanical variant. In this paper, in terms of the stochastic process of quantum-mechanical Markov chain Monte Carlo method (the quantum MCMC), we analytically derive macroscopically deterministic equations of order parameters such as 'overlap' in a quantum-mechanical variant of the Hopfield neural networks (let us call "quantum Hopfield model" or "quantum Hopfield networks"). For the case in which non-extensive number $p$ of patterns are embedded via asymmetric Hebbian connections, namely, $p/N \to 0$ for the number of neuron $N \to \infty$ ('far from saturation'), we evaluate the recalling processes for one of the built-in patterns under the influence of quantum-mechanical noise.Comment: 10 pages, 3 figures, using jpconf.cls, Proc. of Statphys-Kolkata VI

Controlling chaos in diluted networks with continuous neurons

Diluted neural networks with continuous neurons and nonmonotonic transfer function are studied, with both fixed and dynamic synapses. A noisy stimulus with periodic variance results in a mechanism for controlling chaos in neural systems with fixed synapses: a proper amount of external perturbation forces the system to behave periodically with the same period as the stimulus.Comment: 11 pages, 8 figure

Response Functions Improving Performance in Analog Attractor Neural Networks

In the context of attractor neural networks, we study how the equilibrium analog neural activities, reached by the network dynamics during memory retrieval, may improve storage performance by reducing the interferences between the recalled pattern and the other stored ones. We determine a simple dynamics that stabilizes network states which are highly correlated with the retrieved pattern, for a number of stored memories that does not exceed $\alpha_{\star} N$, where $\alpha_{\star}\in[0,0.41]$ depends on the global activity level in the network and $N$ is the number of neurons.Comment: 13 pages (with figures), LaTex (RevTex), to appear on Phys.Rev.E (RC

Influence of synaptic depression on memory storage capacity

Synaptic efficacy between neurons is known to change within a short time scale dynamically. Neurophysiological experiments show that high-frequency presynaptic inputs decrease synaptic efficacy between neurons. This phenomenon is called synaptic depression, a short term synaptic plasticity. Many researchers have investigated how the synaptic depression affects the memory storage capacity. However, the noise has not been taken into consideration in their analysis. By introducing "temperature", which controls the level of the noise, into an update rule of neurons, we investigate the effects of synaptic depression on the memory storage capacity in the presence of the noise. We analytically compute the storage capacity by using a statistical mechanics technique called Self Consistent Signal to Noise Analysis (SCSNA). We find that the synaptic depression decreases the storage capacity in the case of finite temperature in contrast to the case of the low temperature limit, where the storage capacity does not change

Transient dynamics for sequence processing neural networks

An exact solution of the transient dynamics for a sequential associative memory model is discussed through both the path-integral method and the statistical neurodynamics. Although the path-integral method has the ability to give an exact solution of the transient dynamics, only stationary properties have been discussed for the sequential associative memory. We have succeeded in deriving an exact macroscopic description of the transient dynamics by analyzing the correlation of crosstalk noise. Surprisingly, the order parameter equations of this exact solution are completely equivalent to those of the statistical neurodynamics, which is an approximation theory that assumes crosstalk noise to obey the Gaussian distribution. In order to examine our theoretical findings, we numerically obtain cumulants of the crosstalk noise. We verify that the third- and fourth-order cumulants are equal to zero, and that the crosstalk noise is normally distributed even in the non-retrieval case. We show that the results obtained by our theory agree with those obtained by computer simulations. We have also found that the macroscopic unstable state completely coincides with the separatrix.Comment: 21 pages, 4 figure

Spectroscopic signatures of a bandwidth-controlled Mott transition at the surface of 1T-TaSe2_2

High-resolution angle-resolved photoemission (ARPES) data show that a metal-insulator Mott transition occurs at the surface of the quasi-two dimensional compound TaSe$_2$. The transition is driven by the narrowing of the Ta $5d$ band induced by a temperature-dependent modulation of the atomic positions. A dynamical mean-field theory calculation of the spectral function of the half-filled Hubbard model captures the main qualitative feature of the data, namely the rapid transfer of spectral weight from the observed quasiparticle peak at the Fermi surface to the Hubbard bands, as the correlation gap opens up.Comment: 4 pages, 4 figures; one modified figure, added referenc

Toward Generalized Entropy Composition with Different q Indices and H-Theorem

An attempt is made to construct composable composite entropy with different $q$ indices of subsystems and address the H-theorem problem of the composite system. Though the H-theorem does not hold in general situations, it is shown that some composite entropies do not decrease in time in near-equilibrium states and factorized states with negligibly weak interaction between the subsystems.Comment: 25 pages, corrected some typos, to be published in J. Phys. Soc. Ja

Bi-stability of mixed states in neural network storing hierarchical patterns

We discuss the properties of equilibrium states in an autoassociative memory model storing hierarchically correlated patterns (hereafter, hierarchical patterns). We will show that symmetric mixed states (hereafter, mixed states) are bi-stable on the associative memory model storing the hierarchical patterns in a region of the ferromagnetic phase. This means that the first-order transition occurs in this ferromagnetic phase. We treat these contents with a statistical mechanical method (SCSNA) and by computer simulation. Finally, we discuss a physiological implication of this model. Sugase et al. analyzed the time-course of the information carried by the firing of face-responsive neurons in the inferior temporal cortex. We also discuss the relation between the theoretical results and the physiological experiments of Sugase et al.Comment: 18 pages, 6 figure

An associative network with spatially organized connectivity

We investigate the properties of an autoassociative network of threshold-linear units whose synaptic connectivity is spatially structured and asymmetric. Since the methods of equilibrium statistical mechanics cannot be applied to such a network due to the lack of a Hamiltonian, we approach the problem through a signal-to-noise analysis, that we adapt to spatially organized networks. The conditions are analyzed for the appearance of stable, spatially non-uniform profiles of activity with large overlaps with one of the stored patterns. It is also shown, with simulations and analytic results, that the storage capacity does not decrease much when the connectivity of the network becomes short range. In addition, the method used here enables us to calculate exactly the storage capacity of a randomly connected network with arbitrary degree of dilution.Comment: 27 pages, 6 figures; Accepted for publication in JSTA