Pattern-recalling processes in quantum Hopfield networks far from saturation

As a mathematical model of associative memories, the Hopfield model was now well-established and a lot of studies to reveal the pattern-recalling process have been done from various different approaches. As well-known, a single neuron is itself an uncertain, noisy unit with a finite unnegligible error in the input-output relation. To model the situation artificially, a kind of 'heat bath' that surrounds neurons is introduced. The heat bath, which is a source of noise, is specified by the 'temperature'. Several studies concerning the pattern-recalling processes of the Hopfield model governed by the Glauber-dynamics at finite temperature were already reported. However, we might extend the 'thermal noise' to the quantum-mechanical variant. In this paper, in terms of the stochastic process of quantum-mechanical Markov chain Monte Carlo method (the quantum MCMC), we analytically derive macroscopically deterministic equations of order parameters such as 'overlap' in a quantum-mechanical variant of the Hopfield neural networks (let us call "quantum Hopfield model" or "quantum Hopfield networks"). For the case in which non-extensive number $p$ of patterns are embedded via asymmetric Hebbian connections, namely, $p/N \to 0$ for the number of neuron $N \to \infty$ ('far from saturation'), we evaluate the recalling processes for one of the built-in patterns under the influence of quantum-mechanical noise.Comment: 10 pages, 3 figures, using jpconf.cls, Proc. of Statphys-Kolkata VI

The path-integral analysis of an associative memory model storing an infinite number of finite limit cycles

It is shown that an exact solution of the transient dynamics of an associative memory model storing an infinite number of limit cycles with l finite steps by means of the path-integral analysis. Assuming the Maxwell construction ansatz, we have succeeded in deriving the stationary state equations of the order parameters from the macroscopic recursive equations with respect to the finite-step sequence processing model which has retarded self-interactions. We have also derived the stationary state equations by means of the signal-to-noise analysis (SCSNA). The signal-to-noise analysis must assume that crosstalk noise of an input to spins obeys a Gaussian distribution. On the other hand, the path-integral method does not require such a Gaussian approximation of crosstalk noise. We have found that both the signal-to-noise analysis and the path-integral analysis give the completely same result with respect to the stationary state in the case where the dynamics is deterministic, when we assume the Maxwell construction ansatz. We have shown the dependence of storage capacity (alpha_c) on the number of patterns per one limit cycle (l). Storage capacity monotonously increases with the number of steps, and converges to alpha_c=0.269 at l ~= 10. The original properties of the finite-step sequence processing model appear as long as the number of steps of the limit cycle has order l=O(1).Comment: 24 pages, 3 figure

Dynamical Probability Distribution Function of the SK Model at High Temperatures

The microscopic probability distribution function of the Sherrington-Kirkpatrick (SK) model of spin glasses is calculated explicitly as a function of time by a high-temperature expansion. The resulting formula to the third order of the inverse temperature shows that an assumption made by Coolen, Laughton and Sherrington in their recent theory of dynamics is violated. Deviations of their theory from exact results are estimated quantitatively. Our formula also yields explicit expressions of the time dependence of various macroscopic physical quantities when the temperature is suddenly changed within the high-temperature region.Comment: LaTeX, 6 pages, Figures upon request (here revised), To be published in J. Phys. Soc. Jpn. 65 (1996) No.

Synapse efficiency diverges due to synaptic pruning following over-growth

In the development of the brain, it is known that synapses are pruned following over-growth. This pruning following over-growth seems to be a universal phenomenon that occurs in almost all areas -- visual cortex, motor area, association area, and so on. It has been shown numerically that the synapse efficiency is increased by systematic deletion. We discuss the synapse efficiency to evaluate the effect of pruning following over-growth, and analytically show that the synapse efficiency diverges as O(log c) at the limit where connecting rate c is extremely small. Under a fixed synapse number criterion, the optimal connecting rate, which maximize memory performance, exists.Comment: 15 pages, 16 figure

Fundamental properties of Tsallis relative entropy

Fundamental properties for the Tsallis relative entropy in both classical and quantum systems are studied. As one of our main results, we give the parametric extension of the trace inequality between the quantum relative entropy and the minus of the trace of the relative operator entropy given by Hiai and Petz. The monotonicity of the quantum Tsallis relative entropy for the trace preserving completely positive linear map is also shown without the assumption that the density operators are invertible. The generalized Tsallis relative entropy is defined and its subadditivity is shown by its joint convexity. Moreover, the generalized Peierls-Bogoliubov inequality is also proven

Bi-stability of mixed states in neural network storing hierarchical patterns

We discuss the properties of equilibrium states in an autoassociative memory model storing hierarchically correlated patterns (hereafter, hierarchical patterns). We will show that symmetric mixed states (hereafter, mixed states) are bi-stable on the associative memory model storing the hierarchical patterns in a region of the ferromagnetic phase. This means that the first-order transition occurs in this ferromagnetic phase. We treat these contents with a statistical mechanical method (SCSNA) and by computer simulation. Finally, we discuss a physiological implication of this model. Sugase et al. analyzed the time-course of the information carried by the firing of face-responsive neurons in the inferior temporal cortex. We also discuss the relation between the theoretical results and the physiological experiments of Sugase et al.Comment: 18 pages, 6 figure

Associative memory storing an extensive number of patterns based on a network of oscillators with distributed natural frequencies in the presence of external white noise

We study associative memory based on temporal coding in which successful retrieval is realized as an entrainment in a network of simple phase oscillators with distributed natural frequencies under the influence of white noise. The memory patterns are assumed to be given by uniformly distributed random numbers on $[0,2\pi)$ so that the patterns encode the phase differences of the oscillators. To derive the macroscopic order parameter equations for the network with an extensive number of stored patterns, we introduce the effective transfer function by assuming the fixed-point equation of the form of the TAP equation, which describes the time-averaged output as a function of the effective time-averaged local field. Properties of the networks associated with synchronization phenomena for a discrete symmetric natural frequency distribution with three frequency components are studied based on the order parameter equations, and are shown to be in good agreement with the results of numerical simulations. Two types of retrieval states are found to occur with respect to the degree of synchronization, when the size of the width of the natural frequency distribution is changed.Comment: published in Phys. Rev.

Analysis of Bidirectional Associative Memory using SCSNA and Statistical Neurodynamics

Bidirectional associative memory (BAM) is a kind of an artificial neural network used to memorize and retrieve heterogeneous pattern pairs. Many efforts have been made to improve BAM from the the viewpoint of computer application, and few theoretical studies have been done. We investigated the theoretical characteristics of BAM using a framework of statistical-mechanical analysis. To investigate the equilibrium state of BAM, we applied self-consistent signal to noise analysis (SCSNA) and obtained a macroscopic parameter equations and relative capacity. Moreover, to investigate not only the equilibrium state but also the retrieval process of reaching the equilibrium state, we applied statistical neurodynamics to the update rule of BAM and obtained evolution equations for the macroscopic parameters. These evolution equations are consistent with the results of SCSNA in the equilibrium state.Comment: 13 pages, 4 figure