931 research outputs found
Transient dynamics for sequence processing neural networks
An exact solution of the transient dynamics for a sequential associative
memory model is discussed through both the path-integral method and the
statistical neurodynamics. Although the path-integral method has the ability to
give an exact solution of the transient dynamics, only stationary properties
have been discussed for the sequential associative memory. We have succeeded in
deriving an exact macroscopic description of the transient dynamics by
analyzing the correlation of crosstalk noise. Surprisingly, the order parameter
equations of this exact solution are completely equivalent to those of the
statistical neurodynamics, which is an approximation theory that assumes
crosstalk noise to obey the Gaussian distribution. In order to examine our
theoretical findings, we numerically obtain cumulants of the crosstalk noise.
We verify that the third- and fourth-order cumulants are equal to zero, and
that the crosstalk noise is normally distributed even in the non-retrieval
case. We show that the results obtained by our theory agree with those obtained
by computer simulations. We have also found that the macroscopic unstable state
completely coincides with the separatrix.Comment: 21 pages, 4 figure
Analysis of Bidirectional Associative Memory using SCSNA and Statistical Neurodynamics
Bidirectional associative memory (BAM) is a kind of an artificial neural
network used to memorize and retrieve heterogeneous pattern pairs. Many efforts
have been made to improve BAM from the the viewpoint of computer application,
and few theoretical studies have been done. We investigated the theoretical
characteristics of BAM using a framework of statistical-mechanical analysis. To
investigate the equilibrium state of BAM, we applied self-consistent signal to
noise analysis (SCSNA) and obtained a macroscopic parameter equations and
relative capacity. Moreover, to investigate not only the equilibrium state but
also the retrieval process of reaching the equilibrium state, we applied
statistical neurodynamics to the update rule of BAM and obtained evolution
equations for the macroscopic parameters. These evolution equations are
consistent with the results of SCSNA in the equilibrium state.Comment: 13 pages, 4 figure
Storage Capacity Diverges with Synaptic Efficiency in an Associative Memory Model with Synaptic Delay and Pruning
It is known that storage capacity per synapse increases by synaptic pruning
in the case of a correlation-type associative memory model. However, the
storage capacity of the entire network then decreases. To overcome this
difficulty, we propose decreasing the connecting rate while keeping the total
number of synapses constant by introducing delayed synapses. In this paper, a
discrete synchronous-type model with both delayed synapses and their prunings
is discussed as a concrete example of the proposal. First, we explain the
Yanai-Kim theory by employing the statistical neurodynamics. This theory
involves macrodynamical equations for the dynamics of a network with serial
delay elements. Next, considering the translational symmetry of the explained
equations, we re-derive macroscopic steady state equations of the model by
using the discrete Fourier transformation. The storage capacities are analyzed
quantitatively. Furthermore, two types of synaptic prunings are treated
analytically: random pruning and systematic pruning. As a result, it becomes
clear that in both prunings, the storage capacity increases as the length of
delay increases and the connecting rate of the synapses decreases when the
total number of synapses is constant. Moreover, an interesting fact becomes
clear: the storage capacity asymptotically approaches due to random
pruning. In contrast, the storage capacity diverges in proportion to the
logarithm of the length of delay by systematic pruning and the proportion
constant is . These results theoretically support the significance of
pruning following an overgrowth of synapses in the brain and strongly suggest
that the brain prefers to store dynamic attractors such as sequences and limit
cycles rather than equilibrium states.Comment: 27 pages, 14 figure
Analysis of Oscillator Neural Networks for Sparsely Coded Phase Patterns
We study a simple extended model of oscillator neural networks capable of
storing sparsely coded phase patterns, in which information is encoded both in
the mean firing rate and in the timing of spikes. Applying the methods of
statistical neurodynamics to our model, we theoretically investigate the
model's associative memory capability by evaluating its maximum storage
capacities and deriving its basins of attraction. It is shown that, as in the
Hopfield model, the storage capacity diverges as the activity level decreases.
We consider various practically and theoretically important cases. For example,
it is revealed that a dynamically adjusted threshold mechanism enhances the
retrieval ability of the associative memory. It is also found that, under
suitable conditions, the network can recall patterns even in the case that
patterns with different activity levels are stored at the same time. In
addition, we examine the robustness with respect to damage of the synaptic
connections. The validity of these theoretical results is confirmed by
reasonable agreement with numerical simulations.Comment: 23 pages, 11 figure
The path-integral analysis of an associative memory model storing an infinite number of finite limit cycles
It is shown that an exact solution of the transient dynamics of an
associative memory model storing an infinite number of limit cycles with l
finite steps by means of the path-integral analysis. Assuming the Maxwell
construction ansatz, we have succeeded in deriving the stationary state
equations of the order parameters from the macroscopic recursive equations with
respect to the finite-step sequence processing model which has retarded
self-interactions. We have also derived the stationary state equations by means
of the signal-to-noise analysis (SCSNA). The signal-to-noise analysis must
assume that crosstalk noise of an input to spins obeys a Gaussian distribution.
On the other hand, the path-integral method does not require such a Gaussian
approximation of crosstalk noise. We have found that both the signal-to-noise
analysis and the path-integral analysis give the completely same result with
respect to the stationary state in the case where the dynamics is
deterministic, when we assume the Maxwell construction ansatz.
We have shown the dependence of storage capacity (alpha_c) on the number of
patterns per one limit cycle (l). Storage capacity monotonously increases with
the number of steps, and converges to alpha_c=0.269 at l ~= 10. The original
properties of the finite-step sequence processing model appear as long as the
number of steps of the limit cycle has order l=O(1).Comment: 24 pages, 3 figure
Neural network based architectures for aerospace applications
A brief history of the field of neural networks research is given and some simple concepts are described. In addition, some neural network based avionics research and development programs are reviewed. The need for the United States Air Force and NASA to assume a leadership role in supporting this technology is stressed
Platonic model of mind as an approximation to neurodynamics
Hierarchy of approximations involved in simplification of microscopic theories, from sub-cellural to the whole brain level, is presented. A new approximation to neural dynamics is described, leading to a Platonic-like model of mind based on psychological spaces. Objects and events in these spaces correspond to quasi-stable states of brain dynamics and may be interpreted from psychological point of view. Platonic model bridges the gap between neurosciences and psychological sciences. Static and dynamic versions of this model are outlined and Feature Space Mapping, a neurofuzzy realization of the static version of Platonic model, described. Categorization experiments with human subjects are analyzed from the neurodynamical and Platonic model points of view
- …