37,008 research outputs found
Effect of dilution in asymmetric recurrent neural networks
We study with numerical simulation the possible limit behaviors of
synchronous discrete-time deterministic recurrent neural networks composed of N
binary neurons as a function of a network's level of dilution and asymmetry.
The network dilution measures the fraction of neuron couples that are
connected, and the network asymmetry measures to what extent the underlying
connectivity matrix is asymmetric. For each given neural network, we study the
dynamical evolution of all the different initial conditions, thus
characterizing the full dynamical landscape without imposing any learning rule.
Because of the deterministic dynamics, each trajectory converges to an
attractor, that can be either a fixed point or a limit cycle. These attractors
form the set of all the possible limit behaviors of the neural network. For
each network, we then determine the convergence times, the limit cycles'
length, the number of attractors, and the sizes of the attractors' basin. We
show that there are two network structures that maximize the number of possible
limit behaviors. The first optimal network structure is fully-connected and
symmetric. On the contrary, the second optimal network structure is highly
sparse and asymmetric. The latter optimal is similar to what observed in
different biological neuronal circuits. These observations lead us to
hypothesize that independently from any given learning model, an efficient and
effective biologic network that stores a number of limit behaviors close to its
maximum capacity tends to develop a connectivity structure similar to one of
the optimal networks we found.Comment: 31 pages, 5 figure
Storage Capacity Diverges with Synaptic Efficiency in an Associative Memory Model with Synaptic Delay and Pruning
It is known that storage capacity per synapse increases by synaptic pruning
in the case of a correlation-type associative memory model. However, the
storage capacity of the entire network then decreases. To overcome this
difficulty, we propose decreasing the connecting rate while keeping the total
number of synapses constant by introducing delayed synapses. In this paper, a
discrete synchronous-type model with both delayed synapses and their prunings
is discussed as a concrete example of the proposal. First, we explain the
Yanai-Kim theory by employing the statistical neurodynamics. This theory
involves macrodynamical equations for the dynamics of a network with serial
delay elements. Next, considering the translational symmetry of the explained
equations, we re-derive macroscopic steady state equations of the model by
using the discrete Fourier transformation. The storage capacities are analyzed
quantitatively. Furthermore, two types of synaptic prunings are treated
analytically: random pruning and systematic pruning. As a result, it becomes
clear that in both prunings, the storage capacity increases as the length of
delay increases and the connecting rate of the synapses decreases when the
total number of synapses is constant. Moreover, an interesting fact becomes
clear: the storage capacity asymptotically approaches due to random
pruning. In contrast, the storage capacity diverges in proportion to the
logarithm of the length of delay by systematic pruning and the proportion
constant is . These results theoretically support the significance of
pruning following an overgrowth of synapses in the brain and strongly suggest
that the brain prefers to store dynamic attractors such as sequences and limit
cycles rather than equilibrium states.Comment: 27 pages, 14 figure
Discrete and fuzzy dynamical genetic programming in the XCSF learning classifier system
A number of representation schemes have been presented for use within
learning classifier systems, ranging from binary encodings to neural networks.
This paper presents results from an investigation into using discrete and fuzzy
dynamical system representations within the XCSF learning classifier system. In
particular, asynchronous random Boolean networks are used to represent the
traditional condition-action production system rules in the discrete case and
asynchronous fuzzy logic networks in the continuous-valued case. It is shown
possible to use self-adaptive, open-ended evolution to design an ensemble of
such dynamical systems within XCSF to solve a number of well-known test
problems
Global analysis of parallel analog networks with retarded feedback
We analyze the retrieval dynamics of analog ‘‘neural’’ networks with clocked sigmoid elements and multiple signal delays. Proving a conjecture by Marcus and Westervelt, we show that for delay-independent symmetric coupling strengths, the only attractors are fixed points and periodic limit cycles. The same result applies to a larger class of asymmetric networks that may be utilized to store temporal associations with a cyclic structure. We discuss implications for various learning schemes in the space-time domain
Pattern reconstruction and sequence processing in feed-forward layered neural networks near saturation
The dynamics and the stationary states for the competition between pattern
reconstruction and asymmetric sequence processing are studied here in an
exactly solvable feed-forward layered neural network model of binary units and
patterns near saturation. Earlier work by Coolen and Sherrington on a parallel
dynamics far from saturation is extended here to account for finite stochastic
noise due to a Hebbian and a sequential learning rule. Phase diagrams are
obtained with stationary states and quasi-periodic non-stationary solutions.
The relevant dependence of these diagrams and of the quasi-periodic solutions
on the stochastic noise and on initial inputs for the overlaps is explicitly
discussed.Comment: 9 pages, 7 figure
- …