2,254 research outputs found
Random Recurrent Neural Networks Dynamics
This paper is a review dealing with the study of large size random recurrent
neural networks. The connection weights are selected according to a probability
law and it is possible to predict the network dynamics at a macroscopic scale
using an averaging principle. After a first introductory section, the section 1
reviews the various models from the points of view of the single neuron
dynamics and of the global network dynamics. A summary of notations is
presented, which is quite helpful for the sequel. In section 2, mean-field
dynamics is developed.
The probability distribution characterizing global dynamics is computed. In
section 3, some applications of mean-field theory to the prediction of chaotic
regime for Analog Formal Random Recurrent Neural Networks (AFRRNN) are
displayed. The case of AFRRNN with an homogeneous population of neurons is
studied in section 4. Then, a two-population model is studied in section 5. The
occurrence of a cyclo-stationary chaos is displayed using the results of
\cite{Dauce01}. In section 6, an insight of the application of mean-field
theory to IF networks is given using the results of \cite{BrunelHakim99}.Comment: Review paper, 36 pages, 5 figure
Collective stability of networks of winner-take-all circuits
The neocortex has a remarkably uniform neuronal organization, suggesting that
common principles of processing are employed throughout its extent. In
particular, the patterns of connectivity observed in the superficial layers of
the visual cortex are consistent with the recurrent excitation and inhibitory
feedback required for cooperative-competitive circuits such as the soft
winner-take-all (WTA). WTA circuits offer interesting computational properties
such as selective amplification, signal restoration, and decision making. But,
these properties depend on the signal gain derived from positive feedback, and
so there is a critical trade-off between providing feedback strong enough to
support the sophisticated computations, while maintaining overall circuit
stability. We consider the question of how to reason about stability in very
large distributed networks of such circuits. We approach this problem by
approximating the regular cortical architecture as many interconnected
cooperative-competitive modules. We demonstrate that by properly understanding
the behavior of this small computational module, one can reason over the
stability and convergence of very large networks composed of these modules. We
obtain parameter ranges in which the WTA circuit operates in a high-gain
regime, is stable, and can be aggregated arbitrarily to form large stable
networks. We use nonlinear Contraction Theory to establish conditions for
stability in the fully nonlinear case, and verify these solutions using
numerical simulations. The derived bounds allow modes of operation in which the
WTA network is multi-stable and exhibits state-dependent persistent activities.
Our approach is sufficiently general to reason systematically about the
stability of any network, biological or technological, composed of networks of
small modules that express competition through shared inhibition.Comment: 7 Figure
Stability in N-Layer recurrent neural networks
Starting with the theory developed by Hopfield, Cohen-Grossberg and Kosko, the study of associative memories is extended to N - layer re-current neural networks. The stability of different multilayer networks is demonstrated under specified bounding hypotheses. The analysis involves theorems for the additive as well as the multiplicative models for continuous and discrete N - layer networks. These demonstrations are based on contin-uous and discrete Liapunov theory. The thesis develops autoassociative and heteroassociative memories. It points out the link between all recurrent net-works of this type. The discrete case is analyzed using the threshold signal function as the activation function. A general approach for studying the sta-bility and convergence of the multilayer recurrent networks is developed
Computation in Dynamically Bounded Asymmetric Systems
Previous explanations of computations performed by recurrent networks have focused on symmetrically connected saturating neurons and their convergence toward attractors. Here we analyze the behavior of asymmetrical connected networks of linear threshold neurons, whose positive response is unbounded. We show that, for a wide range of parameters, this asymmetry brings interesting and computationally useful dynamical properties. When driven by input, the network explores potential solutions through highly unstable ‘expansion’ dynamics. This expansion is steered and constrained by negative divergence of the dynamics, which ensures that the dimensionality of the solution space continues to reduce until an acceptable solution manifold is reached. Then the system contracts stably on this manifold towards its final solution trajectory. The unstable positive feedback and cross inhibition that underlie expansion and divergence are common motifs in molecular and neuronal networks. Therefore we propose that very simple organizational constraints that combine these motifs can lead to spontaneous computation and so to the spontaneous modification of entropy that is characteristic of living systems
Patterns of consumption in a discrete choice model with asymmetric interactions
We study the consumption behaviour of an asymmetric network of heterogeneous
agents in the framework of discrete choice models with stochastic decision
rules. We assume that the interactions among agents are uniquely specified by
their ``social distance'' and consumption is driven by peering, distinction and
aspiration effects. The utility of each agent is positively or negatively
affected by the choices of other agents and consumption is driven by peering,
imitation and distinction effects. The dynamical properties of the model are
explored, by numerical simulations, using three different evolution algorithms
with: parallel, sequential and random-sequential updating rules. We analyze the
long-time behaviour of the system which, given the asymmetric nature of the
interactions, can either converge into a fixed point or a periodic attractor.
We discuss the role of symmetric versus asymmetric contributions to the utility
function and also that of idiosyncratic preferences, costs and memory in the
consumption decision of the agents.Comment: 11 pages, 9 figures, presented at "Complex Behaviour in Economics"
Aix-en-Provence 3-7 May, 2000. Minor modifications made: references added and
typos corrected. This paper is a fully revised version to the one previously
submitted as cond-mat/990913
Griffiths phases and localization in hierarchical modular networks
We study variants of hierarchical modular network models suggested by Kaiser
and Hilgetag [Frontiers in Neuroinformatics, 4 (2010) 8] to model functional
brain connectivity, using extensive simulations and quenched mean-field theory
(QMF), focusing on structures with a connection probability that decays
exponentially with the level index. Such networks can be embedded in
two-dimensional Euclidean space. We explore the dynamic behavior of the contact
process (CP) and threshold models on networks of this kind, including
hierarchical trees. While in the small-world networks originally proposed to
model brain connectivity, the topological heterogeneities are not strong enough
to induce deviations from mean-field behavior, we show that a Griffiths phase
can emerge under reduced connection probabilities, approaching the percolation
threshold. In this case the topological dimension of the networks is finite,
and extended regions of bursty, power-law dynamics are observed. Localization
in the steady state is also shown via QMF. We investigate the effects of link
asymmetry and coupling disorder, and show that localization can occur even in
small-world networks with high connectivity in case of link disorder.Comment: 18 pages, 20 figures, accepted version in Scientific Report
The stability and attractivity of neural associative memories.
Han-bing Ji.Thesis (Ph.D.)--Chinese University of Hong Kong, 1996.Includes bibliographical references (p. 160-163).Microfiche. Ann Arbor, Mich.: UMI, 1998. 2 microfiches ; 11 x 15 cm
- …