3,820 research outputs found
Artificial Neurons with Arbitrarily Complex Internal Structures
Artificial neurons with arbitrarily complex internal structure are
introduced. The neurons can be described in terms of a set of internal
variables, a set activation functions which describe the time evolution of
these variables and a set of characteristic functions which control how the
neurons interact with one another. The information capacity of attractor
networks composed of these generalized neurons is shown to reach the maximum
allowed bound. A simple example taken from the domain of pattern recognition
demonstrates the increased computational power of these neurons. Furthermore, a
specific class of generalized neurons gives rise to a simple transformation
relating attractor networks of generalized neurons to standard three layer
feed-forward networks. Given this correspondence, we conjecture that the
maximum information capacity of a three layer feed-forward network is 2 bits
per weight.Comment: 22 pages, 2 figure
A three-threshold learning rule approaches the maximal capacity of recurrent neural networks
Understanding the theoretical foundations of how memories are encoded and
retrieved in neural populations is a central challenge in neuroscience. A
popular theoretical scenario for modeling memory function is the attractor
neural network scenario, whose prototype is the Hopfield model. The model has a
poor storage capacity, compared with the capacity achieved with perceptron
learning algorithms. Here, by transforming the perceptron learning rule, we
present an online learning rule for a recurrent neural network that achieves
near-maximal storage capacity without an explicit supervisory error signal,
relying only upon locally accessible information. The fully-connected network
consists of excitatory binary neurons with plastic recurrent connections and
non-plastic inhibitory feedback stabilizing the network dynamics; the memory
patterns are presented online as strong afferent currents, producing a bimodal
distribution for the neuron synaptic inputs. Synapses corresponding to active
inputs are modified as a function of the value of the local fields with respect
to three thresholds. Above the highest threshold, and below the lowest
threshold, no plasticity occurs. In between these two thresholds,
potentiation/depression occurs when the local field is above/below an
intermediate threshold. We simulated and analyzed a network of binary neurons
implementing this rule and measured its storage capacity for different sizes of
the basins of attraction. The storage capacity obtained through numerical
simulations is shown to be close to the value predicted by analytical
calculations. We also measured the dependence of capacity on the strength of
external inputs. Finally, we quantified the statistics of the resulting
synaptic connectivity matrix, and found that both the fraction of zero weight
synapses and the degree of symmetry of the weight matrix increase with the
number of stored patterns.Comment: 24 pages, 10 figures, to be published in PLOS Computational Biolog
Attractor Metadynamics in Adapting Neural Networks
Slow adaption processes, like synaptic and intrinsic plasticity, abound in
the brain and shape the landscape for the neural dynamics occurring on
substantially faster timescales. At any given time the network is characterized
by a set of internal parameters, which are adapting continuously, albeit
slowly. This set of parameters defines the number and the location of the
respective adiabatic attractors. The slow evolution of network parameters hence
induces an evolving attractor landscape, a process which we term attractor
metadynamics. We study the nature of the metadynamics of the attractor
landscape for several continuous-time autonomous model networks. We find both
first- and second-order changes in the location of adiabatic attractors and
argue that the study of the continuously evolving attractor landscape
constitutes a powerful tool for understanding the overall development of the
neural dynamics
Phase Transitions of Neural Networks
The cooperative behaviour of interacting neurons and synapses is studied
using models and methods from statistical physics. The competition between
training error and entropy may lead to discontinuous properties of the neural
network. This is demonstrated for a few examples: Perceptron, associative
memory, learning from examples, generalization, multilayer networks, structure
recognition, Bayesian estimate, on-line training, noise estimation and time
series generation.Comment: Plenary talk for MINERVA workshop on mesoscopics, fractals and neural
networks, Eilat, March 1997 Postscript Fil
Computational physics of the mind
In the XIX century and earlier such physicists as Newton, Mayer, Hooke, Helmholtz and Mach were actively engaged in the research on psychophysics, trying to relate psychological sensations to intensities of physical stimuli. Computational physics allows to simulate complex neural processes giving a chance to answer not only the original psychophysical questions but also to create models of mind. In this paper several approaches relevant to modeling of mind are outlined. Since direct modeling of the brain functions is rather limited due to the complexity of such models a number of approximations is introduced. The path from the brain, or computational neurosciences, to the mind, or cognitive sciences, is sketched, with emphasis on higher cognitive functions such as memory and consciousness. No fundamental problems in understanding of the mind seem to arise. From computational point of view realistic models require massively parallel architectures
- …