1,954 research outputs found
Born to learn: The inspiration, progress, and future of evolved plastic artificial neural networks
Biological plastic neural networks are systems of extraordinary computational
capabilities shaped by evolution, development, and lifetime learning. The
interplay of these elements leads to the emergence of adaptive behavior and
intelligence. Inspired by such intricate natural phenomena, Evolved Plastic
Artificial Neural Networks (EPANNs) use simulated evolution in-silico to breed
plastic neural networks with a large variety of dynamics, architectures, and
plasticity rules: these artificial systems are composed of inputs, outputs, and
plastic components that change in response to experiences in an environment.
These systems may autonomously discover novel adaptive algorithms, and lead to
hypotheses on the emergence of biological adaptation. EPANNs have seen
considerable progress over the last two decades. Current scientific and
technological advances in artificial neural networks are now setting the
conditions for radically new approaches and results. In particular, the
limitations of hand-designed networks could be overcome by more flexible and
innovative solutions. This paper brings together a variety of inspiring ideas
that define the field of EPANNs. The main methods and results are reviewed.
Finally, new opportunities and developments are presented
Seven properties of self-organization in the human brain
The principle of self-organization has acquired a fundamental significance in the newly emerging field of computational philosophy. Self-organizing systems have been described in various domains in science and philosophy including physics, neuroscience, biology and medicine, ecology, and sociology. While system architecture and their general purpose may depend on domain-specific concepts and definitions, there are (at least) seven key properties of self-organization clearly identified in brain systems: 1) modular connectivity, 2) unsupervised learning, 3) adaptive ability, 4) functional resiliency, 5) functional plasticity, 6) from-local-to-global functional organization, and 7) dynamic system growth. These are defined here in the light of insight from neurobiology, cognitive neuroscience and Adaptive Resonance Theory (ART), and physics to show that self-organization achieves stability and functional plasticity while minimizing structural system complexity. A specific example informed by empirical research is discussed to illustrate how modularity, adaptive learning, and dynamic network growth enable stable yet plastic somatosensory representation for human grip force control. Implications for the design of âstrongâ artificial intelligence in robotics are brought forward
Learning Reservoir Dynamics with Temporal Self-Modulation
Reservoir computing (RC) can efficiently process time-series data by
transferring the input signal to randomly connected recurrent neural networks
(RNNs), which are referred to as a reservoir. The high-dimensional
representation of time-series data in the reservoir significantly simplifies
subsequent learning tasks. Although this simple architecture allows fast
learning and facile physical implementation, the learning performance is
inferior to that of other state-of-the-art RNN models. In this paper, to
improve the learning ability of RC, we propose self-modulated RC (SM-RC), which
extends RC by adding a self-modulation mechanism. The self-modulation mechanism
is realized with two gating variables: an input gate and a reservoir gate. The
input gate modulates the input signal, and the reservoir gate modulates the
dynamical properties of the reservoir. We demonstrated that SM-RC can perform
attention tasks where input information is retained or discarded depending on
the input signal. We also found that a chaotic state emerged as a result of
learning in SM-RC. This indicates that self-modulation mechanisms provide RC
with qualitatively different information-processing capabilities. Furthermore,
SM-RC outperformed RC in NARMA and Lorentz model tasks. In particular, SM-RC
achieved a higher prediction accuracy than RC with a reservoir 10 times larger
in the Lorentz model tasks. Because the SM-RC architecture only requires two
additional gates, it is physically implementable as RC, providing a new
direction for realizing edge AI
The Role of Neuromodulators in Cortical Plasticity. A Computational Perspective
Neuromodulators play a ubiquitous role across the brain in regulating plasticity. With recent advances in experimental techniques, it is possible to study the effects of diverse neuromodulatory states in specific brain regions. Neuromodulators are thought to impact plasticity predominantly through two mechanisms: the gating of plasticity and the upregulation of neuronal activity. However, the consequences of these mechanisms are poorly understood and there is a need for both experimental and theoretical exploration. Here we illustrate how neuromodulatory state affects cortical plasticity through these two mechanisms. First, we explore the ability of neuromodulators to gate plasticity by reshaping the learning window for spike-timing-dependent plasticity. Using a simple computational model, we implement four different learning rules and demonstrate their effects on receptive field plasticity. We then compare the neuromodulatory effects of upregulating learning rate versus the effects of upregulating neuronal activity. We find that these seemingly similar mechanisms do not yield the same outcome: upregulating neuronal activity can lead to either a broadening or a sharpening of receptive field tuning, whereas upregulating learning rate only intensifies the sharpening of receptive field tuning. This simple model demonstrates the need for further exploration of the rich landscape of neuromodulator-mediated plasticity. Future experiments, coupled with biologically detailed computational models, will elucidate the diversity of mechanisms by which neuromodulatory state regulates cortical plasticity
Sensitivity and stability: A signal propagation sweet spot in a sheet of recurrent centre crossing neurons
In this paper we demonstrate that signal propagation across a laminar sheet of recurrent neurons is maximised when two conditions are met. First, neurons must be in the so-called centre crossing configuration. Second, the networkâs topology and weights must be such that the network comprises strongly coupled nodes, yet lies within the weakly coupled regime. We develop tools from linear stability analysis with which to describe this regime, and use them to examine the apparent tension between the sensitivity and instability of centre crossing networks
- âŠ