Evolving Plasticity for Autonomous Learning under Changing Environmental Conditions

A fundamental aspect of learning in biological neural networks is the plasticity property which allows them to modify their configurations during their lifetime. Hebbian learning is a biologically plausible mechanism for modeling the plasticity property in artificial neural networks (ANNs), based on the local interactions of neurons. However, the emergence of a coherent global learning behavior from local Hebbian plasticity rules is not very well understood. The goal of this work is to discover interpretable local Hebbian learning rules that can provide autonomous global learning. To achieve this, we use a discrete representation to encode the learning rules in a finite search space. These rules are then used to perform synaptic changes, based on the local interactions of the neurons. We employ genetic algorithms to optimize these rules to allow learning on two separate tasks (a foraging and a prey-predator scenario) in online lifetime learning settings. The resulting evolved rules converged into a set of well-defined interpretable types, that are thoroughly discussed. Notably, the performance of these rules, while adapting the ANNs during the learning tasks, is comparable to that of offline learning methods such as hill climbing.Comment: Evolutionary Computation Journa

Born to learn: The inspiration, progress, and future of evolved plastic artificial neural networks

Biological plastic neural networks are systems of extraordinary computational capabilities shaped by evolution, development, and lifetime learning. The interplay of these elements leads to the emergence of adaptive behavior and intelligence. Inspired by such intricate natural phenomena, Evolved Plastic Artificial Neural Networks (EPANNs) use simulated evolution in-silico to breed plastic neural networks with a large variety of dynamics, architectures, and plasticity rules: these artificial systems are composed of inputs, outputs, and plastic components that change in response to experiences in an environment. These systems may autonomously discover novel adaptive algorithms, and lead to hypotheses on the emergence of biological adaptation. EPANNs have seen considerable progress over the last two decades. Current scientific and technological advances in artificial neural networks are now setting the conditions for radically new approaches and results. In particular, the limitations of hand-designed networks could be overcome by more flexible and innovative solutions. This paper brings together a variety of inspiring ideas that define the field of EPANNs. The main methods and results are reviewed. Finally, new opportunities and developments are presented

A Theory of the Transition to Critical Period Plasticity: Inhibition Selectively Suppresses Spontaneous Activity

SummaryWhat causes critical periods (CPs) to open? For the best-studied case, ocular dominance plasticity in primary visual cortex in response to monocular deprivation (MD), the maturation of inhibition is necessary and sufficient. How does inhibition open the CP? We present a theory: the transition from pre-CP to CP plasticity arises because inhibition preferentially suppresses responses to spontaneous relative to visually driven input activity, switching learning cues from internal to external sources. This differs from previous proposals in (1) arguing that the CP can open without changes in plasticity mechanisms when activity patterns become more sensitive to sensory experience through circuit development, and (2) explaining not simply a transition from no plasticity to plasticity, but a change in outcome of MD-induced plasticity from pre-CP to CP. More broadly, hierarchical organization of sensory-motor pathways may develop through a cascade of CPs induced as circuit maturation progresses from “lower” to “higher” cortical areas

Temporal ordering of input modulates connectivity formation in a developmental neuronal network model of the cortex

Preterm infant brain activity is discontinuous; bursts of activity recorded using EEG (electroencephalography), thought to be driven by subcortical regions, display scale free properties and exhibit a complex temporal ordering known as long-range temporal correlations (LRTCs). During brain development, activity-dependent mechanisms are essential for synaptic connectivity formation, and abolishing burst activity in animal models leads to weak disorganised synaptic connectivity. Moreover, synaptic pruning shares similar mechanisms to spike-timing dependent plasticity (STDP), suggesting that the timing of activity may play a critical role in connectivity formation. We investigated, in a computational model of leaky integrate-and-fire neurones, whether the temporal ordering of burst activity within an external driving input could modulate connectivity formation in the network. Connectivity evolved across the course of simulations using an approach analogous to STDP, from networks with initial random connectivity. Small-world connectivity and hub neurones emerged in the network structure—characteristic properties of mature brain networks. Notably, driving the network with an external input which exhibited LRTCs in the temporal ordering of burst activity facilitated the emergence of these network properties, increasing the speed with which they emerged compared with when the network was driven by the same input with the bursts randomly ordered in time. Moreover, the emergence of small-world properties was dependent on the strength of the LRTCs. These results suggest that the temporal ordering of burst activity could play an important role in synaptic connectivity formation and the emergence of small-world topology in the developing brain

Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of neoHebbian Three-Factor Learning Rules

Most elementary behaviors such as moving the arm to grasp an object or walking into the next room to explore a museum evolve on the time scale of seconds; in contrast, neuronal action potentials occur on the time scale of a few milliseconds. Learning rules of the brain must therefore bridge the gap between these two different time scales. Modern theories of synaptic plasticity have postulated that the co-activation of pre- and postsynaptic neurons sets a flag at the synapse, called an eligibility trace, that leads to a weight change only if an additional factor is present while the flag is set. This third factor, signaling reward, punishment, surprise, or novelty, could be implemented by the phasic activity of neuromodulators or specific neuronal inputs signaling special events. While the theoretical framework has been developed over the last decades, experimental evidence in support of eligibility traces on the time scale of seconds has been collected only during the last few years. Here we review, in the context of three-factor rules of synaptic plasticity, four key experiments that support the role of synaptic eligibility traces in combination with a third factor as a biological implementation of neoHebbian three-factor learning rules

The Applicability of Spike Time Dependent Plasticity to Development

Spike time dependent plasticity (STDP) has been observed in both developing and adult animals. Theoretical studies suggest that it implicitly leads to both competition and homeostasis in addition to correlation-based plasticity, making it a good candidate to explain developmental refinement and plasticity in a number of systems. However, it has only been observed to play a clear role in development in a small number of cases. Because the fast time scales necessary to elicit STDP, it would likely be inefficient in governing synaptic modifications in the absence of fast correlations in neural activity. In contrast, later stages of development often depend on sensory inputs that can drive activity on much faster time scales, suggesting a role in STDP in many sensory systems after opening of the eyes and ear canals. Correlations on fast time scales can be also be present earlier in developing microcircuits, such as those produced by specific transient “teacher” circuits in the cerebral cortex. By reviewing examples of each case, we suggest that STDP is not a universal rule, but rather might be masked or phased in, depending on the information available to instruct refinement in different developing circuits. Thus, this review describes selected cases where STDP has been studied in developmental contexts, and uses these examples to suggest a more general framework for understanding where it could be playing a role in development

Phenomenological models of synaptic plasticity based on spike timing

Synaptic plasticity is considered to be the biological substrate of learning and memory. In this document we review phenomenological models of short-term and long-term synaptic plasticity, in particular spike-timing dependent plasticity (STDP). The aim of the document is to provide a framework for classifying and evaluating different models of plasticity. We focus on phenomenological synaptic models that are compatible with integrate-and-fire type neuron models where each neuron is described by a small number of variables. This implies that synaptic update rules for short-term or long-term plasticity can only depend on spike timing and, potentially, on membrane potential, as well as on the value of the synaptic weight, or on low-pass filtered (temporally averaged) versions of the above variables. We examine the ability of the models to account for experimental data and to fulfill expectations derived from theoretical considerations. We further discuss their relations to teacher-based rules (supervised learning) and reward-based rules (reinforcement learning). All models discussed in this paper are suitable for large-scale network simulation

AMPA Receptor Trafficking in Homeostatic Synaptic Plasticity: Functional Molecules and Signaling Cascades

Homeostatic synaptic plasticity is a negative-feedback response employed to compensate for functional disturbances in the nervous system. Typically, synaptic activity is strengthened when neuronal firing is chronically suppressed or weakened when neuronal activity is chronically elevated. At both the whole cell and entire network levels, activity manipulation leads to a global up- or downscaling of the transmission efficacy of all synapses. However, the homeostatic response can also be induced locally at subcellular regions or individual synapses. Homeostatic synaptic scaling is expressed mainly via the regulation of α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid receptor (AMPAR) trafficking and synaptic expression. Here we review the recently identified functional molecules and signaling pathways that are involved in homeostatic plasticity, especially the homeostatic regulation of AMPAR localization at excitatory synapses