Balancing Feed-Forward Excitation and Inhibition via Hebbian Inhibitory Synaptic Plasticity

It has been suggested that excitatory and inhibitory inputs to cortical cells are balanced, and that this balance is important for the highly irregular firing observed in the cortex. There are two hypotheses as to the origin of this balance. One assumes that it results from a stable solution of the recurrent neuronal dynamics. This model can account for a balance of steady state excitation and inhibition without fine tuning of parameters, but not for transient inputs. The second hypothesis suggests that the feed forward excitatory and inhibitory inputs to a postsynaptic cell are already balanced. This latter hypothesis thus does account for the balance of transient inputs. However, it remains unclear what mechanism underlies the fine tuning required for balancing feed forward excitatory and inhibitory inputs. Here we investigated whether inhibitory synaptic plasticity is responsible for the balance of transient feed forward excitation and inhibition. We address this issue in the framework of a model characterizing the stochastic dynamics of temporally anti-symmetric Hebbian spike timing dependent plasticity of feed forward excitatory and inhibitory synaptic inputs to a single post-synaptic cell. Our analysis shows that inhibitory Hebbian plasticity generates ‘negative feedback’ that balances excitation and inhibition, which contrasts with the ‘positive feedback’ of excitatory Hebbian synaptic plasticity. As a result, this balance may increase the sensitivity of the learning dynamics to the correlation structure of the excitatory inputs

Regulation of circuit organization and function through inhibitory synaptic plasticity

Diverse inhibitory neurons in the mammalian brain shape circuit connectivity and dynamics through mechanisms of synaptic plasticity. Inhibitory plasticity can establish excitation/inhibition (E/I) balance, control neuronal firing, and affect local calcium concentration, hence regulating neuronal activity at the network, single neuron, and dendritic level. Computational models can synthesize multiple experimental results and provide insight into how inhibitory plasticity controls circuit dynamics and sculpts connectivity by identifying phenomenological learning rules amenable to mathematical analysis. We highlight recent studies on the role of inhibitory plasticity in modulating excitatory plasticity, forming structured networks underlying memory formation and recall, and implementing adaptive phenomena and novelty detection. We conclude with experimental and modeling progress on the role of interneuron-specific plasticity in circuit computation and context-dependent learning

Functional consequences of inhibitory plasticity: homeostasis, the excitation-inhibition balance and beyond

Computational neuroscience has a long-standing tradition of investigating the consequences of excitatory synaptic plasticity. In contrast, the functions of inhibitory plasticity are still largely nebulous, particularly given the bewildering diversity of interneurons in the brain. Here, we review recent computational advances that provide first suggestions for the functional roles of inhibitory plasticity, such as a maintenance of the excitation-inhibition balance, a stabilization of recurrent network dynamics and a decorrelation of sensory responses. The field is still in its infancy, but given the existing body of theory for excitatory plasticity, it is likely to mature quickly and deliver important insights into the self-organization of inhibitory circuits in the brain

Bio-mimetic Spiking Neural Networks for unsupervised clustering of spatio-temporal data

Spiking neural networks aspire to mimic the brain more closely than traditional artificial neural networks. They are characterised by a spike-like activation function inspired by the shape of an action potential in biological neurons. Spiking networks remain a niche area of research, perform worse than the traditional artificial networks, and their real-world applications are limited. We hypothesised that neuroscience-inspired spiking neural networks with spike-timing-dependent plasticity demonstrate useful learning capabilities. Our objective was to identify features which play a vital role in information processing in the brain but are not commonly used in artificial networks, implement them in spiking networks without copying constraints that apply to living organisms, and to characterise their effect on data processing. The networks we created are not brain models; our approach can be labelled as artificial life. We performed a literature review and selected features such as local weight updates, neuronal sub-types, modularity, homeostasis and structural plasticity. We used the review as a guide for developing the consecutive iterations of the network, and eventually a whole evolutionary developmental system. We analysed the model’s performance on clustering of spatio-temporal data. Our results show that combining evolution and unsupervised learning leads to a faster convergence on the optimal solutions, better stability of fit solutions than each approach separately. The choice of fitness definition affects the network’s performance on fitness-related and unrelated tasks. We found that neuron type-specific weight homeostasis can be used to stabilise the networks, thus enabling longer training. We also demonstrated that networks with a rudimentary architecture can evolve developmental rules which improve their fitness. This interdisciplinary work provides contributions to three fields: it proposes novel artificial intelligence approaches, tests the possible role of the selected biological phenomena in information processing in the brain, and explores the evolution of learning in an artificial life system

Modulating the Granularity of Category Formation by Global Cortical States

The unsupervised categorization of sensory stimuli is typically attributed to feedforward processing in a hierarchy of cortical areas. This purely sensory-driven view of cortical processing, however, ignores any internal modulation, e.g., by top-down attentional signals or neuromodulator release. To isolate the role of internal signaling on category formation, we consider an unbroken continuum of stimuli without intrinsic category boundaries. We show that a competitive network, shaped by recurrent inhibition and endowed with Hebbian and homeostatic synaptic plasticity, can enforce stimulus categorization. The degree of competition is internally controlled by the neuronal gain and the strength of inhibition. Strong competition leads to the formation of many attracting network states, each being evoked by a distinct subset of stimuli and representing a category. Weak competition allows more neurons to be co-active, resulting in fewer but larger categories. We conclude that the granularity of cortical category formation, i.e., the number and size of emerging categories, is not simply determined by the richness of the stimulus environment, but rather by some global internal signal modulating the network dynamics. The model also explains the salient non-additivity of visual object representation observed in the monkey inferotemporal (IT) cortex. Furthermore, it offers an explanation of a previously observed, demand-dependent modulation of IT activity on a stimulus categorization task and of categorization-related cognitive deficits in schizophrenic patients

Synaptic motility and functional stability in the whisker cortex

The high motility of synaptic weights raises the question of how the brain can retain its functionality in the face of constant synaptic remodeling. Here we used the whisker system of rats and mice to study the interplay between synaptic plasticity (motility) and the transmission of sensory signals downstream. Rats and mice probe their surroundings by rhythmically moving their whiskers back and forth. The azimuthal position of a whisker can be estimated from the activity of whisking neurons that respond selectively to a preferred phase along the whisking cycle. These preferred phases are widely distributed on the ring. However, simple models for the transmission of the whisking signal downstream predict a distribution of preferred phases that is an order of magnitude narrower than empirically observed. Here, we suggest that synaptic plasticity in the form of spike-timing-dependent plasticity (STDP) may provide a solution to this conundrum. This hypothesis is addressed in the framework of a modeling study that investigated the STDP dynamics in a population of synapses that propagates the whisking signal downstream. The findings showed that for a wide range of parameters, STDP dynamics do not relax to a fixed point. As a result, the preferred phases of downstream neurons drift in time at a non-uniform velocity which in turn, induces a non-uniform distribution of the preferred phases of the downstream population. This demonstrates how functionality, in terms of the distribution of preferred phases, can be retained not simply despite, but because of the constant synaptic motility. Our analysis leads to several key empirical predictions to test this hypothesis

Experience-driven formation of parts-based representations in a model of layered visual memory

Growing neuropsychological and neurophysiological evidence suggests that the visual cortex uses parts-based representations to encode, store and retrieve relevant objects. In such a scheme, objects are represented as a set of spatially distributed local features, or parts, arranged in stereotypical fashion. To encode the local appearance and to represent the relations between the constituent parts, there has to be an appropriate memory structure formed by previous experience with visual objects. Here, we propose a model how a hierarchical memory structure supporting efficient storage and rapid recall of parts-based representations can be established by an experience-driven process of self-organization. The process is based on the collaboration of slow bidirectional synaptic plasticity and homeostatic unit activity regulation, both running at the top of fast activity dynamics with winner-take-all character modulated by an oscillatory rhythm. These neural mechanisms lay down the basis for cooperation and competition between the distributed units and their synaptic connections. Choosing human face recognition as a test task, we show that, under the condition of open-ended, unsupervised incremental learning, the system is able to form memory traces for individual faces in a parts-based fashion. On a lower memory layer the synaptic structure is developed to represent local facial features and their interrelations, while the identities of different persons are captured explicitly on a higher layer. An additional property of the resulting representations is the sparseness of both the activity during the recall and the synaptic patterns comprising the memory traces.Comment: 34 pages, 12 Figures, 1 Table, published in Frontiers in Computational Neuroscience (Special Issue on Complex Systems Science and Brain Dynamics), http://www.frontiersin.org/neuroscience/computationalneuroscience/paper/10.3389/neuro.10/015.2009

Experience Dependent Cross-modal Regulation of Cortical Circuitry

Sensory experience is essential not only for the formation and maintenance of cortical circuits during development but also throughout life. Neural networks within the brain regulate activity based on experience, using both synapse specific (Hebbian) and global (homeostatic) mechanisms to achieve optimal signal processing without compromising their overall excitability. The loss of one sense can trigger compensation of spared sensory modalities, which is called cross-modal plasticity. These behavioral enhancements are realized through both Hebbian and homeostatic mechanisms to compensate for the loss of one sense, processing spared senses with heightened sensitivity via alterations in cortical circuit strengths. Specifically, spared cortex enhances the feed-forward signal arriving from the thalamus while deprived cortex remains unchanged, regardless of sensory modality. Loss of vision induces enhanced feed-forward signal propagation throughout layer 4 of auditory cortex and up to layer 2/3. In layer 2/3, Hebbian strengthening of feed-forward signals combine with homeostatic scaling down of spontaneous events and weakened lateral inputs to enhance the signal to noise ratio in auditory cortex after loss of sight. These changes in excitation are complemented by alterations in inhibitory transmission, with an increase in spontaneous event frequency in superficial layers, and an increase in parvalbumin mediated evoked inhibition in layer 4. An increase in spontaneous inhibitory synaptic transmission in layer 2/3 may dampen excitable inputs, allowing only the strong and salient signals to impact the network, while stronger evoked inhibition in layer 4 may serve to sharpen tuning as the signal arrives to auditory cortex. Both cross-modal and uni-modal (within the modality) plasticity require similar molecular mechanisms, as the scaling down of spontaneous events in superficial auditory cortex is abolished without the presence of Arc, an activity regulated protein which is known to regulate synaptic AMPA receptor localization. Arc’s involvement with activity regulated production of amyloid beta (Aβ) indicates that Aβ may play a role in normal physiological maintenance of homeostasis in the network. Here we observed an inability of visual cortex layer 2/3 neurons to homeostatically adapt to loss of vision in mice lacking the main enzyme necessary to produce Aβ. Together these results indicate that cross-modal and uni-modal plasticity may use similar molecular mechanisms to homeostatically adapt to changes in sensory environment. The brain’s ability to undergo cross-modal regulation of synaptic strength in response to loss of a sensory modality extends well beyond the classical critical period, and in some cases may be more readily recruited after uni-modal sensory perturbations. The critical period may reflect an optimal balance of excitation and inhibition, which may be reopened throughout life to enable an organism to adapt to their surroundings

Inhibitory synaptic plasticity: spike timing-dependence and putative network function

While the plasticity of excitatory synaptic connections in the brain has been widely studied, the plasticity of inhibitory connections is much less understood. Here, we present recent experimental and theoretical findings concerning the rules of spike timing-dependent inhibitory plasticity and their putative network function. This is a summary of a workshop at the COSYNE conference 2012