Reinforcement learning in populations of spiking neurons

Population coding is widely regarded as a key mechanism for achieving reliable behavioral responses in the face of neuronal variability. But in standard reinforcement learning a flip-side becomes apparent. Learning slows down with increasing population size since the global reinforcement becomes less and less related to the performance of any single neuron. We show that, in contrast, learning speeds up with increasing population size if feedback about the populationresponse modulates synaptic plasticity in addition to global reinforcement. The two feedback signals (reinforcement and population-response signal) can be encoded by ambient neurotransmitter concentrations which vary slowly, yielding a fully online plasticity rule where the learning of a stimulus is interleaved with the processing of the subsequent one. The assumption of a single additional feedback mechanism therefore reconciles biological plausibility with efficient learning

Synchronized dynamics of cortical neurons with time-delay feedback

The dynamics of three mutually coupled cortical neurons with time delays in the coupling are explored numerically and analytically. The neurons are coupled in a line, with the middle neuron sending a somewhat stronger projection to the outer neurons than the feedback it receives, to model for instance the relay of a signal from primary to higher cortical areas. For a given coupling architecture, the delays introduce correlations in the time series at the time-scale of the delay. It was found that the middle neuron leads the outer ones by the delay time, while the outer neurons are synchronized with zero lag times. Synchronization is found to be highly dependent on the synaptic time constant, with faster synapses increasing both the degree of synchronization and the firing rate. Analysis shows that presynaptic input during the interspike interval stabilizes the synchronous state, even for arbitrarily weak coupling, and independent of the initial phase. The finding may be of significance to synchronization of large groups of cells in the cortex that are spatially distanced from each other.Comment: 21 pages, 11 figure

IP3-dependent, post-tetanic calcium transients induced by electrostimulation of adult skeletal muscle fibers

Tetanic electrical stimulation induces two separate calcium signals in rat skeletal myotubes, a fast one, dependent on Cav 1.1 or dihydropyridine receptors (DHPRs) and ryanodine receptors and related to contraction, and a slow signal, dependent on DHPR and inositol trisphosphate receptors (IP3Rs) and related to transcriptional events. We searched for slow calcium signals in adult muscle fibers using isolated adult flexor digitorum brevis fibers from 5–7-wk-old mice, loaded with fluo-3. When stimulated with trains of 0.3-ms pulses at various frequencies, cells responded with a fast calcium signal associated with muscle contraction, followed by a slower signal similar to one previously described in cultured myotubes. Nifedipine inhibited the slow signal more effectively than the fast one, suggesting a role for DHPR in its onset. The IP3R inhibitors Xestospongin B or C (5 µM) also inhibited it. The amplitude of post-tetanic calcium transients depends on both tetanus frequency and duration, having a maximum at 10–20 Hz. At this stimulation frequency, an increase of the slow isoform of troponin I mRNA was detected, while the fast isoform of this gene was inhibited. All three IP3R isoforms were present in adult muscle. IP3R-1 was differentially expressed in different types of muscle fibers, being higher in a subset of fast-type fibers. Interestingly, isolated fibers from the slow soleus muscle did not reveal the slow calcium signal induced by electrical stimulus. These results support the idea that IP3R-dependent slow calcium signals may be characteristic of distinct types of muscle fibers and may participate in the activation of specific transcriptional programs of slow and fast phenotype

Specificity of Synaptic Connectivity between Layer 1 Inhibitory Interneurons and Layer 2/3 Pyramidal Neurons in the Rat Neocortex

Understanding the structure and function of the neocortical microcircuit requires a description of the synaptic connectivity between identified neuronal populations. Here, we investigate the electrophysiological properties of layer 1 (L1) neurons of the rat somatosensory neocortex (postnatal day 24–36) and their synaptic connectivity with supragranular pyramidal neurons. The active and passive properties of visually identified L1 neurons (n = 266) suggested division into 4 groups according to the Petilla classification scheme with characteristics of neurogliaform cells (NGFCs) (n = 72), classical-accommodating (n = 137), fast-spiking (n = 23), and burst-spiking neurons (n = 34). Anatomical reconstructions of L1 neurons supported the existence of 4 major neuronal groups. Multiparameter unsupervised cluster analysis confirmed the existence of 4 groups, revealing a high degree of similarity with the Petilla scheme. Simultaneous recordings between synaptically connected L1 neurons and L2/3 pyramidal neurons (n = 384) demonstrated neuronal class specificity in both excitatory and inhibitory connectivity and the properties of synaptic potentials. Notably, all groups of L1 neurons received monosynaptic excitatory input from L2/3 pyramidal neurons (n = 33), with the exception of NGFCs (n = 68 pairs tested). In contrast, NGFCs strongly inhibited L2/3 pyramidal neurons (n = 12 out 27 pairs tested). These data reveal a high specificity of excitatory and inhibitory connections in the superficial layers of the neocortex

Early alterations in the electrophysiological properties of rat spinal motoneurones following neonatal axotomy

Early in development, motoneurones are critically dependent on their target muscles for survival and differentiation. Previous studies have shown that neonatal axotomy causes massive motoneurone death and abnormal function in the surviving motoneurones. We have investigated the electrophysiological and morphological properties of motoneurones innervating the flexor tibialis anterior (TA) muscle during the first week after a neonatal axotomy, at a time when the motoneurones would be either in the process of degeneration or attempting to reinnervate their target muscles. We found that a large number (∼75%) of TA motoneurones died within 3 weeks after neonatal axotomy. Intracellular recordings revealed a marked increase in motoneurone excitability, as indicated by changes in passive and active membrane electrical properties. These changes were associated with a shift in the motoneurone firing pattern from a predominantly phasic pattern to a tonic pattern. Morphologically, the dendritic tree of the physiologically characterized axotomized cells was significantly reduced compared with age-matched normal motoneurones. These data demonstrate that motoneurone electrical properties are profoundly altered shortly after neonatal axotomy. In a subpopulation of the axotomized cells, abnormally high motoneurone excitability (input resistance significantly higher compared with control cells) was associated with a severe truncation of the dendritic arbor, suggesting that this excitability may represent an early electrophysiological correlate of motoneurone degeneration

Spatio-Temporal Credit Assignment in Neuronal Population Learning

In learning from trial and error, animals need to relate behavioral decisions to environmental reinforcement even though it may be difficult to assign credit to a particular decision when outcomes are uncertain or subject to delays. When considering the biophysical basis of learning, the credit-assignment problem is compounded because the behavioral decisions themselves result from the spatio-temporal aggregation of many synaptic releases. We present a model of plasticity induction for reinforcement learning in a population of leaky integrate and fire neurons which is based on a cascade of synaptic memory traces. Each synaptic cascade correlates presynaptic input first with postsynaptic events, next with the behavioral decisions and finally with external reinforcement. For operant conditioning, learning succeeds even when reinforcement is delivered with a delay so large that temporal contiguity between decision and pertinent reward is lost due to intervening decisions which are themselves subject to delayed reinforcement. This shows that the model provides a viable mechanism for temporal credit assignment. Further, learning speeds up with increasing population size, so the plasticity cascade simultaneously addresses the spatial problem of assigning credit to synapses in different population neurons. Simulations on other tasks, such as sequential decision making, serve to contrast the performance of the proposed scheme to that of temporal difference-based learning. We argue that, due to their comparative robustness, synaptic plasticity cascades are attractive basic models of reinforcement learning in the brain

Converging Neuronal Activity in Inferior Temporal Cortex during the Classification of Morphed Stimuli

How does the brain dynamically convert incoming sensory data into a representation useful for classification? Neurons in inferior temporal (IT) cortex are selective for complex visual stimuli, but their response dynamics during perceptual classification is not well understood. We studied IT dynamics in monkeys performing a classification task. The monkeys were shown visual stimuli that were morphed (interpolated) between pairs of familiar images. Their ability to classify the morphed images depended systematically on the degree of morph. IT neurons were selected that responded more strongly to one of the 2 familiar images (the effective image). The responses tended to peak ∼120 ms following stimulus onset with an amplitude that depended almost linearly on the degree of morph. The responses then declined, but remained above baseline for several hundred ms. This sustained component remained linearly dependent on morph level for stimuli more similar to the ineffective image but progressively converged to a single response profile, independent of morph level, for stimuli more similar to the effective image. Thus, these neurons represented the dynamic conversion of graded sensory information into a task-relevant classification. Computational models suggest that these dynamics could be produced by attractor states and firing rate adaptation within the population of IT neurons

Differential Regulation of the Excitability of Prefrontal Cortical Fast-Spiking Interneurons and Pyramidal Neurons by Serotonin and Fluoxetine

Serotonin exerts a powerful influence on neuronal excitability. In this study, we investigated the effects of serotonin on different neuronal populations in prefrontal cortex (PFC), a major area controlling emotion and cognition. Using whole-cell recordings in PFC slices, we found that bath application of 5-HT dose-dependently increased the firing of FS (fast spiking) interneurons, and decreased the firing of pyramidal neurons. The enhancing effect of 5-HT in FS interneurons was mediated by 5-HT2 receptors, while the reducing effect of 5-HT in pyramidal neurons was mediated by 5-HT1 receptors. Fluoxetine, the selective serotonin reuptake inhibitor, also induced a concentration-dependent increase in the excitability of FS interneurons, but had little effect on pyramidal neurons. In rats with chronic fluoxetine treatment, the excitability of FS interneurons was significantly increased, while pyramidal neurons remained unchanged. Fluoxetine injection largely occluded the enhancing effect of 5-HT in FS interneurons, but did not alter the reducing effect of 5-HT in pyramidal neurons. These data suggest that the excitability of PFC interneurons and pyramidal neurons is regulated by exogenous 5-HT in an opposing manner, and FS interneurons are the major target of Fluoxetine. It provides a framework for understanding the action of 5-HT and antidepressants in altering PFC network activity

Dendritic release of neurotransmitters

Release of neuroactive substances by exocytosis from dendrites is surprisingly widespread and is not confined to a particular class of transmitters: it occurs in multiple brain regions, and includes a range of neuropeptides, classical neurotransmitters and signaling molecules such as nitric oxide, carbon monoxide, ATP and arachidonic acid. This review is focused on hypothalamic neuroendocrine cells that release vasopressin and oxytocin and midbrain neurons that release dopamine. For these two model systems, the stimuli, mechanisms and physiological functions of dendritic release have been explored in greater detail than is yet available for other neurons and neuroactive substances