Timing is not Everything: Neuromodulation Opens the STDP Gate

Spike timing dependent plasticity (STDP) is a temporally specific extension of Hebbian associative plasticity that has tied together the timing of presynaptic inputs relative to the postsynaptic single spike. However, it is difficult to translate this mechanism to in vivo conditions where there is an abundance of presynaptic activity constantly impinging upon the dendritic tree as well as ongoing postsynaptic spiking activity that backpropagates along the dendrite. Theoretical studies have proposed that, in addition to this pre- and postsynaptic activity, a “third factor” would enable the association of specific inputs to specific outputs. Experimentally, the picture that is beginning to emerge, is that in addition to the precise timing of pre- and postsynaptic spikes, this third factor involves neuromodulators that have a distinctive influence on STDP rules. Specifically, neuromodulatory systems can influence STDP rules by acting via dopaminergic, noradrenergic, muscarinic, and nicotinic receptors. Neuromodulator actions can enable STDP induction or – by increasing or decreasing the threshold – can change the conditions for plasticity induction. Because some of the neuromodulators are also involved in reward, a link between STDP and reward-mediated learning is emerging. However, many outstanding questions concerning the relationship between neuromodulatory systems and STDP rules remain, that once solved, will help make the crucial link from timing-based synaptic plasticity rules to behaviorally based learning

Timing is not Everything: Neuromodulation Opens the STDP Gate

Spike timing dependent plasticity (STDP) is a temporally specific extension of Hebbian associative plasticity that has tied together the timing of presynaptic inputs relative to the postsynaptic single spike. However, it is difficult to translate this mechanism to in vivo conditions where there is an abundance of presynaptic activity constantly impinging upon the dendritic tree as well as ongoing postsynaptic spiking activity that backpropagates along the dendrite. Theoretical studies have proposed that, in addition to this pre- and postsynaptic activity, a “third factor” would enable the association of specific inputs to specific outputs. Experimentally, the picture that is beginning to emerge, is that in addition to the precise timing of pre- and postsynaptic spikes, this third factor involves neuromodulators that have a distinctive influence on STDP rules. Specifically, neuromodulatory systems can influence STDP rules by acting via dopaminergic, noradrenergic, muscarinic, and nicotinic receptors. Neuromodulator actions can enable STDP induction or – by increasing or decreasing the threshold – can change the conditions for plasticity induction. Because some of the neuromodulators are also involved in reward, a link between STDP and reward-mediated learning is emerging. However, many outstanding questions concerning the relationship between neuromodulatory systems and STDP rules remain, that once solved, will help make the crucial link from timing-based synaptic plasticity rules to behaviorally based learning

Contributions to models of single neuron computation in striatum and cortex

A deeper understanding is required of how a single neuron utilizes its nonlinear subcellular devices to generate complex neuronal dynamics. Two compartmental models of cortex and striatum are accurately formulated and firmly grounded in the experimental reality of electrophysiology to address the questions: how striatal projection neurons implement location-dependent dendritic integration to carry out association-based computation and how cortical pyramidal neurons strategically exploit the type and location of synaptic contacts to enrich its computational capacities.Neuronale Zellen transformieren kontinuierliche Signale in diskrete Zeitserien von Aktionspotentialen und kodieren damit Perzeptionen und interne Zustände. Kompartiment-Modelle werden formuliert von Nervenzellen im Kortex und Striatum, die elektrophysiologisch fundiert sind, um spezifische Fragen zu adressieren: i) Inwiefern implementieren Projektionen vom Striatum ortsabhängige dendritische Integration, um Assoziationens-basierte Berechnungen zu realisieren? ii) Inwiefern nutzen kortikale Zellen den Typ und den Ort, um die durch sie realisierten Berechnungen zu optimieren

A Kinetic Model of Dopamine- and Calcium-Dependent Striatal Synaptic Plasticity

Corticostriatal synapse plasticity of medium spiny neurons is regulated by glutamate input from the cortex and dopamine input from the substantia nigra. While cortical stimulation alone results in long-term depression (LTD), the combination with dopamine switches LTD to long-term potentiation (LTP), which is known as dopamine-dependent plasticity. LTP is also induced by cortical stimulation in magnesium-free solution, which leads to massive calcium influx through NMDA-type receptors and is regarded as calcium-dependent plasticity. Signaling cascades in the corticostriatal spines are currently under investigation. However, because of the existence of multiple excitatory and inhibitory pathways with loops, the mechanisms regulating the two types of plasticity remain poorly understood. A signaling pathway model of spines that express D1-type dopamine receptors was constructed to analyze the dynamic mechanisms of dopamine- and calcium-dependent plasticity. The model incorporated all major signaling molecules, including dopamine- and cyclic AMP-regulated phosphoprotein with a molecular weight of 32 kDa (DARPP32), as well as AMPA receptor trafficking in the post-synaptic membrane. Simulations with dopamine and calcium inputs reproduced dopamine- and calcium-dependent plasticity. Further in silico experiments revealed that the positive feedback loop consisted of protein kinase A (PKA), protein phosphatase 2A (PP2A), and the phosphorylation site at threonine 75 of DARPP-32 (Thr75) served as the major switch for inducing LTD and LTP. Calcium input modulated this loop through the PP2B (phosphatase 2B)-CK1 (casein kinase 1)-Cdk5 (cyclin-dependent kinase 5)-Thr75 pathway and PP2A, whereas calcium and dopamine input activated the loop via PKA activation by cyclic AMP (cAMP). The positive feedback loop displayed robust bi-stable responses following changes in the reaction parameters. Increased basal dopamine levels disrupted this dopamine-dependent plasticity. The present model elucidated the mechanisms involved in bidirectional regulation of corticostriatal synapses and will allow for further exploration into causes and therapies for dysfunctions such as drug addiction

Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of neoHebbian Three-Factor Learning Rules

Most elementary behaviors such as moving the arm to grasp an object or walking into the next room to explore a museum evolve on the time scale of seconds; in contrast, neuronal action potentials occur on the time scale of a few milliseconds. Learning rules of the brain must therefore bridge the gap between these two different time scales. Modern theories of synaptic plasticity have postulated that the co-activation of pre- and postsynaptic neurons sets a flag at the synapse, called an eligibility trace, that leads to a weight change only if an additional factor is present while the flag is set. This third factor, signaling reward, punishment, surprise, or novelty, could be implemented by the phasic activity of neuromodulators or specific neuronal inputs signaling special events. While the theoretical framework has been developed over the last decades, experimental evidence in support of eligibility traces on the time scale of seconds has been collected only during the last few years. Here we review, in the context of three-factor rules of synaptic plasticity, four key experiments that support the role of synaptic eligibility traces in combination with a third factor as a biological implementation of neoHebbian three-factor learning rules

Interaction of multiple inputs in plasticity of the corticostriatal synapses

Dopamine-dependent plasticity in synapses between the cortical pyramidal neurons and the spiny projection neurons (SPNs) in the striatum is associated with reinforcement learning. Spike timing-dependent plasticity (STDP), which depends on the relative timing of pre- and postsynaptic activity, has been described in these synapses. Previously the STDP profile has been determined by testing single input-output events in isolation from the context of concurrently occurring multiple inputs into the same neuron. However, interactions among synaptic inputs at the level of the dendrites might influence STDP induction. The overall aim of this thesis is to study whether the activation of multiple synaptic inputs alters the characteristics of STDP in the corticostriatal pathway. Whole-cell electrophysiological recordings of SPNs in the dorsomedial striatum (DMS) of mouse brain slices were made in the presence of two inputs stimulated at different time points relative to postsynaptic firing. This protocol induced LTD depending on the timing of each input in SPNs expressing dopamine D1 receptors but not in SPNs expressing D2 receptors. When two inputs showed interactions, indicated by nonlinear summation of evoked EPSPs, STDP profiles were altered from those seen when single inputs were studied. In addition, pairing of two presynaptic inputs without postsynaptic firing also induced LTD, suggesting that pairing of synaptic inputs alone within a temporal window can induce associative synaptic plasticity. In separate experiments, optogenetic release of dopamine two seconds after each pairing modified STDP, depending on the input timing and interactions. Dopamine also modulated associative synaptic plasticity induced in the absence of postsynaptic firing. These results suggest that the rules for synaptic plasticity observed with multiple inputs to the same neuron are not identical to those observed when inputs are tested one at a time per neuron. This new knowledge helps to place STDP in the context of whole brain activity and adds to current understanding of associative learning in the striatum.Okinawa Institute of Science and Technology Graduate Universit

Modulation of Spike-Timing Dependent Plasticity: Towards the Inclusion of a Third Factor in Computational Models

In spike-timing dependent plasticity (STDP) change in synaptic strength depends on the timing of pre- vs. postsynaptic spiking activity. Since STDP is in compliance with Hebb’s postulate, it is considered one of the major mechanisms of memory storage and recall. STDP comprises a system of two coincidence detectors with N-methyl-D-aspartate receptor (NMDAR) activation often posited as one of the main components. Numerous studies have unveiled a third component of this coincidence detection system, namely neuromodulation and glia activity shaping STDP. Even though dopaminergic control of STDP has most often been reported, acetylcholine, noradrenaline, nitric oxide (NO), brain-derived neurotrophic factor (BDNF) or gamma-aminobutyric acid (GABA) also has been shown to effectively modulate STDP. Furthermore, it has been demonstrated that astrocytes, via the release or uptake of glutamate, gate STDP expression. At the most fundamental level, the timing properties of STDP are expected to depend on the spatiotemporal dynamics of the underlying signaling pathways. However in most cases, due to technical limitations experiments grant only indirect access to these pathways. Computational models carefully constrained by experiments, allow for a better qualitative understanding of the molecular basis of STDP and its regulation by neuromodulators. Recently, computational models of calcium dynamics and signaling pathway molecules have started to explore STDP emergence in ex and in vivo-like conditions. These models are expected to reproduce better at least part of the complex modulation of STDP as an emergent property of the underlying molecular pathways. Elucidation of the mechanisms underlying STDP modulation and its consequences on network dynamics is of critical importance and will allow better understanding of the major mechanisms of memory storage and recall both in health and disease

Dopamine: The Neuromodulator of Long-Term Synaptic Plasticity, Reward and Movement Control

Dopamine (DA) is a key neurotransmitter involved in multiple physiological functions including motor control, modulation of affective and emotional states, reward mechanisms, reinforcement of behavior, and selected higher cognitive functions. Dysfunction in dopaminergic transmission is recognized as a core alteration in several devastating neurological and psychiatric disorders, including Parkinson's disease (PD), schizophrenia, bipolar disorder, attention deficit hyperactivity disorder (ADHD) and addiction. Here we will discuss the current insights on the role of DA in motor control and reward learning mechanisms and its involvement in the modulation of synaptic dynamics through different pathways. In particular, we will consider the role of DA as neuromodulator of two forms of synaptic plasticity, known as long-term potentiation (LTP) and long-term depression (LTD) in several cortical and subcortical areas. Finally, we will delineate how the effect of DA on dendritic spines places this molecule at the interface between the motor and the cognitive systems. Specifically, we will be focusing on PD, vascular dementia, and schizophrenia

Oculomotor learning revisited: a model of reinforcement learning in the basal ganglia incorporating an efference copy of motor actions

In its simplest formulation, reinforcement learning is based on the idea that if an action taken in a particular context is followed by a favorable outcome, then, in the same context, the tendency to produce that action should be strengthened, or reinforced. While reinforcement learning forms the basis of many current theories of basal ganglia (BG) function, these models do not incorporate distinct computational roles for signals that convey context, and those that convey what action an animal takes. Recent experiments in the songbird suggest that vocal-related BG circuitry receives two functionally distinct excitatory inputs. One input is from a cortical region that carries context information about the current “time” in the motor sequence. The other is an efference copy of motor commands from a separate cortical brain region that generates vocal variability during learning. Based on these findings, I propose here a general model of vertebrate BG function that combines context information with a distinct motor efference copy signal. The signals are integrated by a learning rule in which efference copy inputs gate the potentiation of context inputs (but not efference copy inputs) onto medium spiny neurons in response to a rewarded action. The hypothesis is described in terms of a circuit that implements the learning of visually guided saccades. The model makes testable predictions about the anatomical and functional properties of hypothesized context and efference copy inputs to the striatum from both thalamic and cortical sources

On striatum in silico

The basal ganglia are a collection of subcortical nuclei involved in movement and action selection. The striatum is the main input nucleus with extensive projections from the cortex and thalamus, and dopaminergic projections from SNc and VTA. The two main cell types are the striatal projection neurons (SPNs), which are divided into the direct (dSPN) and indirect (iSPN) pathways, based on the downstream projections and the expression of dopamine D1 and D2 receptors, respectively. The remaining 5% consists mainly of GABAergic interneurons, such as parvalbumin-expressing fastspiking interneurons (FS) and low threshold spiking interneurons (LTS). The cholinergic interneuron (ChIN) is spontaneously active and unlike the other interneurons releases acetylcholine. This thesis is focused on investigating the function of the striatum and the role of SPNs and the striatal interneurons. This is achieved by building a platform, tools, and a database of multi-compartmental models of SPN, FS, ChIN, and LTS; and through simulations systematically uncovering the roles of these striatal neuron types and external input and, more specifically, the role of neuromodulation and intrastriatal inhibition. In Paper I, Snudda, a platform for simulating large-scale networks, is developed and includes multicompartmental models of dSPN, iSPN, FS, LTS, and ChIN. The tools include methods to generate external input from the cortex and thalamus; and dopaminergic modulation from SNc. Paper II investigates the relationship between ChIN and LTS. The ChIN releases ACh, which activates both nicotinic and muscarinic receptors within the striatum. The dominating effect on LTS is inhibition caused by muscarinic M4 receptors. LTS, on the other hand, releases NO which excites ChINs. Paper II showed that the interaction between these neuromodulators could control the activity of ChIN and LTS, which are generally spontaneously active. In the subsequent Paper III, Snudda was complemented with the neuromodulation package called Neuromodcell, a Python Package, for creating models of neuromodulation, which can be included in large-scale network simulations in Snudda. The method of simulating neuromodulators in Snudda was expanded to include multiple simultaneously active modulators. This resulted in several simulations with simultaneous ACh pause with DA burst as well as an ACh burst with a DA burst. In Paper IV, the effect of intrastriatal surround inhibition on striatal activity was investigated by utilizing ablations, clustered input, dopaminergic modulation, and other features in Snudda. These simulations demonstrated that shunting inhibition could reduce the amplitude of corticostriatal input onto SPNs. The surround inhibition can further modulate the plateau potentials in SPNs, which is dependent on the GABA reversal. Lastly, the competition between populations of SPNs can be modified by varying the strength, size, and positions of populations. Furthermore, dopaminergic modulation can enhance the effect of dSPNs, while increasing the inhibition onto iSPNs. Overall, this thesis provides an analysis of the striatal microcircuit and a tool for further investigations of the striatum in silico; and demonstrates the importance to consider the different components of the striatal microcircuit and how neuromodulators can reshape microcircuits on both single neuron and network levels