A computational model of how cholinergic interneurons protect striatal-dependent learning

Abstract ■ An essential component of skill acquisition is learning the environmental conditions in which that skill is relevant. This article proposes and tests a neurobiologically detailed theory of how such learning is mediated. The theory assumes that a key component of this learning is provided by the cholinergic interneurons in the striatum known as tonically active neurons (TANs). The TANs are assumed to exert a tonic inhibitory influence over cortical inputs to the striatum that prevents the execution of any striatal-dependent actions. The TANs learn to pause in rewarding environments, and this pause releases the striatal output neurons from this inhibitory effect, thereby facilitating the learning and expression of striataldependent behaviors. When rewards are no longer available, the TANs cease to pause, which protects striatal learning from decay. A computational version of this theory accounts for a variety of single-cell recording data and some classic behavioral phenomena, including fast reacquisition after extinction.

Neural Coordination of Distinct Motor Learning Strategies: Latent Neurofunctional Mechanisms Elucidated via Computational Modeling

In this dissertation, a neurofunctional theory of learning is presented as an extension of functional analysis. This new theory clarifies the distinction— via applied quantitative analysis— between functionally intrinsic (essential) mechanistic structures and irrelevant structural details. This thesis is supported by a review of the relevant literature to provide historical context and sufficient scientific background. Further, the scope of this thesis is elucidated by two questions that are posed from a neurofunctional perspective— (1) how can specialized neuromorphology contribute to the functional dynamics of neural learning processes? (2) Can large-scale neurofunctional pathways emerge via inter-network communication between disparate neural circuits? These questions motivate the specific aims of this dissertation. Each aim is addressed by posing a relevant hypothesis, which is then tested via a neurocomputational experiment. In each experiment, computational techniques are leveraged to elucidate specific mechanisms that underlie neurofunctional learning processes. For instance, the role of specialized neuromorphology is investigated via the development of a computational model that replicates the neurophysiological mechanisms that underlie cholinergic interneurons’ regulation of dopamine in the striatum during reinforcement learning. Another research direction focuses on the emergence of large-scale neurofunctional pathways that connect the cerebellum and basal ganglia— this study also involves the construction of a neurocomputational model. The results of each study illustrate the capability of neurocomputational models to replicate functional learning dynamics of human subjects during a variety of motor adaptation tasks. Finally, the significance— and some potential applications— of neurofunctional theory are discussed

The Functional Role of Striatal Cholinergic Interneurons in Reinforcement Learning From Computational Perspective

In this study, we explore the functional role of striatal cholinergic interneurons, hereinafter referred to as tonically active neurons (TANs), via computational modeling; specifically, we investigate the mechanistic relationship between TAN activity and dopamine variations and how changes in this relationship affect reinforcement learning in the striatum. TANs pause their tonic firing activity after excitatory stimuli from thalamic and cortical neurons in response to a sensory event or reward information. During the pause striatal dopamine concentration excursions are observed. However, functional interactions between the TAN pause and striatal dopamine release are poorly understood. Here we propose a TAN activity-dopamine relationship model and demonstrate that the TAN pause is likely a time window to gate phasic dopamine release and dopamine variations reciprocally modulate the TAN pause duration. Furthermore, this model is integrated into our previously published model of reward-based motor adaptation to demonstrate how phasic dopamine release is gated by the TAN pause to deliver reward information for reinforcement learning in a timely manner. We also show how TAN-dopamine interactions are affected by striatal dopamine deficiency to produce poor performance of motor adaptation

Functional neurochemical imaging of the human striatal cholinergic system during reversal learning

Animal studies have shown that acetylcholine (ACh) levels in the dorsal striatum play a role in reversal learning. However, this has not been studied in humans due to a lack of appropriate non-invasive techniques. Proton magnetic resonance spectroscopy (1H-MRS) can be used to measure metabolite levels in humans in vivo. Although it cannot be used to study ACh directly, 1H-MRS can be used to study choline, an ACh precursor which is linked to activity-dependent ACh release. The aim of this study was to use functional-1H-MRS (fMRS) to measure changes in choline levels in the human dorsal striatum during performance of a probabilistic reversal learning task. We demonstrate a task-dependent decrease in choline, specifically during reversal, but not initial, learning. We interpret this to reflect a sustained increase in ACh levels, which is in line with findings from the animal literature. This task-dependent change was specific to choline and was not observed in control metabolites. These findings provide support for the use of fMRS in the in vivo study of the human cholinergic system

Computational models of basal-ganglia pathway functions: focus on functional neuroanatomy

Over the past 15 years, computational models have had a considerable impact on basal-ganglia research. Most of these models implement multiple distinct basal-ganglia pathways and assume them to fulfill different functions. As there is now a multitude of different models, it has become complex to keep track of their various, sometimes just marginally different assumptions on pathway functions. Moreover, it has become a challenge to oversee to what extent individual assumptions are corroborated or challenged by empirical data. Focusing on computational, but also considering non-computational models, we review influential concepts of pathway functions and show to what extent they are compatible with or contradict each other. Moreover, we outline how empirical evidence favors or challenges specific model assumptions and propose experiments that allow testing assumptions against each other

The functional organisation of basal ganglia inputs

The basal ganglia allow organisms to adjust their behaviour according to changes in their internal state or their environment. One essential prerequisite for the selection and execution of appropriate movements is the convergence of inputs from various sources, conveying sensory information, motor commands, reward value, and more. These diverse inputs are integrated in the striatum, the input structure of the basal ganglia. In the last decades, numerous striatal cell types have been identified, their molecular profiles have been extracted and their local connectivity has been revealed. However, relatively little is known about the functional organisation of striatal inputs innervating these different neuron populations. The aim of this thesis is to examine how striatal inputs are integrated by the main cell types of this microcircuit. In Paper I, we uncover the mechanisms underlying sensory deficits in a mouse model of Parkinson’s disease. We show that one type of striatal projection neurons encodes the laterality of somatosensory inputs better than the other output neuron in healthy mice and that this encoding is lost in the dopamine-depleted state. In Paper II, we map the excitatory synaptic pathways of five striatal input structures (ipsi- and contralateral somatosensory and motor cortex, and the parafascicular nucleus) onto five different classes of striatal neurons. The study characterises the synaptic strength, receptor composition, and shortterm plasticity of each pathway with an unprecedented level of detail and comparability, thereby contributing to the understanding of the role of different striatal cell types. In Paper III, we create an in silico model of the striatum that integrates data from the subcellular to the microcircuit level. This model will be publicly available for testing new hypotheses and continuously updated with novel findings. In summary, the work presented in this thesis provides a further step in untangling the heterogeneous excitatory inputs that drive the activity of the primarily inhibitory microcircuit of the striatum and thus basal ganglia. We show that each striatal input targets a different set of striatal neurons and that the intricate organisation of these afferents is a function of both the presynaptic region and the postsynaptic cell type. Ultimately, knowledge of the functional connectivity of cortico- and thalamostriatal pathways as well as their synaptic properties will be essential for understanding and modelling the cortico- and thalamo-basal ganglia network in health and disease

A mathematical model of levodopa medication effect on basal ganglia in parkinson’s disease: An application to the alternate finger tapping task

Malfunctions in the neural circuitry of the basal ganglia (BG), induced by alterations in the dopaminergic system, are responsible for an array of motor disorders and milder cognitive issues in Parkinson's disease (PD). Recently Baston and Ursino (2015a) presented a new neuroscience mathematical model aimed at exploring the role of basal ganglia in action selection. The model is biologically inspired and reproduces the main BG structures and pathways, modeling explicitly both the dopaminergic and the cholinergic system. The present work aims at interfacing this neurocomputational model with a compartmental model of levodopa, to propose a general model of medicated Parkinson's disease. Levodopa effect on the striatum was simulated with a two-compartment model of pharmacokinetics in plasma joined with a motor effect compartment. The latter is characterized by the levodopa removal rate and by a sigmoidal relationship (Hill law) between concentration and effect. The main parameters of this relationship are saturation, steepness, and the half-maximum concentration. The effect of levodopa is then summed to a term representing the endogenous dopamine effect, and is used as an external input for the neurocomputation model; this allows both the temporal aspects of medication and the individual patient characteristics to be simulated. The frequency of alternate tapping is then used as the outcome of the whole model, to simulate effective clinical scores. Pharmacokinetic-pharmacodynamic modeling was preliminary performed on data of six patients with Parkinson's disease (both “stable” and “wearing-off” responders) after levodopa standardized oral dosing over 4 h. Results show that the model is able to reproduce the temporal profiles of levodopa in plasma and the finger tapping frequency in all patients, discriminating between different patterns of levodopa motor response. The more influential parameters are the Hill coefficient, related with the slope of the effect sigmoidal relationship, the drug concentration at half-maximum effect, and the drug removal rate from the effect compartment. The model can be of value to gain a deeper understanding on the pharmacokinetics and pharmacodynamics of the medication, and on the way dopamine is exploited in the neural circuitry of the basal ganglia in patients at different stages of the disease progression

GABAergic interneurons and prenatal ethanol exposure: from development to aging

Fetal Alcohol Spectrum Disorders are the most common non-genetic cause of neurodevelopmental disability worldwide. Individuals with Fetal Alcohol Spectrum Disorder experience clinical symptoms including differences in physical, cognitive and behavioral development beginning in early childhood, but continue to face challenges into adulthood. There is a critical need to examine the effects of prenatal ethanol exposure across early development, and to establish how the developmental effects of prenatal ethanol exposure may or may not progress in aging individuals. To contribute to these two areas, I asked how a binge-type prenatal ethanol exposure might affect: (1) early postnatal development of striatal neurons and, relate to the development of early motor behaviors over time, and (2) synaptic function in the medial prefrontal cortex, and affect the onset and severity of cognitive deficits in a transgenic mouse model of familial Alzheimer’s disease. I used whole-cell patch clamp electrophysiology to assess the functional and synaptic maturation of two populations of striatal neurons: striatal GABAergic interneurons and spiny striatal projection neurons, and the excitatory-inhibitory balance in deep layer medial prefrontal cortex pyramidal neurons. I found that prenatal ethanol exposure altered the postnatal developmental trajectory of striatal neurons in a sex-dependent manner, that coincided with sex-differences in the development of early motor behaviors, and morphological differences in striatal projection neurons. I also determined that prenatal ethanol exposure resulted in an earlier onset of deficits in GABAergic synaptic activity in cortical pyramidal neurons, that was an associated with a decreased number of parvalbumin expressing GABAergic interneurons, and an increase in intraneuronal APP/β-amyloid. These findings highlight the dynamic effects of prenatal ethanol exposure on synaptic function and behavioral outcomes during early development, and the lasting effects of prenatal ethanol exposure on neural circuits, modifying the aging process