A neural network model of adaptively timed reinforcement learning and hippocampal dynamics

A neural model is described of how adaptively timed reinforcement learning occurs. The adaptive timing circuit is suggested to exist in the hippocampus, and to involve convergence of dentate granule cells on CA3 pyramidal cells, and NMDA receptors. This circuit forms part of a model neural system for the coordinated control of recognition learning, reinforcement learning, and motor learning, whose properties clarify how an animal can learn to acquire a delayed reward. Behavioral and neural data are summarized in support of each processing stage of the system. The relevant anatomical sites are in thalamus, neocortex, hippocampus, hypothalamus, amygdala, and cerebellum. Cerebellar influences on motor learning are distinguished from hippocampal influences on adaptive timing of reinforcement learning. The model simulates how damage to the hippocampal formation disrupts adaptive timing, eliminates attentional blocking, and causes symptoms of medial temporal amnesia. It suggests how normal acquisition of subcortical emotional conditioning can occur after cortical ablation, even though extinction of emotional conditioning is retarded by cortical ablation. The model simulates how increasing the duration of an unconditioned stimulus increases the amplitude of emotional conditioning, but does not change adaptive timing; and how an increase in the intensity of a conditioned stimulus "speeds up the clock", but an increase in the intensity of an unconditioned stimulus does not. Computer simulations of the model fit parametric conditioning data, including a Weber law property and an inverted U property. Both primary and secondary adaptively timed conditioning are simulated, as are data concerning conditioning using multiple interstimulus intervals (ISIs), gradually or abruptly changing ISis, partial reinforcement, and multiple stimuli that lead to time-averaging of responses. Neurobiologically testable predictions are made to facilitate further tests of the model.Air Force Office of Scientific Research (90-0175, 90-0128); Defense Advanced Research Projects Agency (90-0083); National Science Foundation (IRI-87-16960); Office of Naval Research (N00014-91-J-4100

THE SITE-SPECIFIC EFFECTS OF KINDLING ON COGNITION AND ADULT HIPPOCAMPAL NEUROGENESIS

Nearly 1.5% of the general population is affected by epilepsy. Despite a long history of research and clinical endeavors to combat the disease, in 1 of 3 cases, epilepsy is intractable and resistant to medication treatment. Current pharmacological strategies target seizure onset as the primary manifestation of illness; however, patients with advanced stages of the disease suffer from a wide array of psychiatric comorbidities. Anxiety, depression and memory deterioration are the chief complaints of epileptic patients delegated from neurologic to psychiatric care. Temporal lobe epilepsy is the most resistant form of epilepsy and the form that is most complicated by the presence of psychiatric and cognitive comorbidities that are increasingly recognized as critical factors in long-term patient care. These comorbidities are independent risk factors for poor quality of life. In fact, studies have shown that in patients with epilepsy, co- morbid factors correlate more strongly with poor quality of life than does seizure frequency. At present, the behavioural comorbidities associated with temporal lobe epilepsy are poorly understood, and their management is difficult because many commonly prescribed anticonvulsant drugs make them worse. Therefore, even patients with some degree of seizure control often continue to experience debilitating behavioural problems that impair their daily living. This thesis attempts to understand the impact of seizures originating from different brain sites. The research described in this thesis was conducted using an animal model called kindling to investigate the hypothesis that seizure-induced alterations in hippocampal neurogenesis play a significant role in the cognitive deficits associated with epilepsy. The experiment used 25 adult male rats. Rats were divided into four group: hippocampal kindled (n = 6), amygdala kindled (n = 7), caudate nucleus kindled (n = 6) and control rats (n = 6). Kindled rats received 99 electrical stimulations delivered to the appropriate brain region through an implanted bilateral electrode. At the end of the kindling phase, rats were subjected to a fear conditioning paradigm to assess cognitive behavior. After the fear conditioning, rats were sacrificed and immunohistochemical analyses were done to assess the impact of seizures on hippocampal neurogenesis and neuronal activation. The results indicated that there is a significant deterioration of cognitive performance following long-term limbic kindling (i.e., hippocampal and amygdala kindling only). This was accompanied by an increase in hippocampal neurogenesis that was paralleled with low expression of immediate early genes. Collectively, these findings enhance our understanding of mechanisms underlying the behavioral co- morbidities associated with temporal lobe epilepsy and demonstrate the link between adult neurogenesis and cognitive impairments following long-term kindling of limbic brain regions

Identifying Network Correlates of Memory Consolidation

Neuronal spiking activity carries information about our experiences in the waking world but exactly how the brain can quickly and efficiently encode sensory information into a useful neural code and then subsequently consolidate that information into memory remains a mystery. While neuronal networks are known to play a vital role in these processes, detangling the properties of network activity from the complex spiking dynamics observed is a formidable challenge, requiring collaborations across scientific disciplines. In this work, I outline my contributions in computational modeling and data analysis toward understanding how network dynamics facilitate memory consolidation. For experimental perspective, I investigate hippocampal recordings of mice that are subjected to contextual fear conditioning and subsequently undergo sleep-dependent fear memory consolidation. First, I outline the development of a functional connectivity algorithm which rapidly and robustly assesses network structure based on neuronal spike timing. I show that the relative stability of these functional networks can be used to identify global network dynamics, revealing that an increase in functional network stability correlates with successful fear memory consolidation in vivo. Using an attractor-based model to simulate memory encoding and consolidation, I go on to show that dynamics associated with a second-order phase transition, at a critical point in phase-space, are necessary for recruiting additional neurons into network dynamics associated with memory consolidation. I show that successful consolidation subsequently shifts dynamics away from a critical point and towards sub-critical dynamics. Investigations of in vivo spiking dynamics likewise revealed that hippocampal dynamics during non-rapid-eye-movement (NREM) sleep show features of being near a critical point and that fear memory consolidation leads to a shift in dynamics. Finally, I investigate the role of NREM sleep in facilitating memory consolidation using a conductance-based model of neuronal activity that can easily switch between modes of activity loosely representing waking and NREM sleep. Analysis of model simulations revealed that oscillations associated with NREM sleep promote a phase-based coding of information; neurons with high firing rates during periods of wake lead spiking activity during NREM oscillations. I show that when phase-coding is active in both simulations and in vivo, synaptic plasticity selectively strengthens the input to neurons firing late in the oscillation while simultaneously reducing input to neurons firing early in the oscillation. The effect is a net homogenization of firing rates observed in multiple other studies, and subsequently leads to recruitment of new neurons into a memory engram and information transfer from fast firing neurons to slow firing neurons. Taken together, my work outlines important, newly-discovered features of neuronal network dynamics related to memory encoding and consolidation: networks near criticality promote recruitment of additional neurons into stable firing patterns through NREM-associated oscillations and subsequently consolidates information into memories through phase-based coding.PHDBiophysicsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/162991/1/qmskill_1.pd

Apoaequorin Differentially Modulates Fear Conditioning in Adult and Aged Rats

Normal aging is associated with a number of changes in behavioral and cellular function, and is often linked to increased susceptibility to cognitive impairment. The hippocampus has been widely implicated in learning and memory, and many forms of learning that are hippocampus-dependent (e.g. trace fear conditioning) are impaired in aged animals. A proposed contributor to aging-related cognitive impairment is aging-related calcium (Ca2+) dysregulation. This dysregulation is thought to result from changes in specific Ca2+-regulatory mechanisms, including abnormal Ca2+ ion channel activity or expression, as well as reduced Ca2+-binding protein (CaBP) expression, which is associated with cognitive and synaptic impairment. Previous data from our lab indicate that a single hippocampal infusion of the CaBP apoaequorin (AQ) is neuroprotective in the event of an ischemic insult, a process characterized by Ca2+-induced excitotoxicity. However, the effect of AQ on fear memory in adult and aged animals has yet to be examined. The current experiments investigate the effect of AQ infusion on trace fear conditioning in adult and aged rats. We firstly demonstrate that a single infusion of AQ 24 h before trace fear acquisition fails to rescue an aging-related trace fear memory deficit. Second, we found that AQ infusion 1 h prior to trace fear acquisition reduces baseline freezing during a cue test in a novel context, suggesting pre-training AQ infusion may mitigate context fear generalization. Furthermore, AQ infusion 1 h prior to trace fear acquisition and 1 h prior to testing results in a reversal of aging-related context fear memory impairment. The results of these studies suggest a possible role for AQ in modifying cognitive function in adult and aged rats

Cognitive Impairment and Aberrant Plasticity in the Kindling Model of Epilepsy

Epilepsy is a neurological disorder that affects approximately 1% of the population worldwide. Although motor seizures are the best known feature of epilepsy, many patients also experience severe interictal (between-seizure) behavioral and cognitive comorbidities that have a greater negative influence on quality of life than seizure control or frequency. To study the characteristics of these interictal comorbidities and the neural mechanisms that underlie them, I use the kindling model of epilepsy. Kindling refers to the brief electrical stimulation of a discrete brain site that produces a gradual and permanent increase in the severity of elicited seizure activity. The repeated seizures associated with kindling induce robust structural and functional plasticity that appears to be primarily aberrant. Importantly, the aberrant plasticity evoked by repeated seizures is thought to contribute to the pathophysiology of epilepsy and its associated behavioral and cognitive comorbidities. Unfortunately, the relationship between aberrant plasticity and cognition dysfunction following repeated seizures remains poorly understood. The aim of this dissertation is to gain a better understanding of the effects of repeated convulsions on aberrant neural plasticity and interictal behavior. In Chapter 2, I will examine the effect of short- and long-term amygdala kindling on amygdala- and hippocampal-dependent forms of operant fear conditioning. To evaluate whether kindling alters neural circuits important in memory, I will analyze post-mortem measures of neural activity following the retrieval of fearful memories. In Chapter 3, I will evaluate whether deficits in operant fear learning and memory are a general consequence of convulsions induced by kindling stimulations or whether these deficits occur following kindling of specific brain regions. To evaluate whether aberrant plasticity following kindling of different brain regions contributes to learning and memory deficits, I will make post-mortem examinations of the inhibitory neurotransmitter neuropeptide Y and its Y2 receptor. In Chapter 4, I will investigate the relationship between hippocampal neurogenesis and cognition. Specifically, I will determine whether kindling of different brain regions induces an aberrant form of hippocampal neurogenesis that contributes to cognitive dysfunction. In Chapter 5, I will investigate whether kindling of different brain regions alters different subpopulations of hippocampal GABAergic interneurons, in terms of number and morphological features. Finally, Chapter 6 will provide preliminary evidence that the cognitive impairments associated with kindling can be ameliorated through intrahippocampal infusions of recombinant reelin. The collection of studies in this dissertation improves our understanding of the relationship between aberrant plasticity and cognitive impairments associated with repeated convulsions

Systems consolidation requires postlearning activation of NMDA receptors in the medial prefrontal cortex in trace eyeblink conditioning.

The importance of the hippocampus in declarative memory is limited to recently acquired memory, and remotely acquired memory is believed to be stored somewhere in the neocortex. However, it remains unknown how the memory network is reorganized from a hippocampus-dependent form into a neocortex-dependent one. We reported previously that the medial prefrontal cortex (mPFC) is important for this neocortex-dependent remote memory in rat trace eyeblink conditioning. Here, we investigate the involvement of NMDA receptors in the mPFC in this reorganization and determine the time window of their contribution using chronic infusion of an antagonist into the mPFC, specifically during the postlearning consolidation period. The rats with blockade of the mPFC NMDA receptors during the first 1 or 2 weeks after learning showed a marked impairment in memory retention measured 6 weeks after learning, but relearned normally with subsequent conditioning. In contrast, the same treatment had no effect if it was performed during the third to fourth weeks or during the first day just after learning. The specificity of NMDA receptor blockade was confirmed by the reduced long-term potentiation in the hippocampal-prefrontal pathway in these rats. These results suggest that successful establishment of remotely acquired memory requires activation of NMDA receptors in the mPFC during at least the initial week of the postlearning period. Such NMDA receptor-dependent processes may mediate the maturation of neocortical networks that underlies permanent memory storage and serve as a way to reorganize memory circuitry to the neocortex-dependent form

Neurophysiological and Morphological Plasticity in Rat Hippocampus and Medial Prefrontal Cortex Following Trace Fear Conditioning

Pavlovian fear conditioning provides a useful model system for investigating the mechanisms underlying associative learning. In recent years, there has been an increasing interest in trace fear conditioning, which requires conscious awareness of the contingency of CS and US therefore considered as a rodent model of explicit fear. Acquisition of trace fear conditioning requires an intact hippocampus and medial prefrontal cortex (mPFC), but the underlying mechanisms are still unclear. The current set of studies investigated how trace fear conditioning affects neuronal plasticity in both hippocampus and mPFC in adult rats. Trace fear conditioning significantly enhanced both intrinsic excitability and synaptic plasticity (LTP) in hippocampal CA1 neurons. Interestingly, intrinsic excitability and synaptic plasticity were significantly correlated with behavioral performance, suggesting that these changes were learning-specific. The next set of experiments investigated learning-related changes in mPFC. In order to study circuit-specific changes, only neurons that project to the basolateral nucleus of amygdala (BLA) were studied by injecting a retrograde tracer into BLA. Trace fear conditioning significantly enhanced the excitability the layer 5 (L5) projection neurons in the infralimbic (IL) subregion of mPFC whereas it decreased the excitability of L5 projection neurons in the prelimbic (PL) subregion. In both IL and PL, the conditioning effect was time-dependent because it was not observed following a retention (tested 10 days after conditioning). Furthermore, extinction reversed the conditioning effect in both IL and PL, suggesting that these changes are transient and plastic. For comparison, the effects of delay fear conditioning on mPFC neuronal excitability was also studied. These data demonstrated that in adult rats delay fear conditioning significantly enhanced the intrinsic excitability of IL but not PL neurons. However, this conditioning effect was only significant in response to stronger (e.g., larger magnitude) current injections, suggesting that this learning effect was weak. Finally, how trace fear conditioning and extinction modulate dendritic spine density of mPFC-BLA projection neurons was also studied. These data suggest that the spine density is significantly higher in L2/3 neurons than that of L5 neurons, and that extinction facilitates the elimination of spines within L2/3 neurons in both IL and PL. Together these data implicate that both neurophysiological and morphological changes within hippocampus and mPFC are critical for the acquisition and extinction of trace fear conditioning in rats

Il-15/il-15rα signalling and synaptic transmission: a crosstalk between the immune and the nervous system?

Immune and nervous system have been traditionally considered separately, but from ‘90s many studies had unraveled the deep interconnection and interdependence between these two systems, enough to coin the term “neuroimmune system” to define this relationship. While it was well known that central nervous system (CNS) actively communicates with the immune system to control immune responses both centrally and peripherally, the opposite action was just recently discovered. Related to the role of immune system in defending and react, the interactions between immune system and CNS have been classically studied in contexts of neuroinflammation such as trauma, injury and disease [1] [2]. Recent evidences about the neuroinflammatory process in non-pathological conditions and the discovery of the important involvement of adaptive immune system in healthy brain development and activity [3], have opened many questions about physiological neuroimmune cross-talk. In this view, the cytokine network, well known to operate in a bidirectional way affecting both immune and nervous system, has a pivotal role in neuroimmune cross-talk [4]. Traditionally seen as immunomodulators, in the last years has been evident that cytokines are also potent neuromodulators [5]. In the complex cytokine system, interleukin 15 (IL-15) is considered a bridge between adaptive and innate immune system and it is one of the first upregulated cytokines in neuroinflammation [6]. It has many bioregulatory roles which range from those of modulator of selected adaptive immune responses [7] [8] and central player in the development and homeostasis of several immunocyte populations [9] to those of a potent, general inhibitor of apoptosis in multiple systems [9]. Interestingly, has been shown that IL-15 and IL-15Rα deletions affect memory and neurotransmitters concentration suggesting a major role of this signalling in cerebral functions which cannot be compensated during the development [10] [11] [12]. IL-15Rα KO mice, in particular, show decreased retention of spatial memory and contextual fear, both related to hippocampus-dependent memory, and alteration in GABA concentration. Their hippocampal ultrastructure is, however, well preserved, suggesting that the modulatory changes may involve neural plasticity even if the exact role of IL15 in modulating neurotransmission has not been investigated so far. The understandings about the mechanism by which IL-15/IL-15Rα system affect the synaptic transmission may be useful to get insight into the mechanisms of cross talk between the immune and the nervous system and eventually to develop strategies to treat pathologies whose symptoms are memory impairments and neuroinflammation

N-methyl-D-aspartate receptors play important roles in acquisition and expression of the eyeblink conditioned response in glutamate receptor subunit delta2 mutant mice.

Classical eyeblink conditioning has been known to depend critically on the cerebellum. Apparently consistent with this, glutamate receptor subunit delta2 null mutant mice, which have serious morphological and functional deficiencies in the cerebellar cortex, are severely impaired in delay paradigm. However, these mutant mice successfully learn in trace paradigm, even in \u270-trace paradigm,\u27 in which the unconditioned stimulus starts just after the conditioned stimulus terminates. Our previous studies revealed that the hippocampus and the muscarinic acetylcholine receptors play crucial roles in 0-trace paradigm in glutamate receptor subunit delta2 null mutant mice unlike in wild-type mice, suggesting a large contribution of the forebrain to 0-trace conditioning in this type of mutant mice. In the present study, we investigated the role of N-methyl-D-aspartate receptors in 0-trace eyeblink conditioning in glutamate receptor subunit delta2 null mutant mice. Mice were injected intraperitoneally with the noncompetitive N-methyl-d-aspartate receptor antagonist (+)MK-801 (0.1mg/kg) or saline, and conditioned with 350-ms tone conditioned stimulus followed by 100-ms periorbital shock unconditioned stimulus. Glutamate receptor subunit delta2 null mutant mice that received (+)MK-801 injection exhibited a severe impairment in acquisition of the conditioned response, compared with the saline-injected glutamate receptor subunit delta2 null mutant mice. In contrast, wild-type mice were not impaired in acquisition of 0-trace conditioned response by (+)MK-801 injection. After the injection solution was changed from (+)MK-801 to saline, glutamate receptor subunit delta2 null mutant mice showed a rapid and partial recovery of performance of the conditioned response. On the other hand, when the injection solution was changed from saline to (+)MK-801, glutamate receptor subunit delta2 null mutant mice showed a marked impairment in expression of the pre-acquired conditioned response, whereas impairment of the expression was small in wild-type mice. Injection of (+)MK-801 had no significant effects on spontaneous eyeblink frequency or startle eyeblink frequency to the tone conditioned stimulus in either glutamate receptor subunit delta2 null mutant mice or wild-type mice. These results suggest that N-methyl-D-aspartate receptors play critical roles both in acquisition and expression of the conditioned response in 0-trace eyeblink conditioning in glutamate receptor subunit delta2 null mutant mice