Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of neoHebbian Three-Factor Learning Rules

Most elementary behaviors such as moving the arm to grasp an object or walking into the next room to explore a museum evolve on the time scale of seconds; in contrast, neuronal action potentials occur on the time scale of a few milliseconds. Learning rules of the brain must therefore bridge the gap between these two different time scales. Modern theories of synaptic plasticity have postulated that the co-activation of pre- and postsynaptic neurons sets a flag at the synapse, called an eligibility trace, that leads to a weight change only if an additional factor is present while the flag is set. This third factor, signaling reward, punishment, surprise, or novelty, could be implemented by the phasic activity of neuromodulators or specific neuronal inputs signaling special events. While the theoretical framework has been developed over the last decades, experimental evidence in support of eligibility traces on the time scale of seconds has been collected only during the last few years. Here we review, in the context of three-factor rules of synaptic plasticity, four key experiments that support the role of synaptic eligibility traces in combination with a third factor as a biological implementation of neoHebbian three-factor learning rules

Modeling biophysical and neural circuit bases for core cognitive abilities evident in neuroimaging patterns: hippocampal mismatch, mismatch negativity, repetition positivity, and alpha suppression of distractors

This dissertation develops computational models to address outstanding problems in the domain of expectation-related cognitive processes and their neuroimaging markers in functional MRI or EEG. The new models reveal a way to unite diverse phenomena within a common framework focused on dynamic neural encoding shifts, which can arise from robust interactive effects of M-currents and chloride currents in pyramidal neurons. By specifying efficient, biologically realistic circuits that achieve predictive coding (e.g., Friston, 2005), these models bridge among neuronal biophysics, systems neuroscience, and theories of cognition. Chapter one surveys data types and neural processes to be examined, and outlines the Dynamically Labeled Predictive Coding (DLPC) framework developed during the research. Chapter two models hippocampal prediction and mismatch, using the DLPC framework. Chapter three presents extensions to the model that allow its application for modeling neocortical EEG genesis. Simulations of this extended model illustrate how dynamic encoding shifts can produce Mismatch Negativity (MMN) phenomena, including pharmacological effects on MMN reported for humans or animals. Chapters four and five describe new modeling studies of possible neural bases for alpha-induced information suppression, a phenomenon associated with active ignoring of stimuli. Two models explore the hypothesis that in simple rate-based circuits, information suppression might be a robust effect of neural saturation states arising near peaks of resonant alpha oscillations. A new proposal is also introduced for how the basal ganglia may control onset and offset of alpha-induced information suppression. Although these rate models could reproduce many experimental findings, they fell short of reproducing a key electrophysiological finding: phase-dependent reduction in spiking activity correlated with power in the alpha frequency band. Therefore, chapter five also specifies how a DLPC model, adapted from the neocortical model developed in chapter three, can provide an expectation-based model of alpha-induced information suppression that exhibits phase-dependent spike reduction during alpha-band oscillations. The model thus can explain experimental findings that were not reproduced by the rate models. The final chapter summarizes main theses, results, and basic research implications, then suggests future directions, including expanded models of neocortical mismatch, applications to artificial neural networks, and the introduction of reward circuitry

TRPM5 Channels Contribute to Persistent Neural Activity and Working Memory

Working memory is a type of memory that is active only for a short period of time (Fuster and Alexander, 1971; Goldman-Rakic, 1992). A common example of working memory is our ability to hold a phone number in our minds transiently, until it is dialed. Working memory is critical for many cognitive tasks, such as making decisions and guiding subsequent actions (Goldman-Rakic, 1992; Wickelgren, 2001). Deficits in working memory are associated with numerous pathological conditions, including schizophrenia, attention deficit hyperactivity disorder, aging, and stress (Birnbaum et al., 2004; Goldman-Rakic, 1992; Goldman-Rakic and Selemon, 1997). Therefore, it is important to understand the neural basis of working memory. During performance of a working memory task, pyramidal neurons in prefrontal cortex (PFC) are able to maintain sustained firing during a delay period between an informative cue and the appropriate behavioral response (Goldman-Rakic, 1995). Thus, stimulus-specific persistent neural activity is thought to be a neural substrate for holding memories over short time delays (Major and Tank, 2004). Once persistent activity is triggered within a neuron or neural circuit, its activity can be maintained after the stimulus has terminated. Three general (non-mutually exclusive) mechanisms of persistent activity have been hypothesized: recurrent network activity (Compte et al., 2000; Wang, 2001), short-term synaptic plasticity (Mongillo et al., 2008) and intrinsic biophysical cellular properties. Several studies have demonstrated the role of intrinsic biophysical cellular properties in persistent activity (Egorov et al., 2002; Egorov et al., 2006; Fransen et al., 2006). This firing behavior is linked to cholinergic muscarinic receptor activation and phospholipase C (PLC) signaling in the absence of synaptic reverberations. Two fundamental questions are: (1) What mechanism underlies the generation of sustained firing at a single cell level? (2) What role does intrinsic persistent firing play in working memory? Pharmacological studies suggest that persistent activity relies on activity of Ca2+-activated non-selective cation (CAN) current (ICAN) (Egorov et al., 2002; Egorov et al., 2006). However, the molecules that constitute CAN channels in the brain are not well studied, and the importance of CAN channels to working memory is unknown. I seek to identify molecular mechanisms to convert subthreshold input into intrinsic persistent neural firing in PFC layer 5 pyramidal neurons. I hypothesize that CAN channels are responsible for the intrinsic properties that mediate persistent neural activity in PFC layer 5 neurons. During muscarinic receptor activation, bursts of action potentials will lead to Ca2+ influx. CAN channels will be activated due to the increased intercellular Ca2+ and promote a slow afterdepolarization (sADP), a transition state between subthreshold input and suprathreshold sustained firing. If the sADP is strong enough, it will trigger subsequent spikes, causing further opening of voltage-dependent Ca2+ channels and Ca2+ influx, and thus further opening of CAN channels. Therefore, ICAN will be maintained by a positive feedback loop, generating persistent activity. I have combined electrophysiology, pharmacology, genetics and behavioral analyses to address the potential roles of CAN channels and persistent activity in working memory. First, I confirmed that in the presence of the muscarinic agonist carbachol a brief burst of action potentials triggers a prominent sADP and persistent activity in these neurons. Second, I confirmed that this sADP and persistent firing require activation of a PLC signaling cascade and intracellular calcium signaling. Third, I obtained direct evidence that the transient receptor potential melastatin 5 channel (TRPM5), which is thought to function as a CAN channel in non-neural cells, makes an important contribution to sADP and persistent activity in the layer 5 neurons. Importantly, Trpm5-/- mice show deficits in a Delayed-Non-Match-to-Sample maze (DNMTS) task, a working memory task in the mouse model. Furthermore, PFC-specific expression of TRPM5 using a virally-mediated delivery system in Trpm5-/- mice produced a partial rescue of deficits in the working memory tasks, indicating the importance of TRPM5 in mPFC for performance of these tasks. Lastly, I found that PFC-specific expression of TRPM5 partially rescued the electrophysiological defects in Trpm5-/- mice. By identifying an ion channel contributing to working memory, this work opens the possibility of discovering new drugs for treating working memory deficit

A History of Spike-Timing-Dependent Plasticity

How learning and memory is achieved in the brain is a central question in neuroscience. Key to today’s research into information storage in the brain is the concept of synaptic plasticity, a notion that has been heavily influenced by Hebb's (1949) postulate. Hebb conjectured that repeatedly and persistently co-active cells should increase connective strength among populations of interconnected neurons as a means of storing a memory trace, also known as an engram. Hebb certainly was not the first to make such a conjecture, as we show in this history. Nevertheless, literally thousands of studies into the classical frequency-dependent paradigm of cellular learning rules were directly inspired by the Hebbian postulate. But in more recent years, a novel concept in cellular learning has emerged, where temporal order instead of frequency is emphasized. This new learning paradigm – known as spike-timing-dependent plasticity (STDP) – has rapidly gained tremendous interest, perhaps because of its combination of elegant simplicity, biological plausibility, and computational power. But what are the roots of today’s STDP concept? Here, we discuss several centuries of diverse thinking, beginning with philosophers such as Aristotle, Locke, and Ribot, traversing, e.g., Lugaro’s plasticità and Rosenblatt’s perceptron, and culminating with the discovery of STDP. We highlight interactions between theoretical and experimental fields, showing how discoveries sometimes occurred in parallel, seemingly without much knowledge of the other field, and sometimes via concrete back-and-forth communication. We point out where the future directions may lie, which includes interneuron STDP, the functional impact of STDP, its mechanisms and its neuromodulatory regulation, and the linking of STDP to the developmental formation and continuous plasticity of neuronal networks

The cortical states of wakefulness

Cortical neurons process information on a background of spontaneous, ongoing activity with distinct spatiotemporal profiles defining different cortical states. During wakefulness, cortical states alter constantly in relation to behavioral context, attentional level or general motor activity. In this review article, we will discuss our current understanding of cortical states in awake rodents, how they are controlled, their impact on sensory processing, and highlight areas for future research. A common observation in awake rodents is the rapid change in spontaneous cortical activity from high-amplitude, low-frequency (LF) fluctuations, when animals are quiet, to faster and smaller fluctuations when animals are active. This transition is typically thought of as a change in global brain state but recent work has shown variation in cortical states across regions, indicating the presence of a fine spatial scale control system. In sensory areas, the cortical state change is mediated by at least two convergent inputs, one from the thalamus and the other from cholinergic inputs in the basal forebrain. Cortical states have a major impact on the balance of activity between specific subtypes of neurons, on the synchronization between nearby neurons, as well as the functional coupling between distant cortical areas. This reorganization of the activity of cortical networks strongly affects sensory processing. Thus cortical states provide a dynamic control system for the moment-by-moment regulation of cortical processing

Acetylcholine neuromodulation in normal and abnormal learning and memory: vigilance control in waking, sleep, autism, amnesia, and Alzheimer's disease

This article provides a unified mechanistic neural explanation of how learning, recognition, and cognition break down during Alzheimer's disease, medial temporal amnesia, and autism. It also clarifies whey there are often sleep disturbances during these disorders. A key mechanism is how acetylcholine modules vigilance control in cortical layer

Effect of Background Synaptic Activity on Excitatory-Postsynaptic Potential-Spike Coupling

Neurons receive large amount of synaptic inputs in vivo, which may impact the coupling between EPSPs and spikes. We mimicked the in vivo synaptic activity of the cell with the dynamic clamp system. We recorded from pyramidal cells in neocortical slices in vitro to investigate how timing and probability of spike generation in response to an EPSP is affected by background synaptic conductance under these conditions. We found that near threshold, background synaptic conductance improved the precision of spike timing by reducing the depolarization-related prolongation of the EPSP. In cells with ongoing spike activity and background synaptic conductances, an EPSP rapidly increased the probability of firing. The time window of the spike probability increase was comparable to the EPSP rise time and was followed by a long period of reduced firing. We found that the net synaptic gain was determined not only by the amplitude of the EPSP, but also by the firing frequency of the cell. In addition, a background fluctuating conductance reduced the time window of perturbation of spike patterns generated by EPSP related spikes. Taken together, these results indicate that in vivo, the level of the background synaptic activity may regulate spike-timing precision and affect synaptic gain

The computational role of short-term plasticity and the balance of excitation and inhibition in neural microcircuits: experimental and theoretical analysis

The computations performed by the brain ultimately rely on the functional connectivity between neurons embedded in complex networks. It is well known that the neuronal connections, the synapses, are plastic, i.e. the contribution of each presynaptic neuron to the firing of a postsynaptic neuron can be independently adjusted. The modulation of effective synaptic strength can occur on time scales that range from tens or hundreds of milliseconds, to tens of minutes or hours, to days, and may involve pre- and/or post-synaptic modifications. The collection of these mechanisms is generally believed to underlie learning and memory and, hence, it is fundamental to understand their consequences in the behavior of neurons.(...

Mode shifting between storage and recall based on novelty detection in oscillating hippocampal circuits.

ABSTRACT: It has been suggested that hippocampal mode shifting between a storage and a retrieval state might be under the control of acetylcholine (ACh) levels, as set by an autoregulatory hippocampo-septohippocampal loop. The present study investigates how such a mechanism might operate in a large-scale connectionist model of this circuitry that takes into account the major hippocampal subdivisions, oscillatory population dynamics and the time scale on which ACh exerts its effects in the hippocampus. The model assumes that hippocampal mode shifting is regulated by a novelty signal generated in the hippocampus. The simulations suggest that this signal originates in the dentate. Novel patterns presented to this structure lead to brief periods of depressed firing in the hippocampal circuitry. During these periods, an inhibitory influence of the hippocampus on the septum is lifted, leading to increased firing of cholinergic neurons. The resulting increase in ACh release in the hippocampus produces network dynamics that favor learning over retrieval. Resumption of activity in the hippocampus leads to the reinstatement of inhibition. Despite theta-locked rhythmic firing of ACh neurons in the septum, ACh modulation in the model fluctuates smoothly on a time scale of seconds. It is shown that this is compatible with the time scale on which memory processes take place. A number of strong predictions regarding memory function are derived from the model. © 2004 Wiley-Liss, Inc. KEY WORDS: acetylcholine; computational modeling; hippocampus; medial septum; memor

Biophysical foundation and function of depolarizing afterpotentials in principal cells of the medial entorhinal cortex

Neurons in layer II of the rodent medial entorhinal cortex (MEC) encode spatial information. One particular type, grid cells, tends to fire at specific spatial locations that form hexagonal lattices covering the explored environment. Within these firing fields grid cells frequently show short high-frequency spike sequences. Such bursts have received little attention but may contribute substantially to encoding spatial information. On the other hand, in vitro recordings of MEC principal cells have revealed that action potentials are followed by prominent depolarizing afterpotentials (DAP). Their biophysical foundation and function, however, are poorly understood. The objective of this study is to understand the mechanism behind the DAP by creating a biophysical realistic model of a stellate cell and to draw a connection between DAPs and burst firing in vivo. The developed single-compartment model reproduced the main electrophysi- ological characteristics of stellate cells in the MEC layer II, that are a DAP, sag, tonic firing in response to positive step currents and resonance. Using virtual blocking experiments, it was found that for the generation of the DAP only a NaP , KDR and leak current were necessary whereby the NaP current also exhibited a resurgent component. This suggests that for the generation of the DAP a balance between several currents is needed. In addition, the persistent and resurgent sodium current might play an important role. We analyzed the relevance of DAPs in vivo using whole-cell recordings of grid cells from Domnisoru et al. (2013). We found that around 20% of the cells exhibited a DAP. However, the percentage of cells was much lower than estimates from in vitro recordings. We showed that this is partly due to the quality of the recording as selecting APs from presumably good parts of the recording improved the visibility of DAPs. To investigate the relationship between DAPs and burst firing all cells were classified into bursty and non-bursty based on the spike-time autocorrelation. All cells with a DAP were bursty except the cell with the smallest DAP. Moreover, taking the mean of the spike-triggered average of the membrane potential for all bursty and non-bursty cells respectively showed a clear DAP for bursty but not for non-bursty cells. In summary, we found that the DAP can be realized in a single-compartment model by a NaP , KDR and leak current and provided evidence for the relevance of DAPs for burst firing in vivo