Dopamine and the Temporal Dependence of Learning and Memory

Animal behavior is largely influenced by the seeking out of rewards and avoidance of punishments. Positive or negative reinforcements, like a food reward or painful shock, impart meaningful valence onto sensory cues in the animal’s environment. The ability of animals to form associations between a sensory cue and a rewarding or punishing reinforcement permits them to adapt their future behavior to maximize reward and minimize punishments. Animals rely on the timing of events to infer the causal relationships between cues and outcomes –– sensory cues that precede a painful shock in time become associated with its onset and are imparted with negative valence, whereas cues that follow the shock in time are instead associated with its cessation and imparted with positive valence. While the temporal requirements for associative learning have been well characterized at the behavioral level, the molecular and circuit mechanisms for this temporal sensitivity remain incompletely understood. Using the simple architecture of the mushroom body, an olfactory associative learning center in Drosophila, I examined how the relative timing of olfactory inputs and dopaminergic reinforcement signals is encoded at the molecular, synaptic, and circuit level to give rise to learned odor associations. I show that in Drosophila, opposing olfactory associations can be formed and updated on a trial-by-trial basis depending on the temporal relationship between an odor cue and dopaminergic reinforcement during conditioning. Additionally, both negative and positive reinforcements equivalently instruct appetitive and aversive olfactory associations –– odors preceding a negative reinforcement or following a rewarding reinforcement acquire an aversive valence, while odors instead following a negative reinforcement or preceding a rewarding reinforcement become attractive. Furthermore, functional imaging revealed that synapses within the mushroom body are bidirectionally modulated depending on the temporal ordering of odor and dopaminergic reinforcement, leading to synaptic depression when an odor precedes dopaminergic activity or synaptic facilitation when dopaminergic activity instead precedes an odor. Through the synchronous recording of neural activity and behavior, I found that the bidirectional regulation of synaptic transmission within the mushroom body directly correlates with the emergence of learned olfactory behaviors. This temporal sensitivity arises from two dopamine receptors, DopR1 and DopR2, that couple to distinct second-messengers and direct either synaptic depression or potentiation. Loss of either receptor renders the synapses of the mushroom body capable of only unidirectional plasticity and prevents the behavioral flexibility of writing opposing associations depending on the temporal structure of conditioning. Together, these results reveal how the distinct intracellular signaling pathways of two dopamine receptors can detect the order of events within an associative learning circuit to instruct opposing forms of synaptic and behavioral plasticity, providing a mechanism for animals to use both the onset and offset of a reinforcement signal to instruct distinct associations. Additionally, this bidirectional modulation allows animals to flexibly update olfactory associations on a trial-bytrial basis when temporal relationships are altered, permitting them to contend with a complex and changing sensory world

Neurofly 2008 abstracts : the 12th European Drosophila neurobiology conference 6-10 September 2008 Wuerzburg, Germany

This volume consists of a collection of conference abstracts

Rhythmic Action Synchronizes Memory Replay During Reinforcement Learning

Our cognitive abilities - learning from the past, sensing the current environment, planning into the future, executing an action, and infusing value into an experience - all rely on precisely timed and widespread electrical communications across neural networks. The brain’s hippocampal formation receives multimodal input, forges episodic associations, and predicts future state. Oscillating electrical bursts originating from the hippocampus, termed ‘sharp-wave ripples’ (SWR), often contain patterns of previously expressed neural spike sequences, and are necessary for certain forms of learning and memory. The discharge of SWR-replay resonates in remote parts of the brain and displays specific characteristics depending on a subject’s state of awareness and sensory context. In the sleep state, when motoric repertoire is limited, waves of breathing synchronize neural activity in several regions of the brain, including SWRs of the hippocampus. During active sensation of the awake state, cyclic licking dynamically entrains taste-reward networks in subcortical and cortical areas throughout learning. However, the neural correlates linking oromotor movements in the active learning state to the memory system of the hippocampal formation have not yet been established. Given the recurrence of SWR-replay during rhythmic ingestion of reinforcement learning and the hierarchical coupling of orofacial behaviors, we hypothesized that repeated licking could provide the oscillatory framework to synchronize memory reactivation during active learning. We approach this question with new technology development to track licking events at a reward port (P-event) during behavior on a spatial alternation task. Additionally, we developed a modular brain implant to simultaneously record from hippocampal area CA1 and medial entorhinal cortex (MEC) - interconnected brain regions that are crucial to episodic memory processing. Along with the co-modulation of individual neurons by licking and SWRs, we provide the first evidence that SWRs detected in dorsal CA1 synchronize with the phase of P-event cycle during learning. Furthermore, we confirmed that SWRs occurring during licking bouts contain neural reactivation of active navigation and trigger enhanced ripple-frequency power in downstream MEC. These results connect movement with memory and may assist in addressing abnormal ingestion behaviors that negatively affect mental or physical healt

A systems pharmacology approach to modulating spatial memory

Spatial navigation in humans correlates with activity of cells in hippocampus that respond when we traverse specific locations in our environment. Hippocampal pyramidal cells in rodents called "place cells" may contribute to episodic memory by encoding location in physical space. Place cells display plasticity by "remapping" or altering their firing rates and patterns of activity in response to changes in spatial environment. Impaired remapping may underlie age-related deficits in spatial memory tasks. Using in vivo high-density electrophysiology to record place cell activity in awake, behaving rats, we tested the hypothesis that CA3 neuron hyperactivity in aged animals could be normalized by pharmacotherapy. Results show that acute, systemic administration of low dose levetiracetam and sodium valproate ameliorates deficits in the aged hippocampal network by reducing firing rates, decreasing place field area, and increasing the spatial selectivity of CA3 place cells. We then tested the hypothesis that place cell activity, field area, and spatial selectivity may be an indicator for therapeutic enhancement of spatial memory in young adult rats. The results demonstrate that α5IA enhances hippocampal-dependent spatial memory as measured by the location novelty recognition task in rats, consistent with the previously established action of α5IA as an enhancer of spatial memory in the water maze test. Electrophysiological recordings on the same animals carried out in parallel demonstrate that α5IA increases place cell firing rates, reduces field area, and increases spatial selectivity. Together, these results suggest that reducing place field area and enhancing spatial selectivity correlate with the age-independent therapeutic improvement of spatial memory. The increase in place cell firing rates by α5IA likely results from its known action as a negative allosteric modulator of α5-subunit-containing receptors (α), which are located extrasynaptically at the base of dendritic spines on CA1 and CA3 pyramidal cells. Thus, to potentially target extrasynaptic tonic inhibition in the hippocampus, we synthesized and validated two α specific miRNAs as a platform for future attempts to improve spatial memory in young adult and aging animals via molecular genetics

From Biological Synapses to "Intelligent" Robots

This selective review explores biologically inspired learning as a model for intelligent robot control and sensing technology on the basis of specific examples. Hebbian synaptic learning is discussed as a functionally relevant model for machine learning and intelligence, as explained on the basis of examples from the highly plastic biological neural networks of invertebrates and vertebrates. Its potential for adaptive learning and control without supervision, the generation of functional complexity, and control architectures based on self-organization is brought forward. Learning without prior knowledge based on excitatory and inhibitory neural mechanisms accounts for the process through which survival-relevant or task-relevant representations are either reinforced or suppressed. The basic mechanisms of unsupervised biological learning drive synaptic plasticity and adaptation for behavioral success in living brains with different levels of complexity. The insights collected here point toward the Hebbian model as a choice solution for “intelligent” robotics and sensor systems. Keywords: Hebbian learning; synaptic plasticity; neural networks; self-organization; brain; reinforcement; sensory processing; robot contro