CNTRICS Final Task Selection: Long-term Memory

Long-term memory (LTM) is a multifactorial construct, composed of different stages of information processing and different cognitive operations that are mediated by distinct neural systems, some of which may be more responsible for the marked memory problems that limit the daily function of individuals with schizophrenia. From the outset of the CNTRICS initiative, this multidimensionality was appreciated, and an effort was made to identify the specific memory constructs and task paradigms that hold the most promise for immediate translational development. During the second CNTRICS meeting, the LTM group identified item encoding and retrieval and relational encoding and retrieval as key constructs. This article describes the process that the LTM group went through in the third and final CNTRICS meeting to select nominated tasks within the 2 LTM constructs and within a reinforcement learning construct that were judged most promising for immediate development. This discussion is followed by each nominating authors' description of their selected task paradigm, ending with some thoughts about future directions.Psycholog

Reinforcement Learning in Individuals at Risk for Alzheimer\u27s Disease

Explicit memory is the hallmark of impairment in Alzheimer’s disease (AD) while implicit memory has mixed task-dependent results. Models of memory processes have posited that hippocampal function is sensitive to reinforcement learning (RL), which involves both explicit and implicit memory. The hippocampus is also vital for the transfer of learned associations to novel situations. Nevertheless, RL paradigms have been underutilized in assessing memory processes in individuals at risk for AD, which may aid in early identification of cognitive decline. Thirty-six apolipoprotein-E (APOE) genotyped older adults (Male n=8; Mage=80; Meducation=15 years) performed word stem completion, word recognition, and RL tasks. The RL task was comprised of an RL phase, an implicit testing phase, and explicit recognition component. Group comparisons were made based on low risk (APOE ε4-; n=16) vs. high risk (APOE ε4+; n=20) for AD. A series of mixed ANOVAs based on task performance indicated that risk groups did not differ on EM measures (RL, word recognition, and RL recognition). However, high risk participants exhibited significantly poorer IM performance (RL testing and word stem) than the low risk group, p = .03. The pattern of results in the present study was counter to prediction in that risk groups did not differ on explicit memory measures, which was strongly supported by existing literature. However, the exhibited performance of poorer implicit memory in the high risk group is consistent with results implicating the hippocampus in the application learned associations to novel environments. RL paradigms may offer high sensitivity for assessing preclinical decline

Inhibition of Action, Thought, and Emotion: A Selective Neurobiological Review

The neural bases of inhibitory function are reviewed, covering data from paradigms assessing inhibition of motor responses (antisaccade, go/nogo, stop-signal), cognitive sets (e.g., Wisconsin Card Sort Test), and emotion (fear extinction). The frontal cortex supports performance on these paradigms, but the specific neural circuitry varies: response inhibition depends upon fronto-basal ganglia networks, inhibition of cognitive sets is supported by orbitofrontal cortex, and retention of fear extinction reflects ventromedial prefrontal cortex-amygdala interactions. Inhibition is thus neurobiologically heterogeneous, although right ventrolateral prefrontal cortex may support a general inhibitory process. Dysfunctions in these circuits may contribute to psychopathological conditions marked by inhibitory deficits.Psycholog

Brain network mechanisms in learning behavior

The study of learning has been a central focus of psychology and neuroscience since their inception. Cognitive neuroscience’s traditional approach to understanding learn-ing has been to decompose it into discrete cognitive processes with separable and localized underlying neural systems. While this focus on modular cognitive functions for individual brain areas has led to considerable progress, there is increasing evidence that much of learn-ing behavior relies on overlapping cognitive and neural systems, which may be harder to disentangle than previously envisioned. This is not surprising, as the processes underlying learning must involve widespread integration of information from sensory, affective, and motor sources. The standard tools of cognitive neuroscience limit our ability to describe processes that rely on widespread coordination of brain activity. To understand learning, it will be necessary to characterize dynamic co-activation at the circuit level. In this dissertation, I present three studies that seek to describe the roles of distrib-uted brain networks in learning. I begin by giving an overview of our current understand-ing of multiple forms of learning, describing the neural and computational mechanisms thought to underlie incremental feedback-based learning and flexible episodic memory. I will focus in particular on the difficulties in separating these processes at the cognitive level and in localizing them to individual regions at the neural level. I will then describe recent findings that have begun to characterize the brain’s large-scale network structure, emphasiz-ing the potential roles that distributed networks could play in understanding learning and cognition more generally. I will end the introduction by reviewing current attempts to char-acterize the dynamics of large-scale brain networks, which will be essential for providing a mechanistic link to learning behavior. Chapter 2 is a study demonstrating that intrinsic connectivity between the hippo-campus and the ventromedial prefrontal cortex, as well as between these regions and dis-tributed brain networks, is related to individual differences in the transfer of learning on a sensory preconditioning task. The hippocampus and ventromedial prefrontal cortex have both been shown to be involved in this type of learning, and this study represents an early attempt to link connectivity between individual regions and broader networks to learning processes. Chapter 3 is a study that takes advantage of recent developments in mathematical modeling of temporal networks to demonstrate a relationship between large-scale network dynamics and reinforcement learning within individuals. This study shows that the flexibil-ity of network connectivity in the striatum is related to learning performance over time, as well as to individual differences in parameters estimated from computational models of re-inforcement learning. Notably, connectivity between the striatum and visual as well as or-bitofrontal regions increased over the course of the task, which is consistent with an inte-grative role for the region in learning value-based associations. Network flexibility in a dis-tinct set of regions is associated with episodic memory for object images presented during the learning task. Chapter 4 examines the role of dopamine, a neurotransmitter strongly linked to val-ue updating in reinforcement learning, in the dynamic network changes occurring during learning. Patients with Parkinson’s disease, who experience a loss of dopaminergic neu-rons in the substantia nigra, performed a reversal-learning task while undergoing functional magnetic resonance imaging. Patients were scanned on and off of a dopamine precursor medication (levodopa) in a within-subject design in order to examine the impact of dopa-mine on brain network dynamics during learning. The reversal provided an experimental manipulation of dynamic connectivity, and patients on medication showed greater modula-tion of striatal-cortical connectivity. Similar results were found in a number of regions re-ceiving midbrain projections including the prefrontal cortex and medial temporal lobe. This study indicates that dopamine inputs from the midbrain modulate large-scale network dy-namics during learning, providing a direct link between reinforcement learning theories of value updating and network neuroscience accounts of dynamic connectivity. Together, these results indicate that large-scale networks play a critical role in multi-ple forms of learning behavior. Each highlights the potential importance of understanding dynamic routing and integration of information across large-scale circuits for our concep-tion of learning and other cognitive processes. Understanding the when, where, and how of this information flow in the brain may provide an alternative or compliment to traditional theories of distinct learning systems. These studies also illustrate challenges in integrating this perspective with established theories in cognitive neuroscience. Chapter 5 will situate the studies in a broader discussion of how brain activity relates to cognition in general, while pointing out current roadblocks and potential ways forward for a cognitive network neuroscience of learning

Mild Reinforcement Learning Deficits in Patients With First-Episode Psychosis

Numerous studies have identified reinforcement learning (RL) deficits in schizophrenia. Most have focused on chronic patients with longstanding antipsychotic treatment, however, and studies of RL in early-illness patients have produced mixed results, particularly regarding gradual/procedural learning. No study has directly contrasted both rapid and gradual RL in first-episode psychosis (FEP) samples. We examined probabilistic RL in 34 FEP patients and 36 controls, using Go/NoGo (GNG) and Gain vs Loss-Avoidance (GLA) paradigms. Our results were mixed, with FEP patients exhibiting greater impairment in the ability to use positive, as opposed to negative, feedback to drive rapid RL on the GLA, but not the GNG. By contrast, patients and controls showed similar improvement across the acquisition. Finally, we found no significant between-group differences in the postacquisition expression of value-based preference in both tasks. Negative symptoms were modestly associated with RL measures, while the overall bias to engage in Go-responding correlated significantly with psychosis severity in FEP patients, consistent with striatal hyperdopaminergia. Taken together, FEP patients demonstrated more circumscribed RL impairments than previous studies have documented in chronic samples, possibly reflecting differential symptom profiles between first-episode and chronic samples. Our finding of relatively preserved gradual/procedural RL, in briefly medicated FEP patients, might suggest spared or restored basal ganglia function. Our findings of preserved abilities to use representations of expected value to guide decision making, and our mixed results regarding rapid RL, may reflect a lesser degree of prefrontal cortical functional impairment in FEP than in chronic samples. Further longitudinal research, in larger samples, is required.postprin

The Cognitive Role of the Globus Pallidus interna; Insights from Disease States.

The motor symptoms of both Parkinson's disease and focal dystonia arise from dysfunction of the basal ganglia, and are improved by pallidotomy or deep brain stimulation of the Globus Pallidus interna (GPi). However, Parkinson's disease is associated with a greater degree of basal ganglia-dependent learning impairment than dystonia. We attempt to understand this observation in terms of a comparison of the electrophysiology of the output of the basal ganglia between the two conditions. We use the natural experiment offered by Deep Brain Stimulation to compare GPi local field potential responses in subjects with Parkinson's disease compared to subjects with dystonia performing a forced-choice decision-making task with sensory feedback. In dystonic subjects, we found that auditory feedback was associated with the presence of high gamma oscillations nestled on a negative deflection, morphologically similar to sharp wave ripple complexes described in human rhinal cortex. These were not present in Parkinson's disease subjects. The temporal properties of the high gamma burst were modified by incorrect trial performance compared to correct trial performance. Both groups exhibited a robust low frequency response to 'incorrect' trial performance in dominant GPi but not non-dominant GPi at theta frequency. Our results suggest that cellular processes associated with striatum-dependent memory function may be selectively impaired in Parkinson's disease even if dopaminergic drugs are administered, but that error detection mechanisms are preserved

Habits without values

Habits form a crucial component of behavior. In recent years, key computational models have conceptualized habits as arising from model-free reinforcement learning (RL) mechanisms, which typically select between available actions based on the future value expected to result from each. Traditionally, however, habits have been understood as behaviors that can be triggered directly by a stimulus, without requiring the animal to evaluate expected outcomes. Here, we develop a computational model instantiating this traditional view, in which habits develop through the direct strengthening of recently taken actions rather than through the encoding of outcomes. We demonstrate that this model accounts for key behavioral manifestations of habits, including insensitivity to outcome devaluation and contingency degradation, as well as the effects of reinforcement schedule on the rate of habit formation. The model also explains the prevalent observation of perseveration in repeated-choice tasks as an additional behavioral manifestation of the habit system. We suggest that mapping habitual behaviors onto value-free mechanisms provides a parsimonious account of existing behavioral and neural data. This mapping may provide a new foundation for building robust and comprehensive models of the interaction of habits with other, more goal-directed types of behaviors and help to better guide research into the neural mechanisms underlying control of instrumental behavior more generally