Striatal intrinsic reinforcement signals during recognition memory: relationship to response bias and dysregulation in schizophrenia

Ventral striatum (VS) is a critical brain region for reinforcement learning and motivation, and VS hypofunction is implicated in psychiatric disorders including schizophrenia. Providing rewards or performance feedback has been shown to activate VS. Intrinsically motivated subjects performing challenging cognitive tasks are likely to engage reinforcement circuitry even in the absence of external feedback or incentives. However, such intrinsic reinforcement responses have received little attention, have not been examined in relation to behavioral performance, and have not been evaluated for impairment in neuropsychiatric disorders such as schizophrenia. Here we used fMRI to examine a challenging “old” vs. “new” visual recognition task in healthy subjects and patients with schizophrenia. Targets were unique fractal stimuli previously presented as salient distractors in a visual oddball task, producing incidental memory encoding. Based on the prediction error theory of reinforcement learning, we hypothesized that correct target recognition would activate VS in controls, and that this activation would be greater in subjects with lower expectation of responding correctly as indexed by a more conservative response bias. We also predicted these effects would be reduced in patients with schizophrenia. Consistent with these predictions, controls activated VS and other reinforcement processing regions during correct recognition, with greater VS activation in those with a more conservative response bias. Patients did not show either effect, with significant group differences suggesting hyporesponsivity in patients to internally generated feedback. These findings highlight the importance of accounting for intrinsic motivation and reward when studying cognitive tasks, and add to growing evidence of reward circuit dysfunction in schizophrenia that may impact cognition and function

A resting EEG study of neocortical hyperexcitability and altered functional connectivity in fragile X syndrome

(A) Scalp topographies of “local coupling”, showing correlations in each electrode between relative power of activity in the theta, and lower and upper alpha power bands and gamma power for male FXS and male healthy control participants, with significant group differences presented in the bottom row (p < 0.05, corrected), with dark blue reflecting no group difference. (B) Mean and standard error of correlations for all electrodes showing group differences as are plotted in A. * denotes correlations of spectral power in theta and upper alpha bands with gamma band power that are significantly different from zero based on the results of permutation analyses at p < 0.05. (TIF 4297 kb

Juvenile mice show greater flexibility in multiple choice reversal learning than adults

ABSTRACTWe hypothesized that decision-making strategies in juvenile animals, rather than being immature, are optimized to navigate the uncertainty and instability likely to be encountered in the environment at the time of the animal's transition to independence. We tested juvenile and young adult mice on discrimination and reversal of a 4-choice and 2-choice odor-based foraging task. Juvenile mice (P26–27) learned a 4-choice discrimination and reversal faster than adults (P60–70), making fewer perseverative and distraction errors. Juvenile mice had shorter choice latencies and more focused search strategies. In both ages, performance of the task was significantly impaired by a lesion of the dorsomedial frontal cortex. Our data show that the frontal cortex can support highly flexible behavior in juvenile mice at a time coincident with weaning and first independence. The unexpected developmental decline in flexibility of behavior one month later suggests that frontal cortex based executive function may not inevitably become more flexible with age, but rather may be developmentally tuned to optimize exploratory and exploitative behavior for each life stage

Does Feedback-Related Brain Response during Reinforcement Learning Predict Socio-motivational (In-)dependence in Adolescence?

This multi-methodological study applied functional magnetic resonance imaging to investigate neural activation in a group of adolescent students (N = 88) during a probabilistic reinforcement learning task. We related patterns of emerging brain activity and individual learning rates to socio-motivational (in-)dependence manifested in four different motivation types (MTs): (1) peer-dependent MT, (2) teacher-dependent MT, (3) peer-and-teacher-dependent MT, (4) peer-and-teacher-independent MT. A multinomial regression analysis revealed that the individual learning rate predicts students’ membership to the independent MT, or the peer-and-teacher-dependent MT. Additionally, the striatum, a brain region associated with behavioral adaptation and flexibility, showed increased learning-related activation in students with motivational independence. Moreover, the prefrontal cortex, which is involved in behavioral control, was more active in students of the peer-and-teacher-dependent MT. Overall, this study offers new insights into the interplay of motivation and learning with (1) a focus on inter-individual differences in the role of peers and teachers as source of students’ individual motivation and (2) its potential neurobiological basis

The uncertain brain: a co-ordinate based meta-analysis of the neural signatures supporting uncertainty during different contexts

Uncertainty is often inevitable in everyday life and can be both stressful and exciting. Given its relevance to psychopathology and wellbeing, recent research has begun to address the brain basis of uncertainty. In the current review we examined whether there are discrete and shared neural signatures for different uncertain contexts. From the literature we identified three broad categories of uncertainty currently empirically studied using functional MRI (fMRI): basic threat and reward uncertainty, decision-making under uncertainty, and associative learning under uncertainty. We examined the neural basis of each category by using a coordinate based metaanalysis, where brain activation foci from previously published fMRI experiments were drawn together (1998-2017; 87 studies). The analyses revealed shared and discrete patterns of neural activation for uncertainty, such as the insula and amygdala, depending on the category. Such findings will have relevance for researchers attempting to conceptualise uncertainty, as well as clinical researchers examining the neural basis of uncertainty in relation to psychopathology

Dissociable roles for the striatal cholinergic system in different flexibility contexts

The production of behavioural flexibility requires the coordination and integration of information from across the brain, by the dorsal striatum. In particular, the striatal cholinergic system is thought to be important for the modulation of striatal activity. Research from animal literature has shown that chemical inactivation of the dorsal striatum leads to impairments in reversal learning. Furthermore, proton magnetic resonance spectroscopy work has shown that the striatal cholinergic system is also importaiknt for reversal learning in humans. Here, we aim to assess whether the state of the dorsal striatal cholinergic system at rest is related to serial reversal learning in humans. We provide preliminary results showing that variability in choline in the dorsal striatum is significantly related to both the number of perseverative and regressive errors that participants make, and their rate of learning from positive and negative prediction errors. These findings, in line with previous work, suggest the resting state of dorsal striatal cholinergic system has important implications for producing flexible behaviour. However, these results also suggest the system may have heterogeneous functionality across different types of tasks measuring behavioural flexibility. These findings provide a starting point for further interrogation into understanding the functional role of the striatal cholinergic system in flexibility

Inactivation of the Mouse L-Proline Transporter PROT Alters Glutamatergic Synapse Biochemistry and Perturbs Behaviors Required to Respond to Environmental Changes

The endogenous neutral amino acid L-proline exhibits a variety of physiological and behavioral actions in the nervous system, highlighting the importance of accurately regulating its extracellular abundance. The L-proline transporter PROT (Slc6A7) is believed to control the spatial and temporal distribution of L-proline at glutamatergic synapses by rapid uptake of this amino acid into presynaptic terminals. Despite the importance of members of the Slc6 transporter family regulating neurotransmitter signaling and homeostasis in brain, evidence that PROT dysfunction supports risk for mental illness is lacking. Here we report the disruption of the PROT gene by homologous recombination. Mice defective in PROT displayed altered expression of glutamate transmission-related synaptic proteins in cortex and thalamus. PROT deficiency perturbed mouse behavior, such as reduced locomotor activity, decreased approach motivation and impaired memory extinction. Thus, our study demonstrates that PROT regulates behaviors that are needed to respond to environmental changes in vivo and suggests that PROT dysfunctions might contribute to mental disorders showing altered response choice following task contingency changes

Cortical and subcortical contributions to human cognitive flexibility

Cognitive flexibility enables individuals to respond adaptively to an ever-changing world. Neurally, flexibility is underpinned by involvement from across the cerebrum, and there is evidence from animal and human neuroscience suggesting that integration of cortical and thalamic signals in the striatum is necessary for appropriate behavioural control. A commonly used assay of flexibility is reversal learning, an associative learning task with high inter-species translatability. Evidence from animal literature has clearly defined the importance of the striatal cholinergic system in regulating striatal activity and output from the basal ganglia, and there is nascent evidence suggesting this system operates in a similar way in humans. However, there is a need to further disentangle the role of cortical, striatal, and thalamic regions during reversal learning in humans to better understand how the system works, and whether it has heterogeneous functionality in different contexts. Furthermore, as studying these processes is not trivial, further methodological work is required to enable us to understand the system. In chapter two we systematically assess an automated parcellation technique for identifying specific thalamic nuclei. Despite generally being treated as a homologous structure in neuroimaging work, nuclei within the thalamus have dissociable roles, and have diverse contributions to cognitive functioning, including reversal learning. We found mixed efficacy for segmentations across the thalamus, with some regions being more accurately defined relative to a “gold standard” atlas than others. Crucially, we find that the centromedian and parafascicular nuclei, which have an important role in reversal learning, are clearly defined and have little overlap with contiguous regions. These results show we can use this automated parcellation technique to identify specific thalamic nuclei that are relevant for cognitive flexibility and use these parcellations to study functionally relevant processes. Recent work has demonstrated that the functional relevance of the striatal cholinergic system can be studied in vivo using magnetic resonance spectroscopy by separating the peaks of different metabolites. But this non-conventional approach has not yet been widely adopted, and work is needed to determine its reliability. Chapter three presents test-retest reliability data on the use of magnetic resonance spectroscopy to study cholinergic activity in the striatum and cortex. We find measures of choline containing compounds are highly correlated when peaks are separated and when they are not. Across time we find that choline concentrations are relatively inconsistent, and that this was due to changes in the functionally relevant metabolite choline. Conversely, metabolites that we think are not functionally relevant were stable over time. We believe these differences may underly differences in acetylcholine function over time and may explain some intra-individual behavioural variability. In chapter four we use functional magnetic resonance imaging and psychophysiological interaction analysis to study corticostriatal and thalamostriatal connectivity during serial reversal learning. Functional connectivity between the centromedian-parafascicular nuclei of the thalamus and the associative dorsal striatum, and between the lateral-orbitofrontal cortex and the associative dorsal striatum was related to processing feedback during reversal learning. Specifically, thalamostriatal connectivity was found across the task, and may reflect a general error signal used to identify potential changes in context. Conversely, corticostriatal connectivity was found to be specific to when behaviour changed and suggests this may be a mechanism for the implementing adaptive change. We also show findings from exploratory work that may explain further how the cortex supports flexibility during reversal learning. Lastly, we used magnetic resonance spectroscopy to investigate whether the state of the cholinergic system at rest is related to reversal learning performance and latent measures of behaviour using computational modelling. Choline concentrations at rest showed significant functional relevance to our measures of reversal learning. More specifically, we found that errors during reversal learning, and learning rates for positive and negative prediction errors, explained significant variance in choline. However, the relationship between choline levels and task performance presented here differ from previous work which instead used a multi-alternative reversal learning task, and suggests that the striatal cholinergic system may have dissociable roles in different contexts. Overall, we show that the striatum, its cholinergic interneuron system, and its afferent projections from the cortex and thalamus, are associated with performance during serial reversal learning. Moreover, these findings suggest that the system may operate in separable ways in different contexts which may be dependent on internal representations of task structure

The 'How': The role of learning and flexibility in problem solving in grey and red squirrels.

Recent studies have advanced our knowledge of factors that could affect problem solving performance, and also of the positive effects of problem solving ability on fitness measures (the ‘what’ of problem solving). However, a missing linkage exists between this ‘what’ and the corresponding ‘how’. Such linkage requires the understanding of how these factors contribute to problem solving. Therefore, the central aim of this thesis is to examine this ‘how’. The roles of learning and behavioural flexibility in the context of problem solving are shown across the experiments, primarily with laboratory and free-ranging grey squirrels and to a lesser extent with wild red squirrels. Under a recurring change, laboratory grey squirrels showed a rapid decrease in the number of errors they made per reversal phase in a serial spatial reversal learning task. Such efficiency is achieved by a gradual tactic change, from sequential to integrative tactics, with increased experience. It also involves support from cognitive mechanisms such as attention and inhibitory control. In a puzzle box task, wild grey squirrels showed that they were better problem solvers than the wild red squirrels. However, red squirrels that solved the puzzle box were more efficient than the grey solvers. Detailed analysis of the results showed that learning and flexibility play independent roles in problem solving. Each process is associated with particular traits that to increase efficiency. For grey squirrels, behavioural selectivity (effective behaviours) and persistence increased with increased experience. Flexibility, however, showed minimal positive effect for them, given that it decreased behavioural selectivity. In contrast, flexibility primarily provided a positive effect for red squirrels’ solving efficiency. These results showed that the two species appear to use both similar and different cognitive processes in solving the task. The discussion gathers the results and explores how learning and flexibility, along with other behavioural traits, vary in their contributions to problem solving performance. As learning and flexibility are definitely not limited in problem solving, the discussion also addresses how these two processes might be involved a construct of general intelligence (‘g’) in animals, and how they are relevant to wilder ecological aspects