Rodent medial and lateral orbitofrontal cortices represent unique components of cognitive maps of task space.

The orbitofrontal cortex (OFC) has been proposed to function as a cognitive map of task space: a mental model of the steps involved in a task. This idea has proven popular because it provides a cohesive explanation for a number of disparate findings regarding the OFC's role in a broad array of tasks. Concurrently, evidence has begun to reveal the functional heterogeneity of OFC subregions, particularly the medial and lateral OFC. How these subregions uniquely contribute to the OFC's role as a cognitive map of task space, however, has not been explored. Here we propose that, in rodents, the lateral OFC represents the agent's initial position within that task map (i.e. initial state), determining which actions are available as a consequence of that position, whereas the medial OFC represents the agent's future position within the task map (i.e. terminal state), influencing which actions are selected to achieve that position. We argue that these processes are achieved somewhat independently and somewhat interdependently, and are achieved through similar but non-identical circuitry

Fronto-striatal organization: Defining functional and microstructural substrates of behavioural flexibility.

Discrete yet overlapping frontal-striatal circuits mediate broadly dissociable cognitive and behavioural processes. Using a recently developed multi-echo resting-state functional MRI (magnetic resonance imaging) sequence with greatly enhanced signal compared to noise ratios, we map frontal cortical functional projections to the striatum and striatal projections through the direct and indirect basal ganglia circuit. We demonstrate distinct limbic (ventromedial prefrontal regions, ventral striatum - VS, ventral tegmental area - VTA), motor (supplementary motor areas - SMAs, putamen, substantia nigra) and cognitive (lateral prefrontal and caudate) functional connectivity. We confirm the functional nature of the cortico-striatal connections, demonstrating correlates of well-established goal-directed behaviour (involving medial orbitofrontal cortex - mOFC and VS), probabilistic reversal learning (lateral orbitofrontal cortex - lOFC and VS) and attentional shifting (dorsolateral prefrontal cortex - dlPFC and VS) while assessing habitual model-free (SMA and putamen) behaviours on an exploratory basis. We further use neurite orientation dispersion and density imaging (NODDI) to show that more goal-directed model-based learning (MBc) is also associated with higher mOFC neurite density and habitual model-free learning (MFc) implicates neurite complexity in the putamen. This data highlights similarities between a computational account of MFc and conventional measures of habit learning. We highlight the intrinsic functional and structural architecture of parallel systems of behavioural control.VV and NAH are Wellcome Trust (WT) intermediate Clinical Fellows. LM is in receipt of an MRC studentship. The BCNI is supported by a WT and MRC grant. ETB is employed part-time by the University of Cambridge and part-time by GSK PLC and is a shareholder of GSK. TWR is a consultant for Cambridge Cognition, Eli Lilly, GSK, Merck, Sharpe and Dohme, Lundbeck, Teva and Shire Pharmaceuticals. He is or has been in receipt of research grants from Lundbeck, Eli Lilly and GSK and is an editor for Springer-Verlag (Psychopharmacology). The remaining authors declare no competing financial interests. The study was funded by the Wellcome Trust Fellowship grant for VV (093705/Z/10/Z) and Cambridge NIHR Biomedical Research Centre.This is the final version of the article. It was first available from Elsevier via http://dx.doi.org/10.1016/j.cortex.2015.11.00

Patients with basal ganglia damage show preserved learning in an economic game.

Both basal ganglia (BG) and orbitofrontal cortex (OFC) have been widely implicated in social and non-social decision-making. However, unlike OFC damage, BG pathology is not typically associated with disturbances in social functioning. Here we studied the behavior of patients with focal lesions to either BG or OFC in a multi-strategy competitive game known to engage these regions. We find that whereas OFC patients are significantly impaired, BG patients show intact learning in the economic game. By contrast, when information about the strategic context is absent, both cohorts are significantly impaired. Computational modeling further shows a preserved ability in BG patients to learn by anticipating and responding to the behavior of others using the strategic context. These results suggest that apparently divergent findings on BG contribution to social decision-making may instead reflect a model where higher-order learning processes are dissociable from trial-and-error learning, and can be preserved despite BG damage

Determining a Role for Ventromedial Prefrontal Cortex in Encoding Action-Based Value Signals During Reward-Related Decision Making

Considerable evidence has emerged to implicate ventromedial prefrontal cortex in encoding expectations of future reward during value-based decision making. However, the nature of the learned associations upon which such representations depend is much less clear. Here, we aimed to determine whether expected reward representations in this region could be driven by action–outcome associations, rather than being dependent on the associative value assigned to particular discriminative stimuli. Subjects were scanned with functional magnetic resonance imaging while performing 2 variants of a simple reward-related decision task. In one version, subjects made choices between 2 different physical motor responses in the absence of discriminative stimuli, whereas in the other version, subjects chose between 2 different stimuli that were randomly assigned to different responses on a trial-by-trial basis. Using an extension of a reinforcement learning algorithm, we found activity in ventromedial prefrontal cortex tracked expected future reward during the action-based task as well as during the stimulus-based task, indicating that value representations in this region can be driven by action–outcome associations. These findings suggest that ventromedial prefrontal cortex may play a role in encoding the value of chosen actions irrespective of whether those actions denote physical motor responses or more abstract decision options

Dopaminergic and Non-Dopaminergic Value Systems in Conditioning and Outcome-Specific Revaluation

Animals are motivated to choose environmental options that can best satisfy current needs. To explain such choices, this paper introduces the MOTIVATOR (Matching Objects To Internal Values Triggers Option Revaluations) neural model. MOTIVATOR describes cognitiveemotional interactions between higher-order sensory cortices and an evaluative neuraxis composed of the hypothalamus, amygdala, and orbitofrontal cortex. Given a conditioned stimulus (CS), the model amygdala and lateral hypothalamus interact to calculate the expected current value of the subjective outcome that the CS predicts, constrained by the current state of deprivation or satiation. The amygdala relays the expected value information to orbitofrontal cells that receive inputs from anterior inferotemporal cells, and medial orbitofrontal cells that receive inputs from rhinal cortex. The activations of these orbitofrontal cells code the subjective values of objects. These values guide behavioral choices. The model basal ganglia detect errors in CS-specific predictions of the value and timing of rewards. Excitatory inputs from the pedunculopontine nucleus interact with timed inhibitory inputs from model striosomes in the ventral striatum to regulate dopamine burst and dip responses from cells in the substantia nigra pars compacta and ventral tegmental area. Learning in cortical and striatal regions is strongly modulated by dopamine. The model is used to address tasks that examine food-specific satiety, Pavlovian conditioning, reinforcer devaluation, and simultaneous visual discrimination. Model simulations successfully reproduce discharge dynamics of known cell types, including signals that predict saccadic reaction times and CS-dependent changes in systolic blood pressure.Defense Advanced Research Projects Agency and the Office of Naval Research (N00014-95-1-0409); National Institutes of Health (R29-DC02952, R01-DC007683); National Science Foundation (IIS-97-20333, SBE-0354378); Office of Naval Research (N00014-01-1-0624

Thiamine Deficiency and Alcohol Exposure Both Lead To An Impulsive Phenotype

The detrimental effects of alcohol consumption, including impulsivity, cognitive inflexibility, and neuropathology, are due to a confluence of factors that often occur in addition to alcoholism. We examined the effects of chronic alcohol exposure and thiamine deficiency (or supplementation) in isolation, as well as in concert, to further our understanding of the independent effects of each treatment and how they interact. Our results demonstrated that both chronic alcohol exposure and moderate thiamine deficiency induce an impulsive phenotype. Importantly, the increased impulsivity induced by thiamine deficiency was protracted as compared to that exhibited after chronic alcohol alone. Additionally, differential pathology was observed in chronic ethanol treated subjects presenting as potentially increased limbic volume. This study has furthered our understanding of the role of multiple factors in alcohol induced decision making deficits while illustrating the need to look at the interaction of chronic alcohol exposure induced mild thiamine deficiency

Damage to the Ventromedial Prefrontal Cortex Impairs Learning from Observed Outcomes

Individuals learn both from the outcomes of their own internally generated actions ("experiential learning") and from the observation of the consequences of externally generated actions ("observational learning"). While neuroscience research has focused principally on the neural mechanisms by which brain structures such as the ventromedial prefrontal cortex (vmPFC) support experiential learning, relatively less is known regarding how learning proceeds through passive observation. We explored the necessity of the vmPFC for observational learning by testing a group of patients with damage to the vmPFC as well as demographically matched normal comparison and brain-damaged comparison groups-and a single patient with bilateral dorsal prefrontal damage-using several value-learning tasks that required learning from direct experience, observational learning, or both. We found a specific impairment in observational learning in patients with vmPFC damage manifest in the reduced influence of previously observed rewards on current choices, despite a relatively intact capacity for experiential learning. The current study provides evidence that the vmPFC plays a critical role in observational learning, suggests that there are dissociable neural circuits for experiential and observational learning, and offers an important new extension of how the vmPFC contributes to learning and memory

Examining Theories of Ventromedial Prefrontal Cortex Function

The ventromedial prefrontal cortex (VMPFC) is an intriguing brain region which sends output to and receives input from memory, emotion and reward related structures such as the amygdala, hippocampus, and caudate nucleus. Humans with lesions to the VMPFC on the surface seem to function normally and most have normal intelligence. However, in high-level tasks blending affect and decision-making, they are often highly impaired. This thesis concerns three behavioral experiments of patients with VMPFC damage which contrast and examine hypotheses of VMPFC function. In Experiment 1, the hypothesis that the VMPFC is involved in representing social knowledge was tested with more rigorous methods and a non social control task. Results did not support a specific role of the VMPFC in social knowledge. In Experiments 2 & 3, the hypothesis that VMPFC is involved in rapid reversal of stimulus-reinforcer associations was examined in detail. A gambling task and a probabilistic learning task helped discriminate punishment versus reward processing. Experiment 2 revealed normal performance of VMPFC patients in a rewards-only reversal task, in contrast to performance on previous gambling tasks with both reversal and punishment. Experiment 3 added to this evidence for a special function in punishment processing by examining learning from punishment versus learning from reward. Results revealed deficits in punishment learning, but not reward learning, after damage to the VMPFC. In conclusion, these experiments suggest a special role for the VMPFC in punishment processing, especially when a change in stimulus choice is indicated

Lateral orbitofrontal cortex promotes trial-by-trial learning of risky, but not spatial, biases

Individual choices are not made in isolation but are embedded in a series of past experiences, decisions, and outcomes. The effects of past experiences on choices, often called sequential biases, are ubiquitous in perceptual and value-based decision-making, but their neural substrates are unclear. We trained rats to choose between cued guaranteed and probabilistic rewards in a task in which outcomes on each trial were independent. Behavioral variability often reflected sequential effects, including increased willingness to take risks following risky wins, and spatial ‘win-stay/lose-shift’ biases. Recordings from lateral orbitofrontal cortex (lOFC) revealed encoding of reward history and receipt, and optogenetic inhibition of lOFC eliminated rats’ increased preference for risk following risky wins, but spared other sequential effects. Our data show that different sequential biases are neurally dissociable, and the lOFC’s role in adaptive behavior promotes learning of more abstract biases (here, biases for the risky option), but not spatial ones

Sexual Differentiation of the Prefrontal Cortex in Humans: Examining Behavioural Sex Differences and the Modulatory Role of Androgens

Sex hormones are important factors in the establishment of sex differences in the brain and behaviour during the prenatal developmental period and during adulthood. One brain area that has received little attention with respect to the study of sex differences is the prefrontal cortex (PFC). The PFC is involved in cognitive functions not limited to working memory, reinforcement learning, and inhibitory control. Currently, our understanding of the hormonal modulation of the PFC by sex steroids is also limited. The overall objectives of the present thesis were: to test the hypothesis that select cognitive functions known to depend on the PFC exhibit sex differences, to investigate whether some of these functions are influenced by developmental and/or adult androgens, and to begin to determine the functional components of PFC-dependent cognitive tasks that are responsible for eliciting sex differences. In Study 1, there was no evidence for a sex difference on two working memory tasks (Self-Ordered Pointing and the n-back), but males selected more advantageous cards than females on the Iowa Gambling Task (IGT) and were more accurate during the reversal phase of a probabilistic reversal learning task. In Study 2, the relationship between current and developmental androgens and performance on the IGT was investigated. Financial risk-taking was assessed as a potential mediator of the relationships. Circulating testosterone was found to be negatively correlated with the number of good card selections on the IGT, but there was no evidence to suggest that risk-taking was a mediator. On the other hand, there was evidence that developmental levels of androgens (using digit ratio as a proxy measure) may influence IGT performance in adulthood indirectly through an effect on risk-taking. In Study 3, females were more accurate than males on a reinforcement learning task under conditions where learning was based on positive feedback, whereas males were faster on an interference inhibition task than females. Taken together, the set of studies described in the present dissertation advance our knowledge regarding the sexual differentiation of the PFC and add to our current understanding of the modulatory role played by sex steroids on certain cognitive functions dependent on the PFC