Inferring action-dependent outcome representations depends on anterior but not posterior medial orbitofrontal cortex

© 2018 Elsevier Inc. Although studies examining orbitofrontal cortex (OFC) often treat it as though it were functionally homogeneous, recent evidence has questioned this assumption. Not only are the various subregions of OFC (lateral, ventral, and medial) hetereogeneous, but there is further evidence of heterogeneity within those subregions. For example, several studies in both humans and monkeys have revealed a functional subdivision along the anterior-posterior gradient of the medial OFC (mOFC). Given our previous findings suggesting that, in rats, the mOFC is responsible for inferring the likelihood of unobservable action outcomes (Bradfield, Dezfouli, van Holstein, Chieng, & Balleine, 2015), and given the anterior nature of the placements of our prior manipulations, we decided to assess whether the rat mOFC also differs in connection and function along its anteroposterior axis. We first used retrograde tracing to compare the density of efferents from mOFC to several structures known to contribute to goal-directed action: the mediodorsal thalamus, basolateral amygdala, posterior dorsomedial striatum, nucleus accumbens core and ventral tegmental area. We then compared the functional effects of anterior versus posterior mOFC excitotoxic lesions on tests of Pavlovian-instrumental transfer, instrumental outcome devaluation and outcome-specific reinstatement. We found evidence that the anterior mOFC had greater connectivity with the accumbens core and greater functional involvement in goal-directed action than the posterior mOFC. Consistent with previous findings across species, therefore, these results suggest that the anterior and posterior mOFC of the rat are indeed functionally distinct, and that it is the anterior mOFC that is particularly critical for inferring unobservable action outcomes

Prefrontal corticostriatal disconnection blocks the acquisition of goal-directed action

© 2018 the authors. The acquisition of goal-directed action requires encoding of the association between an action and its specific consequences or outcome. At a neural level, this encoding has been hypothesized to involve a prefrontal corticostriatal circuit involving the projection from the prelimbic cortex (PL) to the posterior dorsomedial striatum (pDMS); however, no direct evidence for this claim has been reported. In a series of experiments, we performed functional disconnection of this pathway using targeted lesions of the anterior corpus callosum to disrupt contralateral corticostriatal projections with asymmetrical lesions of the PL and/orpDMSto block plasticity in this circuit in rats. We first demonstrated that unilaterally blocking the PL input to the pDMS prevented the phosphorylation of extracellular signal-related kinase/mitogen activated protein kinase (pERK/pMAPK) induced by instrumental training. Next, we used a full bilateral disconnection of the PL from the pDMS and assessed goal-directed action using an outcome-devaluation test. Importantly, we found evidence that rats maintaining an ipsilateral and/or contralateral connection between thePLand thepDMSwere able to acquire goal-directed actions. In contrast, bilateral PL–pDMS disconnection abolished the acquisition of goal-directed actions. Finally, we used a temporary pharmacological disconnection to disrupt PL inputs to thepDMSby infusing theNMDAantagonist DL-2-amino-5-phosphonopentanoic acid into thepDMSduring instrumental training and found that this manipulation also disrupted goal-directed learning. These results establish that, in rats, the acquisition of new goal-directed actions depends on a prefrontal-corticostriatal circuit involving a connection between the PL and the pDMS

Goal-directed actions transiently depend on dorsal hippocampus.

The role of the hippocampus in goal-directed action is currently unclear; studies investigating this issue have produced contradictory results. Here we reconcile these contradictions by demonstrating that, in rats, goal-directed action relies on the dorsal hippocampus, but only transiently, immediately after initial acquisition. Furthermore, we found that goal-directed action also depends transiently on physical context, suggesting a psychological basis for the hippocampal regulation of goal-directed action control

Medial Orbitofrontal Cortex Regulates Instrumental Conditioned Punishment, but not Pavlovian Conditioned Fear

Abstract Bidirectionally aberrant medial orbitofrontal cortical (mOFC) activity has been consistently linked with compulsive disorders and related behaviors. Although rodent studies have established a causal link between mOFC excitation and compulsive-like actions, no such link has been made with mOFC inhibition. Here we use excitotoxic lesions of mOFC to investigate its role in sensitivity to punishment; a core characteristic of many compulsive disorders. In our first experiment, we demonstrated that mOFC lesions prevented rats from learning to avoid a lever that was punished with a stimulus that co-terminated with footshock. Our second experiment demonstrated that retrieval of punishment learning is also somewhat mOFC-dependent, as lesions prevented the extended retrieval of punishment contingencies relative to shams. In contrast, mOFC lesions did not prevent rats from re-acquiring the ability to avoid a punished lever when it was learned prior to lesions being administered. In both experiments, Pavlovian fear conditioning to the stimulus was intact for all animals. Together, these results reveal that the mOFC regulates punishment learning and retrieval in a manner that is separate from any role in Pavlovian fear conditioning. These results imply that aberrant mOFC activity may contribute to the punishment insensitivity that is observed across multiple compulsive disorders.</jats:p

Retrospective model-based inference guides model-free credit assignment

An extensive reinforcement learning literature shows that organisms assign credit efficiently, even under conditions of state uncertainty. However, little is known about credit-assignment when state uncertainty is subsequently resolved. Here, we address this problem within the framework of an interaction between model-free (MF) and model-based (MB) control systems. We present and support experimentally a theory of MB retrospective-inference. Within this framework, a MB system resolves uncertainty that prevailed when actions were taken thus guiding an MF credit-assignment. Using a task in which there was initial uncertainty about the lotteries that were chosen, we found that when participants’ momentary uncertainty about which lottery had generated an outcome was resolved by provision of subsequent information, participants preferentially assigned credit within a MF system to the lottery they retrospectively inferred was responsible for this outcome. These findings extend our knowledge about the range of MB functions and the scope of system interactions

Food color is in the eye of the beholder: the role of human trichromatic vision in food evaluation

Non-human primates evaluate food quality based on brightness of red and green shades of color, with red signaling higher energy or greater protein content in fruits and leafs. Despite the strong association between food and other sensory modalities, humans, too, estimate critical food features, such as calorie content, from vision. Previous research primarily focused on the effects of color on taste/flavor identification and intensity judgments. However, whether evaluation of perceived calorie content and arousal in humans are biased by color has received comparatively less attention. In this study we showed that color content of food images predicts arousal and perceived calorie content reported when viewing food even when confounding variables were controlled for. Specifically, arousal positively co-varied with red-brightness, while green-brightness was negatively associated with arousal and perceived calorie content. This result holds for a large array of food comprising of natural food - where color likely predicts calorie content - and of transformed food where, instead, color is poorly diagnostic of energy content. Importantly, this pattern does not emerged with nonfood items. We conclude that in humans visual inspection of food is central to its evaluation and seems to partially engage the same basic system as non-human primates

Reward-Related Behavioral Paradigms for Addiction Research in the Mouse: Performance of Common Inbred Strains

The mouse has emerged as a uniquely valuable species for studying the molecular and genetic basis of complex behaviors and modeling neuropsychiatric disease states. While valid and reliable preclinical assays for reward-related behaviors are critical to understanding addiction-related processes, and various behavioral procedures have been developed and characterized in rats and primates, there have been relatively few studies using operant-based addiction-relevant behavioral paradigms in the mouse. Here we describe the performance of the C57BL/6J inbred mouse strain on three major reward-related paradigms, and replicate the same procedures in two other commonly used inbred strains (DBA/2J, BALB/cJ). We examined Pavlovian-instrumental transfer (PIT) by measuring the ability of an auditory cue associated with food reward to promote an instrumental (lever press) response. In a separate experiment, we assessed the acquisition and extinction of a simple stimulus-reward instrumental behavior on a touchscreen-based task. Reinstatement of this behavior was then examined following either continuous exposure to cues (conditioned reinforcers, CRs) associated with reward, brief reward and CR exposure, or brief reward exposure followed by continuous CR exposure. The third paradigm examined sensitivity of an instrumental (lever press) response to devaluation of food reward (a probe for outcome insensitive, habitual behavior) by repeated pairing with malaise. Results showed that C57BL/6J mice displayed robust PIT, as well as clear extinction and reinstatement, but were insensitive to reinforcer devaluation. DBA/2J mice showed good PIT and (rewarded) reinstatement, but were slow to extinguish and did not show reinforcer devaluation or significant CR-reinstatement. BALB/cJ mice also displayed good PIT, extinction and reinstatement, and retained instrumental responding following devaluation, but, unlike the other strains, demonstrated reduced Pavlovian approach behavior (food magazine head entries). Overall, these assays provide robust paradigms for future studies using the mouse to elucidate the neural, molecular and genetic factors underpinning reward-related behaviors relevant to addiction research

Disorders of compulsivity: a common bias towards learning habits.

Why do we repeat choices that we know are bad for us? Decision making is characterized by the parallel engagement of two distinct systems, goal-directed and habitual, thought to arise from two computational learning mechanisms, model-based and model-free. The habitual system is a candidate source of pathological fixedness. Using a decision task that measures the contribution to learning of either mechanism, we show a bias towards model-free (habit) acquisition in disorders involving both natural (binge eating) and artificial (methamphetamine) rewards, and obsessive-compulsive disorder. This favoring of model-free learning may underlie the repetitive behaviors that ultimately dominate in these disorders. Further, we show that the habit formation bias is associated with lower gray matter volumes in caudate and medial orbitofrontal cortex. Our findings suggest that the dysfunction in a common neurocomputational mechanism may underlie diverse disorders involving compulsion.This study was funded by the WT fellowship grant for VV (093705/Z/ 10/Z) and Cambridge NIHR Biomedical Research Centre. VV and NAH are Wellcome Trust (WT) intermediate Clinical Fellows. YW is supported by the Fyssen Fondation and MRC Studentships. PD is supported by the Gatsby Charitable Foundation. JEG has received grants from the National Institute of Drug Abuse and the National Center for Responsible Gaming. TWR and BJS are supported on a WT Programme Grant (089589/Z/09/Z). The BCNI is supported by a WT and MRC grant.This is the final published version. It's also available from Molecular Psychiatry at http://www.nature.com/mp/journal/vaop/ncurrent/full/mp201444a.html

Eliminating the mystery from the concept of emergence

While some branches of complexity theory are advancing rapidly, the same cannot be said for our understanding of emergence. Despite a complete knowledge of the rules underlying the interactions between the parts of many systems, we are often baffled by their sudden transitions from simple to complex. Here I propose a solution to this conceptual problem. Given that emergence is often the result of many interactions occurring simultaneously in time and space, an ability to intuitively grasp it would require the ability to consciously think in parallel. A simple exercise is used to demonstrate that we do not possess this ability. Our surprise at the behaviour of cellular automata models, and the natural cases of pattern formation they mimic, is then explained from this perspective. This work suggests that the cognitive limitations of the mind can be as significant a barrier to scientific progress as the limitations of our senses