Sour grapes and sweet victories: How actions shape preferences

Classical decision theory postulates that choices proceed from subjective values assigned to the probable outcomes of alternative actions. Some authors have argued that opposite causality should also be envisaged, with choices influencing subsequent values expressed in desirability ratings. The idea is that agents may increase their ratings of items that they have chosen in the first place, which has been typically explained by the need to reduce cognitive dissonance. However, evidence in favor of this reverse causality has been the topic of intense debates that have not reached consensus so far. Here, we take a novel approach using Bayesian techniques to compare models in which choices arise from stable (but noisy) underlying values (one-way causality) versus models in which values are in turn influenced by choices (two-way causality). Moreover, we examined whether in addition to choices, other components of previous actions, such as the effort invested and the eventual action outcome (success or failure), could also impact subsequent values. Finally, we assessed whether the putative changes in values were only expressed in explicit ratings, or whether they would also affect other value-related behaviors such as subsequent choices. Behavioral data were obtained from healthy participants in a rating-choice-rating-choice-rating paradigm, where the choice task involves deciding whether or not to exert a given physical effort to obtain a particular food item. Bayesian selection favored two-way causality models, where changes in value due to previous actions affected subsequent ratings, choices and action outcomes. Altogether, these findings may help explain how values and actions drift when several decisions are made successively, hence highlighting some shortcomings of classical decision theory

Liraglutide restores impaired associative learning in individuals with obesity

Survival under selective pressure is driven by the ability of our brain to use sensory information to our advantage to control physiological needs. To that end, neural circuits receive and integrate external environmental cues and internal metabolic signals to form learned sensory associations, consequently motivating and adapting our behaviour. The dopaminergic midbrain plays a crucial role in learning adaptive behaviour and is particularly sensitive to peripheral metabolic signals, including intestinal peptides, such as glucagon-like peptide 1 (GLP-1). In a single-blinded, randomized, controlled, crossover basic human functional magnetic resonance imaging study relying on a computational model of the adaptive learning process underlying behavioural responses, we show that adaptive learning is reduced when metabolic sensing is impaired in obesity, as indexed by reduced insulin sensitivity (participants: N = 30 with normal insulin sensitivity; N = 24 with impaired insulin sensitivity). Treatment with the GLP-1 receptor agonist liraglutide normalizes impaired learning of sensory associations in men and women with obesity. Collectively, our findings reveal that GLP-1 receptor activation modulates associative learning in people with obesity via its central effects within the mesoaccumbens pathway. These findings provide evidence for how metabolic signals can act as neuromodulators to adapt our behaviour to our body’s internal state and how GLP-1 receptor agonists work in clinics

A model of reward- and effort-based optimal decision making and motor control.

Costs (e.g. energetic expenditure) and benefits (e.g. food) are central determinants of behavior. In ecology and economics, they are combined to form a utility function which is maximized to guide choices. This principle is widely used in neuroscience as a normative model of decision and action, but current versions of this model fail to consider how decisions are actually converted into actions (i.e. the formation of trajectories). Here, we describe an approach where decision making and motor control are optimal, iterative processes derived from the maximization of the discounted, weighted difference between expected rewards and foreseeable motor efforts. The model accounts for decision making in cost/benefit situations, and detailed characteristics of control and goal tracking in realistic motor tasks. As a normative construction, the model is relevant to address the neural bases and pathological aspects of decision making and motor control

Compromis entre efforts et récompenses (un modèle unifié de la décision et de la motricité)

Le comportement est fortement déterminé par les coûts (e.g. les efforts) et les bénéfices (e.g. la nourriture, l'argent) de nos actions. Dans les approches actuelles, en économie et en écologie plus particulièrement, l'optimisation de ces valeurs sert de principe pour organiser le comportement. Toutefois, ces approches ne peuvent expliquer comment la décision (le choix d'un objectif à atteindre) est effectivement convertie en action (le contrôle de la bio-mécanique du corps). La raison à cela est que l'optimisation des coûts et des bénéfices sont considérés comme deux processus imperméables l'un à l'autre. Pourtant, ces deux éléments interagissent fortement à tous les niveaux de l'élaboration de nos actes. Ce travail propose un modèle du comportement dans lequel la décision et la production motrice émergent de l'optimisation permanente du compromis prospectif et pondéré entre la récompense et l'effort moteur liés à l'action. Ce modèle permet de rendre compte de la décision dans les situations d'effort ainsi que des caractéristiques détaillées de la coordination motrice et de sa modulation par le contexte comportemental. Le formalisme présenté offre un cadre normatif pour interpréter les bases neurologiques de la décision et du contrôle moteur. Nous proposons également des éléments théoriques pour comprendre le rôle de la dopamine et des ganglions de la base dans la régulation de l'effort, plus particulièrement dans les pathologies telles que la maladie de ParkinsonPARIS-BIUSJ-Biologie recherche (751052107) / SudocSudocFranceF

Valence-Dependent Belief Updating: Computational Validation

People tend to update beliefs about their future outcomes in a valence-dependent way: they are likely to incorporate good news and to neglect bad news. However, belief formation is a complex process which depends not only on motivational factors such as the desire for favorable conclusions, but also on multiple cognitive variables such as prior beliefs, knowledge about personal vulnerabilities and resources, and the size of the probabilities and estimation errors. Thus, we applied computational modeling in order to test for valence-induced biases in updating while formally controlling for relevant cognitive factors. We compared biased and unbiased Bayesian models of belief updating, and specified alternative models based on reinforcement learning. The experiment consisted of 80 trials with 80 different adverse future life events. In each trial, participants estimated the base rate of one of these events and estimated their own risk of experiencing the event before and after being confronted with the actual base rate. Belief updates corresponded to the difference between the two self-risk estimates. Valence-dependent updating was assessed by comparing trials with good news (better-than-expected base rates) with trials with bad news (worse-than-expected base rates). After receiving bad relative to good news, participants' updates were smaller and deviated more strongly from rational Bayesian predictions, indicating a valence-induced bias. Model comparison revealed that the biased (i.e., optimistic) Bayesian model of belief updating better accounted for data than the unbiased (i.e., rational) Bayesian model, confirming that the valence of the new information influenced the amount of updating. Moreover, alternative computational modeling based on reinforcement learning demonstrated higher learning rates for good than for bad news, as well as a moderating role of personal knowledge. Finally, in this specific experimental context, the approach based on reinforcement learning was superior to the Bayesian approach. The computational validation of valence-dependent belief updating represents a novel support for a genuine optimism bias in human belief formation. Moreover, the precise control of relevant cognitive variables justifies the conclusion that the motivation to adopt the most favorable self-referential conclusions biases human judgments.ISSN:1664-107

Brief Report: Reduced Optimism Bias in Self-Referential Belief Updating in High-Functioning Autism

Previous research has demonstrated irrational asymmetry in belief updating: people tend to take into account good news and neglect bad news. Contradicting formal learning principles, belief updates were on average larger after better-than-expected information than after worse-than-expected information. In the present study, typically developing subjects demonstrated this optimism bias in self-referential judgments. In contrast, adults with high-functioning autism spectrum disorder (ASD) were significantly less biased when updating self-referential beliefs (each group n=21, matched for age, gender and IQ). These findings indicate a weaker influence of self-enhancing motives on prospective judgments in ASD. Reduced susceptibility to emotional and motivational biases in reasoning in ASD could elucidate impairments of social cognition, but may also confer important cognitive benefits

Simulation of Liu and Todorov [<b>29</b>]<b>.</b>

<p> <b>A</b>. Simulated trajectories for reaching movements toward a target which jumps unexpectedly up or down, 100 ms, 200 ms or 300 ms after movement onset. <b>B</b>. Corresponding velocity profiles. <b>C</b>. Arrival time as a function of the timing of the perturbation. <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1002716#s2" target="_blank">Results</a> obtained with Object IIIa. Initial arm position (deg): (15,120). Same parameters as in <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1002716#pcbi-1002716-g003" target="_blank">Fig. 3</a>.</p

Objective function and model architecture.

<p><b>A</b>. Objective function (<i>thick</i>) as a function of movement duration, built from the sum of a discounted reward term (<i>thin</i>) and a discounted effort term (<i>dashed</i>). Optimal duration is indicated by a vertical <i>dotted</i> line. <b>B</b>. Architecture of the infinite-horizontal optimal feedback controller. See <b>Text</b> for notations.</p