Better than expected: the influence of option expectations during decision-making

Our choices often arise from a consideration of options presented in a sequence (e.g. the products in a supermarket row). However, whether the precise sequential order of option presentation affects decision-making remains poorly understood. A recent model of choice proposes that, in a set of options presented sequentially, those that are better than expected will be perceived as more valuable, even when options are objectively equivalent within the set. Inspired by this proposal, we devised a novel decision-making task where we manipulated the order of option presentation together with expectations about option value. Even when we compared trials that were exactly equivalent except for option order, we observed a striking preference for options that were better than expected. Our findings show that expectations about options affect which option will be favoured within a sequence, an influence which is manifested as a preference for better-than-expected options. The findings have potential practical implications, as for example they may help policymakers in devising nudge strategies that rely on ad hoc option orders

Reference effects on decision-making elicited by previous rewards

Substantial evidence has highlighted reference effects occurring during decision-making, whereby subjective value is not calculated in absolute terms but relative to the distribution of rewards characterizing a context. Among these, within-choice effects are exerted by options simultaneously available during choice. These should be distinguished from between-choice effects, which depend on the distribution of options presented in the past. Influential theories on between-choice effects include Decision-by-Sampling, Expectation-as-Reference and Divisive Normalization. Surprisingly, previous literature has focused on each theory individually disregarding the others. Thus, similarities and differences among theories remain to be systematically examined. Here we fill this gap by offering an overview of the state-of-the-art of research about between-choice reference effects. Our comparison of alternative theories shows that, at present, none of them is able to account for the full range of empirical data. To address this, we propose a model inspired by previous perspectives and based on a logistic framework, hence called logistic model of subjective value. Predictions of the model are analysed in detail about reference effects and risky decision-making. We conclude that our proposal offers a compelling framework for interpreting the multifaceted manifestations of between-choice reference effects

Learning Contextual Reward Expectations for Value Adaptation

Substantial evidence indicates that subjective value is adapted to the statistics of reward expected within a given temporal context. However, how these contextual expectations are learned is poorly understood. To examine such learning, we exploited a recent observation that participants performing a gambling task adjust their preferences as a function of context. We show that, in the absence of contextual cues providing reward information, an average reward expectation was learned from recent past experience. Learning dependent on contextual cues emerged when two contexts alternated at a fast rate, whereas both cue-independent and cue-dependent forms of learning were apparent when two contexts alternated at a slower rate. Motivated by these behavioral findings, we reanalyzed a previous fMRI data set to probe the neural substrates of learning contextual reward expectations. We observed a form of reward prediction error related to average reward such that, at option presentation, activity in ventral tegmental area/substantia nigra and ventral striatum correlated positively and negatively, respectively, with the actual and predicted value of options. Moreover, an inverse correlation between activity in ventral tegmental area/substantia nigra (but not striatum) and predicted option value was greater in participants showing enhanced choice adaptation to context. The findings help understanding the mechanisms underlying learning of contextual reward expectation

The value of novelty in schizophrenia

Influential models of schizophrenia suggest that patients experience incoming stimuli as excessively novel and motivating, with important consequences for hallucinatory experience and delusional belief. However, whether schizophrenia patients exhibit excessive novelty value and whether this interferes with adaptive behaviour has not yet been formally tested. Here, we employed a three-armed bandit task to investigate this hypothesis. Schizophrenia patients and healthy controls were first familiarised with a group of images and then asked to repeatedly choose between familiar and unfamiliar images associated with different monetary reward probabilities. By fitting a reinforcement-learning model we were able to estimate the values attributed to familiar and unfamiliar images when first presented in the context of the decision-making task. In line with our hypothesis, we found increased preference for newly introduced images (irrespective of whether these were familiar or unfamiliar) in patients compared to healthy controls and this to correlate with severity of hallucinatory experience. In addition, we found a correlation between value assigned to novel images and task performance, suggesting that excessive novelty value may interfere with optimal learning in patients, putatively through the disruption of the mechanisms regulating exploration versus exploitation. Our results suggest excessive novelty value in patients, whereby even previously seen stimuli acquire higher value as the result of their exposure in a novel context – a form of ‘hyper novelty’ which may explain why patients are often attracted by familiar stimuli experienced as new

The Mixed Instrumental Controller: Using Value of Information to Combine Habitual Choice and Mental Simulation

Instrumental behavior depends on both goal-directed and habitual mechanisms of choice. Normative views cast these mechanisms in terms of model-free and model-based methods of reinforcement learning, respectively. An influential proposal hypothesizes that model-free and model-based mechanisms coexist and compete in the brain according to their relative uncertainty. In this paper we propose a novel view in which a single Mixed Instrumental Controller produces both goal-directed and habitual behavior by flexibly balancing and combining model-based and model-free computations. The Mixed Instrumental Controller performs a cost-benefits analysis to decide whether to chose an action immediately based on the available “cached” value of actions (linked to model-free mechanisms) or to improve value estimation by mentally simulating the expected outcome values (linked to model-based mechanisms). Since mental simulation entails cognitive effort and increases the reward delay, it is activated only when the associated “Value of Information” exceeds its costs. The model proposes a method to compute the Value of Information, based on the uncertainty of action values and on the distance of alternative cached action values. Overall, the model by default chooses on the basis of lighter model-free estimates, and integrates them with costly model-based predictions only when useful. Mental simulation uses a sampling method to produce reward expectancies, which are used to update the cached value of one or more actions; in turn, this updated value is used for the choice. The key predictions of the model are tested in different settings of a double T-maze scenario. Results are discussed in relation with neurobiological evidence on the hippocampus – ventral striatum circuit in rodents, which has been linked to goal-directed spatial navigation

Value encoding in the globus pallidus: fMRI reveals an interaction effect between reward and dopamine drive

The external part of the globus pallidus (GPe) is a core nucleus of the basal ganglia (BG) whose activity is disrupted under conditions of low dopamine release, as in Parkinson's disease. Current models assume decreased dopamine release in the dorsal striatum results in deactivation of dorsal GPe, which in turn affects motor expression via a regulatory effect on other nuclei of the BG. However, recent studies in healthy and pathological animal models have reported neural dynamics that do not match with this view of the GPe as a relay in the BG circuit. Thus, the computational role of the GPe in the BG is still to be determined. We previously proposed a neural model that revisits the functions of the nuclei of the BG, and this model predicts that GPe encodes values which are amplified under a condition of low striatal dopaminergic drive. To test this prediction, we used an fMRI paradigm involving a within-subject placebo-controlled design, using the dopamine antagonist risperidone, wherein healthy volunteers performed a motor selection and maintenance task under low and high reward conditions. ROI-based fMRI analysis revealed an interaction between reward and dopamine drive manipulations, with increased BOLD activity in GPe in a high compared to low reward condition, and under risperidone compared to placebo. These results confirm the core prediction of our computational model, and provide a new perspective on neural dynamics in the BG and their effects on motor selection and cognitive disorders

The role of expecting feedback during decision-making under risk

Sometimes choice is followed by outcome feedback and other times it is not. It remains unknown whether humans prefer gambling when they expect feedback to be revealed. Regarding this question, decision-making theories make alternative predictions. Some theories have proposed that choice is influenced by whether one expects to be disappointed in the future. Given that feedback is sometimes disappointing, these theories predict increased aversion towards gambling when feedback is expected compared to when feedback is not expected. The opposite effect is predicted by theories of curiosity, which postulate reduction of uncertainty as an important behavioural drive. Given that feedback reduces uncertainty, these theories predict that gambling will be favoured when feedback is expected. To examine whether expecting feedback influences gambling behaviour, we recorded functional neuroimaging data while participants performed a novel decision-making task requiring to chose between a sure option and a gamble. Crucially, participants expected to receive feedback in some trials but not in other trials. Consistent with theories of curiosity, we found that expecting feedback increased gambling propensity. At the neural level, at option presentation the increased value of gambling during feedback was reflected in activity in the ventral striatum. This suggests that, together with its established role in signalling reward, the ventral striatum also processes a form of epistemic value. Our study demonstrates that gambling becomes more attractive when feedback is expected and suggests that striatal activity could signal the value of feedback information

The Link Between COVID-19, Anxiety, and Religious Beliefs in the United States and the United Kingdom

Research has shown that stress impacts on people’s religious beliefs. However, several aspects of this effect remain poorly understood, for example regarding the role of prior religiosity and stress-induced anxiety. This paper explores these aspects in the context of the recent coronavirus pandemic (COVID-19). The latter has impacted dramatically on many people’s well-being; hence it can be considered a highly stressful event. Through online questionnaires administered to UK (n = 140) and USA (n = 140) citizens professing either Christian faith or no religion, this paper examines the impact of the coronavirus crisis upon common people’s religious beliefs. Anxiety about the coronavirus and prior religiosity showed an interaction effect upon change in religious beliefs (t(276) = 2.27, p = .024): for strong believers higher anxiety about coronavirus was associated with increased strengthening of religious beliefs (r = .249), while for non-believers higher anxiety about coronavirus was associated with increased scepticism towards religious beliefs (r = − .157). These observations are consistent with the notion that stress-induced anxiety enhances support for an individual’s existing ideology already embraced before a stressful event occurs. This study sheds light on the psychological and cultural implications of the coronavirus crisis, which represents one of the most serious health emergencies in recent times

Political motivation: A referent evaluation mathematical model

Mathematical modelling is popular in cognitive psychology because it enables clear and formal descriptions of the processes at play; yet, this approach has rarely been applied to political psychology. Here we adopt mathematical modelling to develop a theory of political motivation, which is a central concept in political psychology. The theory assumes that, in certain contexts, individuals entertain a set of representations of society, for example of the past, present and future (but also of fictive societies such as utopias). To each representation of society, an incentive value is attached which is not absolute, but (following theories of motivation in cognitive psychology) reference-dependent; namely, dependent on the context, corresponding to the whole set of representations of society. In turn, the model proposes that these subjective values determine two central aspects: a motivation for performing an appropriate political action and the ensuing political mood. We discuss the model with respect to theoretical and empirical research (and we examine Marx and Engel’s communist manifesto as an example of the latter). In short, we offer a new mathematical perspective on political motivation which emphasises the role of multiple representations of society in determining political motivation and the ensuing political mood

Prisoner of the present: Borderline personality and a tendency to overweight cues during Bayesian inference

Recent work has examined the computational mechanisms underlying Borderline Personality (BP). However, this research has been confined to specific tasks. A computational analysis of BP’s mental processes as they broadly unfold in everyday life is lacking. Here a computational model of BP is proposed which describes patients’ everyday-life mental experience at large. Grounded on Bayesian inference, the proposal is that BP sufferers attribute excessive weight to cues considered to infer life contexts (e.g., to infer whether a cooperation or competition context is ongoing). Remarkably, model simulations demonstrate that this idea accounts for several characteristics of BP, from extreme oscillations in identity, affect, and behaviour, to dysfunctional interpersonal cycles. Altogether, the paper offers a framework to interpret the broad, everyday life computational mechanisms underlying BP. This can inspire theoretical and empirical research and can help understanding how clinical interventions for BP work, thus contributing to refine such interventions