A Goal-Directed Bayesian Framework for Categorization

Categorization is a fundamental ability for efficient behavioral control. It allows organisms to remember the correct responses to categorical cues and not for every stimulus encountered (hence eluding computational cost or complexity), and to generalize appropriate responses to novel stimuli dependant on category assignment. Assuming the brain performs Bayesian inference, based on a generative model of the external world and future goals, we propose a computational model of categorization in which important properties emerge. These properties comprise the ability to infer latent causes of sensory experience, a hierarchical organization of latent causes, and an explicit inclusion of context and action representations. Crucially, these aspects derive from considering the environmental statistics that are relevant to achieve goals, and from the fundamental Bayesian principle that any generative model should be preferred over alternative models based on an accuracy-complexity trade-off. Our account is a step toward elucidating computational principles of categorization and its role within the Bayesian brain hypothesis

Backwards is the way forward: feedback in the cortical hierarchy predicts the expected future

Clark offers a powerful description of the brain as a prediction machine, which offers progress on two distinct levels. First, on an abstract conceptual level, it provides a unifying framework for perception, action, and cognition (including subdivisions such as attention, expectation, and imagination). Second, hierarchical prediction offers progress on a concrete descriptive level for testing and constraining conceptual elements and mechanisms of predictive coding models (estimation of predictions, prediction errors, and internal models)

Better than expected: the influence of option expectations during decision-making

Our choices often arise from a consideration of options presented in a sequence (e.g. the products in a supermarket row). However, whether the precise sequential order of option presentation affects decision-making remains poorly understood. A recent model of choice proposes that, in a set of options presented sequentially, those that are better than expected will be perceived as more valuable, even when options are objectively equivalent within the set. Inspired by this proposal, we devised a novel decision-making task where we manipulated the order of option presentation together with expectations about option value. Even when we compared trials that were exactly equivalent except for option order, we observed a striking preference for options that were better than expected. Our findings show that expectations about options affect which option will be favoured within a sequence, an influence which is manifested as a preference for better-than-expected options. The findings have potential practical implications, as for example they may help policymakers in devising nudge strategies that rely on ad hoc option orders

A Bayesian model of context-sensitive value attribution

Substantial evidence indicates that incentive value depends on an anticipation of rewards within a given context. However, the computations underlying this context sensitivity remain unknown. To address this question, we introduce a normative (Bayesian) account of how rewards map to incentive values. This assumes that the brain inverts a model of how rewards are generated. Key features of our account include (i) an influence of prior beliefs about the context in which rewards are delivered (weighted by their reliability in a Bayes-optimal fashion), (ii) the notion that incentive values correspond to precision-weighted prediction errors, (iii) and contextual information unfolding at different hierarchical levels. This formulation implies that incentive value is intrinsically context-dependent. We provide empirical support for this model by showing that incentive value is influenced by context variability and by hierarchically nested contexts. The perspective we introduce generates new empirical predictions that might help explaining psychopathologies, such as addiction

Putting bandits into context: How function learning supports decision making

The authors introduce the contextual multi-armed bandit task as a framework to investigate learning and decision making in uncertain environments. In this novel paradigm, participants repeatedly choose between multiple options in order to maximize their rewards. The options are described by a number of contextual features which are predictive of the rewards through initially unknown functions. From their experience with choosing options and observing the consequences of their decisions, participants can learn about the functional relation between contexts and rewards and improve their decision strategy over time. In three experiments, the authors explore participants’ behavior in such learning environments. They predict participants’ behavior by context-blind (mean-tracking, Kalman filter) and contextual (Gaussian process and linear regression) learning approaches combined with different choice strategies. Participants are mostly able to learn about the context-reward functions and their behavior is best described by a Gaussian process learning strategy which generalizes previous experience to similar instances. In a relatively simple task with binary features, they seem to combine this learning with a probability of improvement decision strategy which focuses on alternatives that are expected to lead to an improvement upon a current favorite option. In a task with continuous features that are linearly related to the rewards, participants seem to more explicitly balance exploration and exploitation. Finally, in a difficult learning environment where the relation between features and rewards is nonlinear, some participants are again well-described by a Gaussian process learning strategy, whereas others revert to context-blind strategies

Decision by sampling

We present a theory of decision by sampling (DbS) in which, in contrast with traditional models, there are no underlying psychoeconomic scales. Instead, we assume that an attribute’s subjective value is constructed from a series of binary, ordinal comparisons to a sample of attribute values drawn from memory and is its rank within the sample. We assume that the sample reflects both the immediate distribution of attribute values from the current decision’s context and also the background, real-world distribution of attribute values. DbS accounts for concave utility functions; losses looming larger than gains; hyperbolic temporal discounting; and the overestimation of small probabilities and the underestimation of large probabilities

Reference effects on decision-making elicited by previous rewards

Substantial evidence has highlighted reference effects occurring during decision-making, whereby subjective value is not calculated in absolute terms but relative to the distribution of rewards characterizing a context. Among these, within-choice effects are exerted by options simultaneously available during choice. These should be distinguished from between-choice effects, which depend on the distribution of options presented in the past. Influential theories on between-choice effects include Decision-by-Sampling, Expectation-as-Reference and Divisive Normalization. Surprisingly, previous literature has focused on each theory individually disregarding the others. Thus, similarities and differences among theories remain to be systematically examined. Here we fill this gap by offering an overview of the state-of-the-art of research about between-choice reference effects. Our comparison of alternative theories shows that, at present, none of them is able to account for the full range of empirical data. To address this, we propose a model inspired by previous perspectives and based on a logistic framework, hence called logistic model of subjective value. Predictions of the model are analysed in detail about reference effects and risky decision-making. We conclude that our proposal offers a compelling framework for interpreting the multifaceted manifestations of between-choice reference effects