Observing the Observer (II): Deciding When to Decide

In a companion paper [1], we have presented a generic approach for inferring how subjects make optimal decisions under uncertainty. From a Bayesian decision theoretic perspective, uncertain representations correspond to “posterior” beliefs, which result from integrating (sensory) information with subjective “prior” beliefs. Preferences and goals are encoded through a “loss” (or “utility”) function, which measures the cost incurred by making any admissible decision for any given (hidden or unknown) state of the world. By assuming that subjects make optimal decisions on the basis of updated (posterior) beliefs and utility (loss) functions, one can evaluate the likelihood of observed behaviour. In this paper, we describe a concrete implementation of this meta-Bayesian approach (i.e. a Bayesian treatment of Bayesian decision theoretic predictions) and demonstrate its utility by applying it to both simulated and empirical reaction time data from an associative learning task. Here, inter-trial variability in reaction times is modelled as reflecting the dynamics of the subjects' internal recognition process, i.e. the updating of representations (posterior densities) of hidden states over trials while subjects learn probabilistic audio-visual associations. We use this paradigm to demonstrate that our meta-Bayesian framework allows for (i) probabilistic inference on the dynamics of the subject's representation of environmental states, and for (ii) model selection to disambiguate between alternative preferences (loss functions) human subjects could employ when dealing with trade-offs, such as between speed and accuracy. Finally, we illustrate how our approach can be used to quantify subjective beliefs and preferences that underlie inter-individual differences in behaviour

Observing the Observer (I): Meta-Bayesian Models of Learning and Decision-Making

In this paper, we present a generic approach that can be used to infer how subjects make optimal decisions under uncertainty. This approach induces a distinction between a subject's perceptual model, which underlies the representation of a hidden "state of affairs" and a response model, which predicts the ensuing behavioural (or neurophysiological) responses to those inputs. We start with the premise that subjects continuously update a probabilistic representation of the causes of their sensory inputs to optimise their behaviour. In addition, subjects have preferences or goals that guide decisions about actions given the above uncertain representation of these hidden causes or state of affairs. From a Bayesian decision theoretic perspective, uncertain representations are so-called "posterior" beliefs, which are influenced by subjective "prior" beliefs. Preferences and goals are encoded through a "loss" (or "utility") function, which measures the cost incurred by making any admissible decision for any given (hidden) state of affair. By assuming that subjects make optimal decisions on the basis of updated (posterior) beliefs and utility (loss) functions, one can evaluate the likelihood of observed behaviour. Critically, this enables one to "observe the observer", i.e. identify (context-or subject-dependent) prior beliefs and utility-functions using psychophysical or neurophysiological measures. In this paper, we describe the main theoretical components of this meta-Bayesian approach (i.e. a Bayesian treatment of Bayesian decision theoretic predictions). In a companion paper ('Observing the observer (II): deciding when to decide'), we describe a concrete implementation of it and demonstrate its utility by applying it to simulated and real reaction time data from an associative learning task

Rejoinder to "'Time for a new curriculum for social statistics?' A negative answer."

SummaryDominant individuals report high levels of self-sufficiency, self-esteem, and authoritarianism. The lay stereotype suggests that such individuals ignore information from others, preferring to make their own choices. However, the nonhuman animal literature presents a conflicting view, suggesting that dominant individuals are avid social learners, whereas subordinates focus on learning from private experience. Whether dominant humans are best characterized by the lay stereotype or the animal view is currently unknown. Here, we present a “social dominance paradox”: using self-report scales and computerized tasks, we demonstrate that socially dominant people explicitly value independence, but, paradoxically, in a complex decision-making task, they show an enhanced reliance (relative to subordinate individuals) on social learning. More specifically, socially dominant people employed a strategy of copying other agents when the agents’ responses had a history of being correct. However, in humans, two subtypes of dominance have been identified [1]: aggressive and social. Aggressively dominant individuals, who are as likely to “get their own way” as socially dominant individuals but who do so through the use of aggressive or Machiavellian tactics, did not use social information, even when it was beneficial to do so. This paper presents the first study of dominance and social learning in humans and challenges the lay stereotype in which all dominant individuals ignore others’ views [2]. The more subtle perspective we offer could have important implications for decision making in both the boardroom and the classroom

Aversive Pavlovian control of instrumental behavior in humans

Adaptive behavior involves interactions between systems regulating Pavlovian and instrumental control of actions. Here, we present the first investigation of the neural mechanisms underlying aversive Pavlovian-instrumental transfer using fMRI in humans. Recent evidence indicates that these Pavlovian influences on instrumental actions are action-specific: Instrumental approach is invigorated by appetitive Pavlovian cues but inhibited by aversive Pavlovian cues. Conversely, instrumental withdrawal is inhibited by appetitive Pavlovian cues but invigorated by aversive Pavlovian cues. We show that BOLD responses in the amygdala and the nucleus accumbens were associated with behavioral inhibition by aversive Pavlovian cues, irrespective of action context. Furthermore, BOLD responses in the ventromedial prefrontal cortex differed between approach and withdrawal actions. Aversive Pavlovian conditioned stimuli modulated connectivity between the ventromedial prefrontal cortex and the caudate nucleus. These results show that action-specific aversive control of instrumental behavior involves the modulation of fronto-striatal interactions by Pavlovian conditioned stimuli

Report The Social Dominance Paradox

Summary Dominant individuals report high levels of self-sufficiency, self-esteem, and authoritarianism. The lay stereotype suggests that such individuals ignore information from others, preferring to make their own choices. However, the nonhuman animal literature presents a conflicting view, suggesting that dominant individuals are avid social learners, whereas subordinates focus on learning from private experience. Whether dominant humans are best characterized by the lay stereotype or the animal view is currently unknown. Here, we present a ''social dominance paradox'': using selfreport scales and computerized tasks, we demonstrate that socially dominant people explicitly value independence, but, paradoxically, in a complex decision-making task, they show an enhanced reliance (relative to subordinate individuals) on social learning. More specifically, socially dominant people employed a strategy of copying other agents when the agents' responses had a history of being correct. However, in humans, two subtypes of dominance have been identified [1]: aggressive and social. Aggressively dominant individuals, who are as likely to ''get their own way'' as socially dominant individuals but who do so through the use of aggressive or Machiavellian tactics, did not use social information, even when it was beneficial to do so. This paper presents the first study of dominance and social learning in humans and challenges the lay stereotype in which all dominant individuals ignore others' views Results and Discussion In experiment 1, adult participants (n = 33; age mean = 27.88, SEM = 1.39; 19 males, 14 females; In the decision-making task, participants scored points by using individually experienced (outcome history) and/or social Multiple regression models applied at the group level showed that SD (t(32) = 2.08, p = 0.048, standardized b [stdb] = 0.39) was a significant positive predictor of the social beta values: the higher a participant scored in SD, the more they used the social information, as estimated by the social learner model, to make their choices

A hemodynamic model for layered BOLD signals

High-resolution blood oxygen level dependent (BOLD) functional magnetic resonance imaging (fMRI) at the sub-millimeter scale has become feasible with recent advances in MR technology. In principle, this would enable the study of layered cortical circuits, one of the fundaments of cortical computation. However, the spatial layout of cortical blood supply may become an important confound at such high resolution. In particular, venous blood draining back to the cortical surface perpendicularly to the layered structure is expected to influence the measured responses in different layers. Here, we present an extension of a hemodynamic model commonly used for analyzing fMRI data (in dynamic causal models or biophysical network models) that accounts for such blood draining effects by coupling local hemodynamics across layers. We illustrate the properties of the model and its inversion by a series of simulations and show that it successfully captures layered fMRI data obtained during a simple visual experiment. We conclude that for future studies of the dynamics of layered neuronal circuits with high-resolution fMRI, it will be pivotal to include effects of blood draining, particularly when trying to infer on the layer-specific connections in cortex--a theme of key relevance for brain disorders like schizophrenia and for theories of brain function such as predictive coding

Catecholaminergic modulation of meta-learning

The remarkable expedience of human learning is thought to be underpinned by meta-learning, whereby slow accumulative learning processes are rapidly adjusted to the current learning environment. To date, the neurobiological implementation of meta-learning remains unclear. A burgeoning literature argues for an important role for the catecholamines dopamine and noradrenaline in meta-learning. Here, we tested the hypothesis that enhancing catecholamine function modulates the ability to optimise a meta-learning parameter (learning rate) as a function of environmental volatility. 102 participants completed a task which required learning in stable phases, where the probability of reinforcement was constant, and volatile phases, where probabilities changed every 10–30 trials. The catecholamine transporter blocker methylphenidate enhanced participants’ ability to adapt learning rate: Under methylphenidate, compared with placebo, participants exhibited higher learning rates in volatile relative to stable phases. Furthermore, this effect was significant only with respect to direct learning based on the participants’ own experience, there was no significant effect on inferred-value learning where stimulus values had to be inferred. These data demonstrate a causal link between catecholaminergic modulation and the adjustment of the meta-learning parameter learning rate