2,405 research outputs found
Monte Carlo Planning method estimates planning horizons during interactive social exchange
Reciprocating interactions represent a central feature of all human exchanges. They have been the target of various recent experiments, with healthy participants and psychiatric populations engaging as dyads in multi-round exchanges such as a repeated trust task. Behaviour in such exchanges involves complexities related to each agent's preference for equity with their partner, beliefs about the partner's appetite for equity, beliefs about the partner's model of their partner, and so on. Agents may also plan different numbers of steps into the future. Providing a computationally precise account of the behaviour is an essential step towards understanding what underlies choices. A natural framework for this is that of an interactive partially observable Markov decision process (IPOMDP). However, the various complexities make IPOMDPs inordinately computationally challenging. Here, we show how to approximate the solution for the multi-round trust task using a variant of the Monte-Carlo tree search algorithm. We demonstrate that the algorithm is efficient and effective, and therefore can be used to invert observations of behavioural choices. We use generated behaviour to elucidate the richness and sophistication of interactive inference
Monte Carlo Planning method estimates planning horizons during interactive social exchange
Reciprocating interactions represent a central feature of all human
exchanges. They have been the target of various recent experiments, with
healthy participants and psychiatric populations engaging as dyads in
multi-round exchanges such as a repeated trust task. Behaviour in such
exchanges involves complexities related to each agent's preference for equity
with their partner, beliefs about the partner's appetite for equity, beliefs
about the partner's model of their partner, and so on. Agents may also plan
different numbers of steps into the future. Providing a computationally precise
account of the behaviour is an essential step towards understanding what
underlies choices. A natural framework for this is that of an interactive
partially observable Markov decision process (IPOMDP). However, the various
complexities make IPOMDPs inordinately computationally challenging. Here, we
show how to approximate the solution for the multi-round trust task using a
variant of the Monte-Carlo tree search algorithm. We demonstrate that the
algorithm is efficient and effective, and therefore can be used to invert
observations of behavioural choices. We use generated behaviour to elucidate
the richness and sophistication of interactive inference
Neural signature of fictive learning signals in a sequential investment task
Reinforcement learning models now provide principled guides for a wide range of reward learning experiments in animals and humans. One key learning (error) signal in these models is experiential and reports ongoing temporal differences between expected and experienced reward. However, these same abstract learning models also accommodate the existence of another class of learning signal that takes the form of a fictive error encoding ongoing differences between experienced returns and returns that "could-have-been-experienced" if decisions had been different. These observations suggest the hypothesis that, for all real-world learning tasks, one should expect the presence of both experiential and fictive learning signals. Motivated by this possibility, we used a sequential investment game and fMRI to probe ongoing brain responses to both experiential and fictive learning signals generated throughout the game. Using a large cohort of subjects (n = 54), we report that fictive learning signals strongly predict changes in subjects' investment behavior and correlate with fMRI signals measured in dopaminoceptive structures known to be involved in valuation and choice
- …