2,760 research outputs found
Task complexity interacts with state-space uncertainty in the arbitration between model-based and model-free learning
It has previously been shown that the relative reliability of model-based and model-free reinforcement-learning (RL) systems plays a role in the allocation of behavioral control between them. However, the role of task complexity in the arbitration between these two strategies remains largely unknown. Here, using a combination of novel task design, computational modelling, and model-based fMRI analysis, we examined the role of task complexity alongside state-space uncertainty in the arbitration process. Participants tended to increase model-based RL control in response to increasing task complexity. However, they resorted to model-free RL when both uncertainty and task complexity were high, suggesting that these two variables interact during the arbitration process. Computational fMRI revealed that task complexity interacts with neural representations of the reliability of the two systems in the inferior prefrontal cortex
Neurocognitive basis of model-based decision making and its metacontrol in childhood
Human behavior is supported by both goal-directed (model-based) and habitual (model-free) decision-making, each differing in its flexibility, accuracy, and computational cost. The arbitration between habitual and goal-directed systems is thought to be regulated by a process known as metacontrol. However, how these systems emerge and develop remains poorly understood. Recently, we found that while children between 5 and 11 years displayed robust signatures of model-based decision-making, which increased during this developmental period, there were substantial individual differences in the display of metacontrol. Here, we inspect the neurocognitive basis of model-based decision-making and metacontrol in childhood and focus this investigation on executive functions, fluid reasoning, and brain structure. A total of 69 participants between the ages of 6-13 completed a two-step decision-making task and an extensive behavioral test battery. A subset of 44 participants also completed a structural magnetic resonance imaging scan. We find that individual differences in metacontrol are specifically associated with performance on an inhibition task and individual differences in thickness of dorsolateral prefrontal, temporal, and superior-parietal cortices. These brain regions likely reflect the involvement of cognitive processes crucial to metacontrol, such as cognitive control and contextual processing
Building machines that learn and think about morality
Lake et al. propose three criteria which, they argue, will bring artificial intelligence (AI) systems closer to human cognitive abilities. In this paper, we explore the application of these criteria to a particular domain of human cognition: our capacity for moral reasoning. In doing so, we explore a set of considerations relevant to the development of AI moral decision-making. Our main focus is on the relation between dual-process accounts of moral reasoning and model-free/model-based forms of machine learning. We also discuss how work in embodied and situated cognition could provide a valu- able perspective on future research
Habits and goals in synergy: a variational Bayesian framework for behavior
How to behave efficiently and flexibly is a central problem for understanding
biological agents and creating intelligent embodied AI. It has been well known
that behavior can be classified as two types: reward-maximizing habitual
behavior, which is fast while inflexible; and goal-directed behavior, which is
flexible while slow. Conventionally, habitual and goal-directed behaviors are
considered handled by two distinct systems in the brain. Here, we propose to
bridge the gap between the two behaviors, drawing on the principles of
variational Bayesian theory. We incorporate both behaviors in one framework by
introducing a Bayesian latent variable called "intention". The habitual
behavior is generated by using prior distribution of intention, which is
goal-less; and the goal-directed behavior is generated by the posterior
distribution of intention, which is conditioned on the goal. Building on this
idea, we present a novel Bayesian framework for modeling behaviors. Our
proposed framework enables skill sharing between the two kinds of behaviors,
and by leveraging the idea of predictive coding, it enables an agent to
seamlessly generalize from habitual to goal-directed behavior without requiring
additional training. The proposed framework suggests a fresh perspective for
cognitive science and embodied AI, highlighting the potential for greater
integration between habitual and goal-directed behaviors
Neurostimulation reveals context-dependent arbitration between model-based and model-free reinforcement learning
While it is established that humans use model-based (MB) and model-free (MF) reinforcement learning in a complementary fashion, much less is known about how the brain determines which of these systems should control behavior at any given moment. Here we provide causal evidence for a neural mechanism that acts as a context-dependent arbitrator between both systems. We applied excitatory and inhibitory transcranial direct current stimulation over a region of the left ventrolateral prefrontal cortex previously found to encode the reliability of both learning systems. The opposing neural interventions resulted in a bidirectional shift of control between MB and MF learning. Stimulation also affected the sensitivity of the arbitration mechanism itself, as it changed how often subjects switched between the dominant system over time. Both of these effects depended on varying task contexts that either favored MB or MF control, indicating that this arbitration mechanism is not context-invariant but flexibly incorporates information about current environmental demands
Recommended from our members
Active Inference, homeostatic regulation and adaptive behavioural control
We review a theory of homeostatic regulation and adaptive behavioural control within the Active Inference framework. Our aim is to connect two research streams that are usually considered independently; namely, Active Inference and associative learning theories of animal behaviour. The former uses a probabilistic (Bayesian) formulation of perception and action, while the latter calls on multiple (Pavlovian, habitual, goal-directed) processes for homeostatic and behavioural control. We offer a synthesis these classical processes and cast them as successive hierarchical contextualisations of sensorimotor constructs, using the generative models that underpin Active Inference. This dissolves any apparent mechanistic distinction between the optimization processes that mediate classical control or learning. Furthermore, we generalize the scope of Active Inference by emphasizing interoceptive inference and homeostatic regulation. The ensuing homeostatic (or allostatic) perspective provides an intuitive explanation for how priors act as drives or goals to enslave action, and emphasises the embodied nature of inference
- …