Catecholaminergic modulation of meta-learning

The remarkable expedience of human learning is thought to be underpinned by meta-learning, whereby slow accumulative learning processes are rapidly adjusted to the current learning environment. To date, the neurobiological implementation of meta-learning remains unclear. A burgeoning literature argues for an important role for the catecholamines dopamine and noradrenaline in meta-learning. Here, we tested the hypothesis that enhancing catecholamine function modulates the ability to optimise a meta-learning parameter (learning rate) as a function of environmental volatility. 102 participants completed a task which required learning in stable phases, where the probability of reinforcement was constant, and volatile phases, where probabilities changed every 10–30 trials. The catecholamine transporter blocker methylphenidate enhanced participants’ ability to adapt learning rate: Under methylphenidate, compared with placebo, participants exhibited higher learning rates in volatile relative to stable phases. Furthermore, this effect was significant only with respect to direct learning based on the participants’ own experience, there was no significant effect on inferred-value learning where stimulus values had to be inferred. These data demonstrate a causal link between catecholaminergic modulation and the adjustment of the meta-learning parameter learning rate

Catecholamine Challenge Uncovers Distinct Mechanisms for Direct Versus Indirect, but Not Social Versus Non-Social, Learning

ABSTRACTEvidence that social and individual learning are at least partially dissociable sustains the belief that humans possess adaptive specialisations for social learning. However, in most extant paradigms, social information comprises an indirect source that can be used to supplement one’s own, direct, experience. Thus, social and individual learning differ both in terms of social nature (social versus non-social) and directness (indirect versus direct). To test whether the dissociation between social and individual learning is best explained in terms of social nature or directness, we used a catecholaminergic challenge known to modulate learning. Two groups completed a decision-making task which required direct learning, from own experience, and indirect learning from an additional source. The groups differed in terms of whether the indirect source was social or non-social. The catecholamine transporter blocker, methylphenidate, affected direct learning by improving adaptation to changes in the volatility of the environment but there was no effect of methylphenidate on learning from the social or non-social indirect source. Thus, we report positive evidence for a dissociable effect of methylphenidate on direct and indirect learning, but no evidence for a distinction between social and non-social. These data fail to support the adaptive specialisation view, instead providing evidence for distinct mechanisms for direct versus indirect learning.</jats:p

Effects of methylphenidate on reinforcement learning depend on working memory capacity

RATIONALE: Brain catecholamines have long been implicated in reinforcement learning, exemplified by catecholamine drug and genetic effects on probabilistic reversal learning. However, the mechanisms underlying such effects are unclear. OBJECTIVES AND METHODS: Here we investigated effects of an acute catecholamine challenge with methylphenidate (20 mg, oral) on a novel probabilistic reversal learning paradigm in a within-subject, double-blind randomised design. The paradigm was designed to disentangle effects on punishment avoidance from effects on reward perseveration. Given the known large individual variability in methylphenidate’s effects, we stratified our effects by working memory capacity and trait impulsivity, putatively modulating the effects of methylphenidate, in a large sample (n = 102) of healthy volunteers. RESULTS: Contrary to our prediction, methylphenidate did not alter performance in the reversal phase of the task. Our key finding is that methylphenidate altered learning of choice-outcome contingencies in a manner that depended on individual variability in working memory span. Specifically, methylphenidate improved performance by adaptively reducing the effective learning rate in participants with higher working memory capacity. CONCLUSIONS: This finding emphasises the important role of working memory in reinforcement learning, as reported in influential recent computational modelling and behavioural work, and highlights the dependence of this interplay on catecholaminergic function. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s00213-021-05974-w