Computational modeling of behavioral tasks: An illustration on a classic reinforcement learning paradigm

There has been a growing interest among psychologists, psychiatrists and neuroscientists in applying computational modeling to behavioral data to understand animal and human behavior. Such approaches can be daunting for those without experience. This paper presents a step-by-step tutorial to conduct parameter estimation in R via three techniques: Maximum Likelihood Estimation (MLE), Maximum A Posteriori (MAP) and Expectation-Maximization with Laplace approximation (EML). We first demonstrate how to simulate a classic reinforcement learning paradigm -- the two-armed bandit task -- for N = 100 subjects; and then explain how to develop the computational model and implement the MLE, MAP and EML methods to recover the parameters. By presenting a sufficiently detailed walkthrough on a familiar behavioral task, we hope this tutorial could benefit readers interested in applying parameter estimation methods in their own research

Dopamine D2 receptor stimulation modulates the balance between ignoring and updating according to baseline working memory ability

BACKGROUND:Working memory (WM) deficits in neuropsychiatric disorders have often been attributed to altered dopaminergic signalling. Specifically, D2 receptor stimulation is thought to affect the ease with which items can be gated into and out of WM. In addition, this effect has been hypothesised to vary according to baseline WM ability, a putative index of dopamine synthesis levels. Moreover, whether D2 stimulation affects WM vicariously through modulating relatively WM-free cognitive control processes has not been explored. AIMS:We examined the effect of administering a dopamine agonist on the ability to ignore or update information in WM. METHOD:A single dose of cabergoline (1 mg) was administered to healthy older adult humans in a within-subject, double-blind, placebo-controlled study. In addition, we obtained measures of baseline WM ability and relatively WM-free cognitive control (overcoming response conflict). RESULTS:Consistent with predictions, baseline WM ability significantly modulated the effect that drug administration had on the proficiency of ignoring and updating. High-WM individuals were relatively better at ignoring compared to updating after drug administration. Whereas the opposite occurred in low-WM individuals. Although the ability to overcome response conflict was not affected by cabergoline, a negative relationship between the effect the drug had on response conflict performance and ignoring was observed. Thus, both response conflict and ignoring are coupled to dopaminergic stimulation levels. CONCLUSIONS:Cumulatively, these results provide evidence that dopamine affects subcomponents of cognitive control in a diverse, antagonistic fashion and that the direction of these effects is dependent upon baseline WM

Dopamine guides competition for cognitive control:Common effects of haloperidol on working memory and response conflict

Several lines of evidence suggest that dopamine modulates working memory (the ability to faithfully maintain and efficiently manipulate information over time) but its specific role has not been fully defined. Nor is it clear whether any effects of dopamine are specific to memory processes or whether they reflect more general cognitive mechanisms that extend beyond the working memory domain. Here, we examine the effect of haloperidol, principally a dopamine D2 receptor antagonist, on the ability of humans to ignore distracting information or update working memory contents. We compare these effects to performance on an independent measure of cognitive control (response conflict) which has minimal memory requirements. Haloperidol did not selectively affect the ability to ignore or update, but instead reduced the overall quality of recall. In addition, it impaired the ability to overcome response conflict. The deleterious effect of haloperidol on response conflict was selectively associated with the negative effect of the drug on ignoring - but not updating - suggesting that dopamine affects protection of working memory contents and inhibition in response conflict through a common mechanism. These findings provide new insights into the role of dopamine D2 receptors on human cognition. They suggest that D2 receptor effects on protecting the memory contents from distraction might be related to a more general process that supports inhibitory control in contexts that do not require working memory

Distinct effects of apathy and dopamine on effort-based decision making in Parkinson’s disease

Effort-based decision making is a cognitive process crucial to normal motivated behaviour. Apathy is a common and disabling complication of Parkinson’s disease, but its aetiology remains unclear. Intriguingly, the neural substrates associated with apathy also sub-serve effort-based decision making in animal models and humans. Furthermore, the dopaminergic system plays a core role in motivating effortful behaviour for reward, and its dysfunction has been proposed to play a crucial role in the aetiology of apathy in Parkinson’s disease. We hypothesised that disrupted effort-based decision making underlies the syndrome of apathy in Parkinson’s disease, and that this disruption may be modulated by the dopaminergic system. An effort-based decision making task was administered to 39 patients with Parkinson’s disease, with and without clinical apathy, ON and OFF their normal dopaminergic medications across two separate sessions, as well as 32 healthy age- and gender-matched controls. On a trial-by-trial basis participants decided whether to accept or reject offers of monetary reward in return for exerting different levels of physical effort via handheld, individually calibrated dynamometers. Effort and reward were manipulated independently, such that offers spanned the full range of effort/reward combinations. Apathy was assessed using the Lille apathy rating scale. Motor effects of the dopamine manipulation were assessed using the UPDRS part three motor score. The primary outcome variable was choice (accept/decline offer). analysed using a hierarchical generalised linear mixed effects model, and the vigour of squeeze (Newtons exerted above required force). Both apathy and dopamine depletion were associated with reduced acceptance of offers. However, these effects were driven by dissociable patterns of responding. While apathy was characterised by increased rejection of predominantly low reward offers, dopamine increased responding to high effort, high reward offers, irrespective of underlying motivational state. Dopamine also exerted a main effect on motor vigour, increasing force production independently of reward offered, while apathy did not affect this measure. The findings demonstrate that disrupted effort based decision making underlies Parkinson’s disease apathy, but in a manner distinct to that caused by dopamine depletion. Apathy is associated with reduced incentivisation by the rewarding outcomes of actions. In contrast, dopamine has a general effect in motivating behaviour for high effort, high reward options without altering the response pattern that characterises the apathetic state. Thus the motivational deficit observed in Parkinson’s disease appears not to be simply secondary to dopaminergic depletion of mesocorticolimbic pathways, suggesting non-dopaminergic therapeutic strategies for apathy may be important future targets

Dopamine modulates dynamic decision-making during foraging

The mesolimbic dopaminergic system exerts a crucial influence on incentive processing. However, the contribution of dopamine in dynamic, ecological situations where reward rates vary, and decisions evolve over time, remains unclear. In such circumstances, current (foreground) reward accrual needs to be compared continuously with potential rewards that could be obtained by travelling elsewhere (background reward rate), in order to determine the opportunity cost of staying versus leaving. We hypothesised that dopamine specifically modulates the influence of background – but not foreground – reward information when making a dynamic comparison of these variables for optimal behaviour. On a novel foraging task based on an ecological account of animal behaviour (marginal value theorem), human participants of either sex decided when to leave locations in situations where foreground rewards depleted at different rates, either in rich or poor environments with high or low background rates. In line with theoretical accounts, people’s decisions to move from current locations were independently modulated by changes in both foreground and background reward rates. Pharmacological manipulation of dopamine D2 receptor activity using the agonist cabergoline significantly affected decisions to move on, specifically modulating the effect of background reward rates. In particular, when on cabergoline, people left patches in poor environments much earlier. These results demonstrate a role of dopamine in signalling the opportunity cost of rewards, not value per se. Using this ecologically derived framework we uncover a specific mechanism by which D2 dopamine receptor activity modulates decision-making when foreground and background reward rates are dynamically compared

An exploratory research : the factors affecting the choice of industry by entrepreneurs setting up small and medium enterprises (SMEs).

The purpose of this research is to find out factors affecting the choice of industry by local entrepreneurs setting up SMEs and their approaches to strategic planning before and after setting up a venture. The topics touched and scrutinised by local academics had mainly been on the SMEs' roles, their contributions to the Singapore economy and the problems faced by them. As it had been a particular focal point of the Singapore Government in developing a second wing (an external economy), it would be interesting to study the behavioural aspects of entrepreneurs in venturing into new businesses and the reasons affecting their choices.BUSINES

Individual differences in empathy are associated with apathy-motivation

Abstract Empathy - the capacity to understand and resonate with the experiences of other people - is considered an essential aspect of social cognition. However, although empathy is often thought to be automatic, recent theories have argued that there is a key role for motivation in modulating empathic experiences. Here we administered self-report measures of empathy and apathy-motivation to a large sample of healthy people (n = 378) to test whether people who are more empathic are also more motivated. We then sought to replicate our findings in an independent sample (n = 198) that also completed a behavioural task to measure state affective empathy and emotion recognition. Cognitive empathy was associated with higher levels of motivation generally across behavioural, social and emotional domains. In contrast, affective empathy was associated with lower levels of behavioural motivation, but higher levels of emotional motivation. Factor analyses showed that empathy and apathy are distinct constructs, but that affective empathy and emotional motivation are underpinned by the same latent factor. These results have potentially important clinical applications for disorders associated with reduced empathy and motivation as well as the understanding of these processes in healthy people

Momentary gustative-olfactory sensitivity and tonic heart rate variability are independently associated with motivational behavior.

Deficits in motivational functioning including impairments in reward learning or reward sensitivity are common in psychiatric disorders characterized by anhedonia. Recently, anhedonic symptoms have been exacerbated by the pandemic caused by the Coronavirus disease 2019 (COVID-19) in the general population. The present study examined the putative associations between loss of smell (anosmia) and taste (ageusia) sensitivity, irrespective of COVID-19 infection, and anhedonia, measured by a signal-detection task probing the ability to modify behavior as a function of rewards (Probabilistic Reward Task; PRT). Tonic heart rate variability (HRV) was included in the model, due to its association with both smell and taste sensitivity as well as motivational functioning. The sample included 114 healthy individuals (81 females; mean age 22.2 years), who underwent a laboratory session in which dispositional traits, resting HRV and PRT performance were assessed, followed by a 4-days ecological momentary assessment to obtain daily measures of anosmia and ageusia. Lower levels of tonic HRV and lower momentary levels of smell and taste sensitivity were associated with impaired reward responsiveness and ability to shape future behavioral choices based on prior reinforcement experiences. Overall, the current results provide initial correlational evidence that could be fruitfully used to inform future experimental investigations aimed at elucidating the disruptive worldwide mental health consequences triggered by the pandemic

Covid-19 consequences on motivational functioning: an ecological and laboratory study

Deficits in motivational functioning, including impairments in reward learning or reward sensitivity, are common in psychiatric disorders characterized by anhedonia, the inability to experience pleasure in previously rewarding activities and stimuli. Anhedonic symptoms have been exacerbated by Coronavirus disease 2019 (COVID-19) pandemic in the general population. The present study examined whether consequences of COVID-19 such as anosmia and ageusia (loss of smell and taste sensitivity, respectively), could predict anhedonia assessed by the Probabilistic Reward Task (PRT), a well-validated signal-detection task operationalizing participants’ reward processing. Tonic heart rate variability (HRV) was also assessed as a potential predictor of anhedonic behavior. 114 healthy individuals (81 females) underwent a laboratory session in which dispositional traits, resting HRV and performance on the PRT were assessed, followed by a 4-days ecological momentary assessment of daily anosmia and ageusia. Lower resting HRV and lower momentary levels of smell and taste sensitivity independently predicted impaired performance on the PRT. The present study provides insight into the general exacerbation of mental health issues worldwide due to COVID-19 pandemic

Momentary gustative-olfactory sensitivity and tonic heart rate variability are independently associated with motivational behavior

: Deficits in motivational functioning including impairments in reward learning or reward sensitivity are common in psychiatric disorders characterized by anhedonia. Recently, anhedonic symptoms have been exacerbated by the pandemic caused by the Coronavirus disease 2019 (COVID-19) in the general population. The present study examined the putative associations between loss of smell (anosmia) and taste (ageusia) sensitivity, irrespective of COVID-19 infection, and anhedonia, measured by a signal-detection task probing the ability to modify behavior as a function of rewards (Probabilistic Reward Task; PRT). Tonic heart rate variability (HRV) was included in the model, due to its association with both smell and taste sensitivity as well as motivational functioning. The sample included 114 healthy individuals (81 females; mean age 22.2 years), who underwent a laboratory session in which dispositional traits, resting HRV and PRT performance were assessed, followed by a 4-days ecological momentary assessment to obtain daily measures of anosmia and ageusia. Lower levels of tonic HRV and lower momentary levels of smell and taste sensitivity were associated with impaired reward responsiveness and ability to shape future behavioral choices based on prior reinforcement experiences. Overall, the current results provide initial correlational evidence that could be fruitfully used to inform future experimental investigations aimed at elucidating the disruptive worldwide mental health consequences triggered by the pandemic