Computational modeling of behavioral tasks: An illustration on a classic reinforcement learning paradigm

There has been a growing interest among psychologists, psychiatrists and neuroscientists in applying computational modeling to behavioral data to understand animal and human behavior. Such approaches can be daunting for those without experience. This paper presents a step-by-step tutorial to conduct parameter estimation in R via three techniques: Maximum Likelihood Estimation (MLE), Maximum A Posteriori (MAP) and Expectation-Maximization with Laplace approximation (EML). We first demonstrate how to simulate a classic reinforcement learning paradigm -- the two-armed bandit task -- for N = 100 subjects; and then explain how to develop the computational model and implement the MLE, MAP and EML methods to recover the parameters. By presenting a sufficiently detailed walkthrough on a familiar behavioral task, we hope this tutorial could benefit readers interested in applying parameter estimation methods in their own research

Dopamine D2 receptor stimulation modulates the balance between ignoring and updating according to baseline working memory ability

BACKGROUND:Working memory (WM) deficits in neuropsychiatric disorders have often been attributed to altered dopaminergic signalling. Specifically, D2 receptor stimulation is thought to affect the ease with which items can be gated into and out of WM. In addition, this effect has been hypothesised to vary according to baseline WM ability, a putative index of dopamine synthesis levels. Moreover, whether D2 stimulation affects WM vicariously through modulating relatively WM-free cognitive control processes has not been explored. AIMS:We examined the effect of administering a dopamine agonist on the ability to ignore or update information in WM. METHOD:A single dose of cabergoline (1 mg) was administered to healthy older adult humans in a within-subject, double-blind, placebo-controlled study. In addition, we obtained measures of baseline WM ability and relatively WM-free cognitive control (overcoming response conflict). RESULTS:Consistent with predictions, baseline WM ability significantly modulated the effect that drug administration had on the proficiency of ignoring and updating. High-WM individuals were relatively better at ignoring compared to updating after drug administration. Whereas the opposite occurred in low-WM individuals. Although the ability to overcome response conflict was not affected by cabergoline, a negative relationship between the effect the drug had on response conflict performance and ignoring was observed. Thus, both response conflict and ignoring are coupled to dopaminergic stimulation levels. CONCLUSIONS:Cumulatively, these results provide evidence that dopamine affects subcomponents of cognitive control in a diverse, antagonistic fashion and that the direction of these effects is dependent upon baseline WM

Dopamine guides competition for cognitive control:Common effects of haloperidol on working memory and response conflict

Several lines of evidence suggest that dopamine modulates working memory (the ability to faithfully maintain and efficiently manipulate information over time) but its specific role has not been fully defined. Nor is it clear whether any effects of dopamine are specific to memory processes or whether they reflect more general cognitive mechanisms that extend beyond the working memory domain. Here, we examine the effect of haloperidol, principally a dopamine D2 receptor antagonist, on the ability of humans to ignore distracting information or update working memory contents. We compare these effects to performance on an independent measure of cognitive control (response conflict) which has minimal memory requirements. Haloperidol did not selectively affect the ability to ignore or update, but instead reduced the overall quality of recall. In addition, it impaired the ability to overcome response conflict. The deleterious effect of haloperidol on response conflict was selectively associated with the negative effect of the drug on ignoring - but not updating - suggesting that dopamine affects protection of working memory contents and inhibition in response conflict through a common mechanism. These findings provide new insights into the role of dopamine D2 receptors on human cognition. They suggest that D2 receptor effects on protecting the memory contents from distraction might be related to a more general process that supports inhibitory control in contexts that do not require working memory

Dopamine modulates dynamic decision-making during foraging

The mesolimbic dopaminergic system exerts a crucial influence on incentive processing. However, the contribution of dopamine in dynamic, ecological situations where reward rates vary, and decisions evolve over time, remains unclear. In such circumstances, current (foreground) reward accrual needs to be compared continuously with potential rewards that could be obtained by travelling elsewhere (background reward rate), in order to determine the opportunity cost of staying versus leaving. We hypothesised that dopamine specifically modulates the influence of background – but not foreground – reward information when making a dynamic comparison of these variables for optimal behaviour. On a novel foraging task based on an ecological account of animal behaviour (marginal value theorem), human participants of either sex decided when to leave locations in situations where foreground rewards depleted at different rates, either in rich or poor environments with high or low background rates. In line with theoretical accounts, people’s decisions to move from current locations were independently modulated by changes in both foreground and background reward rates. Pharmacological manipulation of dopamine D2 receptor activity using the agonist cabergoline significantly affected decisions to move on, specifically modulating the effect of background reward rates. In particular, when on cabergoline, people left patches in poor environments much earlier. These results demonstrate a role of dopamine in signalling the opportunity cost of rewards, not value per se. Using this ecologically derived framework we uncover a specific mechanism by which D2 dopamine receptor activity modulates decision-making when foreground and background reward rates are dynamically compared

An exploratory research : the factors affecting the choice of industry by entrepreneurs setting up small and medium enterprises (SMEs).

The purpose of this research is to find out factors affecting the choice of industry by local entrepreneurs setting up SMEs and their approaches to strategic planning before and after setting up a venture. The topics touched and scrutinised by local academics had mainly been on the SMEs' roles, their contributions to the Singapore economy and the problems faced by them. As it had been a particular focal point of the Singapore Government in developing a second wing (an external economy), it would be interesting to study the behavioural aspects of entrepreneurs in venturing into new businesses and the reasons affecting their choices.BUSINES

Individual differences in empathy are associated with apathy-motivation

Abstract Empathy - the capacity to understand and resonate with the experiences of other people - is considered an essential aspect of social cognition. However, although empathy is often thought to be automatic, recent theories have argued that there is a key role for motivation in modulating empathic experiences. Here we administered self-report measures of empathy and apathy-motivation to a large sample of healthy people (n = 378) to test whether people who are more empathic are also more motivated. We then sought to replicate our findings in an independent sample (n = 198) that also completed a behavioural task to measure state affective empathy and emotion recognition. Cognitive empathy was associated with higher levels of motivation generally across behavioural, social and emotional domains. In contrast, affective empathy was associated with lower levels of behavioural motivation, but higher levels of emotional motivation. Factor analyses showed that empathy and apathy are distinct constructs, but that affective empathy and emotional motivation are underpinned by the same latent factor. These results have potentially important clinical applications for disorders associated with reduced empathy and motivation as well as the understanding of these processes in healthy people

Momentary gustative-olfactory sensitivity and tonic heart rate variability are independently associated with motivational behavior.

Deficits in motivational functioning including impairments in reward learning or reward sensitivity are common in psychiatric disorders characterized by anhedonia. Recently, anhedonic symptoms have been exacerbated by the pandemic caused by the Coronavirus disease 2019 (COVID-19) in the general population. The present study examined the putative associations between loss of smell (anosmia) and taste (ageusia) sensitivity, irrespective of COVID-19 infection, and anhedonia, measured by a signal-detection task probing the ability to modify behavior as a function of rewards (Probabilistic Reward Task; PRT). Tonic heart rate variability (HRV) was included in the model, due to its association with both smell and taste sensitivity as well as motivational functioning. The sample included 114 healthy individuals (81 females; mean age 22.2 years), who underwent a laboratory session in which dispositional traits, resting HRV and PRT performance were assessed, followed by a 4-days ecological momentary assessment to obtain daily measures of anosmia and ageusia. Lower levels of tonic HRV and lower momentary levels of smell and taste sensitivity were associated with impaired reward responsiveness and ability to shape future behavioral choices based on prior reinforcement experiences. Overall, the current results provide initial correlational evidence that could be fruitfully used to inform future experimental investigations aimed at elucidating the disruptive worldwide mental health consequences triggered by the pandemic

Covid-19 consequences on motivational functioning: an ecological and laboratory study

Deficits in motivational functioning, including impairments in reward learning or reward sensitivity, are common in psychiatric disorders characterized by anhedonia, the inability to experience pleasure in previously rewarding activities and stimuli. Anhedonic symptoms have been exacerbated by Coronavirus disease 2019 (COVID-19) pandemic in the general population. The present study examined whether consequences of COVID-19 such as anosmia and ageusia (loss of smell and taste sensitivity, respectively), could predict anhedonia assessed by the Probabilistic Reward Task (PRT), a well-validated signal-detection task operationalizing participants’ reward processing. Tonic heart rate variability (HRV) was also assessed as a potential predictor of anhedonic behavior. 114 healthy individuals (81 females) underwent a laboratory session in which dispositional traits, resting HRV and performance on the PRT were assessed, followed by a 4-days ecological momentary assessment of daily anosmia and ageusia. Lower resting HRV and lower momentary levels of smell and taste sensitivity independently predicted impaired performance on the PRT. The present study provides insight into the general exacerbation of mental health issues worldwide due to COVID-19 pandemic

Distinct Subtypes of Apathy Revealed by the Apathy Motivation Index

<div><p>Apathy is a debilitating but poorly understood disorder characterized by a reduction in motivation. As well as being associated with several brain disorders, apathy is also prevalent in varying degrees in healthy people. Whilst many tools have been developed to assess levels of apathy in clinical disorders, surprisingly there are no measures of apathy suitable for healthy people. Moreover, although apathy is commonly comorbid with symptoms of depression, anhedonia and fatigue, how and why these symptoms are associated is unclear. Here we developed the Apathy-Motivation Index (AMI), a brief self-report index of apathy and motivation. Using exploratory factor analysis (in a sample of 505 people), and then confirmatory analysis (in a different set of 479 individuals), we identified subtypes of apathy in <i>behavioural</i>, <i>social</i> and <i>emotional</i> domains. Latent profile analyses showed four different profiles of apathy that were associated with varying levels of depression, anhedonia and fatigue. The AMI is a novel and reliable measure of individual differences in apathy and might provide a useful means of probing different mechanisms underlying sub-clinical lack of motivation in otherwise healthy individuals. Moreover, associations between apathy and comorbid states may be reflective of problems in different emotional, social and behavioural domains.</p></div

Distribution of apathy along the AMI subscales and conditional response means of the 4-class solution.

<p>The AMI consists of three subscales: Behavioural Activation, Social Motivation and Emotional Sensitivity. Every subscale contains 6 items that is each scored from 0–4, with a higher mean score indicating greater apathy. <b>(A)</b> 3D scatterplot illustrating the distribution of each healthy individual’s mean rating along the three AMI subscales. The four classes were labelled generally motivated (orange), behaviourally/socially apathetic (green), emotionally apathetic (blue), and generally apathetic (red). <b>(B)</b> Conditional response mean value greater than overall sample means (black line) indicates apathy on that AMI subscale.</p