Nonhuman gamblers: lessons from rodents, primates, and robots

The search for neuronal and psychological underpinnings of pathological gambling in humans would benefit from investigating related phenomena also outside of our species. In this paper, we present a survey of studies in three widely different populations of agents, namely rodents, non-human primates, and robots. Each of these populations offers valuable and complementary insights on the topic, as the literature demonstrates. In addition, we highlight the deep and complex connections between relevant results across these different areas of research (i.e., cognitive and computational neuroscience, neuroethology, cognitive primatology, neuropsychiatry, evolutionary robotics), to make the case for a greater degree of methodological integration in future studies on pathological gambling

Toward a multiscale modeling framework for understanding serotonergic function

Despite its importance in regulating emotion and mental wellbeing, the complex structure and function of the serotonergic system present formidable challenges toward understanding its mechanisms. In this paper, we review studies investigating the interactions between serotonergic and related brain systems and their behavior at multiple scales, with a focus on biologically-based computational modeling. We first discuss serotonergic intracellular signaling and neuronal excitability, followed by neuronal circuit and systems levels. At each level of organization, we will discuss the experimental work accompanied by related computational modeling work. We then suggest that a multiscale modeling approach that integrates the various levels of neurobiological organization could potentially transform the way we understand the complex functions associated with serotonin

Modeling Neuromodulation as a Framework to Integrate Uncertainty in General Cognitive Architectures

International audienceOne of the most critical properties of a versatile intelligent agent is its capacity to adapt autonomously to any change in the environment without overly complexifying its cognitive architecture. In this paper, we propose that understanding the role of neuromodulation in the brain is of central interest for this purpose. More precisely, we propose that an accurate estimation of the nature of uncertainty present in the environment is performed by specific brain regions and broadcast throughout the cerebral network by neuromodulators, resulting in appropriate changes in cerebral functioning and learning modes. Better understanding the principles of these mechanisms in the brain might tremendously inspire the field of Artificial General Intelligence. The original contribution of this paper is to relate the four major neuromodulators to four fundamental dimensions of uncertainty

Elucidating the neural correlates of cost-benefit decisions in a rat cortico-basal ganglia network

Adaptive value-guided decision-making often requires weighing-up of costs and benefits of pursuing an available opportunity. Several brain areas, particularly in frontal- striatal circuits, have been reported to be important for this behaviour. However, little is currently known regarding how the decision variables are represented and evolve on the single cell and population levels across these circuits within the confines of a single behavioural paradigm. We developed a novel rodent cost-benefit “accept-reject” task, in which food-restricted rats choose whether or not to run to the end of a corridor to collect sucrose pellets based on the prospective reward (4 different levels, varied trial- by-trial, signalled by auditory cues) and effort cost (3 different levels, varied over blocks of trials, implemented with barriers placed in the corridor that needed to be scaled to reach the reward magazine). Behavioural data (n=12 rats, average 1119 trials per animal) demonstrate a positive effect of reward and a negative influence of effort on the likelihood of accepting an offer, without an interaction between the two. Outcome devaluation strongly reduces acceptance of any offer. A subset of rats (n=5) were also implanted with a bespoke driveable micro-electrode array targeting, anterior cingulate cortex (ACC), medial orbitofrontal cortex (mOFC), dorsomedial striatum (DMS), ventral pallidum (VP), and subthalamic area (STA) and electrophysiological recordings were collected as they performed the task. Analysis of spiking data (n=4 rats; average 119 cells per region per animal) suggests that individual neurons in the 3 targeted basal ganglia areas (DMS, VP and STA) encode reward and/or, to a lesser extent, effort with high fidelity during cue presentation, while neurons in the two cortical areas (ACC and mOFC) display weaker tuning to reward/effort across multiple task points. By contrast, decision is progressively signalled by mOFC, DMS and VP units after action initiation. Decoding analyses using all the units from individual areas in a given session (ensembles) revealed that reward is signalled with similar spatial and temporal characteristics to single units. By contrast, effort is represented by DMS ensembles in a time-distributed manner and is dependent on diverse underlying neuronal activity profiles. These results are consistent with a parallel and distributed system for processing cost-benefit decision variables in the frontal cortical-basal ganglia network

An investigation into serotonergic and environmental interventions against depression in a simulated delayed reward paradigm

The disruption of the serotonergic (5HT) system has been implicated in causing major depression and the standard view is that a lack of serotonin is to blame for the resulting symptoms. Consequently, pharmacological interventions aim to increase serotonin concentration in its target areas or stimulating excitatory 5HT receptors. A standard approach is to use serotonin reuptake inhibitors (SSRIs) which cause a higher accumulation of serotonin. Another approach is to stimulate excitatory serotonin receptors with psychedelic drugs. This article compares these two approaches by first setting up a system-level limbic system model of the relevant brain areas and then modelling a delayed reward paradigm which is known to be disrupted by a lack of 5HT. Central to our model is how serotonin changes the response characteristics of decision-making neurons where low levels of 5HT allow small signals to pass through, whereas high levels of 5HT create a barrier for smaller signals but amplifying the larger ones. We show with both standard behavioural simulations and model checking that SSRIs perform significantly better against interventions with psychedelics. However, psychedelics might work better in other paradigms where a high level of exploration is beneficial to obtain rewards

Representations of Reward and Movement in Drosophila Dopaminergic Neurons

The neuromodulator dopamine is known to influence both immediate and future behavior, motivating and invigorating an animal’s ongoing movement but also serving as a reinforcement signal to instruct learning. Yet it remains unclear whether this dual role of dopamine involves the same dopaminergic pathways. Although reward-responsive dopaminergic neurons display movement-related activity, debate continues as to what features of an individual’s experience these motor-correlates correspond and how they influence concurrent behavior. The mushroom body, a prominent neuropil in the brain of the fruit fly Drosophila melanogaster, is richly innervated by dopaminergic neurons that play an essential role in the formation of olfactory associations. While dopaminergic neurons respond to reward and punishment to drive associative learning, they have also been implicated in a number of adaptive behaviors and their activity correlates with the behavioral state of an animal and its coarse motor actions. Here, we take advantage of the concise circuit architecture of the Drosophila mushroom body to investigate the nature of motor-related signals in dopaminergic neurons that drive associative learning. In vivo functional imaging during naturalistic tethered locomotion reveals that the activity of different subsets of mushroom body dopaminergic neurons reflects distinct aspects of movement. To gain insight into what facets of an animal’s experience are represented by these movement-related signals, we employed a closed loop virtual reality paradigm to monitor neural activity as animals track an olfactory stimulus and are actively engaged in a goal-directed and sensory-motivated behavior. We discover that odor responses in dopaminergic neurons correlate with the extent to which an animal tracks upwind towards the fictive odor source. In different experimental contexts where distinct motor actions were required to track the odor, dopaminergic neurons become emergently linked to the behavioral metric most relevant for effective olfactory navigation. Subsets of dopaminergic neurons were correlated with the strength of upwind tracking regardless of the identity of the odor and remained so even after the satiety state of an animal was altered. We proceed to demonstrate that transient inhibition of dopaminergic neurons that are positively correlated with upwind tracking significantly diminishes the normal approach responses to an appetitive olfactory cue. Accordingly, activation of those same dopaminergic neurons enhances approach to an odor and even drives upwind tracking in clean air alone. Together, these results reveal that the same dopaminergic pathways that convey reinforcements to instruct learning also carry representations of an animal’s moment-by-moment movements and actively influence behavior. The complex activity patterns of mushroom body dopaminergic neurons therefore represent neither purely sensory nor motor variables but rather reflect the goal or motivation underlying an animal’s movements. Our data suggest a fundamental coupling between reinforcement signals and motivation-related locomotor representations within dopaminergic circuitry, drawing a striking parallel between the mushroom body dopaminergic neurons described here and the emerging understanding of mammalian dopaminergic pathways. The apparent conservation in dopaminergic neuromodulatory networks between mammals and insects suggests a shared logic for how neural circuits assign meaning to both sensory stimuli and motor actions to generate flexible and adaptive behavior

Motivation in motor and cognitive control: Effects of dopamine and monetary reward and penalty

Dopamine has been identified as a key player in reward signalling and motivational processes and has been linked to apathy in Parkinson’s disease (PD), its hallmark being dopamine depletion. Direct characterisation of how dopamine modulates reward sensitivity especially in the presence of aversive stimuli is, however, still a matter of controversy. Saccadic eye movements have long been considered reward insensitive due to their high level of stereotypy, but in recent years have been recognised as a precise tool to study motor and cognitive control processes and measure reward sensitivity. This thesis investigates how oculomotor properties are influenced by different dopamine levels and motivation through both reward anticipation and penalty avoidance. Thereby I seek to shed light on the underlying pathomechanisms responsible for motor and non-motor symptoms in diseases characterized by dopamine depletion (e.g., PD). Data from the first experimental chapter suggest a common “net-value” for both incentive valences and confirms similar effects of both incentives on saccadic properties in healthy participants. The second part investigates the role of dopamine in signalling incentive values, which indicates a similar role of dopamine in both rewarding and aversive incentives. Both drugs (haloperidol and levodopa) decreased motor vigour, while having different effects on preparatory and inhibitory processes, which ultimately led to antagonistic effects on precision. Most intriguingly we also found increased reward sensitivity after a single dose of levodopa independent of incentive valence. As some of these effects might reflect motor effects of dopamine, I next examined the high-level cognitive effects using a visual working memory task. This was assessed in health as well as in a cohort of patients who had undergone VTA DBS surgery. No effect of Madopar or motivation was found on working memory in a tablet-based task, while haloperidol was detrimental to memory precision. DBS stimulation in the VTA improved performance potentially by increasing dopamine levels in the mesocorticolimbic pathway. In conclusion, this thesis aims provide a comprehensive picture of the role of nigrostriatal as well as mesolimibic dopamine on motor and cognitive control potentially aiding early diagnosis and optimising treatment strategies in disease

CNTRICS Final Task Selection: Long-term Memory

Long-term memory (LTM) is a multifactorial construct, composed of different stages of information processing and different cognitive operations that are mediated by distinct neural systems, some of which may be more responsible for the marked memory problems that limit the daily function of individuals with schizophrenia. From the outset of the CNTRICS initiative, this multidimensionality was appreciated, and an effort was made to identify the specific memory constructs and task paradigms that hold the most promise for immediate translational development. During the second CNTRICS meeting, the LTM group identified item encoding and retrieval and relational encoding and retrieval as key constructs. This article describes the process that the LTM group went through in the third and final CNTRICS meeting to select nominated tasks within the 2 LTM constructs and within a reinforcement learning construct that were judged most promising for immediate development. This discussion is followed by each nominating authors' description of their selected task paradigm, ending with some thoughts about future directions.Psycholog

Suicide in Parkinson’s Disease: The Role of Dopamine Deficiency

openParkinson’s disease is the second most common neurodegenerative disease markedly lowering the quality of life. Suicidal ideation is elevated in Parkinson’s disease, and affected patients may be at a higher risk for suicide, especially after Deep Brain Stimulation (DBS). Lower quality of life does not show to be enough to explain this increased risk for suicidality. A number of major risk factors for suicide overlap with some of the most common non-motor symptoms preceding or accompanying Parkinson’s disease. This thesis explores these overlapping factors - notably depression, impulsivity, problems with executive function, and an unbalanced HPA axis - and demonstrates that dopamine is an essential part of the mechanisms in each of the aforementioned factors. Thus, dopamine plays a crucial role in suicide in Parkinson’s disease, and possibly among other populations as well.Parkinson’s disease is the second most common neurodegenerative disease markedly lowering the quality of life. Suicidal ideation is elevated in Parkinson’s disease, and affected patients may be at a higher risk for suicide, especially after Deep Brain Stimulation (DBS). Lower quality of life does not show to be enough to explain this increased risk for suicidality. A number of major risk factors for suicide overlap with some of the most common non-motor symptoms preceding or accompanying Parkinson’s disease. This thesis explores these overlapping factors - notably depression, impulsivity, problems with executive function, and an unbalanced HPA axis - and demonstrates that dopamine is an essential part of the mechanisms in each of the aforementioned factors. Thus, dopamine plays a crucial role in suicide in Parkinson’s disease, and possibly among other populations as well