Bandit Models of Human Behavior: Reward Processing in Mental Disorders

Drawing an inspiration from behavioral studies of human decision making, we propose here a general parametric framework for multi-armed bandit problem, which extends the standard Thompson Sampling approach to incorporate reward processing biases associated with several neurological and psychiatric conditions, including Parkinson's and Alzheimer's diseases, attention-deficit/hyperactivity disorder (ADHD), addiction, and chronic pain. We demonstrate empirically that the proposed parametric approach can often outperform the baseline Thompson Sampling on a variety of datasets. Moreover, from the behavioral modeling perspective, our parametric framework can be viewed as a first step towards a unifying computational model capturing reward processing abnormalities across multiple mental conditions.Comment: Conference on Artificial General Intelligence, AGI-1

Neurocognitive Predictors of ADHD Outcome:A 6-Year Follow-up Study

Contains fulltext : 169764.pdf (publisher's version ) (Open Access)Although a broad array of neurocognitive dysfunctions are associated with ADHD, it is unknown whether these dysfunctions play a role in the course of ADHD symptoms. The present longitudinal study investigated whether neurocognitive functions assessed at study-entry (mean age = 11.5 years, SD = 2.7) predicted ADHD symptom severity and overall functioning 6 years later (mean age = 17.4 years, 82.6 % = male) in a carefully phenotyped large sample of 226 Caucasian participants from 182 families diagnosed with ADHD-combined type. Outcome measures were dimensional measures of ADHD symptom severity and the Kiddie-Global Assessment Scale (K-GAS) for overall functioning. Predictors were derived from component scores for 8 domains of neurocognitive functioning: working memory, motor inhibition, cognitive inhibition, reaction time variability, timing, information processing speed, motor control, intelligence. Effects of age, gender, and pharmacological treatment were considered. Results showed that better working memory predicted lower ADHD symptom severity (R 2 = 3.0 %), and less reaction time variability predicted better overall functioning (higher K-GAS-score, R 2 = 5.6 %). Predictors were still significant with baseline behavior included in the models. The role of neurocognitive functioning in the long term outcome of ADHD behavior is discussed

Can Motivation Normalize Working Memory and Task Persistence in Children with Attention-Deficit/Hyperactivity Disorder? The Effects of Money and Computer-Gaming

Visual-spatial Working Memory (WM) is the most impaired executive function in children with Attention-Deficit/Hyperactivity Disorder (ADHD). Some suggest that deficits in executive functioning are caused by motivational deficits. However, there are no studies that investigate the effects of motivation on the visual-spatial WM of children with- and without ADHD. Studies examining this in executive functions other than WM, show inconsistent results. These inconsistencies may be related to differences in the reinforcement used. The effects of different reinforcers on WM performance were investigated in 30 children with ADHD and 31 non-ADHD controls. A visual-spatial WM task was administered in four reinforcement conditions: Feedback-only, 1 euro, 10 euros, and a computer-game version of the task. In the Feedback-only condition, children with ADHD performed worse on the WM measure than controls. Although incentives significantly improved the WM performance of children with ADHD, even the strongest incentives (10 euros and Gaming) were unable to normalize their performance. Feedback-only provided sufficient reinforcement for controls to reach optimal performance, while children with ADHD required extra reinforcement. Only children with ADHD showed a decrease in performance over time. Importantly, the strongest incentives (10 euros and Gaming) normalized persistence of performance in these children, whereas 1 euro had no such effect. Both executive and motivational deficits give rise to visual-spatial WM deficits in ADHD. Problems with task-persistence in ADHD result from motivational deficits. In ADHD-reinforcement studies and clinical practice (e.g., assessment), reinforcement intensity can be a confounding factor and should be taken into account. Gaming can be a cost-effective way to maximize performance in ADHD

Reward and Punishment Sensitivity in Children with ADHD: Validating the Sensitivity to Punishment and Sensitivity to Reward Questionnaire for Children (SPSRQ-C)

This study validates the Sensitivity to Punishment and Sensitivity to Reward Questionnaire for children (SPSRQ-C), using a Dutch sample of 1234 children between 6–13 years old. Factor analysis determined that a 4-factor and a 5-factor solution were best fitting, explaining 41% and 50% of the variance respectively. The 4-factor model was highly similar to the original SPSRQ factors found in adults (Punishment Sensitivity, Reward Responsivity, Impulsivity/Fun-Seeking, and Drive). The 5-factor model was similar to the 4-factor model, with the exception of a subdivision of the Punishment Sensitivity factor into a factor with ‘social-fear’ items and a factor with ‘anxiety’ items. To determine external validity, scores of three groups of children with attention deficit hyperactivity disorder (ADHD) were compared on the EFA models: ADHD-only (n = 34), ADHD and autism spectrum disorder (ADHD+ASD; n = 22), ADHD and oppositional defiant disorder (ADHD+ODD; n = 22). All ADHD groups scored higher than typical controls on Reward Responsivity and on the ‘anxiety’ factor (n = 75). The ADHD-only and ADHD+ODD group scored higher than other groups on Impulsivity/Fun-Seeking and Drive, while the ADHD+ASD group scored higher on Punishment Sensitivity. The findings emphasize the value of the SPSRQ-C to quickly and reliably assess a child’s sensitivity to reinforcement, with the aim to provide individually-tailored behavioral interventions that utilize reward and reprimands

Predictors of gastrointestinal lesions on endoscopy in iron deficiency anemia without gastrointestinal symptoms

<p>Abstract</p> <p>Background</p> <p>Iron deficiency anaemia (IDA) due to occult gastrointestinal (GI) blood loss usually remains unnoticed until patient become symptomatic. There is sparse data in IDA patients without gastrointestinal symptoms. This study was designed to find out the frequency and predictors of endoscopic lesions in IDA without gastrointestinal symptoms. Cross-sectional study performed on a convenience sample of consecutive subjects.</p> <p>Methods</p> <p>Ninety five consecutive patients with laboratory based diagnosis of IDA having no gastrointestinal symptoms were interviewed and their clinical and biochemical variables were recorded. All the study patients underwent esophago-gastroduodenoscopy (EGD) and colonoscopy. Endoscopic findings were documented as presence/absence of bleeding related lesion and presence/absence of cause of IDA. Multiple logistic regressions were performed to identify variables significantly related to outcome variables.</p> <p>Results</p> <p>Possible cause of anaemia was found in 71% and bleeding related lesions were found in 53% of patients. Upper gastrointestinal tract lesions were found in 41% of patients with bleeding related lesions. On multivariable logistic regression; advancing age, low mean corpuscular volume (MCV ≤ 60 fl), and positive fecal occult blood test were predictive factors for bleeding related GI lesions and cause of IDA</p> <p>Conclusion</p> <p>Clinical and Biochemical markers can predict gastrointestinal lesions on endoscopy in IDA patients without gastrointestinal symptoms. High proportion of upper gastrointestinal involvement warrants EGD as initial endoscopic procedure however, this needs validation by further studies.</p

Inhibition, Reinforcement Sensitivity and Temporal Information Processing in ADHD and ADHD+ODD: Evidence of a Separate Entity?

This study compared children with ADHD-only, ADHD+ODD and normal controls (age 8–12) on three key neurocognitive functions: response inhibition, reinforcement sensitivity, and temporal information processing. The goal was twofold: (a) to investigate neurocognitive impairments in children with ADHD-only and children with ADHD+ODD, and (b) to test whether ADHD+ODD is a more severe from of ADHD in terms of neurocognitive performance. In Experiment 1, inhibition abilities were measured using the Stop Task. In Experiment 2, reinforcement sensitivity and temporal information processing abilities were measured using a Timing Task with both a reward and penalty condition. Compared to controls, children with ADHD-only demonstrated impaired inhibitory control, showed more time underestimations, and showed performance deterioration in the face of reward and penalty. Children with ADHD+ODD performed in-between children with ADHD-only and controls in terms of inhibitory controls and the tendency to underestimate time, but were more impaired than controls and children with ADHD-only in terms of timing variability. In the face of reward and penalty children with ADHD+ODD improved their performance compared to a neutral condition, in contrast to children with ADHD-only. In the face of reward, the performance improvement in the ADHD+ODD group was disproportionally larger than that of controls. Taken together the findings suggest that, in terms of neurocognitive functioning, comorbid ADHD+ODD is a substantial different entity than ADHD-only