Sufficient reliability of the behavioral and computational readouts of a probabilistic reversal learning task

Task-based measures that capture neurocognitive processes can help bridge the gap between brain and behavior. To transfer tasks to clinical application, reliability is a crucial benchmark because it imposes an upper bound to potential correlations with other variables (e.g., symptom or brain data). However, the reliability of many task readouts is low. In this study, we scrutinized the retest reliability of a probabilistic reversal learning task (PRLT) that is frequently used to characterize cognitive flexibility in psychiatric populations. We analyzed data from N = 40 healthy subjects, who completed the PRLT twice. We focused on how individual metrics are derived, i.e., whether data were partially pooled across participants and whether priors were used to inform estimates. We compared the reliability of the resulting indices across sessions, as well as the internal consistency of a selection of indices. We found good to excellent reliability for behavioral indices as derived from mixed-effects models that included data from both sessions. The internal consistency was good to excellent. For indices derived from computational modeling, we found excellent reliability when using hierarchical estimation with empirical priors and including data from both sessions. Our results indicate that the PRLT is well equipped to measure individual differences in cognitive flexibility in reinforcement learning. However, this depends heavily on hierarchical modeling of the longitudinal data (whether sessions are modeled separately or jointly), on estimation methods, and on the combination of parameters included in computational models. We discuss implications for the applicability of PRLT indices in psychiatric research and as diagnostic tools

Reliance on model-based and model-free control in obesity

Consuming more energy than is expended may reflect a failure of control over eating behaviour in obesity. Behavioural control arises from a balance between two dissociable strategies of reinforcement learning: model-free and model-based. We hypothesized that weight status relates to an imbalance in reliance on model-based and model-free control, and that it may do so in a linear or quadratic manner. To test this, 90 healthy participants in a wide BMI range (normal-weight (n=31), overweight (n=29), obese (n=30)) performed a sequential decision-making task. The primary analysis indicated that obese participants relied less on model-based control than overweight and normal-weight participants, with no difference between overweight and normal-weight participants. In line, secondary continuous analyses revealed a negative linear, but not quadratic, relationship between BMI and model-based control. Computational modelling of choice behaviour suggested that a mixture of both strategies was shifted towards less model-based control in obese participants. Furthermore, exploratory analyses of separate weights for model-free and model-based control showed stronger reliance on model-free control with increased BMI. Our findings suggest that obesity may indeed be related to an imbalance in behavioural control as expressed in a phenotype of less model-based control potentially resulting from enhanced reliance on model-free computations

Striatal dopamine and reward prediction error signaling in unmedicated schizophrenia patients

Increased striatal dopamine synthesis capacity has consistently been reported in patients with schizophrenia. However, the mechanism translating this into behavior and symptoms remains unclear. It has been proposed that heightened striatal dopamine may blunt dopaminergic reward prediction error signaling during reinforcement learning. In this study, we investigated striatal dopamine synthesis capacity, reward prediction errors, and their association in unmedicated schizophrenia patients (n = 19) and healthy controls (n = 23). They took part in FDOPA-PET and underwent functional magnetic resonance imaging (fMRI) scanning, where they performed a reversal-learning paradigm. The groups were compared regarding dopamine synthesis capacity (Kicer), fMRI neural prediction error signals, and the correlation of both. Patients did not differ from controls with respect to striatal Kicer. Taking into account, comorbid alcohol abuse revealed that patients without such abuse showed elevated Kicer in the associative striatum, while those with abuse did not differ from controls. Comparing all patients to controls, patients performed worse during reversal learning and displayed reduced prediction error signaling in the ventral striatum. In controls, Kicer in the limbic striatum correlated with higher reward prediction error signaling, while there was no significant association in patients. Kicer in the associative striatum correlated with higher positive symptoms and blunted reward prediction error signaling was associated with negative symptoms. Our results suggest a dissociation between striatal subregions and symptom domains, with elevated dopamine synthesis capacity in the associative striatum contributing to positive symptoms while blunted prediction error signaling in the ventral striatum related to negative symptoms

Levothyroxine effects on depressive symptoms and limbic glucose metabolism in bipolar disorder: a randomized, placebo-controlled positron emission tomography study.

Adding supraphysiologic doses of levothyroxine (L-T4) to standard treatment for bipolar depression shows promise, but the mechanisms underlying clinical improvement are unknown. In a previous pilot study, L-T4 treatment reduced depression scores and activity within the anterior limbic network. Here we extended this work in a randomized, double-blind, placebo-controlled study of patients with bipolar depression. Cerebral glucose metabolism was assessed with positron emission tomography and [F-18]fluorodeoxyglucose before and after 6 weeks of treatment with L-T4 (n=15) or placebo (n=10) in 12 volumes of interest (VOIs): the bilateral thalamus, amygdala, hippocampus, dorsal striatum and ventral striatum, and midline cerebellar vermis and subgenual cingulate cortex. Radioactivity in the VOIs, normalized to whole-brain radioactivity was taken as a surrogate index of glucose metabolism, and markers of thyroid function were assayed. Changes in brain activity and their association with clinical response were assessed using statistical parametric mapping. Adjunctive L-T4 treatment produced a significant decline in depression scores during the 6-week treatment. In patients treated with L-T4, we found a significant decrease in regional activity at P<0.05 after Bonferroni correction in the left thalamus, right amygdala, right hippocampus, left ventral striatum and the right dorsal striatum. Decreases in the left thalamus, left dorsal striatum and the subgenual cingulate were correlated with a reduction in depression scores (P<0.05 after Bonferroni correction). Placebo treatment was associated with a significant decrease in activity only in the right amygdala, and no region had a change in activity that was correlated with change in depression scores. The groups differed significantly in the relationship between the changes in depression scores and in activity in the thalamus bilaterally and the left ventral striatum. The findings provide evidence that administration of supraphysiologic thyroid hormone improves depressive symptoms in patients with bipolar disorder by modulating function in components of the anterior limbic network

Anodal tDCS over the medial prefrontal cortex enhances behavioral adaptation after punishments during reversal learning through increased updating of unchosen choice options

The medial prefrontal cortex (mPFC) is thought to be central for flexible behavioral adaptation. However, the causal relationship between mPFC activity and this behavior is incompletely understood. We investigated whether transcranial direct current stimulation (tDCS) over the mPFC alters flexible behavioral adaptation during reward-based decision-making, targeting Montreal Neurological Institute (MNI) coordinates X = -8, Y = 62, Z = 12, which has previously been associated with impaired behavioral adaptation in alcohol-dependent patients. Healthy human participants (n = 61) received either anodal (n = 30) or cathodal (n = 31) tDCS versus sham tDCS while performing a reversal learning task. To assess the mechanisms of reinforcement learning (RL) underlying our behavioral observations, we applied computational models that varied with respect to the updating of the unchosen choice option. We observed that anodal stimulation over the mPFC induced increased choice switching after punishments compared with sham stimulation, whereas cathodal stimulation showed no effect on participants' behavior compared with sham stimulation. RL revealed increased updating of the unchosen choice option under anodal as compared with sham stimulation, which accounted well for the increased tendency to switch after punishments. Our findings provide a potential model for tDCS interventions in conditions related to flexible behavioral adaptation, such as addiction

Modeling subjective relevance in schizophrenia and its relation to aberrant salience

In schizophrenia, increased aberrant salience to irrelevant events and reduced learning of relevant information may relate to an underlying deficit in relevance detection. So far, subjective estimates of relevance have not been probed in schizophrenia patients. The mechanisms underlying belief formation about relevance and their translation into decisions are unclear. Using novel computational methods, we investigated relevance detection during implicit learning in 42 schizophrenia patients and 42 healthy individuals. Participants underwent functional magnetic resonance imaging while detecting the outcomes in a learning task. These were preceded by cues differing in color and shape, which were either relevant or irrelevant for outcome prediction. We provided a novel definition of relevance based on Bayesian precision and modeled reaction times as a function of relevance weighted unsigned prediction errors (UPE). For aberrant salience, we assessed responses to subjectively irrelevant cue manifestations. Participants learned the contingencies and slowed down their responses following unexpected events. Model selection revealed that individuals inferred the relevance of cue features and used it for behavioral adaption to the relevant cue feature. Relevance weighted UPEs correlated with dorsal anterior cingulate cortex activation and hippocampus deactivation. In patients, the aberrant salience bias to subjectively task-irrelevant information was increased and correlated with decreased striatal UPE activation and increased negative symptoms. This study shows that relevance estimates based on Bayesian precision can be inferred from observed behavior. This underscores the importance of relevance detection as an underlying mechanism for behavioral adaptation in complex environments and enhances the understanding of aberrant salience in schizophrenia

Dealloying of Platinum-Aluminum Thin Films Part II. Electrode Performance

Highly porous Pt/Al thin film electrodes on yttria stabilized zirconia electrolytes were prepared by dealloying of co-sputtered Pt/Al films. The oxygen reduction capability of the resulting electrodes was analyzed in a solid oxide fuel cell setup at elevated temperatures. During initial heating to 523 K exceptionally high performances compared to conventional Pt thin film electrodes were measured. This results from the high internal surface area and large three phase boundary length obtained by the dealloying process. Exposure to elevated temperatures of 673 K or 873 K gave rise to degradation of the electrode performance, which was primarily attributed to the oxidation of remaining Al in the thin films.Comment: 5 pages, 4 figure

The effects of life stress and neural learning signals on fluid intelligence.

Fluid intelligence (fluid IQ), defined as the capacity for rapid problem solving and behavioral adaptation, is known to be modulated by learning and experience. Both stressful life events (SLES) and neural correlates of learning [specifically, a key mediator of adaptive learning in the brain, namely the ventral striatal representation of prediction errors (PE)] have been shown to be associated with individual differences in fluid IQ. Here, we examine the interaction between adaptive learning signals (using a well-characterized probabilistic reversal learning task in combination with fMRI) and SLES on fluid IQ measures. We find that the correlation between ventral striatal BOLD PE and fluid IQ, which we have previously reported, is quantitatively modulated by the amount of reported SLES. Thus, after experiencing adversity, basic neuronal learning signatures appear to align more closely with a general measure of flexible learning (fluid IQ), a finding complementing studies on the effects of acute stress on learning. The results suggest that an understanding of the neurobiological correlates of trait variables like fluid IQ needs to take socioemotional influences such as chronic stress into account

Dealloying of Platinum-Aluminum Thin Films Part I. Dynamics of Pattern Formation

Applying focused ion beam (FIB) nanotomography and Rutherford backscattering spectroscopy (RBS) to dealloyed platinum-aluminum thin films an in-depth analysis of the dominating physical mechanisms of porosity formation during the dealloying process is performed. The dynamical porosity formation due to the dissolution of the less noble aluminum in the alloy is treated as result of a reaction-diffusion system. The RBS analysis yields that the porosity formation is mainly caused by a linearly propagating diffusion front, i.e. the liquid/solid interface, with a uniform speed of 42(3) nm/s when using a 4M aqueous NaOH solution at room temperature. The experimentally observed front evolution is captured by the normal diffusive Fisher-Kolmogorov-Petrovskii-Piskounov (FKPP) equation and can be interpreted as a branching random walk phenomenon. The etching front produces a gradual porosity with an enhanced porosity in the surface-near regions of the thin film due to prolonged exposure of the alloy to the alkaline solution.Comment: 4 pages, 5 figure

Lateral prefrontal model-based signatures are reduced in healthy individuals with high trait impulsivity

High impulsivity is an important risk factor for addiction with evidence from endophenotype studies. In addiction, behavioral control is shifted toward the habitual end. Habitual control can be described by retrospective updating of reward expectations in ‘model-free’ temporal-difference algorithms. Goal- directed control relies on the prospective consideration of actions and their outcomes, which can be captured by forward-planning ‘model-based’ algorithms. So far, no studies have examined behavioral and neural signatures of model- free and model-based control in healthy high-impulsive individuals. Fifty healthy participants were drawn from the upper and lower ends of 452 individuals, completing the Barratt Impulsiveness Scale. All participants performed a sequential decision-making task during functional magnetic resonance imaging (fMRI) and underwent structural MRI. Behavioral and fMRI data were analyzed by means of computational algorithms reflecting model-free and model-based control. Both groups did not differ regarding the balance of model-free and model-based control, but high-impulsive individuals showed a subtle but significant accentuation of model-free control alone. Right lateral prefrontal model-based signatures were reduced in high-impulsive individuals. Effects of smoking, drinking, general cognition or gray matter density did not account for the findings. Irrespectively of impulsivity, gray matter density in the left dorsolateral prefrontal cortex was positively associated with model-based control. The present study supports the idea that high levels of impulsivity are accompanied by behavioral and neural signatures in favor of model-free behavioral control. Behavioral results in healthy high-impulsive individuals were qualitatively different to findings in patients with the same task. The predictive relevance of these results remains an important target for future longitudinal studies