Adults with autism overestimate the volatility of the sensory environment.

Insistence on sameness and intolerance of change are among the diagnostic criteria for autism spectrum disorder (ASD), but little research has addressed how people with ASD represent and respond to environmental change. Here, behavioral and pupillometric measurements indicated that adults with ASD are less surprised than neurotypical adults when their expectations are violated, and decreased surprise is predictive of greater symptom severity. A hierarchical Bayesian model of learning suggested that in ASD, a tendency to overlearn about volatility in the face of environmental change drives a corresponding reduction in learning about probabilistically aberrant events, thus putatively rendering these events less surprising. Participant-specific modeled estimates of surprise about environmental conditions were linked to pupil size in the ASD group, thus suggesting heightened noradrenergic responsivity in line with compromised neural gain. This study offers insights into the behavioral, algorithmic and physiological mechanisms underlying responses to environmental volatility in ASD

Structure Learning in Human Sequential Decision-Making

Studies of sequential decision-making in humans frequently find suboptimal performance relative to an ideal actor that has perfect knowledge of the model of how rewards and events are generated in the environment. Rather than being suboptimal, we argue that the learning problem humans face is more complex, in that it also involves learning the structure of reward generation in the environment. We formulate the problem of structure learning in sequential decision tasks using Bayesian reinforcement learning, and show that learning the generative model for rewards qualitatively changes the behavior of an optimal learning agent. To test whether people exhibit structure learning, we performed experiments involving a mixture of one-armed and two-armed bandit reward models, where structure learning produces many of the qualitative behaviors deemed suboptimal in previous studies. Our results demonstrate humans can perform structure learning in a near-optimal manner

Pharmacological Fingerprints of Contextual Uncertainty

Successful interaction with the environment requires flexible updating of our beliefs about the world. By estimating the likelihood of future events, it is possible to prepare appropriate actions in advance and execute fast, accurate motor responses. According to theoretical proposals, agents track the variability arising from changing environments by computing various forms of uncertainty. Several neuromodulators have been linked to uncertainty signalling, but comprehensive empirical characterisation of their relative contributions to perceptual belief updating, and to the selection of motor responses, is lacking. Here we assess the roles of noradrenaline, acetylcholine, and dopamine within a single, unified computational framework of uncertainty. Using pharmacological interventions in a sample of 128 healthy human volunteers and a hierarchical Bayesian learning model, we characterise the influences of noradrenergic, cholinergic, and dopaminergic receptor antagonism on individual computations of uncertainty during a probabilistic serial reaction time task. We propose that noradrenaline influences learning of uncertain events arising from unexpected changes in the environment. In contrast, acetylcholine balances attribution of uncertainty to chance fluctuations within an environmental context, defined by a stable set of probabilistic associations, or to gross environmental violations following a contextual switch. Dopamine supports the use of uncertainty representations to engender fast, adaptive responses. \ua9 2016 Marshall et al

Learning and generalization under ambiguity: an fMRI study.

Adaptive behavior often exploits generalizations from past experience by applying them judiciously in new situations. This requires a means of quantifying the relative importance of prior experience and current information, so they can be balanced optimally. In this study, we ask whether the brain generalizes in an optimal way. Specifically, we used Bayesian learning theory and fMRI to test whether neuronal responses reflect context-sensitive changes in ambiguity or uncertainty about experience-dependent beliefs. We found that the hippocampus expresses clear ambiguity-dependent responses that are associated with an augmented rate of learning. These findings suggest candidate neuronal systems that may be involved in aberrations of generalization, such as over-confidence

Mapping anhedonia onto reinforcement learning: A behavioural meta-analysis

BACKGROUND: Depression is characterised partly by blunted reactions to reward. However, tasks probing this deficiency have not distinguished insensitivity to reward from insensitivity to the prediction errors for reward that determine learning and are putatively reported by the phasic activity of dopamine neurons. We attempted to disentangle these factors with respect to anhedonia in the context of stress, Major Depressive Disorder (MDD), Bipolar Disorder (BPD) and a dopaminergic challenge. METHODS: Six behavioural datasets involving 392 experimental sessions were subjected to a model-based, Bayesian meta-analysis. Participants across all six studies performed a probabilistic reward task that used an asymmetric reinforcement schedule to assess reward learning. Healthy controls were tested under baseline conditions, stress or after receiving the dopamine D2 agonist pramipexole. In addition, participants with current or past MDD or BPD were evaluated. Reinforcement learning models isolated the contributions of variation in reward sensitivity and learning rate. RESULTS: MDD and anhedonia reduced reward sensitivity more than they affected the learning rate, while a low dose of the dopamine D2 agonist pramipexole showed the opposite pattern. Stress led to a pattern consistent with a mixed effect on reward sensitivity and learning rate. CONCLUSION: Reward-related learning reflected at least two partially separable contributions. The first related to phasic prediction error signalling, and was preferentially modulated by a low dose of the dopamine agonist pramipexole. The second related directly to reward sensitivity, and was preferentially reduced in MDD and anhedonia. Stress altered both components. Collectively, these findings highlight the contribution of model-based reinforcement learning meta-analysis for dissecting anhedonic behavior

Tuning Catalytic Performance through a Single or Sequential Post-Synthesis Reaction(s) in a Gas Phase

Catalytic performance of a bimetallic catalyst is determined by geometric structure and electronic state of the surface or even the near-surface region of the catalyst. Here we report that single and sequential postsynthesis reactions of an as-synthesized bimetallic nanoparticle catalyst in one or more gas phases can tailor surface chemistry and structure of the catalyst in a gas phase, by which catalytic performance of this bimetallic catalyst can be tuned. Pt–Cu regular nanocube (Pt–Cu RNC) and concave nanocube (Pt–Cu CNC) are chosen as models of bimetallic catalysts. Surface chemistry and catalyst structure under different reaction conditions and during catalysis were explored in gas phase of one or two reactants with ambient-pressure X-ray photoelectron spectroscopy (AP-XPS) and extended X-ray absorption fine structure (EXAFS) spectroscopy. The newly formed surface structures of Pt–Cu RNC and Pt–Cu CNC catalysts strongly depend on the reactive gas­(es) used in the postsynthesis reaction(s). A reaction of Pt–Cu RNC-as synthesized with H₂ at 200 °C generates a near-surface alloy consisting of a Pt skin layer, a Cu-rich subsurface, and a Pt-rich deep layer. This near-surface alloy of Pt–Cu RNC-as synthesized-H₂ exhibits a much higher catalytic activity in CO oxidation in terms of a low activation barrier of 39 ± 4 kJ/mol in contrast to 128 ± 7 kJ/mol of Pt–Cu RNC-as synthesized. Here the significant decrease of activation barrier demonstrates a method to tune catalytic performances of as-synthesized bimetallic catalysts. A further reaction of Pt–Cu RNC-as synthesized-H₂ with CO forms a Pt–Cu alloy surface, which exhibits quite different catalytic performance in CO oxidation. It suggests the capability of generating a different surface by using another gas. The capability of tuning surface chemistry and structure of bimetallic catalysts was also demonstrated in restructuring of Pt–Cu CNC-as synthesized

Stress, genotype and norepinephrine in the prediction of mouse behavior using reinforcement learning

Individual behavioral performance during learning is known to be affected by modulatory factors, such as stress and motivation, and by genetic predispositions that influence sensitivity to these factors. Despite numerous studies, no integrative framework is available that could predict how a given animal would perform a certain learning task in a realistic situation. We found that a simple reinforcement learning model can predict mouse behavior in a hole-box conditioning task if model metaparameters are dynamically controlled on the basis of the mouse's genotype and phenotype, stress conditions, recent performance feedback and pharmacological manipulations of adrenergic alpha-2 receptors. We find that stress and motivation affect behavioral performance by altering the exploration-exploitation balance in a genotype-dependent manner. Our results also provide computational insights into how an inverted U-shape relation between stress/arousal/norepinephrine levels and behavioral performance could be explained through changes in task performance accuracy and future reward discounting