Incentivizing Exploration with Heterogeneous Value of Money

Recently, Frazier et al. proposed a natural model for crowdsourced exploration of different a priori unknown options: a principal is interested in the long-term welfare of a population of agents who arrive one by one in a multi-armed bandit setting. However, each agent is myopic, so in order to incentivize him to explore options with better long-term prospects, the principal must offer the agent money. Frazier et al. showed that a simple class of policies called time-expanded are optimal in the worst case, and characterized their budget-reward tradeoff. The previous work assumed that all agents are equally and uniformly susceptible to financial incentives. In reality, agents may have different utility for money. We therefore extend the model of Frazier et al. to allow agents that have heterogeneous and non-linear utilities for money. The principal is informed of the agent's tradeoff via a signal that could be more or less informative. Our main result is to show that a convex program can be used to derive a signal-dependent time-expanded policy which achieves the best possible Lagrangian reward in the worst case. The worst-case guarantee is matched by so-called "Diamonds in the Rough" instances; the proof that the guarantees match is based on showing that two different convex programs have the same optimal solution for these specific instances. These results also extend to the budgeted case as in Frazier et al. We also show that the optimal policy is monotone with respect to information, i.e., the approximation ratio of the optimal policy improves as the signals become more informative.Comment: WINE 201

Galactic and Extragalactic Samples of Supernova Remnants: How They Are Identified and What They Tell Us

Supernova remnants (SNRs) arise from the interaction between the ejecta of a supernova (SN) explosion and the surrounding circumstellar and interstellar medium. Some SNRs, mostly nearby SNRs, can be studied in great detail. However, to understand SNRs as a whole, large samples of SNRs must be assembled and studied. Here, we describe the radio, optical, and X-ray techniques which have been used to identify and characterize almost 300 Galactic SNRs and more than 1200 extragalactic SNRs. We then discuss which types of SNRs are being found and which are not. We examine the degree to which the luminosity functions, surface-brightness distributions and multi-wavelength comparisons of the samples can be interpreted to determine the class properties of SNRs and describe efforts to establish the type of SN explosion associated with a SNR. We conclude that in order to better understand the class properties of SNRs, it is more important to study (and obtain additional data on) the SNRs in galaxies with extant samples at multiple wavelength bands than it is to obtain samples of SNRs in other galaxiesComment: Final 2016 draft of a chapter in "Handbook of Supernovae" edited by Athem W. Alsabti and Paul Murdin. Final version available at https://doi.org/10.1007/978-3-319-20794-0_90-

Using fractional exhaled nitric oxide (FeNO) to diagnose steroid-responsive disease and guide asthma management in routine care

Acknowledgements We thank Robin Taylor for his informative thinking and publications on FeNO, which have helped to influence and direct the thinking of the authors. Funding Extraction of the real-life dataset was funded by Research in Real Life Limited, the analysis of the dataset and the writing of this manuscript were co-funded (50:50) by Research in Real Life Limited and Aerocrine.Peer reviewedPublisher PD

Molecular and cellular mechanisms underlying the evolution of form and function in the amniote jaw.

The amniote jaw complex is a remarkable amalgamation of derivatives from distinct embryonic cell lineages. During development, the cells in these lineages experience concerted movements, migrations, and signaling interactions that take them from their initial origins to their final destinations and imbue their derivatives with aspects of form including their axial orientation, anatomical identity, size, and shape. Perturbations along the way can produce defects and disease, but also generate the variation necessary for jaw evolution and adaptation. We focus on molecular and cellular mechanisms that regulate form in the amniote jaw complex, and that enable structural and functional integration. Special emphasis is placed on the role of cranial neural crest mesenchyme (NCM) during the species-specific patterning of bone, cartilage, tendon, muscle, and other jaw tissues. We also address the effects of biomechanical forces during jaw development and discuss ways in which certain molecular and cellular responses add adaptive and evolutionary plasticity to jaw morphology. Overall, we highlight how variation in molecular and cellular programs can promote the phenomenal diversity and functional morphology achieved during amniote jaw evolution or lead to the range of jaw defects and disease that affect the human condition

DEVELOPMENT QUALIFICATION AND DISPOSAL OF AN ALTERNATIVE IMMOBILIZED LOW-ACTIVITY WASTE FORM AT THE HANFORD SITE

Demonstrating that a waste form produced by a given immobilization process is chemically and physically durable as well as compliant with disposal facility acceptance criteria is critical to the success of a waste treatment program, and must be pursued in conjunction with the maturation of the waste processing technology. Testing of waste forms produced using differing scales of processing units and classes of feeds (simulants versus actual waste) is the crux of the waste form qualification process. Testing is typically focused on leachability of constituents of concern (COCs), as well as chemical and physical durability of the waste form. A principal challenge regarding testing immobilized low-activity waste (ILAW) forms is the absence of a standard test suite or set of mandatory parameters against which waste forms may be tested, compared, and qualified for acceptance in existing and proposed nuclear waste disposal sites at Hanford and across the Department of Energy (DOE) complex. A coherent and widely applicable compliance strategy to support characterization and disposal of new waste forms is essential to enhance and accelerate the remediation of DOE tank waste. This paper provides a background summary of important entities, regulations, and considerations for nuclear waste form qualification and disposal. Against this backdrop, this paper describes a strategy for meeting and demonstrating compliance with disposal requirements emphasizing the River Protection Project (RPP) Integrated Disposal Facility (IDF) at the Hanford Site and the fluidized bed steam reforming (FBSR) mineralized low-activity waste (LAW) product stream

Transient activation of dopaminergic neurons during development modulates visual responsiveness, locomotion and brain activity in a dopamine ontogeny model of schizophrenia

It has been observed that certain developmental environmental risk factors for schizophrenia when modeled in rodents alter the trajectory of dopaminergic development, leading to persistent behavioural changes in adults. This has recently been articulated as the "dopamine ontogeny hypothesis of schizophrenia". To test one aspect of this hypothesis, namely that transient dopaminergic effects during development modulate attention-like behavior and arousal in adults, we turned to a small-brain model, Drosophila melanogaster. By applying genetic tools allowing transient activation or silencing of dopaminergic neurons in the fly brain, we investigated whether a critical window exists during development when altered dopamine (DA) activity levels could lead to impairments in arousal states in adult animals. We found that increased activity in dopaminergic neurons in later stages of development significantly increased visual responsiveness and locomotion, especially in adult males. This misallocation of visual salience and hyperactivity mimicked the effect of acute methamphetamine feeding to adult flies, suggesting up-regulated DA signaling could result from developmental manipulations. Finally, brain recordings revealed significantly reduced gamma-band activity in adult animals exposed to the transient developmental insult. Together, these data support the idea that transient alterations in DA signaling during development can permanently alter behavior in adults, and that a reductionist model such as Drosophila can be used to investigate potential mechanisms underlying complex cognitive disorders such as schizophrenia

Disorders of compulsivity: a common bias towards learning habits.

Why do we repeat choices that we know are bad for us? Decision making is characterized by the parallel engagement of two distinct systems, goal-directed and habitual, thought to arise from two computational learning mechanisms, model-based and model-free. The habitual system is a candidate source of pathological fixedness. Using a decision task that measures the contribution to learning of either mechanism, we show a bias towards model-free (habit) acquisition in disorders involving both natural (binge eating) and artificial (methamphetamine) rewards, and obsessive-compulsive disorder. This favoring of model-free learning may underlie the repetitive behaviors that ultimately dominate in these disorders. Further, we show that the habit formation bias is associated with lower gray matter volumes in caudate and medial orbitofrontal cortex. Our findings suggest that the dysfunction in a common neurocomputational mechanism may underlie diverse disorders involving compulsion.This study was funded by the WT fellowship grant for VV (093705/Z/ 10/Z) and Cambridge NIHR Biomedical Research Centre. VV and NAH are Wellcome Trust (WT) intermediate Clinical Fellows. YW is supported by the Fyssen Fondation and MRC Studentships. PD is supported by the Gatsby Charitable Foundation. JEG has received grants from the National Institute of Drug Abuse and the National Center for Responsible Gaming. TWR and BJS are supported on a WT Programme Grant (089589/Z/09/Z). The BCNI is supported by a WT and MRC grant.This is the final published version. It's also available from Molecular Psychiatry at http://www.nature.com/mp/journal/vaop/ncurrent/full/mp201444a.html

Antenatal glucocorticoid treatment induces adaptations in adult midbrain dopamine neurons, which underpin sexually dimorphic behavioral resilience

We demonstrated previously that antenatal glucocorticoid treatment (AGT, gestational days 16-19) altered the size and organization of the adult rat midbrain dopaminergic (DA) populations. Here we investigated the consequences of these AGT-induced cytoarchitectural disturbances on indices of DA function in adult rats. We show that in adulthood, enrichment of striatal DA fiber density paralleled AGT-induced increases in the numbers of midbrain DA neurons, which retained normal basal electrophysiological properties. This was co-incident with changes in (i) striatal D2-type receptor levels (increased, both sexes); (ii) D1-type receptor levels (males decreased; females increased); (iii) DA transporter levels (males increased; females decreased) in striatal regions; and (iv) amphetamine-induced mesolimbic DA release (males increased; females decreased). However, despite these profound, sexually dimorphic changes in markers of DA neurotransmission, in-utero glucocorticoid overexposure had a modest or no effect on a range of conditioned and unconditioned appetitive behaviors known to depend on mesolimbic DA activity. These findings provide empirical evidence for enduring AGT-induced adaptive mechanisms within the midbrain DA circuitry, which preserve some, but not all, functions, thereby casting further light on the vulnerability of these systems to environmental perturbations. Furthermore, they demonstrate these effects are achieved by different, often opponent, adaptive mechanisms in males and females, with translational implications for sex biases commonly found in midbrain DA-associated disorders

Attribution-based motivation treatment efficacy in an online learning environment for students who differ in cognitive elaboration

Attribution-based motivation treatments can boost performance in competitive achievement settings (Perry and Hamm 2017), yet their efficacy relative to mediating processes and affect-based treatments remains largely unexamined. In a two-semester, pre-post, randomized treatment study (n = 806), attributional retraining (AR) and stress-reduction (SR) treatments were administered in an online learning environment to first-year college students who differed in cognitive elaboration (low, high). Low elaborators who received AR outperformed their SR peers by nearly a letter grade on a class test assessed 5 months post-treatment. Path analysis revealed this AR-performance linkage was mediated by causal attributions, perceived control, and positive and negative achievement emotions in a hypothesized causal sequence. Results advance the literature by showing AR (vs. SR) improved performance indirectly via cognitive and affective process variables specified by Weiner’s (1985a, 2012) attribution theory of motivation and emotion

Sex-specific disruption of murine midbrain astrocytic and dopaminergic developmental trajectories following antenatal GC treatment

The mammalian midbrain dopaminergic systems arising in the substantia nigra pars compacta (SNc) and ventral tegmental area (VTA) are critical for coping behaviours and are implicated in neuropsychiatric disorders where early life challenges comprise significant risk factors. Here, we aimed to advance our hypothesis that glucocorticoids (GCs), recognised key players in neurobiological programming, target development within these systems, with a novel focus on the astrocytic population. Mice received antenatal GC treatment (AGT) by including the synthetic GC, dexamethasone, in the mothers' drinking water on gestational days 16-19; controls received normal drinking water. Analyses of regional shapes and volumes of the adult SNc and VTA demonstrated that AGT induced long-term, dose-dependent, structural changes that were accompanied by profound effects on astrocytes (doubling/tripling of numbers and/or density). Additionally, AGT induced long-term changes in the population size and distribution of SNc/VTA dopaminergic neurons, confirming and extending our previous observations made in rats. Furthermore, glial/neuronal structural remodelling was sexually dimorphic and depended on the AGT dose and sub-region of the SNc/VTA. Investigations within the neonatal brain revealed that these long-term organisational effects of AGT depend, at least in part, on targeting perinatal processes that determine astrocyte density and programmed cell death in dopaminergic neurons. Collectively, our characterisation of enduring, AGT-induced, sex-specific cytoarchitectural disturbances suggests novel mechanistic links for the strong association between early environmental challenge (inappropriate exposure to excess GCs) and vulnerability to developing aberrant behaviours in later life, with translational implications for dopamine-associated disorders (such as schizophrenia, ADHD, autism, depression), which typically show a sex bia