Feasible Adversarial Robust Reinforcement Learning for Underspecified Environments

Robust reinforcement learning (RL) considers the problem of learning policies that perform well in the worst case among a set of possible environment parameter values. In real-world environments, choosing the set of possible values for robust RL can be a difficult task. When that set is specified too narrowly, the agent will be left vulnerable to reasonable parameter values unaccounted for. When specified too broadly, the agent will be too cautious. In this paper, we propose Feasible Adversarial Robust RL (FARR), a novel problem formulation and objective for automatically determining the set of environment parameter values over which to be robust. FARR implicitly defines the set of feasible parameter values as those on which an agent could achieve a benchmark reward given enough training resources. By formulating this problem as a two-player zero-sum game, optimizing the FARR objective jointly produces an adversarial distribution over parameter values with feasible support and a policy robust over this feasible parameter set. We demonstrate that approximate Nash equilibria for this objective can be found using a variation of the PSRO algorithm. Furthermore, we show that an optimal agent trained with FARR is more robust to feasible adversarial parameter selection than with existing minimax, domain-randomization, and regret objectives in a parameterized gridworld and three MuJoCo control environments.Comment: Added new theory sections. Added comparison to self-play. Added adversary mixed-strategy analysi

Selective Perception: Optimizing State Descriptions with Reinforcement Learning for Language Model Actors

Large language models (LLMs) are being applied as actors for sequential decision making tasks in domains such as robotics and games, utilizing their general world knowledge and planning abilities. However, previous work does little to explore what environment state information is provided to LLM actors via language. Exhaustively describing high-dimensional states can impair performance and raise inference costs for LLM actors. Previous LLM actors avoid the issue by relying on hand-engineered, task-specific protocols to determine which features to communicate about a state and which to leave out. In this work, we propose Brief Language INputs for DEcision-making Responses (BLINDER), a method for automatically selecting concise state descriptions by learning a value function for task-conditioned state descriptions. We evaluate BLINDER on the challenging video game NetHack and a robotic manipulation task. Our method improves task success rate, reduces input size and compute costs, and generalizes between LLM actors

Discovering unique tobacco use patterns among Alaska Native people

Background . Alaska Native people are disproportionately impacted by tobacco-related diseases in comparison to non-Native Alaskans. Design. We used Alaska's Behavioral Risk Factor Surveillance System (BRFSS) to describe tobacco use among more than 4,100 Alaska Native adults, stratified by geographic region and demographic groups. Results . Overall tobacco use was high: approximately 2 out of every 5 Alaska Native adults reported smoking cigarettes (41.2%) and 1 in 10 reported using smokeless tobacco (SLT, 12.3%). A small percentage overall (4.8%) reported using iq'mik, an SLT variant unique to Alaska Native people. When examined by geographic region, cigarette smoking was highest in remote geographic regions; SLT use was highest in the southwest region of the state. Use of iq'mik was primarily confined to a specific area of the state; further analysis showed that 1 in 3 women currently used iq'mik in this region. Conclusion . Our results suggest that different types of tobacco use are epidemic among diverse Alaska Native communities. Our results also illustrate that detailed analysis within racial/ethnic groups can be useful for public health programme planning to reduce health disparities

Mena deficiency delays tumor progression and decreases metastasis in polyoma middle-T transgenic mouse mammary tumors

Introduction The actin binding protein Mammalian enabled (Mena), has been implicated in the metastatic progression of solid tumors in humans. Mena expression level in primary tumors is correlated with metastasis in breast, cervical, colorectal and pancreatic cancers. Cells expressing high Mena levels are part of the tumor microenvironment for metastasis (TMEM), an anatomical structure that is predictive for risk of breast cancer metastasis. Previously we have shown that forced expression of Mena adenocarcinoma cells enhances invasion and metastasis in xenograft mice. Whether Mena is required for tumor progression is still unknown. Here we report the effects of Mena deficiency on tumor progression, metastasis and on normal mammary gland development. Methods To investigate the role of Mena in tumor progression and metastasis, Mena deficient mice were intercrossed with mice carrying a transgene expressing the polyoma middle T oncoprotein, driven by the mouse mammary tumor virus. The progeny were investigated for the effects of Mena deficiency on tumor progression via staging of primary mammary tumors and by evaluation of morbidity. Stages of metastatic progression were investigated using an in vivo invasion assay, intravital multiphoton microscopy, circulating tumor cell burden, and lung metastases. Mammary gland development was studied in whole mount mammary glands of wild type and Mena deficient mice. Results Mena deficiency decreased morbidity and metastatic dissemination. Loss of Mena increased mammary tumor latency but had no affect on mammary tumor burden or histologic progression to carcinoma. Elimination of Mena also significantly decreased epidermal growth factor (EGF) induced in vivo invasion, in vivo motility, intravasation and metastasis. Non-tumor bearing mice deficient for Mena also showed defects in mammary gland terminal end bud formation and branching. Conclusions Deficiency of Mena decreases metastasis by slowing tumor progression and reducing tumor cell invasion and intravasation. Mena deficiency during development causes defects in invasive processes involved in mammary gland development. These findings suggest that functional intervention targeting Mena in breast cancer patients may provide a valuable treatment option to delay tumor progression and decrease invasion and metastatic spread leading to an improved prognostic outcome.National Cancer Institute (U.S.). Integrative Cancer Biology Program (grant U54 CA112967)Virginia and D.K. Ludwig Fund for Cancer Researc

PIASγ Is Required for Faithful Chromosome Segregation in Human Cells

BACKGROUND: The precision of the metaphase-anaphase transition ensures stable genetic inheritance. The spindle checkpoint blocks anaphase onset until the last chromosome biorients at metaphase plate, then the bonds between sister chromatids are removed and disjoined chromatids segregate to the spindle poles. But, how sister separation is triggered is not fully understood. PRINCIPAL FINDINGS: We identify PIASγ as a human E3 sumo ligase required for timely and efficient sister chromatid separation. In cells lacking PIASγ, normal metaphase plates form, but the spindle checkpoint is activated, leading to a prolonged metaphase block. Sister chromatids remain cohered even if cohesin is removed by depletion of hSgo1, because DNA catenations persist at centromeres. PIASγ-depleted cells cannot properly localize Topoisomerase II at centromeres or in the cores of mitotic chromosomes, providing a functional link between PIASγ and Topoisomerase II. CONCLUSIONS: PIASγ directs Topoisomerase II to specific chromosome regions that require efficient removal of DNA catenations prior to anaphase. The lack of this activity activates the spindle checkpoint, protecting cells from non-disjunction. Because DNA catenations persist without PIASγ in the absence of cohesin, removal of catenations and cohesin rings must be regulated in parallel

Major histocompatibility complex class I molecules protect motor neurons from astrocyte-induced toxicity in amyotrophic lateral sclerosis

Astrocytes isolated from individuals with amyotrophic lateral sclerosis (ALS) are toxic to motor neurons (MNs) and play a non–cell autonomous role in disease pathogenesis. The mechanisms underlying the susceptibility of MNs to cell death remain unclear. Here we report that astrocytes derived from either mice bearing mutations in genes associated with ALS or human subjects with ALS reduce the expression of major histocompatibility complex class I (MHCI) molecules on MNs; reduced MHCI expression makes these MNs susceptible to astrocyte-induced cell death. Increasing MHCI expression on MNs increases survival and motor performance in a mouse model of ALS and protects MNs against astrocyte toxicity. Overexpression of a single MHCI molecule, HLA-F, protects human MNs from ALS astrocyte–mediated toxicity, whereas knockdown of its receptor, the killer cell immunoglobulin-like receptor KIR3DL2, on human astrocytes results in enhanced MN death. Thus, our data indicate that, in ALS, loss of MHCI expression on MNs renders them more vulnerable to astrocyte-mediated toxicity

Sequence of a complete chicken BG haplotype shows dynamic expansion and contraction of two gene lineages with particular expression patterns.

Many genes important in immunity are found as multigene families. The butyrophilin genes are members of the B7 family, playing diverse roles in co-regulation and perhaps in antigen presentation. In humans, a fixed number of butyrophilin genes are found in and around the major histocompatibility complex (MHC), and show striking association with particular autoimmune diseases. In chickens, BG genes encode homologues with somewhat different domain organisation. Only a few BG genes have been characterised, one involved in actin-myosin interaction in the intestinal brush border, and another implicated in resistance to viral diseases. We characterise all BG genes in B12 chickens, finding a multigene family organised as tandem repeats in the BG region outside the MHC, a single gene in the MHC (the BF-BL region), and another single gene on a different chromosome. There is a precise cell and tissue expression for each gene, but overall there are two kinds, those expressed by haemopoietic cells and those expressed in tissues (presumably non-haemopoietic cells), correlating with two different kinds of promoters and 5' untranslated regions (5'UTR). However, the multigene family in the BG region contains many hybrid genes, suggesting recombination and/or deletion as major evolutionary forces. We identify BG genes in the chicken whole genome shotgun sequence, as well as by comparison to other haplotypes by fibre fluorescence in situ hybridisation, confirming dynamic expansion and contraction within the BG region. Thus, the BG genes in chickens are undergoing much more rapid evolution compared to their homologues in mammals, for reasons yet to be understood.This is the final published version. It was originally published by PLOS in PLOS Genetics here: http://www.plosgenetics.org/article/info%3Adoi%2F10.1371%2Fjournal.pgen.1004417

Current and prospective pharmacological targets in relation to antimigraine action

Migraine is a recurrent incapacitating neurovascular disorder characterized by unilateral and throbbing headaches associated with photophobia, phonophobia, nausea, and vomiting. Current specific drugs used in the acute treatment of migraine interact with vascular receptors, a fact that has raised concerns about their cardiovascular safety. In the past, α-adrenoceptor agonists (ergotamine, dihydroergotamine, isometheptene) were used. The last two decades have witnessed the advent of 5-HT1B/1D receptor agonists (sumatriptan and second-generation triptans), which have a well-established efficacy in the acute treatment of migraine. Moreover, current prophylactic treatments of migraine include 5-HT2 receptor antagonists, Ca2+ channel blockers, and β-adrenoceptor antagonists. Despite the progress in migraine research and in view of its complex etiology, this disease still remains underdiagnosed, and available therapies are underused. In this review, we have discussed pharmacological targets in migraine, with special emphasis on compounds acting on 5-HT (5-HT1-7), adrenergic (α1, α2, and β), calcitonin gene-related peptide (CGRP 1 and CGRP2), adenosine (A1, A2, and A3), glutamate (NMDA, AMPA, kainate, and metabotropic), dopamine, endothelin, and female hormone (estrogen and progesterone) receptors. In addition, we have considered some other targets, including gamma-aminobutyric acid, angiotensin, bradykinin, histamine, and ionotropic receptors, in relation to antimigraine therapy. Finally, the cardiovascular safety of current and prospective antimigraine therapies is touched upon

Lawson criterion for ignition exceeded in an inertial fusion experiment

For more than half a century, researchers around the world have been engaged in attempts to achieve fusion ignition as a proof of principle of various fusion concepts. Following the Lawson criterion, an ignited plasma is one where the fusion heating power is high enough to overcome all the physical processes that cool the fusion plasma, creating a positive thermodynamic feedback loop with rapidly increasing temperature. In inertially confined fusion, ignition is a state where the fusion plasma can begin "burn propagation" into surrounding cold fuel, enabling the possibility of high energy gain. While "scientific breakeven" (i.e., unity target gain) has not yet been achieved (here target gain is 0.72, 1.37 MJ of fusion for 1.92 MJ of laser energy), this Letter reports the first controlled fusion experiment, using laser indirect drive, on the National Ignition Facility to produce capsule gain (here 5.8) and reach ignition by nine different formulations of the Lawson criterion