Joint Goal and Strategy Inference across Heterogeneous Demonstrators via Reward Network Distillation

Reinforcement learning (RL) has achieved tremendous success as a general framework for learning how to make decisions. However, this success relies on the interactive hand-tuning of a reward function by RL experts. On the other hand, inverse reinforcement learning (IRL) seeks to learn a reward function from readily-obtained human demonstrations. Yet, IRL suffers from two major limitations: 1) reward ambiguity - there are an infinite number of possible reward functions that could explain an expert's demonstration and 2) heterogeneity - human experts adopt varying strategies and preferences, which makes learning from multiple demonstrators difficult due to the common assumption that demonstrators seeks to maximize the same reward. In this work, we propose a method to jointly infer a task goal and humans' strategic preferences via network distillation. This approach enables us to distill a robust task reward (addressing reward ambiguity) and to model each strategy's objective (handling heterogeneity). We demonstrate our algorithm can better recover task reward and strategy rewards and imitate the strategies in two simulated tasks and a real-world table tennis task.Comment: In Proceedings of the 2020 ACM/IEEE In-ternational Conference on Human-Robot Interaction (HRI '20), March 23 to 26, 2020, Cambridge, United Kingdom.ACM, New York, NY, USA, 10 page

The long noncoding RNA lncNB1 promotes tumorigenesis by interacting with ribosomal protein RPL35

The majority of patients with neuroblastoma due to MYCN oncogene amplification and consequent N-Myc oncoprotein over-expression die of the disease. Here our analyses of RNA sequencing data identify the long noncoding RNA lncNB1 as one of the transcripts most over-expressed in MYCN-amplified, compared with MYCN-non-amplified, human neuroblastoma cells and also the most over-expressed in neuroblastoma compared with all other cancers. lncNB1 binds to the ribosomal protein RPL35 to enhance E2F1 protein synthesis, leading to DEPDC1B gene transcription. The GTPase-activating protein DEPDC1B induces ERK protein phosphorylation and N-Myc protein stabilization. Importantly, lncNB1 knockdown abolishes neuroblastoma cell clonogenic capacity in vitro and leads to neuroblastoma tumor regression in mice, while high levels of lncNB1 and RPL35 in human neuroblastoma tissues predict poor patient prognosis. This study therefore identifies lncNB1 and its binding protein RPL35 as key factors for promoting E2F1 protein synthesis, N-Myc protein stability and N-Myc-driven oncogenesis, and as therapeutic targets

A Non-Coding RNA Within the Rasgrf1 Locus in Mouse Is Imprinted and Regulated by Its Homologous Chromosome in Trans

BACKGROUND: Rasgrf1 is imprinted in mouse, displaying paternal allele specific expression in neonatal brain. Paternal expression is accompanied by paternal-specific DNA methylation at a differentially methylated domain (DMD) within the locus. The cis-acting elements necessary for Rasgrf1 imprinting are known. A series of tandem DNA repeats control methylation of the adjacent DMD, which is a methylation sensitive enhancer-blocking element. These two sequences constitute a binary switch that controls imprinting and represents the Imprinting Control Region (ICR). One paternally transmitted mutation, which helped define the ICR, induced paramutation, in trans, on the maternal allele. Like many imprinted genes, Rasgrf1 lies within an imprinted cluster. One of four noncoding transcripts in the cluster, AK015891, is known to be imprinted. METHODOLOGY/PRINCIPAL FINDINGS: Here we demonstrate that an additional noncoding RNA, AK029869, is imprinted and paternally expressed in brain throughout development. Intriguingly, any of several maternally inherited ICR mutations affected expression of the paternal AK029869 transcript in trans. Furthermore, we found that the ICR mutations exert different trans effects on AK029869 at different developmental times. CONCLUSIONS/SIGNIFICANCE: Few trans effects have been defined in mammals and, those that exist, do not show the great variation seen at the Rasgrf1 imprinted domain, either in terms of the large number of mutations that produce the effects or the range of phenotypes that emerge when they are seen. These results suggest that trans regulation of gene expression may be more common than originally appreciated and that where trans regulation occurs it can change dynamically during development

Use of an Electrochemical Split Cell Technique to Evaluate the Influence of Shewanella oneidensis Activities on Corrosion of Carbon Steel

Microbially induced corrosion (MIC) is a complex problem that affects various industries. Several techniques have been developed to monitor corrosion and elucidate corrosion mechanisms, including microbiological processes that induce metal deterioration. We used zero resistance ammetry (ZRA) in a split chamber configuration to evaluate the effects of the facultatively anaerobic Fe(III) reducing bacterium Shewanella oneidensis MR-1 on the corrosion of UNS G10180 carbon steel. We show that activities of S. oneidensis inhibit corrosion of steel with which that organism has direct contact. However, when a carbon steel coupon in contact with S. oneidensis was electrically connected to a second coupon that was free of biofilm (in separate chambers of the split chamber assembly), ZRA-based measurements indicated that current moved from the S. oneidensis-containing chamber to the cell-free chamber. This electron transfer enhanced the O2 reduction reaction on the coupon deployed in the cell free chamber, and consequently, enhanced oxidation and corrosion of that electrode. Our results illustrate a novel mechanism for MIC in cases where metal surfaces are heterogeneously covered by biofilms

Gray Matter Changes in Parkinson's and Alzheimer's Disease and Relation to Cognition

Purpose of Review We summarize structural (s)MRI findings of gray matter (GM) atrophy related to cognitive impairment in Alzheimer's disease (AD) and Parkinson's disease (PD) in light of new analytical approaches and recent longitudinal studies results. Recent Findings The hippocampus-to-cortex ratio seems to be the best sMRI biomarker to discriminate between various AD subtypes, following the spatial distribution of tau pathology, and predict rate of cognitive decline. PD is clinically far more variable than AD, with heterogeneous underlying brain pathology. Novel multivariate approaches have been used to describe patterns of early subcortical and cortical changes that relate to more malignant courses of PD. New emerging analytical approaches that combine structural MRI data with clinical and other biomarker outcomes hold promise for detecting specific GM changes in the early stages of PD and preclinical AD that may predict mild cognitive impairment and dementia conversion

Sloan Digital Sky Survey IV: Mapping the Milky Way, Nearby Galaxies, and the Distant Universe

We describe the Sloan Digital Sky Survey IV (SDSS-IV), a project encompassing three major spectroscopic programs. The Apache Point Observatory Galactic Evolution Experiment 2 (APOGEE-2) is observing hundreds of thousands of Milky Way stars at high resolution and high signal-to-noise ratios in the near-infrared. The Mapping Nearby Galaxies at Apache Point Observatory (MaNGA) survey is obtaining spatially resolved spectroscopy for thousands of nearby galaxies (median $z\sim 0.03$). The extended Baryon Oscillation Spectroscopic Survey (eBOSS) is mapping the galaxy, quasar, and neutral gas distributions between $z\sim 0.6$ and 3.5 to constrain cosmology using baryon acoustic oscillations, redshift space distortions, and the shape of the power spectrum. Within eBOSS, we are conducting two major subprograms: the SPectroscopic IDentification of eROSITA Sources (SPIDERS), investigating X-ray AGNs and galaxies in X-ray clusters, and the Time Domain Spectroscopic Survey (TDSS), obtaining spectra of variable sources. All programs use the 2.5 m Sloan Foundation Telescope at the Apache Point Observatory; observations there began in Summer 2014. APOGEE-2 also operates a second near-infrared spectrograph at the 2.5 m du Pont Telescope at Las Campanas Observatory, with observations beginning in early 2017. Observations at both facilities are scheduled to continue through 2020. In keeping with previous SDSS policy, SDSS-IV provides regularly scheduled public data releases; the first one, Data Release 13, was made available in 2016 July

Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research.Peer reviewe

Sloan Digital Sky Survey IV: mapping the Milky Way, nearby galaxies, and the distant universe

We describe the Sloan Digital Sky Survey IV (SDSS-IV), a project encompassing three major spectroscopic programs. The Apache Point Observatory Galactic Evolution Experiment 2 (APOGEE-2) is observing hundreds of thousands of Milky Way stars at high resolution and high signal-to-noise ratios in the near-infrared. The Mapping Nearby Galaxies at Apache Point Observatory (MaNGA) survey is obtaining spatially resolved spectroscopy for thousands of nearby galaxies (median ). The extended Baryon Oscillation Spectroscopic Survey (eBOSS) is mapping the galaxy, quasar, and neutral gas distributions between and 3.5 to constrain cosmology using baryon acoustic oscillations, redshift space distortions, and the shape of the power spectrum. Within eBOSS, we are conducting two major subprograms: the SPectroscopic IDentification of eROSITA Sources (SPIDERS), investigating X-ray AGNs and galaxies in X-ray clusters, and the Time Domain Spectroscopic Survey (TDSS), obtaining spectra of variable sources. All programs use the 2.5 m Sloan Foundation Telescope at the Apache Point Observatory; observations there began in Summer 2014. APOGEE-2 also operates a second near-infrared spectrograph at the 2.5 m du Pont Telescope at Las Campanas Observatory, with observations beginning in early 2017. Observations at both facilities are scheduled to continue through 2020. In keeping with previous SDSS policy, SDSS-IV provides regularly scheduled public data releases; the first one, Data Release 13, was made available in 2016 July