Unsupervised Behavior Extraction via Random Intent Priors

Reward-free data is abundant and contains rich prior knowledge of human behaviors, but it is not well exploited by offline reinforcement learning (RL) algorithms. In this paper, we propose UBER, an unsupervised approach to extract useful behaviors from offline reward-free datasets via diversified rewards. UBER assigns different pseudo-rewards sampled from a given prior distribution to different agents to extract a diverse set of behaviors, and reuse them as candidate policies to facilitate the learning of new tasks. Perhaps surprisingly, we show that rewards generated from random neural networks are sufficient to extract diverse and useful behaviors, some even close to expert ones. We provide both empirical and theoretical evidence to justify the use of random priors for the reward function. Experiments on multiple benchmarks showcase UBER's ability to learn effective and diverse behavior sets that enhance sample efficiency for online RL, outperforming existing baselines. By reducing reliance on human supervision, UBER broadens the applicability of RL to real-world scenarios with abundant reward-free data.Comment: Thirty-seventh Conference on Neural Information Processing System

The Neurobase of ambiguity loss aversion about decision making

In our daily decision-making, there are two confusing problems: risk and ambiguity. Many psychological studies and neuroscience studies have shown that the prefrontal cortex (PFC) is an important neural mechanism for modulating the human brain in risk and ambiguity decision-making, especially the dorsolateral prefrontal cortex (DLPFC). We used transcranial direct current stimulation (tDCS) to reveal the causal relationship between the DLPFC and ambiguity decision-making. We design two experimental tasks involving ambiguity to gain and ambiguity to loss. The results of our study show that there is a significant effect on left DLPFC stimulation about ambiguity to loss, there is an insignificant effect on left DLPFC stimulation about ambiguity to gain, and there is an insignificant effect on right DLPFC stimulation about ambiguity to gain and ambiguity to loss. This result indicates that people are more sensitive to ambiguity loss than ambiguity gain. Further analysis found that the degree of participants’ attitudes toward ambiguity loss who received anodal simulation was lower than that who received sham stimulation across the left DLPFC, which means that the subjects had a strong ambiguity loss aversion after the participants received the anodal simulation of the left DLPFC

A genome-wide association study identifies GRK5 and RASGRP1 as type 2 diabetes loci in Chinese Hans.

Substantial progress has been made in identification of type 2 diabetes (T2D) risk loci in the past few years, but our understanding of the genetic basis of T2D in ethnically diverse populations remains limited. We performed a genome-wide association study and a replication study in Chinese Hans comprising 8,569 T2D case subjects and 8,923 control subjects in total, from which 10 single nucleotide polymorphisms were selected for further follow-up in a de novo replication sample of 3,410 T2D case and 3,412 control subjects and an in silico replication sample of 6,952 T2D case and 11,865 control subjects. Besides confirming seven established T2D loci (CDKAL1, CDKN2A/B, KCNQ1, CDC123, GLIS3, HNF1B, and DUSP9) at genome-wide significance, we identified two novel T2D loci, including G-protein-coupled receptor kinase 5 (GRK5) (rs10886471: P = 7.1 × 10(-9)) and RASGRP1 (rs7403531: P = 3.9 × 10(-9)), of which the association signal at GRK5 seems to be specific to East Asians. In nondiabetic individuals, the T2D risk-increasing allele of RASGRP1-rs7403531 was also associated with higher HbA(1c) and lower homeostasis model assessment of β-cell function (P = 0.03 and 0.0209, respectively), whereas the T2D risk-increasing allele of GRK5-rs10886471 was also associated with higher fasting insulin (P = 0.0169) but not with fasting glucose. Our findings not only provide new insights into the pathophysiology of T2D, but may also shed light on the ethnic differences in T2D susceptibility

Theoretical calculation of cesium deposition and co-deposition with electronegative elements on the plasma grid in negative ion sources

We studied the work function of cesium deposition and co-deposition with the electronegative element on the plasma grid (PG) using the first-principles calculations. The impurity particles may exist in the background plasma and vacuum chamber wall, and the work function of the PG will be affected. The results indicate that the minimum work functions of pure cesium deposition on Mo (110), W (110), and Mo (112) are reached at a partial monolayer. They are 1.66 eV (σ = 0.56 θ), 1.69 eV (σ = 0.75 θ), and 1.75 eV (σ = 0.88 θ), respectively. An appropriate co-deposition model consisting of cesium with electronegative elements can further decrease the work function. The coverage of cesium and electronegative elements are both 0.34 θ in all the co-deposition models. The F-Cs co-deposition model where the Cs atom and F atom are aligned along the surface normal obtains the lowest work function. They are 1.31 eV for F-Cs on Mo (110), and 1.23 eV for F-Cs on W (110), respectively. The change in work function is linearly related to the change in dipole moment density with a slope of −167.03 VÅ. For pure cesium deposition, two factors control the change in dipole-moment density, one is the electron transfer between adsorbates and the substrate, and another one is the restructuring of surface atoms. There are two additional factors for the co-deposition model. One is the intrinsic dipole moment of the double layer, the other is the angle between the intrinsic dipole moment and the surface. The latter two factors play important roles in increasing the total dipole moment

A Genome Wide Association Study Identifies Common Variants Associated with Lipid Levels in the Chinese Population

Plasma lipid levels are important risk factors for cardiovascular disease and are influenced by genetic and environmental factors. Recent genome wide association studies (GWAS) have identified several lipid-associated loci, but these loci have been identified primarily in European populations. In order to identify genetic markers for lipid levels in a Chinese population and analyze the heterogeneity between Europeans and Asians, especially Chinese, we performed a meta-analysis of two genome wide association studies on four common lipid traits including total cholesterol (TC), triglycerides (TG), low-density lipoprotein cholesterol (LDL) and high-density lipoprotein cholesterol (HDL) in a Han Chinese population totaling 3,451 healthy subjects. Replication was performed in an additional 8,830 subjects of Han Chinese ethnicity. We replicated eight loci associated with lipid levels previously reported in a European population. The loci genome wide significantly associated with TC were near DOCK7, HMGCR and ABO; those genome wide significantly associated with TG were near APOA1/C3/A4/A5 and LPL; those genome wide significantly associated with LDL were near HMGCR, ABO and TOMM40; and those genome wide significantly associated with HDL were near LPL, LIPC and CETP. In addition, an additive genotype score of eight SNPs representing the eight loci that were found to be associated with lipid levels was associated with higher TC, TG and LDL levels (P = 5.52×10-16, 1.38×10-6 and 5.59×10-9, respectively). These findings suggest the cumulative effects of multiple genetic loci on plasma lipid levels. Comparisons with previous GWAS of lipids highlight heterogeneity in allele frequency and in effect size for some loci between Chinese and European populations. The results from our GWAS provided comprehensive and convincing evidence of the genetic determinants of plasma lipid levels in a Chinese population

A Rice Plastidial Nucleotide Sugar Epimerase Is Involved in Galactolipid Biosynthesis and Improves Photosynthetic Efficiency

Photosynthesis is the final determinator for crop yield. To gain insight into genes controlling photosynthetic capacity, we selected from our large T-DNA mutant population a rice stunted growth mutant with decreased carbon assimilate and yield production named photoassimilate defective1 (phd1). Molecular and biochemical analyses revealed that PHD1 encodes a novel chloroplast-localized UDP-glucose epimerase (UGE), which is conserved in the plant kingdom. The chloroplast localization of PHD1 was confirmed by immunoblots, immunocytochemistry, and UGE activity in isolated chloroplasts, which was approximately 50% lower in the phd1-1 mutant than in the wild type. In addition, the amounts of UDP-glucose and UDP-galactose substrates in chloroplasts were significantly higher and lower, respectively, indicating that PHD1 was responsible for a major part of UGE activity in plastids. The relative amount of monogalactosyldiacylglycerol (MGDG), a major chloroplast membrane galactolipid, was decreased in the mutant, while the digalactosyldiacylglycerol (DGDG) amount was not significantly altered, suggesting that PHD1 participates mainly in UDP-galactose supply for MGDG biosynthesis in chloroplasts. The phd1 mutant showed decreased chlorophyll content, photosynthetic activity, and altered chloroplast ultrastructure, suggesting that a correct amount of galactoglycerolipids and the ratio of glycolipids versus phospholipids are necessary for proper chloroplast function. Downregulated expression of starch biosynthesis genes and upregulated expression of sucrose cleavage genes might be a result of reduced photosynthetic activity and account for the decreased starch and sucrose levels seen in phd1 leaves. PHD1 overexpression increased photosynthetic efficiency, biomass, and grain production, suggesting that PHD1 plays an important role in supplying sufficient galactolipids to thylakoid membranes for proper chloroplast biogenesis and photosynthetic activity. These findings will be useful for improving crop yields and for bioenergy crop engineering

Impact of Climate Change on Agricultural Total Factor Productivity Based on Spatial Panel Data Model: Evidence from China

To respond to the adverse impact of climate change on agricultural total factor productivity, the question of how to adopt actively appropriate strategies is particularly critical for the stakeholders. However, the previous researchers have paid more attention to investigating the measure methods, regional differences, and determinants of Chinese agricultural total factor productivity, but the possible impact of climate change factors like rainfall, temperature, and evaporation on regional agricultural total factor productivity in China have not yet received the attention that they deserve. Furthermore, more importantly, the study on how to take active measures to reduce and mitigate the negative effects from climate change is relatively small. Therefore, in allusion to the above-mentioned problems, using the data envelopment analysis and building a spatial panel data model embedded with climate change factors, this paper calculated Chinese agricultural total factor productivity and then explored the possible impact of climate change on regional agricultural total factor productivity at a provincial level in China. Results mainly show that the impact of some factors, like annual total precipitation, average temperature in the growing season, and evaporation intensity on regional agricultural total factor productivity, are all very significant and negative, which suggests that the more precipitation, the higher the temperature is, and the higher evaporation intensity would lower agricultural total factor productivity in China. Furthermore, in order to response to mitigate the adverse effects from climate change on agricultural total factor productivity, local governments should continue to increase financial support for the local agricultural economic development, because this action could be beneficial for the related stakeholders in improving agricultural total factor productivity. Summing up, our evidence study would provide an important basic theory basis in terms of increasing agricultural total factor productivity and promoting regional agricultural economic development in China