ED2: Environment Dynamics Decomposition World Models for Continuous Control

Model-based reinforcement learning (MBRL) achieves significant sample efficiency in practice in comparison to model-free RL, but its performance is often limited by the existence of model prediction error. To reduce the model error, standard MBRL approaches train a single well-designed network to fit the entire environment dynamics, but this wastes rich information on multiple sub-dynamics which can be modeled separately, allowing us to construct the world model more accurately. In this paper, we propose the Environment Dynamics Decomposition (ED2), a novel world model construction framework that models the environment in a decomposing manner. ED2 contains two key components: sub-dynamics discovery (SD2) and dynamics decomposition prediction (D2P). SD2 discovers the sub-dynamics in an environment automatically and then D2P constructs the decomposed world model following the sub-dynamics. ED2 can be easily combined with existing MBRL algorithms and empirical results show that ED2 significantly reduces the model error, increases the sample efficiency, and achieves higher asymptotic performance when combined with the state-of-the-art MBRL algorithms on various continuous control tasks. Our code is open source and available at https://github.com/ED2-source-code/ED2.Comment: 10 pages, 13 figure

Overexpression of SIRT1 in Mouse Forebrain Impairs Lipid/Glucose Metabolism and Motor Function

SIRT1 plays crucial roles in glucose and lipid metabolism, and has various functions in different tissues including brain. The brain-specific SIRT1 knockout mice display defects in somatotropic signaling, memory and synaptic plasticity. And the female mice without SIRT1 in POMC neuron are more sensitive to diet-induced obesity. Here we created transgenic mice overexpressing SIRT1 in striatum and hippocampus under the control of CaMKIIα promoter. These mice, especially females, exhibited increased fat accumulation accompanied by significant upregulation of adipogenic genes in white adipose tissue. Glucose tolerance of the mice was also impaired with decreased Glut4 mRNA levels in muscle. Moreover, the SIRT1 overexpressing mice showed decreased energy expenditure, and concomitantly mitochondria-related genes were decreased in muscle. In addition, these mice showed unusual spontaneous physical activity pattern, decreased activity in open field and rotarod performance. Further studies demonstrated that SIRT1 deacetylated IRS-2, and upregulated phosphorylation level of IRS-2 and ERK1/2 in striatum. Meanwhile, the neurotransmitter signaling in striatum and the expression of endocrine hormones in hypothalamus and serum T3, T4 levels were altered. Taken together, our findings demonstrate that SIRT1 in forebrain regulates lipid/glucose metabolism and motor function

AlignDiff: Aligning Diverse Human Preferences via Behavior-Customisable Diffusion Model

Aligning agent behaviors with diverse human preferences remains a challenging problem in reinforcement learning (RL), owing to the inherent abstractness and mutability of human preferences. To address these issues, we propose AlignDiff, a novel framework that leverages RL from Human Feedback (RLHF) to quantify human preferences, covering abstractness, and utilizes them to guide diffusion planning for zero-shot behavior customizing, covering mutability. AlignDiff can accurately match user-customized behaviors and efficiently switch from one to another. To build the framework, we first establish the multi-perspective human feedback datasets, which contain comparisons for the attributes of diverse behaviors, and then train an attribute strength model to predict quantified relative strengths. After relabeling behavioral datasets with relative strengths, we proceed to train an attribute-conditioned diffusion model, which serves as a planner with the attribute strength model as a director for preference aligning at the inference phase. We evaluate AlignDiff on various locomotion tasks and demonstrate its superior performance on preference matching, switching, and covering compared to other baselines. Its capability of completing unseen downstream tasks under human instructions also showcases the promising potential for human-AI collaboration. More visualization videos are released on https://aligndiff.github.io/

Modeling of CO2 absorption into 4-diethylamino-2-butanol solution in a membrane contactor under wetting or non-wetting conditions

© 2022 The Authors. Published by Elsevier. This is an open access article available under a Creative Commons licence. The published version can be accessed at the following link on the publisher’s website: https://doi.org/10.1016/j.ccst.2022.100069In this work, 4-diethylamino-2-butanol (DEAB) as a new type of alkanolamine solvent is used for CO2 capture in a hollow fiber membrane contactor (HFMC). A model describing the gas and liquid reactions and transport inside the membrane contactor under the wetting or non-wetting conditions was built. The countercurrent flow of natural gas and solvent was considered in the model. To investigate the influence of solvent type on decarburization efficiency, DEAB was used and compared with other common solvents such as potassium carbonate (K2CO3), triethylamine (TEA) and diethanolamine (DEA). Under the same operating conditions, the impact of parameters such as humidity, gas flow rate, liquid concentration, membrane length on the decarburization performance was examined. The results indicate that DEAB solvent has the best overall performance especially under the wetting conditions. It was noted that increasing liquid concentration, membrane length and decreasing gas flow rate enhance decarburization.Published versio

Morphology tailoring and temperature sensitivity control of waist cross-linked micelles and evaluation of their application as intelligent drug carriers

A novel type of waist cross-linked micelle (WCM) was developed as an intelligent drug carrier via the self-assembly guiding free radical polymerization of an amphiphilic oligomer: octadecyl, polyethylene glycol butenedioates (O-B-EGs). By changing the concentration of O-B-EG reaction solution, WCMs with monolayer, compound and vesicle-like morphologies were obtained. These WCMs showed controllable temperature responsive properties. DLS and UV-vis analyses indicate that the critical temperatures at which WCMs show an abrupt change in particle size evidently increases with the increase in the molecular weight of the PEG chains. Direct switching of the release of pyrene in WCMs is also realized by a slight change of temperature. Pyrene is released rapidly at the temperatures around the critical temperature of the WCMs, but a further increase in temperature shuts down the release of pyrene. More importantly, these WCMs exhibit reversible and rapid pyrene releasing-absorbing behavior. We suggest that these excellent properties endow WCMs with great potential in drug encapsulation and controlled releaseNational Natural Science Foundation of China [50873082, 30700020]; Research Fund for the Doctoral Program of Higher Education [20070384047]; Scientific and Technical Project of Fujian Province of China [2009J1009

Clinical characterization and proteomic profiling of lean nonalcoholic fatty liver disease

IntroductionObesity has been historically associated with nonalcoholic fatty liver disease (NAFLD), but it can also occur in lean individuals. However, limited data is available on this special group. To investigate the clinical and proteomic characteristics of lean subjects with NAFLD, and to identify potential clinical variables and plasma proteins for diagnosing NAFLD in lean individuals, we collected clinical data from a large cohort of 2,236 subjects.MethodsDiagnosis of NAFLD relied on detecting pronounced hepatic steatosis through abdominal ultrasonography. Participants were categorized into four groups based on body mass index: overweight NAFLD, overweight control, lean NAFLD, and lean control. Plasma proteomic profiling was performed on samples from 20 subjects in each group. The lean NAFLD group was compared to both lean healthy and obese NAFLD groups across all data.Results and discussionThe results indicated that the lean NAFLD group exhibited intermediate metabolic profiles, falling between those of the lean healthy and overweight NAFLD groups. Proteomic profiling of plasma in lean subjects with or without NAFLD revealed 45 statistically significant changes in proteins, of which 37 showed high diagnostic value (AUC > 0.7) for lean NAFLD. These potential biomarkers primarily involved lipid metabolism, the immune and complement systems, and platelet degranulation. Furthermore, AFM, GSN, CFH, HGFAC, MMP2, and MMP9 have been previously associated with NAFLD or NAFLD-related factors such as liver damage, insulin resistance, metabolic syndromes, and extracellular homeostasis. Overall, lean individuals with NAFLD exhibit distinct clinical profiles compared to overweight individuals with NAFLD. Despite having worse metabolic profiles than their healthy counterparts, lean NAFLD patients generally experience milder systemic metabolic disturbances compared to obese NAFLD patients. Additionally, the plasma proteomic profile is significantly altered in lean NAFLD, highlighting the potential of differentially expressed proteins as valuable biomarkers or therapeutic targets for diagnosing and treating NAFLD in this population

Urbanization affects spatial variation and species similarity of bird diversity distribution

Although cities are human-dominated systems, they provide habitat for many other species. Because of the lack of long-term observation data, it is challenging to assess the impacts of rapid urbanization on biodiversity in Global South countries. Using multisource data, we provided the first analysis of the impacts of urbanization on bird distribution at the continental scale and found that the distributional hot spots of threatened birds overlapped greatly with urbanized areas, with only 3.90% of the threatened birds’ preferred land cover type in urban built-up areas. Bird ranges are being reshaped differently because of their different adaptations to urbanization. While green infrastructure can improve local bird diversity, the homogeneous urban environment also leads to species compositions being more similar across regions. More attention should be paid to narrow-range species for the formulation of biodiversity conservation strategies, and conservation actions should be further coordinated among cities from a global perspective

Heterogeneous Impact of Economic Policy Uncertainty on Provincial Environmental Pollution Emissions in China

With China’s proposal of carbon peak and carbon neutral goals, its trend of economic development has shifted from pursuing high-speed economic development to high-quality development. However, for the past few years, with the increasing global economic policy uncertainty, fluctuations in the world economy, especially emergent through public events such as COVID-19, affect investment and consumption, and thus indirectly affect the realization of the dual carbon target. Economic policy uncertainty plays an increasingly important role in many factors affecting environmental pollution. We conducted an empirical test on sample data, which are from 30 provinces and autonomous regions in China from 2008 to 2020, to further study the impact of economic policy uncertainty on environmental pollution emissions. We found that: (1) Economic policy uncertainty is inversely related to the emission of environmental pollution, and the consumption effect brought by economic policy uncertainty is more than the investment effect. This means that, with the economic policy uncertainty index increasing, the comprehensive index of environmental pollution emissions is lower, and the environmental pollution emissions are lower; (2) Compared with provinces with an average level of economic development, the impact of economic policy uncertainty on environmental emissions is deeper in developed provinces