Goal-Conditioned Reinforcement Learning with Disentanglement-based Reachability Planning

Goal-Conditioned Reinforcement Learning (GCRL) can enable agents to spontaneously set diverse goals to learn a set of skills. Despite the excellent works proposed in various fields, reaching distant goals in temporally extended tasks remains a challenge for GCRL. Current works tackled this problem by leveraging planning algorithms to plan intermediate subgoals to augment GCRL. Their methods need two crucial requirements: (i) a state representation space to search valid subgoals, and (ii) a distance function to measure the reachability of subgoals. However, they struggle to scale to high-dimensional state space due to their non-compact representations. Moreover, they cannot collect high-quality training data through standard GC policies, which results in an inaccurate distance function. Both affect the efficiency and performance of planning and policy learning. In the paper, we propose a goal-conditioned RL algorithm combined with Disentanglement-based Reachability Planning (REPlan) to solve temporally extended tasks. In REPlan, a Disentangled Representation Module (DRM) is proposed to learn compact representations which disentangle robot poses and object positions from high-dimensional observations in a self-supervised manner. A simple REachability discrimination Module (REM) is also designed to determine the temporal distance of subgoals. Moreover, REM computes intrinsic bonuses to encourage the collection of novel states for training. We evaluate our REPlan in three vision-based simulation tasks and one real-world task. The experiments demonstrate that our REPlan significantly outperforms the prior state-of-the-art methods in solving temporally extended tasks.Comment: Accepted by 2023 RAL with ICR

Tumor Tissue-Derived Formaldehyde and Acidic Microenvironment Synergistically Induce Bone Cancer Pain

Background: There is current interest in understanding the molecular mechanisms of tumor-induced bone pain. Accumulated evidence shows that endogenous formaldehyde concentrations are elevated in the blood or urine of patients with breast, prostate or bladder cancer. These cancers are frequently associated with cancer pain especially after bone metastasis. It is well known that transient receptor potential vanilloid receptor 1 (TRPV1) participates in cancer pain. The present study aims to demonstrate that the tumor tissue-derived endogenous formaldehyde induces bone cancer pain via TRPV1 activation under tumor acidic environment. Methodology/Principal Findings: Endogenous formaldehyde concentration increased significantly in the cultured breast cancer cell lines in vitro, in the bone marrow of breast MRMT-1 bone cancer pain model in rats and in tissues from breast cancer and lung cancer patients in vivo. Low concentrations (1 similar to 5 mM) of formaldehyde induced pain responses in rat via TRPV1 and this pain response could be significantly enhanced by pH 6.0 (mimicking the acidic tumor microenvironment). Formaldehyde at low concentrations (1 mM to 100 mM) induced a concentration-dependent increase of [Ca(2+)]i in the freshly isolated rat dorsal root ganglion neurons and TRPV1-transfected CHO cells. Furthermore, electrophysiological experiments showed that low concentration formaldehyde-elicited TRPV1 currents could be significantly potentiated by low pH (6.0). TRPV1 antagonists and formaldehyde scavengers attenuated bone cancer pain responses. Conclusions/Significance: Our data suggest that cancer tissues directly secrete endogenous formaldehyde, and this formaldehyde at low concentration induces metastatic bone cancer pain through TRPV1 activation especially under tumor acidic environment.Multidisciplinary SciencesSCI(E)PubMed24ARTICLE4e10234

A forecast of research octane number of FCC gasoline with changing weight hour space velocity

Research Octane Number (RON) of FCC gasoline has been studied using Lanlian FCC gasoline and Lanlian aromatization products as feedstocks and LBO-A as catalyst, and research octane number have been put forward with changing weight hour space velocity (WHSV). The mathematical model forecasts research octane number of modified FCC gasoline with changing weight hour space velocity. The results from experimental data are in accordance with the quantitative analytical conclusions drawn from the calculated data

A forecast of research octane number of FCC gasoline with changing weight hour space velocity

323-325Research Octane Number (RON) of FCC gasoline has been studied using Lanlian FCC gasoline and Lanlian aromatization products as feedstocks and LBO-A as catalyst, and research octane number have been put forward with changing weight hour space velocity (WHSV). The mathematical model forecasts research octane number of modified FCC gasoline with changing weight hour space velocity. The results from experimental data are in accordance with the quantitative analytical conclusions drawn from the calculated data

sj-pdf-3-imr-10.1177_03000605231213781 - Supplemental material for Prediction of diagnostic gene biomarkers for hypertrophic cardiomyopathy by integrated machine learning

Supplemental material, sj-pdf-3-imr-10.1177_03000605231213781 for Prediction of diagnostic gene biomarkers for hypertrophic cardiomyopathy by integrated machine learning by Hongjun You and Mengya Dong in Journal of International Medical Research</p

sj-pdf-1-imr-10.1177_03000605231213781 - Supplemental material for Prediction of diagnostic gene biomarkers for hypertrophic cardiomyopathy by integrated machine learning

Supplemental material, sj-pdf-1-imr-10.1177_03000605231213781 for Prediction of diagnostic gene biomarkers for hypertrophic cardiomyopathy by integrated machine learning by Hongjun You and Mengya Dong in Journal of International Medical Research</p

sj-pdf-2-imr-10.1177_03000605231213781 - Supplemental material for Prediction of diagnostic gene biomarkers for hypertrophic cardiomyopathy by integrated machine learning

Supplemental material, sj-pdf-2-imr-10.1177_03000605231213781 for Prediction of diagnostic gene biomarkers for hypertrophic cardiomyopathy by integrated machine learning by Hongjun You and Mengya Dong in Journal of International Medical Research</p