Look Beneath the Surface: Exploiting Fundamental Symmetry for Sample-Efficient Offline RL

Offline reinforcement learning (RL) offers an appealing approach to real-world tasks by learning policies from pre-collected datasets without interacting with the environment. However, the performance of existing offline RL algorithms heavily depends on the scale and state-action space coverage of datasets. Real-world data collection is often expensive and uncontrollable, leading to small and narrowly covered datasets and posing significant challenges for practical deployments of offline RL. In this paper, we provide a new insight that leveraging the fundamental symmetry of system dynamics can substantially enhance offline RL performance under small datasets. Specifically, we propose a Time-reversal symmetry (T-symmetry) enforced Dynamics Model (TDM), which establishes consistency between a pair of forward and reverse latent dynamics. TDM provides both well-behaved representations for small datasets and a new reliability measure for OOD samples based on compliance with the T-symmetry. These can be readily used to construct a new offline RL algorithm (TSRL) with less conservative policy constraints and a reliable latent space data augmentation procedure. Based on extensive experiments, we find TSRL achieves great performance on small benchmark datasets with as few as 1% of the original samples, which significantly outperforms the recent offline RL algorithms in terms of data efficiency and generalizability.Comment: The first two authors contributed equall

Soyasaponin Ag inhibits triple-negative breast cancer progression via targeting the DUSP6/MAPK signaling

Introduction. Soyasaponins are triterpenoid glycosides discovered in soybean and have anti-cancer properties. Soyasaponin A was reported to repress estrogen-insensitive breast cancer cell proliferation. This study intends to explore the role of one isomer of soyasaponin A, i.e. soyasaponin Ag (Ssa Ag), in triple-negative breast cancer (TNBC) development. Material and methods. Bioinformatic databases were used to predict DUSP6 expression in breast cancer (BC) as well as the correlation between the expression of DUSP6 (or MAPK1, MAPK14) with the prognosis of patients with BC. The expression of DUSP6/MAPK signaling-related genes (DUSP6, MAPK1, and MAPK14) in TNBC cell lines was assessed via Western blot analysis and RT-qPCR. Levels of cell apoptosis proteins (Bax and Bcl-2) in TNBC cells were assessed via Western blot analysis. CCK-8 assay, colony formation assay, and flow cytometry analysis were conducted for the measurement of TNBC cell growth and apoptosis. In vivo xenograft assay was employed for investigating the biological influence of Ssa Ag on tumor growth. Results. The poor prognosis of BC patients was linked to the aberrant expression of DUSP6/MAPK pathwaygenes. Low expression of DUSP6 or high expression of MAPK1 (or MAPK14) was correlated to poor prognosis. DUSP6 was downregulated while MAPK1 and MAPK14 were upregulated in TNBC cells versus normal cells. Ssa Ag upregulated DUSP6 expression while downregulated MAPK1 and MAPK14 expression, inhibiting the MAPK signaling pathway. Additionally, Ssa Ag promoted in vitro TNBC cell apoptosis and restrained cell growth, and repressed in vivo tumor growth. Conclusions. Ssa Ag inhibited TNBC progression via upregulating DUSP6 and inactivating the MAPK signaling pathway

Effect of carboxymethyl cellulose and gibberellic acid-enriched biochar on osmotic stress tolerance in cotton

Abstract The deleterious impact of osmotic stress, induced by water deficit in arid and semi-arid regions, poses a formidable challenge to cotton production. To protect cotton farming in dry areas, it’s crucial to create strong plans to increase soil water and reduce stress on plants. The carboxymethyl cellulose (CMC), gibberellic acid (GA3) and biochar (BC) are individually found effective in mitigating osmotic stress. However, combine effect of CMC and GA3 with biochar on drought mitigation is still not studied in depth. The present study was carried out using a combination of GA3 and CMC with BC as amendments on cotton plants subjected to osmotic stress levels of 70 (70 OS) and 40 (40 OS). There were five treatment groups, namely: control (0% CMC-BC and 0% GA3-BC), 0.4%CMC-BC, 0.4%GA3-BC, 0.8%CMC-BC, and 0.8%GA3-BC. Each treatment was replicated five times with a completely randomized design (CRD). The results revealed that 0.8 GA3-BC led to increase in cotton shoot fresh weight (99.95%), shoot dry weight (95.70%), root fresh weight (73.13%), and root dry weight (95.74%) compared to the control group under osmotic stress. There was a significant enhancement in cotton chlorophyll a (23.77%), chlorophyll b (70.44%), and total chlorophyll (35.44%), the photosynthetic rate (90.77%), transpiration rate (174.44%), and internal CO2 concentration (57.99%) compared to the control group under the 40 OS stress. Thus 0.8GA3-BC can be potential amendment for reducing osmotic stress in cotton cultivation, enhancing agricultural resilience and productivity