Offline Prioritized Experience Replay

Offline reinforcement learning (RL) is challenged by the distributional shift problem. To address this problem, existing works mainly focus on designing sophisticated policy constraints between the learned policy and the behavior policy. However, these constraints are applied equally to well-performing and inferior actions through uniform sampling, which might negatively affect the learned policy. To alleviate this issue, we propose Offline Prioritized Experience Replay (OPER), featuring a class of priority functions designed to prioritize highly-rewarding transitions, making them more frequently visited during training. Through theoretical analysis, we show that this class of priority functions induce an improved behavior policy, and when constrained to this improved policy, a policy-constrained offline RL algorithm is likely to yield a better solution. We develop two practical strategies to obtain priority weights by estimating advantages based on a fitted value network (OPER-A) or utilizing trajectory returns (OPER-R) for quick computation. OPER is a plug-and-play component for offline RL algorithms. As case studies, we evaluate OPER on five different algorithms, including BC, TD3+BC, Onestep RL, CQL, and IQL. Extensive experiments demonstrate that both OPER-A and OPER-R significantly improve the performance for all baseline methods. Codes and priority weights are availiable at https://github.com/sail-sg/OPER.Comment: preprin

Templated formation of porous Mn2O3 octahedra from Mn-MIL-100 for lithium-ion battery anode materials

Octahedral Mn-MIL-100 metal-organic frameworks (MOFs) are first synthesized, which are then used as templates to fabricate the porous Mn2O3 octahedra through a post-calcination strategy. The morphologies and crystalline structures of as-prepared Mn2O3 octahedra are performed by using field-emission scanning electron microscopy (FESEM), X-ray diffraction (XRD) and transmission electron microscopy (TEM). A reversible lithium storage capacity as high as 755 mA h/g at 0.2 C after 100 cycles is measured from Lithium-ion batteries (LIBs) where the porous Mn2O3 octahedra are acted as anode. Such a high performance indicates that the porous Mn2O3 structure is an excellent anode candidate of LIBs with high capacity and long-life cycling stability.MOE (Min. of Education, S’pore

Organophotoredox-catalyzed intermolecular formal Grob fragmentation of cyclic alcohols with activated allylic acetates

We have developed an efficient method that employs organophotoredox-catalyzed relay Grob fragmentation to facilitate the smooth ring-opening allylation of cyclic alcohols in an environmentally friendly manner. This protocol directly incorporates a wide spectrum of cyclic alcohols and activated allylic acetates into the cross-coupling reaction, eliminating the need for metal catalysts. The process yields a variety of distally unsaturated ketones with good to excellent outcomes and stereoselectivity, while acetic acid is the sole byproduct.We gratefully acknowledge financial support from the National Natural Science Foundation of China (21702108), the Natural Science Foundation of Jiangsu Province, China (BK20211257), and the Six Talent Peaks Project in Jiangsu Province (YY-033)

Secondary Metabolites from <i>Dendrobium nobile</i> and Their Activities Induce Metabolites Apoptosis in OSC-19 Cells

To identify potential drug candidates, secondary metabolites of Dendrobium nobile were performed. As a result, two previously undescribed phenanthrene derivatives with a spirolactone ring (1 and 2), along with four known compounds, N-trans-cinnamoyltyramine (3), N-trans-p-coumaroyltyramine (4), N-trans-feruloyltyramine (5), and moscatilin (6), were isolated from Dendrobium nobile. The structures of the undescribed compounds were elucidated using NMR spectroscopy, electronic circular dichroism (ECD) calculations, and extensive spectroscopic data analysis. The cytotoxic effects of compounds on human tongue squamous cells OSC-19 were determined using MTT at concentrations of 2.5 μM, 5 μM, 10 μM, and 20 μM. Compound 6 exhibited potent inhibitory activity against OSC-19 cells with an IC50 of 1.32 μM. Migration assays and western blot assays demonstrated that compound 6 effectively inhibited migration by down-regulating MMP2 and MMP9 at concentrations of 0.5 μM and 1 μM. To investigate its effect on apoptosis, we performed AO/PI staining, flow cytometry, and WB experiments. The results showed that increasing concentrations led to increased red fluorescence, decreased green fluorescence, increased apoptosis rate, decreased expression of bcl-2, caspase 3, caspase 9, and parp proteins, and increased bax expression. Furthermore, the phosphorylation of JNK and P38 was activated, suggesting that compound 6 may induce apoptosis via the MAPK pathway

Chemical constituents from Tsuga longibracteata and its chemotaxonomic study

Phytochemical investigation into the bark of Tsuga longibracteata led to the isolation of three lignans (1-3), two triterpenoids (4, 5), nine flavonoids (6-14), three sterols (15-17) and five phenolic acids (18-22). Their structures were determined by spectroscopic analysis of their spectral data with those reported in literature. Among them, compounds 1-3, 6-22 were obtained from this species for the first time. The chemotaxonomic significance of the isolated compounds is discussed. (C) 2017 Elsevier Ltd. All rights reserved

Callus growth kinetics and accumulation of secondary metabolites of Bletilla striata Rchb.f. using a callus suspension culture.

Bletilla striata is an endangered traditional Chinese medicinal plant with multiple uses and a slow regeneration rate of its germplasm resources. To evaluate the callus growth kinetics and accumulation of secondary metabolites (SMs), a callus suspension culture was proven to be a valuable approach for acquiring high yields of medicinal compounds. An effective callus suspension culture for obtaining B. striata callus growth and its SMs was achieved with the in vitro induction of calluses from B. striata seeds. The callus growth kinetics and accumulation of SMs were analyzed using a mathematical model. The resulting callus growth kinetic model revealed that the growth curves of B. striata suspension-cultured calluses were sigmoidal, indicating changes in the growth of the suspension-cultured calluses. Improved Murashige and Skoog callus growth medium was the most favorable medium for B. striata callus formation, with the highest callus growth occurring during the stationary phase of the cultivation period. Callus growth acceleration started after 7 days and thereafter gradually decreased until day 24 of the cultivation period and reached its highest at day 36 period in both the dry weight and fresh weight analyses. The coelonin concentration peaked during the exponential growth stage and decreased afterward during the stationary stage of the callus suspension culture. The maximum content of coelonin (approximately 0.3323 mg/g callus dry weight) was observed on the 18th day of the cultivation cycle, while dactylorhin A and militarine reached the highest concentrations at day 24, and p-hydroxybenzyl alcohol at day 39. This investigation also laid a foundation for a multimathematical model to better describe the accumulation variation of SMs. The production of SMs showed great specificity during callus growth and development. This research provided a well-organized way to increase the accumulation and production of SMs during the scaled-up biosynthesis of calluses in B. striata callus suspension cultures

Total biflavonoids extraction from Selaginella chaetoloma utilizing ultrasound-assisted deep eutectic solvent: Optimization of conditions, extraction mechanism, and biological activity in vitro

In this study, the deep eutectic solvent based ultrasound-assisted extraction (DES-UAE) was investigated for the efficient and environmentally friendly extraction of Selaginella chaetoloma total biflavonoids (SCTB). As an extractant for optimization, tetrapropylaminium bromide-1,4-butanediol (Tpr-But) was employed for the first time. 36 DESs were created, with Tpr-But producing the most effective results. Based on response surface methodology (RSM), the greatest extraction rate of SCTB was determined to be 21.68 ± 0.78 mg/g, the molar ratio of HBD to HBA was 3.70:1, the extraction temperature was 57 °C, and the water content of DES was 22 %. In accordance with Fick's second rule, a kinetic model for the extraction of SCTB by DES-UAE has been derived. With correlation coefficients 0.91, the kinetic model of the extraction process was significantly correlated with the general and exponential equations of kinetics, and some important kinetic parameters such as rate constants, energy of activation and raffinate rate were determined. In addition, molecular dynamics simulations were used to study the extraction mechanisms generated by different solvents. Comparing the effect of several extraction methods on S.chaetoloma using ultrasound-assisted extraction and conventional methods, together with SEM examination, revealed that DES-UAE not only saved time but also enhanced SCTB extraction rate by 1.5–3 folds. SCTB demonstrated superior antioxidant activity in three studies in vitro. Furthermore, the extract could suppress the growth of A549, HCT-116, HepG2, and HT-29 cancer cells. Alpha-Glucosidase (AG) inhibition experiment and molecular docking studies suggested that SCTB exhibited strong inhibitory activity against AG and potential hypoglycemic effects. The results of this study indicated that a Tpr-But-based UAE method was suitable for the efficient and environmentally friendly extraction of SCTB, and also shed light on the mechanisms responsible for the increased extraction efficiency, which could aid in the application of S.chaetoloma and provide insight into the extraction mechanism of DES