H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gaps

Solving real-world complex tasks using reinforcement learning (RL) without high-fidelity simulation environments or large amounts of offline data can be quite challenging. Online RL agents trained in imperfect simulation environments can suffer from severe sim-to-real issues. Offline RL approaches although bypass the need for simulators, often pose demanding requirements on the size and quality of the offline datasets. The recently emerged hybrid offline-and-online RL provides an attractive framework that enables joint use of limited offline data and imperfect simulator for transferable policy learning. In this paper, we develop a new algorithm, called H2O+, which offers great flexibility to bridge various choices of offline and online learning methods, while also accounting for dynamics gaps between the real and simulation environment. Through extensive simulation and real-world robotics experiments, we demonstrate superior performance and flexibility over advanced cross-domain online and offline RL algorithms

Constrained Nuclear-Electronic Orbital QM/MM Approach for Simulating Complex Systems with Quantum Nuclear Delocalization Effects Incorporated

The hybrid quantum mechanics/molecular mechanics (QM/MM) approach, which combines the accuracy of quantum mechanical (QM) methods with the efficiency of molecular mechanics (MM) methods, is widely used in the study of complex systems. However, past QM/MM implementations often neglect or face challenges in addressing nuclear quantum effects, despite their crucial role in many key chemical and biological processes. Recently, our group developed the constrained nuclear-electronic orbital (CNEO) theory, a cost-efficient approach that accurately addresses nuclear quantum effects, especially quantum nuclear delocalization effects. In this work, we integrate CNEO with QM/MM methods through the electrostatic embedding scheme and apply the resulting CNEO QM/MM to two hydrogen-bonded complexes in both gas and aqueous phases. We find that both solvation effects and nuclear quantum effects significantly impact hydrogen bond structures and dynamics. Notably, in the glutamic acid - glutamate complex, which mimics a low barrier hydrogen bond in human transketolase, CNEO QM/MM accurately predicts nearly equal proton sharing between the two residues, with predicted oxygen-hydrogen distance in excellent agreement with experimental results. With an accurate description of both nuclear quantum effects and environmental effects, CNEO QM/MM is a promising new approach for simulating complex chemical and biological systems

Visible-Light-Induced Living Radical Polymerization (LRP) Mediated by (salen)Co(II)/TPO at Ambient Temperature

Visible-light-induced living radical polymerization of acrylates (MA, <i>n</i>BA, <i>t</i>BA), acrylamides (DMA, AMO), and vinyl acetate (VAc) at ambient temperature mediated by (salen)­Co­(II)/TPO was described. Effects of light intensity, feeding ratio of monomer and equivalent of TPO for the polymerization of MA were investigated. Well-defined homopolymers and block polymers with predetermined molecular weight and narrow polydispersity were obtained under mild conditions. The mechanism of the polymerization was proposed based on polymerization behavior and polymer structure analysis. The (salen)­Co­(II)/TPO system was suitable for both conjugated and unconjugated monomers under mild conditions

Lipopolysaccharide Induces Lung Fibroblast Proliferation through Toll-Like Receptor 4 Signaling and the Phosphoinositide3-Kinase-Akt Pathway

Pulmonary fibrosis is characterized by lung fibroblast proliferation and collagen secretion. In lipopolysaccharide (LPS)induced acute lung injury (ALI), aberrant proliferation of lung fibroblasts is initiated in early disease stages, but the underlying mechanism remains unknown. In this study, we knocked down Toll-like receptor 4 (TLR4) expression in cultured mouse lung fibroblasts using TLR4-siRNA-lentivirus in order to investigate the effects of LPS challenge on lung fibroblast proliferation, phosphoinositide3-kinase (PI3K)-Akt pathway activation, and phosphatase and tensin homolog (PTEN) expression. Lung fibroblast proliferation, detected by BrdU assay, was unaffected by 1 mug/mL LPS challenge up to 24 hours, but at 72 hours, cell proliferation increased significantly. This proliferation was inhibited by siRNA-mediated TLR4 knockdown or treatment with the PI3K inhibitor, Ly294002. In addition, siRNA-mediated knockdown of TLR4 inhibited the LPS-induced up-regulation of TLR4, down-regulation of PTEN, and activation of the PI3K-Akt pathway (overexpression of phospho-Akt) at 72 hours, as detected by real-time PCR and Western blot analysis. Treatment with the PTEN inhibitor, bpV(phen), led to activation of the PI3K-Akt pathway. Neither the baseline expression nor LPS-induced down-regulation of PTEN in lung fibroblasts was influenced by PI3K activation state. PTEN inhibition was sufficient to exert the LPS effect on lung fibroblast proliferation, and PI3K-Akt pathway inhibition could reverse this process. Collectively, these results indicate that LPS can promote lung fibroblast proliferation via a TLR4 signaling mechanism that involves PTEN expression downregulatio

Facile synthesis of an efficient and robust cathode interface material for polymer solar cells

In this study, an alcohol soluble novel naphthalene diimide (NDI)–thiophene-based cathode interface layer (CIL), PNDIT10N, is reported. PNDIT10N was synthesized in a facile three-step method, processed from environmentally friendly benzyl alcohol (BnOH) and employed in inverted polymer solar cells (PSCs). The three polymer donors TQ1, PTNT, and PTB7-Th were paired with the fullerene acceptor PC71BM for bulk heterojunction (BHJ) layers to evaluate the CIL. The modification of the indium tin oxide (ITO) electrode with a ∼3 nm thin layer of PNDIT10N yielded a significant reduction of 0.8 eV in the work function, reducing it from 4.6 to 3.8 eV, effectively transforming ITO to a functioning cathode. PSCs with a TQ1/PC71BM BHJ layer and incorporating a PNDIT10N interlayer were found to have a high Jsc value of 10.5 mA cm–2, Voc value of 909 mV, and an FF value of 68%, resulting in the highest PCE of 6.7% for TQ1 donor in the inverted device structure. Of note, the interface layer showed a good stability in ambient atmosphere for a 10 d indoor aging period, both in darkness and exposed to direct sunlight. Additionally, flexible PSCs incorporating slot-die coated PNDIT10N, processed from a BnOH–acetone solution, and BHJ layer in air achieved a PCE of 1.6%

Control of Interfacial Cl₂ and N₂O₅ Reactivity by a Zwitterionic Phospholipid in Comparison with Ionic and Uncharged Surfactants

Gas–liquid scattering experiments reveal that charge-separated but neutral (zwitterionic) surfactants catalyze the oxidation of dissolved Br^– to Br₂ by gaseous Cl₂ at the surface of a 0.3 M NaBr/glycerol solution. Solutions of NaBr dissolved in glycerol with no surfactant were compared with solutions coated with zwitterionic, cationic, and anionic surfactants at dilute surface concentrations of 1.1 to 1.5 × 10¹⁴ cm^–2 (less than 65% of maximum chain packing). The zwitterionic phospholipid enhances Cl₂ conversion of Br^– to Br₂ by a factor of 1.61 ± 0.15, in comparison with a 14-fold enhancement by a cationic surfactant (tetrahexylammonium) and a five-fold suppression by an anionic surfactant (dodecyl sulfate). Further studies indicate that even an uncharged surfactant, monododecanoylglycerol, enhances Cl₂ → Br₂ production. Similar behavior is observed for the oxidation of Br^– to Br₂ by N₂O₅; it is just slightly suppressed by the phospholipid and strongly enhanced by the cationic surfactant. Collectively, these results suggest that attractions and repulsions between the negative Br^– ions and the positive and negative charges of the surfactant headgroups draw Br^– ions to the surface or repel them away. At low coverages, ion-induced dipole and dispersion interactions between the CH₂ groups and Br^– or Cl₂ may also enhance reactivity. These results demonstrate that the hydrocarbon chains of loosely packed surfactants do not necessarily block gas–liquid reactions but that positively charged, and even uncharged, groups can instead facilitate reactions by bringing gas-phase and solution-phase reagents together in the interfacial region

Expression of TLR4 in lung fibroblasts and its effect on lung fibroblast proliferation.

<p>TLR4 mRNA (A, real-time PCR) and protein (B, Western blot) expression in lung fibroblasts at 72 hours after 1 µg/mL LPS challenge. Effect of siRNA-mediated knockdown of TLR4 (1×10⁸ TU/mL for 48 hours) on lung fibroblast proliferation (C, BrdU assay). * <i>p</i><0.05 <i>vs.</i> negative control group (Column 2); ^†<i>p</i><0.05 <i>vs.</i> positive control group (Column 3). Columns represent mean values and error bars represent SD. Blots are representative of three independent experiments.</p

Effect of LPS on lung fibroblast proliferation.

<p>DNA synthesis in lung fibroblasts was detected by BrdU assay after LPS challenge at 0, 6, 24, and 72 hours. * <i>p</i><0.05 for percentage of OD₄₅₀ absorbance compared to the control group at the same time point. Columns represent mean values (n = 3) and error bars represent SD.</p

Expression of PTEN in lung fibroblasts after LPS challenge or PI3K inhibition.

<p>PTEN mRNA (A and B, real-time PCR) and protein (C and D, Western blot) expression in lung fibroblasts at 72 hours after 1 µg/mL LPS challenge (A, C) or PI3K-Akt pathway inhibition (B, D). siRNA-mediated knockdown of TLR4 (1×10⁸ TU/mL for 48 hours) was used to assess the effect of TLR4 on PTEN expression in lung fibroblasts. Ly294002 (50 µmol/L for one hour) was applied to determine whether PTEN expression is regulated by PI3K activation. * <i>p</i><0.05 <i>vs.</i> positive control group (A and C, Column 3). * <i>p</i><0.05 <i>vs.</i> negative control group (B and D, Column 1); ^†<i>p</i><0.05 <i>vs.</i> positive control group (Column 2). Columns represent mean values and error bars represent SD. Blots are representative of three independent experiments.</p