Agent-Aware Training for Agent-Agnostic Action Advising in Deep Reinforcement Learning

Action advising endeavors to leverage supplementary guidance from expert teachers to alleviate the issue of sampling inefficiency in Deep Reinforcement Learning (DRL). Previous agent-specific action advising methods are hindered by imperfections in the agent itself, while agent-agnostic approaches exhibit limited adaptability to the learning agent. In this study, we propose a novel framework called Agent-Aware trAining yet Agent-Agnostic Action Advising (A7) to strike a balance between the two. The underlying concept of A7 revolves around utilizing the similarity of state features as an indicator for soliciting advice. However, unlike prior methodologies, the measurement of state feature similarity is performed by neither the error-prone learning agent nor the agent-agnostic advisor. Instead, we employ a proxy model to extract state features that are both discriminative (adaptive to the agent) and generally applicable (robust to agent noise). Furthermore, we utilize behavior cloning to train a model for reusing advice and introduce an intrinsic reward for the advised samples to incentivize the utilization of expert guidance. Experiments are conducted on the GridWorld, LunarLander, and six prominent scenarios from Atari games. The results demonstrate that A7 significantly accelerates the learning process and surpasses existing methods (both agent-specific and agent-agnostic) by a substantial margin. Our code will be made publicly available

An Input-Series-Output-Parallel Cascaded Converter System Applied to DC Microgrids

Direct current transformer (DCT) is a key piece of equipment in direct current (DC) microgrids, and the mainstream topologies mainly include LLC resonant converter (LLC) and dual active bridge (DAB). In this paper, a novel bi-directional buck/boost + CLLLC cascade topology is proposed for the input-series-output-parallel cascade converter system of a DC microgrid. To solve the problem that frequency variation causes the converter to deviate from the optimal operating point, resulting in low efficiency, and the inability to achieve a soft switching function. The CLLLC converter operates near the resonant frequency point as a DCT, only providing electrical isolation and voltage matching, while the buck/boost converter controls the output voltage and the voltage and current sharing of each module. Compared to other cascaded converter systems, the cascaded converter proposed in this paper has high efficiency, simplifies the parameter design, and is suitable for wide input and wide output operating conditions. The system adopts a three-loop control strategy, establishes the small-signal modeling of the system, and its stability is verified by theoretical analysis and simulation. The simulation and experimental results verify the correctness of the proposed cascaded converter based on buck/boost + CLLLC and the effectiveness of the control strategy

Independent of EPR effect : a smart delivery nanosystem for tracking and treatment of nonvascularized intra-abdominal metastases

Nanoparticle-based delivery systems (NDS) have impacted the field of cancer therapy on account of the enhanced permeability and retention (EPR) effect that promotes passive accumulation in tumors through the tumor vasculature after intravenous (IV) administration. However, transplanted tumor xenografts on animal models used to justify the feasibility of EPR effect are quite different from clinical tumors in many aspects, a fact that becomes an impediment for NDS to succeed clinical trials. Particularly, early-stage tumor metastases are usually nonvascularized and incapable of conforming the EPR effect after IV injection. Therefore, it is necessary to develop smart NDS to deliver drugs in an EPR-independent route. Herein, an NDS-based treatment approach for intra-abdominal metastases from ovarian carcinoma is reported. Instead of IV injection, intraperitoneal (IP) injection was employed to directly apply the NDS to the metastatic lesions. The NDS was tailor-made with targeting groups to actively target the tumor nidus and redox-responsive drug release to reduce systematic toxicity. Comparing with IV administration, the IP injected NDS could be enriched in metastatic tumor more efficiently, leading to superior therapeutic outcome in vivo. This study provides a successful protocol of EPR independent NDS-based cancer treatment, which may facilitate the clinical translation of nanoparticle-based cancer therapeutics.NRF (Natl Research Foundation, S’pore)Accepted versio

Discovery of Highly Potent Pinanamine-Based Inhibitors against Amantadine- and Oseltamivir-Resistant Influenza A Viruses

Influenza pandemic is a constant major threat to public health caused by influenza A viruses (IAVs). IAVs are subcategorized by the surface proteins hemagglutinin (HA) and neuraminidase (NA), in which they are both essential targets for drug discovery. While it is of great concern that NA inhibitor oseltamivir resistant strains are frequently identified from human or avian influenza virus, structural and functional characterization of influenza HA has raised hopes for new antiviral therapies. In this study, we explored a structure–activity relationship (SAR) of pinanamine-based antivirals and discovered a potent inhibitor <b>M090</b> against amantadine-resistant viruses, including the 2009 H1N1 pandemic strains, and oseltamivir-resistant viruses. Mechanism of action studies, particularly hemolysis inhibition, indicated that <b>M090</b> targets influenza HA and it occupied a highly conserved pocket of the HA₂ domain and inhibited virus-mediated membrane fusion by “locking” the bending state of HA₂ during the conformational rearrangement process. This work provides new binding sites within the HA protein and indicates that this pocket may be a promising target for broad-spectrum anti-influenza A drug design and development