Adaptive Policy Learning for Offline-to-Online Reinforcement Learning

Conventional reinforcement learning (RL) needs an environment to collect fresh data, which is impractical when online interactions are costly. Offline RL provides an alternative solution by directly learning from the previously collected dataset. However, it will yield unsatisfactory performance if the quality of the offline datasets is poor. In this paper, we consider an offline-to-online setting where the agent is first learned from the offline dataset and then trained online, and propose a framework called Adaptive Policy Learning for effectively taking advantage of offline and online data. Specifically, we explicitly consider the difference between the online and offline data and apply an adaptive update scheme accordingly, that is, a pessimistic update strategy for the offline dataset and an optimistic/greedy update scheme for the online dataset. Such a simple and effective method provides a way to mix the offline and online RL and achieve the best of both worlds. We further provide two detailed algorithms for implementing the framework through embedding value or policy-based RL algorithms into it. Finally, we conduct extensive experiments on popular continuous control tasks, and results show that our algorithm can learn the expert policy with high sample efficiency even when the quality of offline dataset is poor, e.g., random dataset.Comment: AAAI202

Protecting the Future: Neonatal Seizure Detection with Spatial-Temporal Modeling

A timely detection of seizures for newborn infants with electroencephalogram (EEG) has been a common yet life-saving practice in the Neonatal Intensive Care Unit (NICU). However, it requires great human efforts for real-time monitoring, which calls for automated solutions to neonatal seizure detection. Moreover, the current automated methods focusing on adult epilepsy monitoring often fail due to (i) dynamic seizure onset location in human brains; (ii) different montages on neonates and (iii) huge distribution shift among different subjects. In this paper, we propose a deep learning framework, namely STATENet, to address the exclusive challenges with exquisite designs at the temporal, spatial and model levels. The experiments over the real-world large-scale neonatal EEG dataset illustrate that our framework achieves significantly better seizure detection performance.Comment: Accepted in IEEE International Conference on Systems, Man, and Cybernetics (SMC) 202

LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression

In long context scenarios, large language models (LLMs) face three main challenges: higher computational/financial cost, longer latency, and inferior performance. Some studies reveal that the performance of LLMs depends on both the density and the position of the key information (question relevant) in the input prompt. Inspired by these findings, we propose LongLLMLingua for prompt compression towards improving LLMs' perception of the key information to simultaneously address the three challenges. We conduct evaluation on a wide range of long context scenarios including single-/multi-document QA, few-shot learning, summarization, synthetic tasks, and code completion. The experimental results show that LongLLMLingua compressed prompt can derive higher performance with much less cost. The latency of the end-to-end system is also reduced. For example, on NaturalQuestions benchmark, LongLLMLingua gains a performance boost of up to 17.1% over the original prompt with ~4x fewer tokens as input to GPT-3.5-Turbo. It can derive cost savings of \$28.5 and \$27.4 per 1,000 samples from the LongBench and ZeroScrolls benchmark, respectively. Additionally, when compressing prompts of ~10k tokens at a compression rate of 2x-10x, LongLLMLingua can speed up the end-to-end latency by 1.4x-3.8x. Our code is available at https://aka.ms/LLMLingua

The Bcl-2/xL inhibitor ABT-263 increases the stability of Mcl-1 mRNA and protein in hepatocellular carcinoma cells

Background Hepatocellular carcinoma (HCC) is one of the major causes of mortality. ABT-263 is a newly synthesized, orally available Bcl-2/xL inhibitor that shows promising efficacy in HCC therapy. ABT-263 inhibits the anti-apoptotic activity of Bcl-2 and Bcl-xL, but not Mcl-1. Previous reports have shown that ABT-263 upregulates Mcl-1 in various cancer cells, which contributes to ABT-263 resistance in cancer therapy. However, the associated mechanisms are not well known. Methods Western blot, RNAi and CCK-8 assays were used to investigate the relationship between Mcl-1 upregulation and ABT-263 sensitivity in HCC cells. Real-time PCR and Western blot were used to detect Mcl-1 mRNA and protein levels. Luciferase reporter assay and RNA synthesis inhibition assay were adopted to analyze the mechanism of Mcl-1 mRNA upregulation. Western blot and the inhibition assays for protein synthesis and proteasome were used to explore the mechanisms of ABT-263-enhanced Mcl-1 protein stability. Trypan blue exclusion assay and flow cytometry were used to examine cell death and apoptosis. Results ABT-263 upregulated Mcl-1 mRNA and protein levels in HCC cells, which contributes to ABT-263 resistance. ABT-263 increased the mRNA level of Mcl-1 in HCC cells by enhancing the mRNA stability without influencing its transcription. Furthermore, ABT-263 increased the protein stability of Mcl-1 through promoting ERK- and JNK-induced phosphorylation of Mcl-1Thr163 and increasing the Akt-mediated inactivation of GSK-3β. Additionally, the inhibitors of ERK, JNK or Akt sensitized ABT-263-induced apoptosis in HCC cells. Conclusions ABT-263 increases Mcl-1 stability at both mRNA and protein levels in HCC cells. Inhibition of ERK, JNK or Akt activity sensitizes ABT-263-induced apoptosis. This study may provide novel insights into the Bcl-2-targeted cancer therapeutics.This study was supported in part by Chongqing Natural Science Foundation (cstc2011BB5030 and 2013jjB10015), the National Natural Science Foundation of China (31201068, 81273226 and 81228005) and the Scientific Funds of Third Military Medical University (2011XHG02 and 2012XZH01)

Unified Medical Image Pre-training in Language-Guided Common Semantic Space

Vision-Language Pre-training (VLP) has shown the merits of analysing medical images, by leveraging the semantic congruence between medical images and their corresponding reports. It efficiently learns visual representations, which in turn facilitates enhanced analysis and interpretation of intricate imaging data. However, such observation is predominantly justified on single-modality data (mostly 2D images like X-rays), adapting VLP to learning unified representations for medical images in real scenario remains an open challenge. This arises from medical images often encompass a variety of modalities, especially modalities with different various number of dimensions (e.g., 3D images like Computed Tomography). To overcome the aforementioned challenges, we propose an Unified Medical Image Pre-training framework, namely UniMedI, which utilizes diagnostic reports as common semantic space to create unified representations for diverse modalities of medical images (especially for 2D and 3D images). Under the text's guidance, we effectively uncover visual modality information, identifying the affected areas in 2D X-rays and slices containing lesion in sophisticated 3D CT scans, ultimately enhancing the consistency across various medical imaging modalities. To demonstrate the effectiveness and versatility of UniMedI, we evaluate its performance on both 2D and 3D images across 10 different datasets, covering a wide range of medical image tasks such as classification, segmentation, and retrieval. UniMedI has demonstrated superior performance in downstream tasks, showcasing its effectiveness in establishing a universal medical visual representation

Characterization of nonstructural protein 3 of a neurovirulent Japanese encephalitis virus strain isolated from a pig

<p>Abstract</p> <p>Background</p> <p>Japanese encephalitis virus (JEV), as a re-emerging virus that causes 10,000-15,000 human deaths from encephalitis in the world each year, has had a significant impact on public health. Pigs are the natural reservoirs of JEV and play an important role in the amplification, dispersal and epidemiology of JEV. The nonstructural protein 3 (NS3) of JEV possesses enzymatic activities of serine protease, helicase and nucleoside 5'-triphosphatase, and plays important roles in viral replication and pathogenesis.</p> <p>Results</p> <p>We characterized the NS3 protein of a neurovirulent strain of JEV (SH-JEV01) isolated from a field-infected pig. The NS3 gene of the JEV SH-JEV01 strain is 1857 bp in length and encodes protein of approximately 72 kDa with 99% amino acid sequence identity to that of the representative immunotype strain JaGAr 01. The NS3 protein was detectable 12 h post-infection in a mouse neuroblastoma cell line, Neuro-2a, and was distributed in the cytoplasm of cells infected with the SH-JEV01 strain of JEV. In the brain of mice infected with the SH-JEV01 strain of JEV, NS3 was detected in the cytoplasm of neuronal cells, including pyramidal neurons of the cerebrum, granule cells, small cells and Purkinje cells of the cerebellum.</p> <p>Conclusions</p> <p>The NS3 protein of a neurovirulent strain of JEV isolated from a pig was characterized. It is an approximately 72 kDa protein and distributed in the cytoplasm of infected cells. The Purkinje cell of the cerebellum is one of the target cells of JEV infection. Our data should provide some basic information for the study of the role of NS3 in the pathogenesis of JEV and the immune response.</p

The Meq oncoprotein of Marek's disease virus interacts with p53 and inhibits its transcriptional and apoptotic activities

<p>Abstract</p> <p>Background</p> <p>Marek's disease virus (MDV) is an oncogenic herpesvirus, which causes malignant lymphoma in chickens. The Meq protein of MDV, which is expressed abundantly in MDV-infected cells and in Marek's disease (MD) tumor cells, functions as a transcriptional activator and has been proposed to play an important role in oncogenic transformation. Preliminary studies demonstrated that Meq is able to bind p53 <it>in vitro</it>, as demonstrated using a protein-binding assay. This observation prompted us to examine whether the interaction between Meq and p53 occurs in cells, and to investigate the biological significance of this interaction.</p> <p>Results</p> <p>We confirmed first that Meq interacted directly with p53 using a yeast two-hybrid assay and an immunoprecipitation assay, and we investigated the biological significance of this interaction subsequently. Exogenous expression of Meq resulted in the inhibition of p53-mediated transcriptional activity and apoptosis, as analyzed using a p53 luciferase reporter assay and a TUNEL assay. The inhibitory effect of Meq on transcriptional activity mediated by p53 was dependent on the physical interaction between these two proteins, because a Meq deletion mutant that lacked the p53-binding region lost the ability to inhibit p53-mediated transcriptional activity and apoptosis. The Meq variants L-Meq and S-Meq, but not VS-Meq and ∆Meq, which were expressed in MD tumor cells and MDV-infected cells, exerted an inhibitory effect on p53 transcriptional activity. In addition, ∆Meq was found to act as a negative regulator of Meq.</p> <p>Conclusions</p> <p>The Meq oncoprotein interacts directly with p53 and inhibits p53-mediated transcriptional activity and apoptosis. These findings provide valuable insight into the molecular basis for the function of Meq in MDV oncogenesis.</p