Large Language Models Are Semi-Parametric Reinforcement Learning Agents

Inspired by the insights in cognitive science with respect to human memory and reasoning mechanism, a novel evolvable LLM-based (Large Language Model) agent framework is proposed as REMEMBERER. By equipping the LLM with a long-term experience memory, REMEMBERER is capable of exploiting the experiences from the past episodes even for different task goals, which excels an LLM-based agent with fixed exemplars or equipped with a transient working memory. We further introduce Reinforcement Learning with Experience Memory (RLEM) to update the memory. Thus, the whole system can learn from the experiences of both success and failure, and evolve its capability without fine-tuning the parameters of the LLM. In this way, the proposed REMEMBERER constitutes a semi-parametric RL agent. Extensive experiments are conducted on two RL task sets to evaluate the proposed framework. The average results with different initialization and training sets exceed the prior SOTA by 4% and 2% for the success rate on two task sets and demonstrate the superiority and robustness of REMEMBERER

Research on Beijing Manufacturing Green-Oriented Transition Path under “Double Carbon” Goal-Based on the GML-SD Model

Accelerating the “double carbon” transformation of the manufacturing industry is the key to achieving the strategic goal of “double carbon” in China, among which the low-carbon development of the Beijing manufacturing industry is the top priority. To achieve high-quality development of the manufacturing industry in Beijing, the main task is to reduce carbon emissions and improve efficiency. Therefore, on the basis of the Global Malmquist–Luenberger (GML) index and the system dynamics (SD) method, this study analyzes the green total-factor energy efficiency trends of 25 manufacturing sub-sectors in Beijing from 2006 to 2020 and incorporates the GML index as a key variable into the construction of the SD model. It then explores the path optimization of Beijing’s manufacturing low-carbon development in the 14th Five-Year Plan period via scenario simulation and offers policy recommendations for the green-oriented transition of Beijing’s manufacturing industry. The study finds that Beijing’s manufacturing industry needs to prioritize energy structure optimization and efficient energy utilization and give full play to the city’s advantages in technological innovation and investment while supporting high-end scientific research and innovation projects to confirm the capital city’s strategic position

A Novel RNA-Seq-Based Model for Preoperative Prediction of Lymph Node Metastasis in Oral Squamous Cell Carcinoma

Objective. To develop and validate a novel RNA-seq-based nomogram for preoperative prediction of lymph node metastasis (LNM) for patients with oral squamous cell carcinoma (OSCC). Methods. RNA-seq data for 276 OSCC patients (including 157 samples with LNM and 119 without LNM) were downloaded from TCGA database. Differential expression analysis was performed between LNM and non-LNM of OSCC. These samples were divided into a training set and a test set by a ratio of 9 : 1 while the relative proportion of the non-LNM and LNM groups was kept balanced within each dataset. Based on clinical features and seven candidate RNAs, we established a prediction model of LNM for OSCC using logistic regression analysis. Tenfold crossvalidation was utilized to examine the accuracy of the nomogram. Decision curve analysis was performed to evaluate the clinical utility of the nomogram. Results. A total of 139 differentially expressed RNAs were identified between LNM and non-LNM of OSCC. Seven candidate RNAs were screened based on FPKM values, including NEURL1, AL162581.1 (miscRNA), AP002336.2 (lncRNA), CCBE1, CORO6, RDH12, and AC129492.6 (pseudogene). Logistic regression analysis revealed that the clinical N stage (p<0.001) was an important factor to predict LNM. Moreover, three RNAs including RDH12 (p value < 0.05), CCBE1 (p value < 0.01), and AL162581.1 (p value < 0.05) could be predictive biomarkers for LNM in OSCC patients. The average accuracy rate of the model was 0.7661, indicating a good performance of the model. Conclusion. Our findings constructed an RNA-seq-based nomogram combined with clinicopathology, which could potentially provide clinicians with a useful tool for preoperative prediction of LNM and be tailored for individualized therapy in patients with OSCC

Severe Burn Injury Significantly Alters the Gene Expression and m6A Methylation Tagging of mRNAs and lncRNAs in Human Skin

N6-methyladenosine (m6A) modulates RNA metabolism and functions in cell differentiation, tissue development, and immune response. After acute burns, skin wounds are highly susceptible to infection and poor healing. However, our understanding of the effect of burn injuries on m6A methylation and their potential mechanism is still limited. Human m6A-mRNA&lncRNA Epitranscriptomic microarray was used to obtain comprehensive mRNA and lncRNA transcriptome m6A profiling and gene expression patterns after burn injuries in human skin tissue. Bioinformatic and functional analyses were conducted to find molecular functions. Microarray profiling showed that 65 mRNAs and 39 lncRNAs were significantly hypermethylated; 5492 mRNAs and 754 lncRNAs were significantly hypomethylated. Notably, 3989 hypomethylated mRNAs were down-expressed and inhibited many wound healing biological processes and pathways including in the protein catabolic process and supramolecular fiber organization pathway; 39 hypermethylated mRNAs were up-expressed and influenced the cell surface receptor signaling pathway and inflammatory response. Moreover, we validated that m6A regulators (METTL14, METTL16, ALKBH5, FMR1, and HNRNPC) were significantly downregulated after burn injury which may be responsible for the alteration of m6A modification and gene expression. In summary, we found that homeostasis in the skin was disrupted and m6A modification may be a potential mechanism affecting trauma infection and wound healing