Enabling Efficient Interaction between an Algorithm Agent and an LLM: A Reinforcement Learning Approach

Large language models (LLMs) encode a vast amount of world knowledge acquired from massive text datasets. Recent studies have demonstrated that LLMs can assist an algorithm agent in solving complex sequential decision making tasks in embodied environments by providing high-level instructions. However, interacting with LLMs can be time-consuming, as in many practical scenarios, they require a significant amount of storage space that can only be deployed on remote cloud server nodes. Additionally, using commercial LLMs can be costly since they may charge based on usage frequency. In this paper, we explore how to enable efficient and cost-effective interactions between the agent and an LLM. We propose a reinforcement learning based mediator model that determines when it is necessary to consult LLMs for high-level instructions to accomplish a target task. Experiments on 4 MiniGrid environments that entail planning sub-goals demonstrate that our method can learn to solve target tasks with only a few necessary interactions with an LLM, significantly reducing interaction costs in testing environments, compared with baseline methods. Experimental results also suggest that by learning a mediator model to interact with the LLM, the agent's performance becomes more robust against both exploratory and stochastic environments.Comment: 10 page

Development of Space-Time-Controlled Multi-Stage Pulsed Magnetic Field Forming and Manufacturing Technology at the WHMFC*

In November 2011, the Project of Basic Research of Forming by Space-Time-Controlled Multi-Stage Pulsed Magnetic Field (Stic-Must-PMF) was supported by the National Basic Research Program of China (973 Project, 2011.11-2016.08). It is aimed at achieving breakthroughs in manufacturing technology to solve current problems in forming largescale and complex sheet and tube parts and components, imposed by the limitations of existing equipment and materials forming properties. The objective of our research group focuses on the design principles and structural layout optimization of Stic-Must-PMF facility. And this paper will report the development of Stic-Must-PMF forming and manufacturing technology at the Wuhan National High Magnetic Field Center (WHMFC) including numerical modeling, experimental setup and experimental studies

A nine months follow-up study of hemodynamic effect on bioabsorbable coronary stent implantation

Coronary artery disease has emerged as one of the major diseases causing death worldwide. Coronary stent has great effect to improve blood flow to the myocardium subtended by that artery, in which bioresorbable vascular scaffolds are new-generation stents used by people. However, Coronary stents implantation has a risk of restenosis, which is relative to hemodynamic parameters. Most of existing literatures studied in this issue have not taken into account such important factors as the strut thickness and lumen profile, and has yet to analyze the time effects among hemodynamic parameters over a certain period of time based on individual models. In this research, we proposed a framework to assess the chronic impact of hemodynamic on coronary stent implantation. In the framework, the optical coherence tomography (OCT) is combined with angiography to reconstruct patient-specific models of bioresorbable vascular scaffolds. Then, the hemodynamics parameters are extracted through the simulated 3D models, obtaining the distribution of wall shear stress (WSS), relative residence time (RRT) and oscillatory shear index (OSI). Finally, the changes of these parameters representing the effectiveness of hemodynamics exerted on the implanted stent can be assessed to estimate the chronic impacts. By a 9-month follow-up case study, it is observed that the difference of hemodynamic parameters are not significance. Both at baseline and 9-month follow-up experiments show that the hemodynamic parameters remain normal and similar, proving that the coronary stent implantation nowadays appears to have a robust and everlasting curative effect

QTL-seq analysis identified the genomic regions of plant height and days to heading in high-latitude rice

Introduction: Rice (Oryza sativa L.) is one of the most extensive crops in the world. China’s Heilongjiang Province is the northernmost rice-growing region in the world. However, rice cultivars suitable for growth in low-latitude regions may not mature normally due to their distinct climate and short frost-free period. It is necessary to precisely determine the frost-free period for each region to make the best use of the rice growth stage so as to ensure the maturity and yield of different rice cultivars in Heilongjiang Province. The time span of the heading stage is a key parameter for evaluating the adaptability of a rice cultivar to a specific rice-growing region. Given the above facts, it is of high importance to study the associated genes and sites controlling days to heading (DH) and plant height (PH) of rice in Heilongjiang Province. Bulked segregant analysis (BSA) combined with high-throughput sequencing can effectively exclude interferences from background genomic differences, making it suitable for analyzing the associated sites of complex agronomic traits in early generations.Methods: In this study, an F3 segregating population was obtained by crossing two main cultivars that are grown under different temperatures and day-light conditions in Heilongjiang. Two pools of extreme phenotypes were built for the DH and PH of the population. For SNP and InDel variants obtained from whole-genome resequencing in the pools, an association analysis was performed using the Euclidean distance (ED) algorithm and the SNP/InDel index algorithm.Results: The intersection of SNP and InDel regions associated with the phenotypes was considered to obtain the final associated sites. After excluding interferences from the cloned genes on chromosomes 2 and 7, a total length of 6.34 Mb on chromosomes 1, 3, and 10 and 3.16 Mb on chromosomes 1 and 10 were left associated with PH and DH, respectively. Then, we performed a gene annotation analysis for candidate genes in the remaining regions using multiple genome annotation databases. Our research provides basic data for subsequent gene mapping and cloning.Discussion: By mining more genetic loci associated with the days to heading and plant height of rice, we may provide abundant genetic resources for refined molecular breeding in Heilongjiang Province

Heterogeneous N2O5 reactions on atmospheric aerosols at four Chinese sites : improving model representation of uptake parameters

Heterogeneous reactivity of N2O5 on aerosols is a critical parameter in assessing NOx fate, nitrate production, and particulate chloride activation. Accurate measurement of its uptake coefficient (gamma N2O5) and representation in air quality models are challenging, especially in the polluted environment. With an in situ aerosol flow-tube system, the gamma N2O5 was directly measured on ambient aerosols at two rural sites in northern and southern China. The results were analyzed together with the gamma N2O5 derived from previous field studies in China to obtain a holistic picture of gamma N2O5 uptake and the influencing factors under various climatic and chemical conditions. The field-derived or measured gamma N2O5 was generally promoted by the aerosol water content and suppressed by particle nitrate. Significant discrepancies were found between the measured gamma N2O5 and that estimated from laboratory-determined parameterizations. An observation-based empirical parameterization was derived in the present work, which better reproduced the mean value and variability of the observed gamma N2O5. Incorporating this new parameterization into a regional air quality model (WRF-CMAQ) has improved the simulation of N2O5, nitrogen oxides, and secondary nitrate in the polluted regions of China.Peer reviewe

Breath-, air- and surface-borne SARS-CoV-2 in hospitals

The COVID-19 pandemic has brought an unprecedented crisis to the global health sector. When discharging COVID-19 patients in accordance with throat or nasal swab protocols using RT-PCR, the potential risk of reintroducing the infection source to humans and the environment must be resolved. Here, 14 patients including 10 COVID-19 subjects were recruited; exhaled breath condensate (EBC), air samples and surface swabs were collected and analyzed for SARS-CoV-2 using reverse transcription-polymerase chain reaction (RT-PCR) in four hospitals with applied natural ventilation and disinfection practices in Wuhan. Here we discovered that 22.2% of COVID-19 patients (n = 9), who were ready for hospital discharge based on current guidelines, had SARS-CoV-2 in their exhaled breath (~10⁵ RNA copies/m³). Although fewer surface swabs (3.1%, n = 318) tested positive, medical equipment such as face shield frequently contacted/used by healthcare workers and the work shift floor were contaminated by SARS-CoV-2 (3–8 viruses/cm²). Three of the air samples (n = 44) including those collected using a robot-assisted sampler were detected positive by a digital PCR with a concentration level of 9–219 viruses/m³. RT-PCR diagnosis using throat swab specimens had a failure rate of more than 22% in safely discharging COVID-19 patients who were otherwise still exhaling the SARS-CoV-2 by a rate of estimated ~1400 RNA copies per minute into the air. Direct surface contact might not represent a major transmission route, and lower positive rate of air sample (6.8%) was likely due to natural ventilation (1.6–3.3 m/s) and regular disinfection practices. While there is a critical need for strengthening hospital discharge standards in preventing re-emergence of COVID-19 spread, use of breath sample as a supplement specimen could further guard the hospital discharge to ensure the safety of the public and minimize the pandemic re-emergence risk