MOELoRA: An MOE-based Parameter Efficient Fine-Tuning Method for Multi-task Medical Applications

The recent surge in the field of Large Language Models (LLMs) has gained significant attention in numerous domains. In order to tailor an LLM to a specific domain such as a web-based healthcare system, fine-tuning with domain knowledge is necessary. However, two issues arise during fine-tuning LLMs for medical applications. The first is the problem of task variety, where there are numerous distinct tasks in real-world medical scenarios. This diversity often results in suboptimal fine-tuning due to data imbalance and seesawing problems. Additionally, the high cost of fine-tuning can be prohibitive, impeding the application of LLMs. The large number of parameters in LLMs results in enormous time and computational consumption during fine-tuning, which is difficult to justify. To address these two issues simultaneously, we propose a novel parameter-efficient fine-tuning framework for multi-task medical applications called MOELoRA. The framework aims to capitalize on the benefits of both MOE for multi-task learning and LoRA for parameter-efficient fine-tuning. To unify MOE and LoRA, we devise multiple experts as the trainable parameters, where each expert consists of a pair of low-rank matrices to maintain a small number of trainable parameters. Additionally, we propose a task-motivated gate function for all MOELoRA layers that can regulate the contributions of each expert and generate distinct parameters for various tasks. To validate the effectiveness and practicality of the proposed method, we conducted comprehensive experiments on a public multi-task Chinese medical dataset. The experimental results demonstrate that MOELoRA outperforms existing parameter-efficient fine-tuning methods. The implementation is available online for convenient reproduction of our experiments

Large Language Model Distilling Medication Recommendation Model

The recommendation of medication is a vital aspect of intelligent healthcare systems, as it involves prescribing the most suitable drugs based on a patient's specific health needs. Unfortunately, many sophisticated models currently in use tend to overlook the nuanced semantics of medical data, while only relying heavily on identities. Furthermore, these models face significant challenges in handling cases involving patients who are visiting the hospital for the first time, as they lack prior prescription histories to draw upon. To tackle these issues, we harness the powerful semantic comprehension and input-agnostic characteristics of Large Language Models (LLMs). Our research aims to transform existing medication recommendation methodologies using LLMs. In this paper, we introduce a novel approach called Large Language Model Distilling Medication Recommendation (LEADER). We begin by creating appropriate prompt templates that enable LLMs to suggest medications effectively. However, the straightforward integration of LLMs into recommender systems leads to an out-of-corpus issue specific to drugs. We handle it by adapting the LLMs with a novel output layer and a refined tuning loss function. Although LLM-based models exhibit remarkable capabilities, they are plagued by high computational costs during inference, which is impractical for the healthcare sector. To mitigate this, we have developed a feature-level knowledge distillation technique, which transfers the LLM's proficiency to a more compact model. Extensive experiments conducted on two real-world datasets, MIMIC-III and MIMIC-IV, demonstrate that our proposed model not only delivers effective results but also is efficient. To ease the reproducibility of our experiments, we release the implementation code online

Chemical structure, properties and potential applications of surfactin, as well as advanced strategies for improving its microbial production

Surfactin, a cyclic lipopeptide produced by microbes belonging to the genus Bacillus, is one of the most effective biosurfactants available in many industrial fields. However, its low production and high cost have intensively constrained its commercial applications. In this review, we first summarize the molecular structure, biological properties, beneficial roles and potential applications of surfactin in the fields of medical care and food safety, highlighting the great medical and commercial values of making its industrial production into reality. Further, genetic regulation for surfactin biosynthesis and advanced strategies for enhancing its microbial production, including optimizing fermentation conditions, rational genetic engineering and synthetic biology combined with metabolic engineering approaches, are elucidated. Finally, prospects for improving surfactin biosynthesis are discussed, and the establishment of suitable chassis hosts for exogenous production of surfactin might serve as an important strategy in future research

Limiting Magnitudes of the Wide Field Survey Telescope (WFST)

Expected to be of the highest survey power telescope in the northern hemisphere, the Wide Field Survey Telescope (WFST) will begin its routine observations of the northern sky since 2023. WFST will produce a lot of scientific data to support the researches of time-domain astronomy, asteroids and the solar system, galaxy formation and cosmology and so on. We estimated that the 5 $\sigma$ limiting magnitudes of WFST with 30 second exposure are $u=22.31$ mag, $g=23.42$ mag, $r=22.95$ mag, $i=22.43$ mag, $z=21.50$ mag, $w=23.61$ mag. The above values are calculated for the conditions of $airmass=1.2$, seeing = 0.75 arcsec, precipitable water vapour (PWV) = 2.5 mm and Moon-object separation = $45^{\circ}$ at the darkest New Moon night of the Lenghu site (V=22.30 mag, Moon phase $\theta=0^{\circ}$). The limiting magnitudes in different Moon phase conditions are also calculated. The calculations are based on the empirical transmittance data of WFST optics, the vendor provided CCD quantum efficiency, the atmospherical model transmittance and spectrum of the site. In the absence of measurement data such as sky transmittance and spectrum, we use model data.Comment: 12 pages, 5 figures, accepted by RAA (Research in Astronomy and Astrophysics

Effect of Corilagin on the Proliferation and NF- κ

Background. This study is to explore the effect of corilagin on the proliferation and NF-κB signaling pathway in U251 glioblastoma cells and U251 glioblastoma stem-like cells. Methods. CD133 positive U251 glioblastoma cells were separated by immunomagnetic beads to isolate glioblastoma stem-like cells. U251 cells and stem-like cells were intervened by different corilagin concentrations (0, 25, 50, and 100 μg/mL) for 48 h, respectively. Cell morphology, cell counting kit-8 assay, flow cytometry, dual luciferase reporter assay, and a western blot were used to detect and analyze the cell proliferation and cell cycle and investigate the expression of IKBα protein in cytoplasm and NF-κB/p65 in nucleus. Results. Corilagin inhibited the cell proliferation of U251 cells and their stem-like cells and the inhibition role was stronger in U251 stem-like cells (P<0.05). The cell cycle was arrested at G2/M phase in the U251 cells following corilagin intervention; the proportion of cells in G2/M phase increased as the concentration of corilagin increased (P<0.05). The U251 stem-like cells were arrested at the S phase following treatment with corilagin; the proportion of cells in the S phase increased as the concentration of corilagin increased (P<0.05). The ratio of dual luciferase activities of U251 stem-like cells was lower than that of U251 cells in the same corilagin concentration. With increasing concentrations of corilagin, the IKBα expression in cytoplasm of U251 cells and U251 stem-like cells was increased, but the p65 expression in nucleus of U251 cells and U251 stem-like cells was decreased (P<0.05). Conclusion. Corilagin can inhibit the proliferation of glioblastoma cells and glioblastoma stem-like cells; the inhibition on glioblastoma stem-like cell proliferation is stronger than glioblastoma cells. This different result indicates that the effect of corilagin on U251 cells and U251 stem-like cells may have close relationships with mechanism of cell cycle and NF-κB signaling pathway; however, the real antitumor mechanism of corilagin is not yet clear and requires further study

A new tool for in vitro culture of porcine eggs

Mineral oil is usually used to cover the microdrops of medium in oocytes or embryos culture system here designated as oil method. A large number of oocytes are needed for the production of porcine embryos for in vitro fertilization or somatic cell nuclear transfer (SCNT). The oil method not only wastes a lot of mineral oil, but needs tedious steps in the transferring of embryos. Here we designed a new method called nest dish, which need not mineral oil, to replace the oil method and improve the development rates of porcine eggs in vitro. The oocyte maturation rate with the mTCM199 (83.2%) was significantly higher than with the NCSU23 (75.5%, P﹤ 0.05), although the parthenogenetic cleavage rates with two media were not significantly different (77.7 and 72.4%, P﹤ 0.05 ). Chosing mTCM199 as base medium, the rate of maturation with concave dish (90.1%) was significantly higher than with the flat dish (82.6%, P﹥ 0.05) in nest method, although no significant differences in the oocyte maturation were found between flat dish (82.6%) in nest method and oil method (80.0%). Parthenogenetic cleavage from nest method (80.1% for concave dish, 78.0% for flat dish) did not show any decrease compared to oil method (76.2%), but the developmental rate to blastocysts in the nest groups(17.9 and 19.5%) were significantly higher than the oil method (12.3%, P﹤ 0.05). These results showed that mTCM199 presented higher maturation rate than that NCSU-23 did, and the nest method with concave dish significantly improved the maturation rate of porcine oocytes in vitro and can replace the conventional oil method.Keywords: Porcine oocytes, in vitro maturation (IVM), microdrop method, nest dish metho