MOELoRA: An MOE-based Parameter Efficient Fine-Tuning Method for Multi-task Medical Applications

The recent surge in the field of Large Language Models (LLMs) has gained significant attention in numerous domains. In order to tailor an LLM to a specific domain such as a web-based healthcare system, fine-tuning with domain knowledge is necessary. However, two issues arise during fine-tuning LLMs for medical applications. The first is the problem of task variety, where there are numerous distinct tasks in real-world medical scenarios. This diversity often results in suboptimal fine-tuning due to data imbalance and seesawing problems. Additionally, the high cost of fine-tuning can be prohibitive, impeding the application of LLMs. The large number of parameters in LLMs results in enormous time and computational consumption during fine-tuning, which is difficult to justify. To address these two issues simultaneously, we propose a novel parameter-efficient fine-tuning framework for multi-task medical applications called MOELoRA. The framework aims to capitalize on the benefits of both MOE for multi-task learning and LoRA for parameter-efficient fine-tuning. To unify MOE and LoRA, we devise multiple experts as the trainable parameters, where each expert consists of a pair of low-rank matrices to maintain a small number of trainable parameters. Additionally, we propose a task-motivated gate function for all MOELoRA layers that can regulate the contributions of each expert and generate distinct parameters for various tasks. To validate the effectiveness and practicality of the proposed method, we conducted comprehensive experiments on a public multi-task Chinese medical dataset. The experimental results demonstrate that MOELoRA outperforms existing parameter-efficient fine-tuning methods. The implementation is available online for convenient reproduction of our experiments

Design and Implementation of Recursive Model Predictive Control for Permanent Magnet Synchronous Motor Drives

In order to control the permanent-magnet synchronous motor system (PMSM) with different disturbances and nonlinearity, an improved current control algorithm for the PMSM systems using recursive model predictive control (RMPC) is developed in this paper. As the conventional MPC has to be computed online, its iterative computational procedure needs long computing time. To enhance computational speed, a recursive method based on recursive Levenberg-Marquardt algorithm (RLMA) and iterative learning control (ILC) is introduced to solve the learning issue in MPC. RMPC is able to significantly decrease the computation cost of traditional MPC in the PMSM system. The effectiveness of the proposed algorithm has been verified by simulation and experimental results

The Impacts of Emission Control and Regional Transport on PM2.5 Ions and Carbon Components in Nanjing during the 2014 Nanjing Youth Olympic Games

Highly time-resolved measurements of water soluble ions, organic and elemental carbon concentrations in the particle diameter size range D-p <2.5 mu m (PM2.5) were performed at a downwind urban site in Nanjing in the western part of the Yangtze River Delta (YRD) in eastern China during the 2014 Youth Olympic Games (YOG). In this study, we discuss the impacts of emission control in Nanjing and the surrounding areas during the YOG and regional/long-range transport on PM2.5 pollution in Nanjing. The average concentrations of NO3-, SO42-, NH4+ were 12.1 +/- 9.9, 16.5 +/- 9.2, 9.0 +/- 5.4 mu g m(-3) during the YOG, and increased 34.3%, 53.7%, 43.9% after the YOG, respectively. The control of construction or on-road soil dust and control of industry led to the decrease of Ca2+ concentration by 55% and SO2 concentration by 46%. However, SO42- concentrations remained at relatively high levels, suggesting a significant impact of regional pollution to secondary fine particles in Nanjing. Strong correlations between OC and EC were observed during and after the YOG. A higher percentage (41%) of secondary organic carbon in Nanjing during the YOG periods was consistent with high potential photochemistry and low contributions from coal combustion. Lagrangian dispersion modelling results proved that the city clusters along the Nanjing and Shanghai axis were the major source region for high PM2.5 pollution in upwind Nanjing. This work shows that short-term strict control measures could improve the air quality, especially that affected by the primary pollutants; however, regional collaborative control strategy across administrative borders in the YRD is needed for a substantial improvement of air quality.Peer reviewe

Intrinsic Surface Effects of Tantalum and Titanium on Integrin α5β1/ ERK1/2 Pathway-Mediated Osteogenic Differentiation in Rat Bone Mesenchymal Stromal Cells

Background/Aims: Accumulating evidence demonstrates the superior osteoinductivity of tantalum (Ta) to that of titanium (Ti); however, the mechanisms underlying these differences are unclear. Thus, the objective of the present study was to examine the effects of Ta and Ti surfaces on osteogenesis using rat bone mesenchymal stromal cells (rBMSCs) as a model. Methods: Ta and Ti substrates were polished to a mirror finish to minimize the influences of structural factors, and the intrinsic surface effects of the two materials on the integrin α5β1/mitogen-activated protein kinases 3 and 1 (ERK1/2) cascade-mediated osteogenesis of rBMSCs were evaluated. Alkaline phosphatase (ALP) activity, Alizarin Red staining, real-time polymerase chain reaction, and western blot assays of critical osteogenic markers were conducted to evaluate the effects of the two substrates on cell osteogenesis. Moreover, the role of the integrin α5β1/ERK1/2 pathway on the osteoinductive performance of Ta and Ti was assessed by up- and down-regulation of integrin α5 and β1 with RNA interference, as well as through ERK1/2 inhibition with U0126. Results: Osteogenesis of rBMSCs seeded on the Ta surface was superior to that of cells seeded on the Ti surface in terms of ALP activity, extracellular matrix calcification, and the expression of integrin α5, integrin β1, ERK1/2, Runt-related transcription factor 2, osteocalcin, collagen type I, and ALP at both the mRNA and protein levels. Moreover, down-regulation of integrin α5 or integrin β1, or ERK1/2 inhibition severely impaired the osteoblastic differentiation on the Ta surface. By contrast, over-expression of integrin α5 or integrin β1 improved osteogenesis on the Ti substrates, while subsequent ERK1/2 inhibition abrogated this effect. Conclusion: The integrin α5β1/ERK1/2 pathway plays a crucial role in regulating rBMSCs osteogenic differentiation; thus, the greater ability of a Ta surface to trigger integrin α5β1/ERK1/2 signaling may explain its better osteoinductivity. The different effects of Ta and Ti surfaces on rBMSC osteogenesis are considered to be related to the conductive behaviors between integrin α5β1 and the oxides spontaneously formed on the two metals. These results should facilitate the development of engineering strategies with Ta and Ti surfaces for improved osteogenesis in endosteal implants

Preparation and Application of Starch/Polyvinyl Alcohol/Citric Acid Ternary Blend Antimicrobial Functional Food Packaging Films

Ternary blend films were prepared with different ratios of starch/polyvinyl alcohol (PVA)/citric acid. The films were characterized by field emission scanning electron microscopy (FE-SEM), thermogravimetric analysis, as well as Fourier transform infrared (FTIR) analysis. The influence of different ratios of starch/polyvinyl alcohol (PVA)/citric acid and different drying times on the performance properties, transparency, tensile strength (TS), water vapor permeability (WVP), water solubility (WS), color difference (ΔE), and antimicrobial activity of the ternary blends films were investigated. The starch/polyvinyl alcohol/citric acid (S/P/C1:1:0, S/P/C3:1:0.08, and S/P/C3:3:0.08) films were all highly transparent. The S/P/C3:3:0.08 had a 54.31 times water-holding capacity of its own weight and its mechanical tensile strength was 46.45 MPa. In addition, its surface had good uniformity and compactness. The S/P/C3:1:0.08 and S/P/C3:3:0.08 showed strong antimicrobial activity to Listeria monocytogenes and Escherichia coli, which were the food-borne pathogenic bacteria used. The freshness test results of fresh figs showed that all of the blends prevented the formation of condensed water on the surface of the film, and the S/P/C3:1:0.08 and S/P/C3:3:0.08 prevented the deterioration of figs during storage. The films can be used as an active food packaging system due to their strong antibacterial effect

An Overview of Plant Phenolic Compounds and Their Importance in Human Nutrition and Management of Type 2 Diabetes

In this paper, the biosynthesis process of phenolic compounds in plants is summarized, which includes the shikimate, pentose phosphate and phenylpropanoid pathways. Plant phenolic compounds can act as antioxidants, structural polymers (lignin), attractants (flavonoids and carotenoids), UV screens (flavonoids), signal compounds (salicylic acid and flavonoids) and defense response chemicals (tannins and phytoalexins). From a human physiological standpoint, phenolic compounds are vital in defense responses, such as anti-aging, anti-inflammatory, antioxidant and anti-proliferative activities. Therefore, it is beneficial to eat such plant foods that have a high antioxidant compound content, which will cut down the incidence of certain chronic diseases, for instance diabetes, cancers and cardiovascular diseases, through the management of oxidative stress. Furthermore, berries and other fruits with low-amylase and high-glucosidase inhibitory activities could be regarded as candidate food items in the control of the early stages of hyperglycemia associated with type 2 diabetes