MOELoRA: An MOE-based Parameter Efficient Fine-Tuning Method for Multi-task Medical Applications

The recent surge in the field of Large Language Models (LLMs) has gained significant attention in numerous domains. In order to tailor an LLM to a specific domain such as a web-based healthcare system, fine-tuning with domain knowledge is necessary. However, two issues arise during fine-tuning LLMs for medical applications. The first is the problem of task variety, where there are numerous distinct tasks in real-world medical scenarios. This diversity often results in suboptimal fine-tuning due to data imbalance and seesawing problems. Additionally, the high cost of fine-tuning can be prohibitive, impeding the application of LLMs. The large number of parameters in LLMs results in enormous time and computational consumption during fine-tuning, which is difficult to justify. To address these two issues simultaneously, we propose a novel parameter-efficient fine-tuning framework for multi-task medical applications called MOELoRA. The framework aims to capitalize on the benefits of both MOE for multi-task learning and LoRA for parameter-efficient fine-tuning. To unify MOE and LoRA, we devise multiple experts as the trainable parameters, where each expert consists of a pair of low-rank matrices to maintain a small number of trainable parameters. Additionally, we propose a task-motivated gate function for all MOELoRA layers that can regulate the contributions of each expert and generate distinct parameters for various tasks. To validate the effectiveness and practicality of the proposed method, we conducted comprehensive experiments on a public multi-task Chinese medical dataset. The experimental results demonstrate that MOELoRA outperforms existing parameter-efficient fine-tuning methods. The implementation is available online for convenient reproduction of our experiments

Large Language Model Distilling Medication Recommendation Model

The recommendation of medication is a vital aspect of intelligent healthcare systems, as it involves prescribing the most suitable drugs based on a patient's specific health needs. Unfortunately, many sophisticated models currently in use tend to overlook the nuanced semantics of medical data, while only relying heavily on identities. Furthermore, these models face significant challenges in handling cases involving patients who are visiting the hospital for the first time, as they lack prior prescription histories to draw upon. To tackle these issues, we harness the powerful semantic comprehension and input-agnostic characteristics of Large Language Models (LLMs). Our research aims to transform existing medication recommendation methodologies using LLMs. In this paper, we introduce a novel approach called Large Language Model Distilling Medication Recommendation (LEADER). We begin by creating appropriate prompt templates that enable LLMs to suggest medications effectively. However, the straightforward integration of LLMs into recommender systems leads to an out-of-corpus issue specific to drugs. We handle it by adapting the LLMs with a novel output layer and a refined tuning loss function. Although LLM-based models exhibit remarkable capabilities, they are plagued by high computational costs during inference, which is impractical for the healthcare sector. To mitigate this, we have developed a feature-level knowledge distillation technique, which transfers the LLM's proficiency to a more compact model. Extensive experiments conducted on two real-world datasets, MIMIC-III and MIMIC-IV, demonstrate that our proposed model not only delivers effective results but also is efficient. To ease the reproducibility of our experiments, we release the implementation code online

Analysis of genetic and nongenetic factors influencing triglycerides-lowering drug effects based on paired observations

Abstract Obesity is a risk factor for heart disease, stroke, diabetes, high blood pressure, and other chronic diseases. Some drugs, including fenofibrate, are used to treat obesity or excessive weight by lowering the level of specific triglycerides. However, different groups have different drug sensitivities and, consequently, there are differences in drug effects. In this study, we assessed both genetic and nongenetic factors that influence drug responses and stratified patients into groups based on differential drug effect and sensitivity. Our methodology of investigating genetic factors and nongenetic factors is applicable to studying differential effects of other drugs, such as statins, and provides an approach to the development of personalized medicine

Analysis of genetic and nongenetic factors influencing triglycerides-lowering drug effects based on paired observations

Obesity is a risk factor for heart disease, stroke, diabetes, high blood pressure, and other chronic diseases. Some drugs, including fenofibrate, are used to treat obesity or excessive weight by lowering the level of specific triglycerides. However, different groups have different drug sensitivities and, consequently, there are differences in drug effects. In this study, we assessed both genetic and nongenetic factors that influence drug responses and stratified patients into groups based on differential drug effect and sensitivity. Our methodology of investigating genetic factors and nongenetic factors is applicable to studying differential effects of other drugs, such as statins, and provides an approach to the development of personalized medicine

Analysis of genetic and nongenetic factors influencing triglycerides-lowering drug effects based on paired observations

Obesity is a risk factor for heart disease, stroke, diabetes, high blood pressure, and other chronic diseases. Some drugs, including fenofibrate, are used to treat obesity or excessive weight by lowering the level of specific triglycerides. However, different groups have different drug sensitivities and, consequently, there are differences in drug effects. In this study, we assessed both genetic and nongenetic factors that influence drug responses and stratified patients into groups based on differential drug effect and sensitivity. Our methodology of investigating genetic factors and nongenetic factors is applicable to studying differential effects of other drugs, such as statins, and provides an approach to the development of personalized medicine

Resilience-oriented planning method of local emergency power supply considering V2B

Private electric vehicles (EVs) have great potential to conduct emergency power supply, considering the rapid development of EVs and vehicle-to-building (V2B) technologies. To enhance the resilience of the building power supply, charging piles can be upgraded to support bi-directional power supply, thus enabling EVs to help restore the buildings affected by disasters. A planning method for the charging piles’ upgrade is proposed. First, a scenario set is generated to consider the influence of uncertainties during the planning period. The uncertainties of disasters, EVs, and building load are included. Then, a two-stage stochastic programming model is established to decide the upgrade plan. Pre-disaster decisions are made in the first stage and the building is restored in the second stage. The method is applied to two different types of buildings and the test results verify the effectiveness of the proposed method

Synthesis of ethyl 4-(2-fluoro-4-nitrophenoxy) picolinate

Cancer has seriously affected people's production and life. The appearance of anti-cancer drugs has brought good news to people. Ethyl 4-(2-fluoro-4-nitrophenoxy) picolinate is an important basic skeleton of a small molecule inhibitor of c-Met and a major intermediate in cancer therapy. A rapid and efficient method for the synthesis of compound 8 was established. Compound 8 was synthesized from picolinic acid by acylation and substitution. These steps were weight gain reaction. The synthesis method was optimized and the structure was confirmed by hydrogen NMR spectroscopy

Synthesis of ethyl 4-(2-fluoro-4-nitrophenoxy) picolinate

Cancer has seriously affected people's production and life. The appearance of anti-cancer drugs has brought good news to people. Ethyl 4-(2-fluoro-4-nitrophenoxy) picolinate is an important basic skeleton of a small molecule inhibitor of c-Met and a major intermediate in cancer therapy. A rapid and efficient method for the synthesis of compound 8 was established. Compound 8 was synthesized from picolinic acid by acylation and substitution. These steps were weight gain reaction. The synthesis method was optimized and the structure was confirmed by hydrogen NMR spectroscopy

Design, Synthesis, Activity and Docking Study of Sorafenib Analogs Bearing Sulfonylurea Unit

Two series of novel sorafenib analogs containing a sulfonylurea unit were synthesized and their chemical structures were confirmed by 1H-NMR, 13C-NMR, MS spectrum and elemental analysis. The synthesized compounds were evaluated for the cytotoxicity against A549, Hela, MCF-7, and PC-3 cancer cell lines. Some of the compounds showed moderate cytotoxic activity, especially compounds 1-(2,4-difluorophenylsulfonyl)-3-(4-(2-(methylcarbamoyl)pyridin-4-yloxy)phenyl)urea (6c) and 1-(4-bromophenylsulfonyl)-3-(4-(2-(methylcarbamoyl)pyridin-4-yloxy)phenyl)urea (6f) with the IC50 values against four cancer cell lines ranging from 16.54 ± 1.22 to 63.92 ± 1.81 μM, respectively. Inhibitory rates against vascular endothelial growth factor receptor-2 (VEGFR2/KDR) kinase at 10 μM of target compounds were further carried out in this paper in order to investigate the target of these compounds. Structure-activity relationships (SARs) and docking studies indicated that the sulfonylurea unit was important to these kinds of compounds. None of the substitutions in the phenoxy group and small halogen atoms such as 2,4-difluoro substitution of the aryl group contributed to the activity. The results suggested that sulfonylurea sorafenib analogs are worthy of further study

Analysis of genetic and nongenetic factors influencing triglycerides-lowering drug effects based on paired observations

Abstract Obesity is a risk factor for heart disease, stroke, diabetes, high blood pressure, and other chronic diseases. Some drugs, including fenofibrate, are used to treat obesity or excessive weight by lowering the level of specific triglycerides. However, different groups have different drug sensitivities and, consequently, there are differences in drug effects. In this study, we assessed both genetic and nongenetic factors that influence drug responses and stratified patients into groups based on differential drug effect and sensitivity. Our methodology of investigating genetic factors and nongenetic factors is applicable to studying differential effects of other drugs, such as statins, and provides an approach to the development of personalized medicine