MOELoRA: An MOE-based Parameter Efficient Fine-Tuning Method for Multi-task Medical Applications

The recent surge in the field of Large Language Models (LLMs) has gained significant attention in numerous domains. In order to tailor an LLM to a specific domain such as a web-based healthcare system, fine-tuning with domain knowledge is necessary. However, two issues arise during fine-tuning LLMs for medical applications. The first is the problem of task variety, where there are numerous distinct tasks in real-world medical scenarios. This diversity often results in suboptimal fine-tuning due to data imbalance and seesawing problems. Additionally, the high cost of fine-tuning can be prohibitive, impeding the application of LLMs. The large number of parameters in LLMs results in enormous time and computational consumption during fine-tuning, which is difficult to justify. To address these two issues simultaneously, we propose a novel parameter-efficient fine-tuning framework for multi-task medical applications called MOELoRA. The framework aims to capitalize on the benefits of both MOE for multi-task learning and LoRA for parameter-efficient fine-tuning. To unify MOE and LoRA, we devise multiple experts as the trainable parameters, where each expert consists of a pair of low-rank matrices to maintain a small number of trainable parameters. Additionally, we propose a task-motivated gate function for all MOELoRA layers that can regulate the contributions of each expert and generate distinct parameters for various tasks. To validate the effectiveness and practicality of the proposed method, we conducted comprehensive experiments on a public multi-task Chinese medical dataset. The experimental results demonstrate that MOELoRA outperforms existing parameter-efficient fine-tuning methods. The implementation is available online for convenient reproduction of our experiments

Observation on A-PRF promoting regeneration of osteochondral defects in rabbit knee joints

Objective·To explore the role of advanced platelet-rich fibrin (A-PRF) in osteochondral regeneration.Methods·Bone-marrow mesenchymal stem cells (BMSCs) and knee joint chondrocytes were obtained from New Zealand rabbits. A-PRF was obtained by low-speed centrifugation of the heart blood of rabbits. The histological structure of A-PRF was observed by an optical microscope. The release of growth factors in A-PRF was detected by ELISA, including platelet-derived growth factor, transforming growth factor-β, insulin-like growth factor, vascular endothelial growth factor, epidermal growth factor and fibroblast growth factor. A-PRF's cytotoxicity and capability for promoting the proliferation of rabbit BMSCs were detected by live/dead double staining and MTT methods. The effect of A-PRF on the gene expression of type Ⅱ collagen, aggrecan, alkaline phosphatase (ALP) and osteocalcin (OCN) in rabbit BMSCs was detected by real-time fluorescence quantitative polymerase chain reaction (qRT-PCR). Transwell chambers were used to determine the effect of A-PRF on the migration ability of rabbit BMSCs and the chondrocytes. Rabbit knee osteochondral defect models were established, and 18 rabbits were randomly divided into 3 groups. The A-PRF group (n=6) was implanted with A-PRF in the defect, the A-PRF+BMSCs group (n=6) was implanted with rabbit BMSCs on A-PRF, and the control group (n=6) did not undergo implantation. The rabbits were sacrificed 12 weeks after surgery and the knee joint specimens were stained with hematoxylin-eosin (H-E), toluidine blue and safranin O/fast green. Based on the surface morphology and histology of the knee joints, the International Cartilage Repair Society (ICRS) scoring system was used for macroscopic and histological scoring.Results·A-PRF had a loose network structure and can slowly release growth factors. No cytotoxicity to rabbit BMSCs was observed after adding A-PRF, and the the capability for promoting the proliferation of rabbit BMSCs was significantly increased at 24, 48 and 72 h after adding A-PRF (all P<0.05). Chondrogenesis-related gene Ⅱ collagen and aggrecan, as well as osteogenesis-related genes ALP and OCN were significantly up-regulated (all P<0.05). After adding A-PRF, the migration abilities of rabbit BMSCs and chondrocytes were significantly enhanced (both P<0.05), and the migration ability of rabbit BMSCs was significantly higher than that of chondrocytes (P=0.025). The joint surface morphology in the rabbit knee joint defect models was observed. It can be seen that the defects in the A-PRF group and the A-PRF+BMSCs group were basically restored, while the the defects in the control group were only covered by soft tissue. In the ICRS macroscopic score, there was no statistical difference between the A-PRF group and the A-PRF+BMSCs group, but the scores of the two groups were all significantly higher than those of the control group (all P<0.05). According to the histological results, both the A-PRF group and the A-PRF+BMSCs group formed osteochondral repair, but the cartilage in the A-PRF group was more mature, while the control group formed fibrous repair. In the ICRS histological score, there was no statistical difference between the A-PRF group and the A-PRF+BMSCs group, but the scores of both the groups were significantly higher than those of the control group (both P<0.05).Conclusion·Autologous A-PRF has good biocompatibility and the capability for promoting the proliferation of BMSCs. It can promote the repair of cartilage and subchondral bone both in vitro and in vivo

Formation and Identification of Unresolved Complex Mixtures in Lacustrine Biodegraded Oil from Nanxiang Basin, China

A comprehensive two-dimensional gas chromatography/time-of-flight mass spectrometry (GC × GC/TOFMS) method has been developed for the formation and identification of unresolved complex mixtures (UCMs) in lacustrine biodegraded oils that with the same source rock, similar maturity, and increasing degradation rank from Nanxiang Basin, China. Normal alkanes, light hydrocarbons, isoprenoids, steranes, and terpanes are degraded gradually from oil B330 to oil G574. The compounds in biodegraded oil (oil G574) have fewer types, the polarity difference of compounds in different types is minor, and the relative content of individual compounds is similar. All the features make the compounds in biodegraded oil coelute in GC analysis and form the raised “baseline hump” named UCMs. By injecting standard materials and analyzing mass spectrums of target compounds, it is shown that cyclic alkanes with one to five rings are the major components of UCMs. Furthermore, UCMs were divided into six classes. Classes I and II, composed of alkyl-cyclohexanes, alkyl-naphthanes, and their isomers, are originated from the enrichment of hydrocarbons resistant to degradation in normal oils. Classes III ~ VI, composed of sesquiterpenoids, tricyclic terpanes, low molecular steranes, diasteranes, norhopanes, and their isomers, are probably from some newly formed compounds during the microbial transformation of oil

Fully endoscopic transforaminal discectomy for thoracolumbar junction disc herniation with or without calcification under general anesthesia: Technical notes and preliminary outcomes

ObjectiveTo evaluate the feasibility, safety, and outcomes of percutaneous endoscopic transforaminal discectomy (PETD) for thoracolumbar junction disc herniation (TLDH) with or without calcification.MethodsThis study included 12 patients diagnosed with TLDH with or without calcification who met the inclusion criteria and underwent surgery for PETD from January 2019 to December 2021. The mean patient age, operation time, hospitalization time, time in bed, and complications were recorded. Patients were followed up for at least 9 months. Visual analog scale (VAS) scores for low-back and leg or thoracic radicular pain and modified Japanese Orthopedic Association score (m-JOA) scores were preoperatively evaluated, at 1 day and 3, 6, and 12 months postoperatively or at last follow-up. The modified MacNab criteria were used to evaluate clinical efficacy at 12 months postoperatively or at last follow-up.ResultsThe mean patient age, operation time, hospitalization time, and time in bed were 53 ± 13.9 years, 101.3 ± 9.2 min, 4.5 ± 1.3 days, and 18.0 ± 7.0 h, respectively. The mean VAS scores of low-back and leg or thoracic radicular pain improved from 5.8 ± 1.5 and 6.5 ± 1.4 to 2.0 ± 0.9 and 1.3 ± 0.5, respectively (P &lt; 0.05). The m-JOA score improved from 7.5 ± 1.2 to 10.0 ± 0.7 (P &lt; 0.05). The overall excellent–good rate of the modified MacNab criteria was 83.3%. No severe complications occurred.ConclusionFully endoscopic transforaminal discectomy and ventral decompression under general anesthesia is a safe, feasible, effective, and minimally invasive method for treating herniated discs with or without calcification at thoracolumbar junction zone

Room temperature molten salts as media for the development of negativeelectrodes in lithium ion batteries and the electrochemical formationof high temperature superconductor precursor

published_or_final_versionChemistryDoctoralDoctor of Philosoph

Research on kinematics analysis of spherical single-cone PDC compound bit and rock breaking simulation verification

The single-cone bit has become the first choice for slim hole sidetracking and deep well drilling with its unique rock breaking method and high ROP (Rate Of Penetration), with its main failure mode being of early excessive wear of the cutting teeth. In order to improve the adaptability of single-cone bits to hard and highly abrasive formations, a spherical single-cone Polycrystalline Diamond Compact (PDC) compound bit is designed. According to the characteristics of the tooth profile, the way of tooth arrangement and the way of contact between the cutting teeth and the rock, the acceleration equation to the cutting teeth of the spherical single-cone PDC compound bit is established. The acceleration of the single-cone bit is verified by numerical simulation experiment of rock-breaking. The shaft inclination angle of the cone, the position and height of the PDC teeth, the radius of the PDC teeth, the lateral rotation angle and the front inclination angle on the acceleration are studied. The results show that as the shaft inclination angle increases, the bit transmission ratio gradually increases, and the harder the rock formation, the larger the transmission ratio of the single-cone bit; the shaft inclination angle and the position of the PDC tooth have a greater influence on the acceleration of the PDC tooth, and the radius, lateral rotation angle and front inclination angle of the PDC tooth have a small influence on the acceleration of the PDC tooth; rock properties have an impact on the acceleration of the cutting teeth, with the acceleration of the cutting teeth in hard rock formations being higher than that in soft rock formations; near the top of the cone, the absolute acceleration of the cutting teeth will fluctuate sharply and cause severe wear of the cutting teeth, so the tooth distribution in this area should be strengthened; on the premise that the bearing life of the single-cone bit is allowed, the value of the shaft inclination angle β can be approached to 70°. The relative error between the theoretical analysis results of the acceleration of the PDC cutter and the rock-breaking simulation experiment results is between −0.95% and −2.24%. This research lays a theoretical foundation for the dynamic research of spherical single-cone PDC compound bit

Adaptive dynamic programming for robust neural control of unknown continuous-time non-linear systems

The design of robust controllers for continuous-time (CT) non-linear systems with completely unknown non-linearities is a challenging task. The inability to accurately identify the non-linearities online or offline motivates the design of robust controllers using adaptive dynamic programming (ADP). In this study, an ADP-based robust neural control scheme is developed for a class of unknown CT non-linear systems. To begin with, the robust non-linear control problem is converted into a non-linear optimal control problem via constructing a value function for the nominal system. Then an ADP algorithm is developed to solve the non-linear optimal control problem. The ADP algorithm employs actor-critic dual networks to approximate the control policy and the value function, respectively. Based on this architecture, only system data is necessary to update simultaneously the actor neural network (NN) weights and the critic NN weights. Meanwhile, the persistence of excitation assumption is no longer required by using the Monte Carlo integration method. The closed-loop system with unknown non-linearities is demonstrated to be asymptotically stable under the obtained optimal control. Finally, two examples are provided to validate the developed method

Nanoparticle-Based Systems for T1-Weighted Magnetic Resonance Imaging Contrast Agents

Because magnetic resonance imaging (MRI) contrast agents play a vital role in diagnosing diseases, demand for new MRI contrast agents, with an enhanced sensitivity and advanced functionalities, is very high. During the past decade, various inorganic nanoparticles have been used as MRI contrast agents due to their unique properties, such as large surface area, easy surface functionalization, excellent contrasting effect, and other size-dependent properties. This review provides an overview of recent progress in the development of nanoparticle-based T1-weighted MRI contrast agents. The chemical synthesis of the nanoparticle-based contrast agents and their potential applications were discussed and summarized. In addition, the recent development in nanoparticle-based multimodal contrast agents including T1-weighted MRI/computed X-ray tomography (CT) and T1-weighted MRI/optical were also described, since nanoparticles may curtail the shortcomings of single mode contrast agents in diagnostic and clinical settings by synergistically incorporating functionality

Numerical Simulation and Analysis of Diffusion Process for the Leakages of a Tunnel LNG Pipeline

Taking a practical project as an example, based on the computational fluid dynamics (CFD), standard k-ε model and finite element method, a mathematical model for the diffusion due to liquefied natural gas (LNG) pipeline leakage in a tunnel was established, and the diffusion process was numerically simulated for three LNG leakage cases. From the simulation results, the variation of CH4 concentration field and explosive gas cloud with time within the tunnel, and the influence of leakage location on the diffusion was analyzed for the three cases. It was shown that the variation of CH4 concentration field had a similar trend for the three cases, but the CH4 explosive gas cloud length increased rapidly with the LNG leakage intensity so that dangerous situations would occur for the medium and large leakages, and a leak location closer to air inlet would lead to a more dangerous situation. When the amount of LNG leakage in the tunnel is large, the effect of mechanical ventilation is obviously weakened. Furthermore, a nitrogen seal precaution was proposed for the situations

Investigation of the Ordered Structure in Partially Melted Isotactic Polypropylene

The ordered structure of partially melted isotactic polypropylene (iPP) was investigated using polarized optical microscopy (POM) and small-/wide-angle X-ray scattering (SAXS/WAXS) measurements. The crystalline morphology was first examined by means of pulling a glass fiber through the iPP melt, which was generated by partially melting a preformed spherulite. The results from the POM experiments indicated that, even at a minimal pulling rate, the surviving ordered structure could also relocate along the direction of fiber pulling and grow into cylindrites eventually. In addition, during the quiescent crystallization from the partially melted sample, which had the same thermal history of fiber-pulling experiments, the obvious memory effect of melt was also observed from the results of X-ray experiments. Moreover, the SAXS profile derived from the partially melted iPP at 170 °C was fitted by the theory of scattering amplitude with the cylindrical form factor. The fit result implied that the surviving ordered structure is of cylindrical nanocrystals with a diameter D ≈ 30 ± 3 nm and height h ≈ 45 ± 3 nm, which can significantly influence the crystallization morphology and kinetics during the subsequent crystallization process