Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization

Large foundation models are becoming ubiquitous, but training them from scratch is prohibitively expensive. Thus, efficiently adapting these powerful models to downstream tasks is increasingly important. In this paper, we study a principled finetuning paradigm -- Orthogonal Finetuning (OFT) -- for downstream task adaptation. Despite demonstrating good generalizability, OFT still uses a fairly large number of trainable parameters due to the high dimensionality of orthogonal matrices. To address this, we start by examining OFT from an information transmission perspective, and then identify a few key desiderata that enable better parameter-efficiency. Inspired by how the Cooley-Tukey fast Fourier transform algorithm enables efficient information transmission, we propose an efficient orthogonal parameterization using butterfly structures. We apply this parameterization to OFT, creating a novel parameter-efficient finetuning method, called Orthogonal Butterfly (BOFT). By subsuming OFT as a special case, BOFT introduces a generalized orthogonal finetuning framework. Finally, we conduct an extensive empirical study of adapting large vision transformers, large language models, and text-to-image diffusion models to various downstream tasks in vision and language.Comment: Technical Report (33 pages, 18 figures

Sepsis promotes splenic production of a protective platelet pool with high CD40 ligand expression.

Platelets have a wide range of functions including critical roles in hemostasis, thrombosis, and immunity. We hypothesized that during acute inflammation, such as in life-threatening sepsis, there are fundamental changes in the sites of platelet production and phenotypes of resultant platelets. Here, we showed during sepsis that the spleen was a major site of megakaryopoiesis and platelet production. Sepsis provoked an adrenergic-dependent mobilization of megakaryocyte-erythrocyte progenitors (MEPs) from the bone marrow to the spleen, where IL-3 induced their differentiation into megakaryocytes (MKs). In the spleen, immune-skewed MKs produced a CD40 ligandhi platelet population with potent immunomodulatory functions. Transfusions of post-sepsis platelets enriched from splenic production enhanced immune responses and reduced overall mortality in sepsis-challenged animals. These findings identify a spleen-derived protective platelet population that may be broadly immunomodulatory in acute inflammatory states such as sepsis.NHS Blood and Transplant European Union's Horizon 2020 (ERC-2014-CoG-648765) MRC-AMED (MR/V005421/1

ADAM8 signaling drives neutrophil migration and ARDS severity

Acute respiratory distress syndrome (ARDS) results in catastrophic lung failure and has an urgent, unmet need for improved early recognition and therapeutic development. Neutrophil influx is a hallmark of ARDS and is associated with the release of tissue-destructive immune effectors, such as matrix metalloproteinases (MMPs) and membrane-anchored metalloproteinase disintegrins (ADAMs). Here, we observed using intravital microscopy that Adam8–/– mice had impaired neutrophil transmigration. In mouse pneumonia models, both genetic deletion and pharmacologic inhibition of ADAM8 attenuated neutrophil infiltration and lung injury while improving bacterial containment. Unexpectedly, the alterations of neutrophil function were not attributable to impaired proteolysis but resulted from reduced intracellular interactions of ADAM8 with the actin-based motor molecule Myosin1f that suppressed neutrophil motility. In 2 ARDS cohorts, we analyzed lung fluid proteolytic signatures and identified that ADAM8 activity was positively correlated with disease severity. We propose that in acute inflammatory lung diseases such as pneumonia and ARDS, ADAM8 inhibition might allow fine-tuning of neutrophil responses for therapeutic gain

Integrated Impacts of Non-Ideal Factors on the Vibration Characteristics of Permanent Magnet Synchronous Motors for Electric Vehicles

The nonlinear electromagnetic vibration of the motor is a major factor that deteriorates the noise, vibration, and hardness (NVH) performance of a vehicle’s electric drive system. Considering the nonlinear characteristics of the inverter, the nonsinusoidal distribution of the air-gap magnetic field, the cogging torque, and the current measurement error, a mathematical model of a permanent magnet synchronous motor of an electric vehicle was established, and its dynamic and electromagnetic vibration characteristics under different speed–load conditions were simulated and analyzed. The results show that the nonlinear characteristics of the inverter and nonsinusoidal distribution of the air-gap magnetic field cause the odd current harmonics, such as the 5th, 7th, 11th, and 13th, which lead to the 6th and its integer multiple order fluctuations of the electromagnetic torque. Moreover, the vibration amplitude is intensified under the coupling action of the nonlinear characteristics of the inverter and the nonsinusoidal distribution of the air-gap magnetic field. The current measurement error produces the 1st and 2nd harmonics of the d- and q-axes currents, which result in the 1st and 2nd order fluctuations of the electromagnetic torque. The cogging torque mainly leads to a 12th order torque ripple of the electromagnetic torque. In addition, the non-ideal factors cause a sharp deterioration in the system vibration state under high-speed and heavy-load conditions. This study provides a theoretical reference for the mathematical modeling and electromagnetic vibration research of permanent magnet synchronous motors, considering non-ideal factors comprehensively

Academic Emotion Classification and Recognition Method for Large-scale Online Learning Environment—Based on A-CNN and LSTM-ATT Deep Learning Pipeline Method

Subjective well-being is a comprehensive psychological indicator for measuring quality of life. Studies have found that emotional measurement methods and measurement accuracy are important for well-being-related research. Academic emotion is an emotion description in the field of education. The subjective well-being of learners in an online learning environment can be studied by analyzing academic emotions. However, in a large-scale online learning environment, it is extremely challenging to classify learners’ academic emotions quickly and accurately for specific comment aspects. This study used literature analysis and data pre-analysis to build a dimensional classification system of academic emotion aspects for students’ comments in an online learning environment, as well as to develop an aspect-oriented academic emotion automatic recognition method, including an aspect-oriented convolutional neural network (A-CNN) and an academic emotion classification algorithm based on the long short-term memory with attention mechanism (LSTM-ATT) and the attention mechanism. The experiments showed that this model can provide quick and effective identification. The A-CNN model accuracy on the test set was 89%, and the LSTM-ATT model accuracy on the test set was 71%. This research provides a new method for the measurement of large-scale online academic emotions, as well as support for research related to students’ well-being in online learning environments

Molecularly imprinted solid phase extraction in a syringe needle packed with polypyrrole-encapsulated carbon nanotubes for determination of ochratoxin A in red wine

A novel micro-solid phase preconcentration (μSPP) device is developed by electrochemically depositing molecularly imprinted polypyrrole (MIPPy) over carbon nanotubes (CNTs) packed inside a 22-gauge syringe needle. The ochratoxin A (OTA) template is removed with 1% triethylamine (TEA) in 20:80 v/v acetonitrile-ammonia buffer (20 mM NH4Cl/NH3, pH 9.2). This syringe needle is used to extract trace OTA in a red wine sample, and the preconcentrated OTA is eluted with 1% (v/v) TEA in 20:80 v/v acetonitrile-ammonia buffer (20 mM NH4Cl/NH3, pH 9.2). The eluate is analyzed by high performance liquid chromatography (HPLC) with fluorescence detection (FD). The results demonstrated a significantly selective enrichment of OTA at sub-ppb levels in the presence of red wine matrix components. Using a sample volume of 0.5mL red wine for preconcentration, it is possible to determine OTA down to a detection limit of 0.04 ng/mL (at 3σB) or a quantification limit of 0.10 ng/mL (at 10σB). The total MIPPy/CNTs-μSPP-HPLC-FD analysis took only 40min, including a μSPP time of 30min, elution time of 20 s, and HPLC analysis time of 10 min. This needle can be reused and hence readily adapted in an autosampler for the processing of multiple samples in series