17 research outputs found
OpenDelta: A Plug-and-play Library for Parameter-efficient Adaptation of Pre-trained Models
The scale of large pre-trained models (PTMs) poses significant challenges in
adapting to downstream tasks due to the high optimization overhead and storage
costs associated with full-parameter fine-tuning. To address this, many studies
explore parameter-efficient tuning methods, also framed as "delta tuning",
which updates only a small subset of parameters, known as "delta modules",
while keeping the backbone model's parameters fixed. However, the practicality
and flexibility of delta tuning have been limited due to existing
implementations that directly modify the code of the backbone PTMs and
hard-code specific delta tuning methods for each PTM. In this paper, we present
OpenDelta, an open-source library that overcomes these limitations by providing
a plug-and-play implementation of various delta tuning methods. Our novel
techniques eliminate the need to modify the backbone PTMs' code, making
OpenDelta compatible with different, even novel PTMs. OpenDelta is designed to
be simple, modular, and extensible, providing a comprehensive platform for
researchers and practitioners to adapt large PTMs efficiently.Comment: Accepted to ACL 2023 Demo trac
SARS-CoV-2-Specific Adaptive Immunity in COVID-19 Survivors With Asthma
BackgroundAsthma patients potentially have impaired adaptive immunity to virus infection. The levels of SARS-CoV-2-specific adaptive immunity between COVID-19 survivors with and without asthma are presently unclear.MethodsCOVID-19 survivors (patients with asthma n=11, with allergies n=8, and COVID-19 only n=17) and non-COVID-19 individuals (asthmatic patients n=10 and healthy controls n=9) were included. The COVID-19 patients were followed up at about 8 months and 16 months after discharge. The clinical characteristics, lymphocyte subsets, memory T cells, and humoral immunity including SARS-CoV-2 specific antibodies, SARS-CoV-2 pseudotyped virus neutralization assay, and memory B cells were analyzed in these subjects.ResultsThe strength of virus-specific T cell response in COVID-19 survivors was positively correlated with the percentage of blood eosinophils and Treg cells (r=0.4007, p=0.0188; and r=0.4435, p=0.0086 respectively) at 8-month follow-up. There were no statistical differences in the levels of SARS-CoV-2-specific T cell response between the COVID-19 survivors with, and without, asthma. Compared to those without asthma, the COVID-19 with asthma survivors had higher levels of SARS-CoV-2-specific neutralizing antibodies (NAbs) at the 8-month follow-up (p<0.05). Moreover, the level of NAbs in COVID-19 survivors was positively correlated with the percentage of Treg and cTfh2 cells (r=0.5037, p=0.002; and r=0.4846, p=0.0141), and negatively correlated with the percentage of Th1 and Th17 cells (r=-0.5701, p=0.0003; and r=-0.3656, p=0.0308), the ratio of Th1/Th2, Th17/Treg, and cTfh1/cTfh2 cell (r=-0.5356, r=-0.5947, r=-0.4485; all p<0.05). The decay rate of NAbs in the COVID-19 survivors with asthma was not significantly different from that of those without asthma at 16-month follow-up.ConclusionThe level of SARS-CoV-2-specific NAbs in COVID-19 survivors with asthma was higher than that of those without asthma at 8-month follow-up. The SARS-CoV-2-specific T cell immunity was associated with blood eosinophils and Treg percentages. The SARS-CoV-2-specific humoral immunity was closely associated with cTfh2/cTfh1 imbalance and Treg/Th17 ratio. According to the findings, asthmatic patients in COVID-19 convalescent period may benefit from an enhanced specific humoral immunity, which associates with skewed Th2/Th1 and Treg/Th17 immune
Sparse Structure Search for Parameter-Efficient Tuning
Adapting large pre-trained models (PTMs) through fine-tuning imposes
prohibitive computational and storage burdens. Recent studies of
parameter-efficient tuning (PET) find that only optimizing a small portion of
parameters conditioned on PTMs could yield on-par performance compared to
conventional fine-tuning. Generally, PET methods exquisitely design
parameter-efficient modules (PET modules) which could be applied to arbitrary
fine-grained positions inside PTMs. However, the effectiveness of these
fine-grained positions largely relies on sophisticated manual designation,
thereby usually producing sub-optimal results. In contrast to the manual
designation, we explore constructing PET modules in an automatic manner. We
automatically \textbf{S}earch for the \textbf{S}parse \textbf{S}tructure of
\textbf{P}arameter-\textbf{E}fficient \textbf{T}uning (SPET). Based on a
unified framework of various PET methods, SPET conducts the differentiable
PET structure search through bi-level optimization and proposes shifted global
sigmoid method to explicitly control the number of trainable parameters.
Extensive experiments show that SPET surpasses manual and random structures
with less trainable parameters. The searched structures preserve more than 99\%
fine-tuning performance with 0.01\% trainable parameters. Moreover, the
advantage of SPET is amplified with extremely low trainable parameters
budgets (0.0009\%0.01\%). The searched structures are transferable and
explainable, providing suggestions and guidance for the future design of PET
methods
In-Situ Growth of Au on KTaO3 Sub-Micron Cubes via Wet Chemical Approach for Enhanced Photodegradation of p-Nitrophenol
Multiple linear regression analysis of the factors related with miR-199a-5p expression.
<p>Multiple linear regression analysis of the factors related with miR-199a-5p expression.</p
Novel imaging phenotypes of naïve asthma patients with distinctive clinical characteristics and T2 inflammation traits
Objective:This study aims to describe the imaging features of naïve asthma patients, defined as not receiving corticosteroids or other asthma medications for at least 1 month, and their association with therapeutic response, and to discover novel unbiased imaging phenotypes.Methods:A total of 109 naïve asthma patients and 50 healthy controls were enrolled in this study. Clinical data and imaging indices of high-resolution computed tomography were collected. The correlation between imaging indices and clinical features was analyzed. Cluster analyses were adopted to determine three novel imaging phenotypes.Results:Compared with healthy controls, naïve asthma patients presented higher scores of airway remodeling, bronchiectasis, and mucus plugs. Mean airway wall area (WA)% was inversely correlated with mid-expiratory flow velocity% predicted. The extent score of bronchiectasis was positively correlated with smoking history and significantly increased in the high mucus group. Mucus plugs were related to improving lung function and type 2 (T2) inflammation, as assessed by sputum and blood eosinophils and fraction of exhaled nitric oxide. Cluster 1 patients had a high proportion of emphysema, the best lung function, and the lowest T2 inflammation; cluster 2 patients had severe airway remodeling, relatively good lung function, and moderate T2 inflammation; cluster 3 patients had severe airway remodeling, mucus plugs, and bronchiectasis, and showed the worst lung function and highest T2 inflammation.Conclusion:Naïve asthma patients had the imaging traits of airway remodeling, bronchiectasis, and mucus plugs. The unbiased imaging phenotypes had good consistency with clinical characteristics, therapeutic response, and T2 inflammation expression in naïve asthma patients
miR-199a-5p expression levels from different cells following LPS stimulation.
<p>(A-C) miR-199a-5p expression levels in PB-derived macrophages (A), lymphocytes (B) and neutrophils (C) isolated from three healthy subjects treated or untreated with LPS (100 ng/mL) for 24 hours. (D) miR-199a-5p expression levels in BEAS-2B cell lines, untreated or treated with LPS (100 ng/mL) for 24 hours. Relative miR-199a-5p levels were calculated over untreated controls.</p
Clinical characteristics of participants.
<p>Clinical characteristics of participants.</p
In silico analysis into the functional role of miRNA.
<p>Overview of the GO(A) and KEGG(B) pathways that contain at least 1 of the genes targeted by miR-199a-5p. Target genes are indicated on both sides. Targets expressed in the pathway are indicated in blue, and white indicates that the gene was not expressed in the gene set. Arrows represent pathways with potential interest in the proliferation of smooth muscle cells.</p
Relative miR-199a-5p levels in sputum in different groups.
<p>(A) Normalized miR-199a-5p expression levels in sputum from different inflammatory phenotypes and healthy controls. (B) The correlation between the miR-199a-5p expression levels in plasma and sputum. Data are presented as dot plots with fitted regression lines. The miR-199a-5p expression levels were normalized to U6. Spearman R-value and p-value are indicated.</p