1,485 research outputs found
Improving Adaptability and Generalizability of Efficient Transfer Learning for Vision-Language Models
Vision-Language Models (VLMs) like CLIP have demonstrated remarkable
applicability across a variety of downstream tasks, including zero-shot image
classification. Recently, the use of prompts or adapters for efficient transfer
learning has gained significant attention for effectively adapting to
downstream tasks. However, the roles of vision and text prompts, as well as
adapters in terms of generalization and transfer difficulty, have been
overlooked, limiting performance on unseen tasks. In this paper, we empirically
analyze how VLMs behave when using vision and text prompts, adapters, and a
combination of these components, marking a novel exploration by our study. Our
observations find that utilizing vision prompts for class separability and text
adapters for task adaptation is crucial for adaptability and generalizability.
Moreover, to improve generalization across every domain, we propose an adaptive
ensemble method that effectively combines the general knowledge of VLMs with
task-specific knowledge according to transfer difficulty. Upon experimenting
with extensive benchmarks, our method consistently outperforms all baselines,
particularly on unseen tasks, demonstrating the effectiveness of our proposed
approach.Comment: 11 pages (19 pages including supplementary), 10 figures (12 figures
including supplementary), 6 tables (17 tables including supplementary
Fine tuning Pre trained Models for Robustness Under Noisy Labels
The presence of noisy labels in a training dataset can significantly impact
the performance of machine learning models. To tackle this issue, researchers
have explored methods for Learning with Noisy Labels to identify clean samples
and reduce the influence of noisy labels. However, constraining the influence
of a certain portion of the training dataset can result in a reduction in
overall generalization performance. To alleviate this, recent studies have
considered the careful utilization of noisy labels by leveraging huge
computational resources. Therefore, the increasing training cost necessitates a
reevaluation of efficiency. In other areas of research, there has been a focus
on developing fine-tuning techniques for large pre-trained models that aim to
achieve both high generalization performance and efficiency. However, these
methods have mainly concentrated on clean datasets, and there has been limited
exploration of the noisy label scenario. In this research, our aim is to find
an appropriate way to fine-tune pre-trained models for noisy labeled datasets.
To achieve this goal, we investigate the characteristics of pre-trained models
when they encounter noisy datasets. Through empirical analysis, we introduce a
novel algorithm called TURN, which robustly and efficiently transfers the prior
knowledge of pre-trained models. The algorithm consists of two main steps: (1)
independently tuning the linear classifier to protect the feature extractor
from being distorted by noisy labels, and (2) reducing the noisy label ratio
and fine-tuning the entire model based on the noise-reduced dataset to adapt it
to the target dataset. The proposed algorithm has been extensively tested and
demonstrates efficient yet improved denoising performance on various benchmarks
compared to previous methods.Comment: 10 pages (17 pages including supplementary
Comparison of volume-controlled and pressure-controlled ventilation using a laryngeal mask airway during gynecological laparoscopy
Background: Several publications have reported the successful, safe use of Laryngeal Mask Airway (LMA)-Classic devices in patients undergoing laparoscopic surgery. However, there have been no studies that have examined the application of volume-controlled ventilation (VCV) or pressure-controlled ventilation (PCV) using a LMA during gynecological laparoscopy. The aim of this study is to compare how the VCV and PCV modes and using a LMA affect the pulmonary mechanics, the gas exchange and the cardiovascular responses in patients who are undergoing gynecological laparoscopy. Methods: Sixty female patients were randomly allocated to one of two groups, (the VCV or PCV groups). In the VCV group, baseline ventilation of the lung was performed with volume-controlled ventilation and a tidal volume of 10 ml/kg ideal body weight (IBW). In the PCV group, baseline ventilation of the lung using pressure-controlled ventilation was initiated with a peak airway pressure that provided a tidal volume of 10 ml/kg IBW and an upper limit of 35 cmH2O. The end-tidal CO2, the peak airway pressures (Ppeak), the compliance, the airway resistance and the arterial oxygen saturation were recorded at T1: 5 minutes after insertion of the laryngeal airway, and at T2 and T3: 5 and 15 minutes, respectively, after CO2 insufflation. Results: The Ppeak at 5 minutes and 15 minutes after CO2 insufflation were significantly increased compared to the baseline values in both groups. Also, at 5 minutes and 15 minutes after CO2 insufflation, there were significant differences of the Ppeak between the two groups. The compliance decreased in both groups after creating the pneumopertoneim (P < 0.05). Conclusions: Our results demonstrate that PCV may be an effective method of ventilation during gynecological laparoscopy, and it ensures oxygenation while minimizing the increases of the peak airway pressure after CO2 insufflation. ��� the Korean Society of Anesthesiologists, 2011
Self-Contrastive Learning: Single-viewed Supervised Contrastive Framework using Sub-network
Contrastive loss has significantly improved performance in supervised
classification tasks by using a multi-viewed framework that leverages
augmentation and label information. The augmentation enables contrast with
another view of a single image but enlarges training time and memory usage. To
exploit the strength of multi-views while avoiding the high computation cost,
we introduce a multi-exit architecture that outputs multiple features of a
single image in a single-viewed framework. To this end, we propose
Self-Contrastive (SelfCon) learning, which self-contrasts within multiple
outputs from the different levels of a single network. The multi-exit
architecture efficiently replaces multi-augmented images and leverages various
information from different layers of a network. We demonstrate that SelfCon
learning improves the classification performance of the encoder network, and
empirically analyze its advantages in terms of the single-view and the
sub-network. Furthermore, we provide theoretical evidence of the performance
increase based on the mutual information bound. For ImageNet classification on
ResNet-50, SelfCon improves accuracy by +0.6% with 59% memory and 48% time of
Supervised Contrastive learning, and a simple ensemble of multi-exit outputs
boosts performance up to +1.5%. Our code is available at
https://github.com/raymin0223/self-contrastive-learning.Comment: AAAI 202
Outcomes of endovascular treatment for aortic pseudoaneurysm in Behcet's disease
ObjectiveTo evaluate the effectiveness of endovascular stent grafting for surgical management of aortic pseudoaneurysm in patients with Behcet's disease (BD).MethodsWe present a single-institution retrospective cohort of patients with aortic pseudoaneurysm and BD treated with aortic stent grafting. Computed tomography imaging was obtained preoperatively in all patients and once within 2 weeks postoperatively, and then annually. Clinical follow-up and erythrocyte sedimentation rate were used to follow BD activity. Immunosuppressant therapy was instituted prior to endovascular treatment unless a contraindication existed.ResultsFrom 1998 to 2012, 10 patients (eight male, two female; median age, 39) with BD and aortic pseudoaneurysm were treated with endovascular stent grafting at this institution. Ninety percent of these patients received immunosuppressive therapy before and after surgical treatment. The median follow-up period was 57 months (interquartile range, 43-72). The locations of the 12 pseudoaneurysms treated in this cohort were infrarenal abdominal aorta (seven), descending thoracic aorta (four), and aortic arch (one). Median pseudoaneurysm size was 4.5 cm (interquartile range, 3.4-5.9). At long-term follow-up, complete resolution of the aortic pseudoaneurysm was noted in all patients. No endoleaks occurred. Newly developed pseudoaneurysm at the distal margin of the stent graft was noted in one patient 17 months after the stent graft procedure. One patient required a subsequent stent graft placement for an expanding pseudoaneurysm of the subclavian artery. No patient deaths occurred during the follow-up period.ConclusionsEndovascular treatment of aortic pseudoaneurysm with stent-grafting in patients with BD is safe and effective with long-term durability
Active site phosphoryl groups in the biphosphorylated phosphotransferase complex reveal dynamics in a millisecond time scale
AbstractThe N-terminal domain of Enzyme I (EIN) and phosphocarrier HPr can form a biphosphorylated complex when they are both phosphorylated by excess cellular phosphoenolpyruvate. Here we show that the electrostatic repulsion between the phosphoryl groups in the biphosphorylated complex results in characteristic dynamics at the active site in a millisecond time scale. The dynamics is localized to phospho-His15 and the stabilizing backbone amide groups of HPr, and does not impact on the phospho-His189 of EIN. The dynamics occurs with the kex of ∼500s−1 which compares to the phosphoryl transfer rate of ∼850s−1 between EIN and HPr. The conformational dynamics in HPr may be important for its phosphotransfer reactions with multiple partner proteins.Structured summary of protein interactionsEIN and HPr bind by nuclear magnetic resonance (View Interaction)
- …