Pre-trained Vision and Language Transformers Are Few-Shot Incremental Learners

Few-Shot Class Incremental Learning (FSCIL) is a task that requires a model to learn new classes incrementally without forgetting when only a few samples for each class are given. FSCIL encounters two significant challenges: catastrophic forgetting and overfitting, and these challenges have driven prior studies to primarily rely on shallow models, such as ResNet-18. Even though their limited capacity can mitigate both forgetting and overfitting issues, it leads to inadequate knowledge transfer during few-shot incremental sessions. In this paper, we argue that large models such as vision and language transformers pre-trained on large datasets can be excellent few-shot incremental learners. To this end, we propose a novel FSCIL framework called PriViLege, Pre-trained Vision and Language transformers with prompting functions and knowledge distillation. Our framework effectively addresses the challenges of catastrophic forgetting and overfitting in large models through new pre-trained knowledge tuning (PKT) and two losses: entropy-based divergence loss and semantic knowledge distillation loss. Experimental results show that the proposed PriViLege significantly outperforms the existing state-of-the-art methods with a large margin, e.g., +9.38% in CUB200, +20.58% in CIFAR-100, and +13.36% in miniImageNet. Our implementation code is available at https://github.com/KHU-AGI/PriViLege.Comment: Accepted by CVPR 202

Online Class Incremental Learning on Stochastic Blurry Task Boundary via Mask and Visual Prompt Tuning

Continual learning aims to learn a model from a continuous stream of data, but it mainly assumes a fixed number of data and tasks with clear task boundaries. However, in real-world scenarios, the number of input data and tasks is constantly changing in a statistical way, not a static way. Although recently introduced incremental learning scenarios having blurry task boundaries somewhat address the above issues, they still do not fully reflect the statistical properties of real-world situations because of the fixed ratio of disjoint and blurry samples. In this paper, we propose a new Stochastic incremental Blurry task boundary scenario, called Si-Blurry, which reflects the stochastic properties of the real-world. We find that there are two major challenges in the Si-Blurry scenario: (1) inter- and intra-task forgettings and (2) class imbalance problem. To alleviate them, we introduce Mask and Visual Prompt tuning (MVP). In MVP, to address the inter- and intra-task forgetting issues, we propose a novel instance-wise logit masking and contrastive visual prompt tuning loss. Both of them help our model discern the classes to be learned in the current batch. It results in consolidating the previous knowledge. In addition, to alleviate the class imbalance problem, we introduce a new gradient similarity-based focal loss and adaptive feature scaling to ease overfitting to the major classes and underfitting to the minor classes. Extensive experiments show that our proposed MVP significantly outperforms the existing state-of-the-art methods in our challenging Si-Blurry scenario

Visfatin exerts angiogenic effects on human umbilical vein endothelial cells through the mTOR signaling pathway

AbstractThe biologically active factors known as adipocytokines are secreted primarily by adipose tissues and can act as modulators of angiogenesis. Visfatin, an adipocytokine that has recently been reported to have angiogenic properties, is upregulated in diabetes, cancer, and inflammatory diseases. Because maintenance of an angiogenic balance is critically important in the management of these diseases, understanding the molecular mechanism by which visfatin promotes angiogenesis is very important. In this report, we describe our findings demonstrating that visfatin stimulates the mammalian target of the rapamycin (mTOR) pathway, which plays important roles in angiogenesis. Visfatin induced the expression of hypoxia-inducible factor 1α (HIF1α) and vascular endothelial growth factor (VEGF) in human endothelial cells. Inhibition of the mTOR pathway by rapamycin eliminated the angiogenic and proliferative effects of visfatin. The visfatin-induced increase in VEGF expression was also eliminated by RNA interference-mediated knockdown of the 70-kDa ribosomal protein S6 kinase (p70S6K), a downstream target of mTOR. Visfatin inactivated glycogen synthase kinase 3β (GSK3β) by phosphorylating it at Ser-9, leading to the nuclear translocation of β-catenin. Both rapamycin co-treatment and p70S6K knockdown inhibited visfatin-induced GSK3β phosphorylation at Ser-9 and nuclear translocation of β-catenin. Taken together, these results indicate that mTOR signaling is involved in visfatin-induced angiogenesis, and that this signaling leads to visfatin-induced VEGF expression and nuclear translocation of β-catenin

Systemic EBV+ T-cell lymphoma in elderly patients: comparison with children and young adult patients

Fulminant Epstein–Barr virus (EBV+) T-cell lymphoma in immunocompetent elderly patients is rare and its character has not been well defined. This study analyzed the clinicopathological features of five elderly patients (group A: 50–84 years) and compared them with those of eight children and young adult patients with systemic T-cell lymphomas (group B: 10–34 years). Group A more commonly presented with generalized lymphadenopathy (n = 3) than did group B (n = 1). Chronic active EBV infection (n = 3) and hydroa vacciniforme-like eruptions (n = 1) were seen in group B, while group A showed no evidence of chronic EBV infection, but did show chronic hepatitis B or C virus infections (n = 3). The histological and immunophenotypical findings were similar. All patients died within 1 to 14 months of diagnosis. These findings suggest that EBV+ T-cell lymphoma in elderly patients is a unique disease with an underlying derangement of T-cell immunity and failure to eradicate infected virus. Additional factors related to senility may play a role in the disruption of homeostasis between the virus and the host’s immune system

Sequencing and characterization of Varicella-Zoster virus vaccine strain SuduVax

<p>Abstract</p> <p>Background</p> <p>Varicella-zoster virus (VZV) causes chickenpox in children and shingles in older people. Currently, live attenuated vaccines based on the Oka strain are available worldwide. In Korea, an attenuated VZV vaccine has been developed from a Korean isolate and has been commercially available since 1994. Despite this long history of use, the mechanism for the attenuation of the vaccine strain is still elusive. We attempted to understand the molecular basis of attenuation mechanism by full genome sequencing and comparative genomic analyses of the Korean vaccine strain SuduVax.</p> <p>Results</p> <p>SuduVax was found to contain a genome that was 124,759 bp and possessed 74 open reading frames (ORFs). SuduVax was genetically most close to Oka strains and these Korean-Japanese strains formed a strong clade in phylogenetic trees. SuduVax, similar to the Oka vaccine strains, underwent T- > C substitution at the stop codon of ORF0, resulting in a read-through mutation to code for an extended form of ORF0 protein. SuduVax also shared certain deletion and insertion mutations in ORFs 17, 29, 56 and 60 with Oka vaccine strains and some clinical strains.</p> <p>Conclusions</p> <p>The Korean VZV vaccine strain SuduVax is genetically similar to the Oka vaccine strains. Further comparative genomic and bioinformatics analyses will help to elucidate the molecular basis of the attenuation of the VZV vaccine strains.</p

Characterization of diverse natural variants of CYP102A1 found within a species of Bacillus megaterium

An extreme diversity of substrates and catalytic reactions of cytochrome P450 (P450) enzymes is considered to be the consequence of evolutionary adaptation driven by different metabolic or environmental demands. Here we report the presence of numerous natural variants of P450 BM3 (CYP102A1) within a species of Bacillus megaterium. Extensive amino acid substitutions (up to 5% of the total 1049 amino acid residues) were identified from the variants. Phylogenetic analyses suggest that this P450 gene evolve more rapidly than the rRNA gene locus. It was found that key catalytic residues in the substrate channel and active site are retained. Although there were no apparent variations in hydroxylation activity towards myristic acid (C14) and palmitic acid (C16), the hydroxylation rates of lauric acid (C12) by the variants varied in the range of >25-fold. Interestingly, catalytic activities of the variants are promiscuous towards non-natural substrates including human P450 substrates. It can be suggested that CYP102A1 variants can acquire new catalytic activities through site-specific mutations distal to the active site

Association between humidifier disinfectant exposure during infancy and subsequent neuropsychiatric outcomes during childhood: a nation-wide cross-sectional study

Background The purpose was to determine the association between infant exposure to humidifier disinfectant (HD) with neuropsychiatric problems in pre-school children. Methods A total of 2,150 children (age 4–11 months) were enrolled in the Panel Study of Korean Children (PSKC) study. The Korean version of the Child Behavior Checklist (CBCL) was used for assessments of neuropsychiatric problems. 1,113 children who participated in all the first to third PSKC studies and answered a question about HD exposure were finally enrolled. Results There were 717 (64.5%) children in non-HD group who were not exposed to HD and 396 (35.5%) in HD group with former exposure to HD. Exposure to HD was associated with total neuropsychiatric problems (adjusted odds ratio, aOR = 1.54, 95% CI = 1.15–2.06), being emotionally reactive (aOR = 1.55, 95% CI = 1.00–2.39), having attention problems (aOR = 1.96, 95% CI = 1.10–3.47), having oppositional defiant problems (aOR = 1.70, 95% CI = 1.07–2.71), and having attention deficit/hyperactivity problems (aOR = 11.57, 95% CI = 1.03–2.38). The risks for neuropsychiatric problems were clearly increased in boy, firstborn, and secondary smoker. Conclusions Exposure to HD during early childhood had a potential association with subsequent behavioral abnormalities.This study was supported by a grant from the Seongnam Atopy Project of the Seongnam City Government, Republic of Korea