LFS-GAN: Lifelong Few-Shot Image Generation

We address a challenging lifelong few-shot image generation task for the first time. In this situation, a generative model learns a sequence of tasks using only a few samples per task. Consequently, the learned model encounters both catastrophic forgetting and overfitting problems at a time. Existing studies on lifelong GANs have proposed modulation-based methods to prevent catastrophic forgetting. However, they require considerable additional parameters and cannot generate high-fidelity and diverse images from limited data. On the other hand, the existing few-shot GANs suffer from severe catastrophic forgetting when learning multiple tasks. To alleviate these issues, we propose a framework called Lifelong Few-Shot GAN (LFS-GAN) that can generate high-quality and diverse images in lifelong few-shot image generation task. Our proposed framework learns each task using an efficient task-specific modulator - Learnable Factorized Tensor (LeFT). LeFT is rank-constrained and has a rich representation ability due to its unique reconstruction technique. Furthermore, we propose a novel mode seeking loss to improve the diversity of our model in low-data circumstances. Extensive experiments demonstrate that the proposed LFS-GAN can generate high-fidelity and diverse images without any forgetting and mode collapse in various domains, achieving state-of-the-art in lifelong few-shot image generation task. Surprisingly, we find that our LFS-GAN even outperforms the existing few-shot GANs in the few-shot image generation task. The code is available at Github.Comment: 20 pages, 19 figures, 14 tables, ICCV 2023 Poste

Online Class Incremental Learning on Stochastic Blurry Task Boundary via Mask and Visual Prompt Tuning

Continual learning aims to learn a model from a continuous stream of data, but it mainly assumes a fixed number of data and tasks with clear task boundaries. However, in real-world scenarios, the number of input data and tasks is constantly changing in a statistical way, not a static way. Although recently introduced incremental learning scenarios having blurry task boundaries somewhat address the above issues, they still do not fully reflect the statistical properties of real-world situations because of the fixed ratio of disjoint and blurry samples. In this paper, we propose a new Stochastic incremental Blurry task boundary scenario, called Si-Blurry, which reflects the stochastic properties of the real-world. We find that there are two major challenges in the Si-Blurry scenario: (1) inter- and intra-task forgettings and (2) class imbalance problem. To alleviate them, we introduce Mask and Visual Prompt tuning (MVP). In MVP, to address the inter- and intra-task forgetting issues, we propose a novel instance-wise logit masking and contrastive visual prompt tuning loss. Both of them help our model discern the classes to be learned in the current batch. It results in consolidating the previous knowledge. In addition, to alleviate the class imbalance problem, we introduce a new gradient similarity-based focal loss and adaptive feature scaling to ease overfitting to the major classes and underfitting to the minor classes. Extensive experiments show that our proposed MVP significantly outperforms the existing state-of-the-art methods in our challenging Si-Blurry scenario

RADIO: Reference-Agnostic Dubbing Video Synthesis

One of the most challenging problems in audio-driven talking head generation is achieving high-fidelity detail while ensuring precise synchronization. Given only a single reference image, extracting meaningful identity attributes becomes even more challenging, often causing the network to mirror the facial and lip structures too closely. To address these issues, we introduce RADIO, a framework engineered to yield high-quality dubbed videos regardless of the pose or expression in reference images. The key is to modulate the decoder layers using latent space composed of audio and reference features. Additionally, we incorporate ViT blocks into the decoder to emphasize high-fidelity details, especially in the lip region. Our experimental results demonstrate that RADIO displays high synchronization without the loss of fidelity. Especially in harsh scenarios where the reference frame deviates significantly from the ground truth, our method outperforms state-of-the-art methods, highlighting its robustness. Pre-trained model and codes will be made public after the review.Comment: Under revie

GLAD: Global-Local View Alignment and Background Debiasing for Unsupervised Video Domain Adaptation with Large Domain Gap

In this work, we tackle the challenging problem of unsupervised video domain adaptation (UVDA) for action recognition. We specifically focus on scenarios with a substantial domain gap, in contrast to existing works primarily deal with small domain gaps between labeled source domains and unlabeled target domains. To establish a more realistic setting, we introduce a novel UVDA scenario, denoted as Kinetics->BABEL, with a more considerable domain gap in terms of both temporal dynamics and background shifts. To tackle the temporal shift, i.e., action duration difference between the source and target domains, we propose a global-local view alignment approach. To mitigate the background shift, we propose to learn temporal order sensitive representations by temporal order learning and background invariant representations by background augmentation. We empirically validate that the proposed method shows significant improvement over the existing methods on the Kinetics->BABEL dataset with a large domain gap. The code is available at https://github.com/KHUVLL/GLAD.Comment: This is an accepted WACV 2024 paper. Our code is available at https://github.com/KHUVLL/GLA

Reflex Movements in Patients with Brain Death: A Prospective Study in A Tertiary Medical Center

Reflex movements have been reported to occur in up to 75% of brain-dead patients, but this issue has not been addressed in Korea. The patients admitted to our hospital who met the criteria for brain death were enrolled between March 2003 and February 2005. The frequency and type of reflex movements in these patients were evaluated prospectively using a standardized protocol. Brain death was determined according to the guideline of Korean Medical Association. Of 26 patients who were included, five (19.2%) exhibited reflex movements such as the pronation-extension reflex, abdominal reflex, flexion reflex, the Lazarus sign, and periodic leg movements. This finding suggests that the frequency of spinal reflex movements is not rare and the awareness of these movements may prevent delays in brain-dead diagnosis and misinterpretations

Multi-dimensional histone methylations for coordinated regulation of gene expression under hypoxia

Hypoxia increases both active and repressive histone methylation levels via decreased activity of histone demethylases. However, how such increases coordinately regulate induction or repression of hypoxia-responsive genes is largely unknown. Here, we profiled active and repressive histone tri-methylations (H3K4me3, H3K9me3, and H3K27me3) and analyzed gene expression profiles in human adipocyte-derived stem cells under hypoxia. We identified differentially expressed genes (DEGs) and differentially methylated genes (DMGs) by hypoxia and clustered the DEGs and DMGs into four major groups. We found that each group of DEGs was predominantly associated with alterations in only one type among the three histone tri-methylations. Moreover, the four groups of DEGs were associated with different TFs and localization patterns of their predominant types of H3K4me3, H3K9me3 and H3K27me3. Our results suggest that the association of altered gene expression with prominent single-type histone tri-methylations characterized by different localization patterns and with different sets of TFs contributes to regulation of particular sets of genes, which can serve as a model for coordinated epigenetic regulation of gene expression under hypoxia.111Ysciescopu

Preparation and Characterization of Self-Emulsified Docetaxel

The aim of this paper was to prepare a self-microemulsifying docetaxel (Dtx) using PLGA, Tetraglycol, Labrasol, and Cremophor ELP. The prepared Dtx-loaded self-microemulsifying system (SMES) showed the initial size of the range of 80–100 nm with narrow size distribution and the negative zeta-potential values. Its morphology was a spherical shape by atomic force microscopy. In experiment of stability, Dtx-loaded SMES prepared in DW and BSA condition showed good stability at 37∘C for 7 days. The viability of the B16F10 cells incubated with Dtx-loaded SMES, Dtx-solution, and Taxol were decreased as a function of incubation time. In conclusion, we confirmed that Dtx-loaded SMES showed an inhibitory effect for proliferation of B16F10 melanoma cells