Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning

In this work, we tackle the challenging problem of arbitrary image style transfer using a novel style feature representation learning method. A suitable style representation, as a key component in image stylization tasks, is essential to achieve satisfactory results. Existing deep neural network based approaches achieve reasonable results with the guidance from second-order statistics such as Gram matrix of content features. However, they do not leverage sufficient style information, which results in artifacts such as local distortions and style inconsistency. To address these issues, we propose to learn style representation directly from image features instead of their second-order statistics, by analyzing the similarities and differences between multiple styles and considering the style distribution. Specifically, we present Contrastive Arbitrary Style Transfer (CAST), which is a new style representation learning and style transfer method via contrastive learning. Our framework consists of three key components, i.e., a multi-layer style projector for style code encoding, a domain enhancement module for effective learning of style distribution, and a generative network for image style transfer. We conduct qualitative and quantitative evaluations comprehensively to demonstrate that our approach achieves significantly better results compared to those obtained via state-of-the-art methods. Code and models are available at https://github.com/zyxElsa/CAST_pytorchComment: Accepted by SIGGRAPH 202

DiffStyler: Controllable Dual Diffusion for Text-Driven Image Stylization

Despite the impressive results of arbitrary image-guided style transfer methods, text-driven image stylization has recently been proposed for transferring a natural image into the stylized one according to textual descriptions of the target style provided by the user. Unlike previous image-to-image transfer approaches, text-guided stylization progress provides users with a more precise and intuitive way to express the desired style. However, the huge discrepancy between cross-modal inputs/outputs makes it challenging to conduct text-driven image stylization in a typical feed-forward CNN pipeline. In this paper, we present DiffStyler on the basis of diffusion models. The cross-modal style information can be easily integrated as guidance during the diffusion progress step-by-step. In particular, we use a dual diffusion processing architecture to control the balance between the content and style of the diffused results. Furthermore, we propose a content image-based learnable noise on which the reverse denoising process is based, enabling the stylization results to better preserve the structure information of the content image. We validate the proposed DiffStyler beyond the baseline methods through extensive qualitative and quantitative experiments

ProSpect: Expanded Conditioning for the Personalization of Attribute-aware Image Generation

Personalizing generative models offers a way to guide image generation with user-provided references. Current personalization methods can invert an object or concept into the textual conditioning space and compose new natural sentences for text-to-image diffusion models. However, representing and editing specific visual attributes like material, style, layout, etc. remains a challenge, leading to a lack of disentanglement and editability. To address this, we propose a novel approach that leverages the step-by-step generation process of diffusion models, which generate images from low- to high-frequency information, providing a new perspective on representing, generating, and editing images. We develop Prompt Spectrum Space P*, an expanded textual conditioning space, and a new image representation method called ProSpect. ProSpect represents an image as a collection of inverted textual token embeddings encoded from per-stage prompts, where each prompt corresponds to a specific generation stage (i.e., a group of consecutive steps) of the diffusion model. Experimental results demonstrate that P* and ProSpect offer stronger disentanglement and controllability compared to existing methods. We apply ProSpect in various personalized attribute-aware image generation applications, such as image/text-guided material/style/layout transfer/editing, achieving previously unattainable results with a single image input without fine-tuning the diffusion models

Arbitrary Video Style Transfer via Multi-Channel Correlation

Video style transfer is getting more attention in AI community for its numerous applications such as augmented reality and animation productions. Compared with traditional image style transfer, performing this task on video presents new challenges: how to effectively generate satisfactory stylized results for any specified style, and maintain temporal coherence across frames at the same time. Towards this end, we propose Multi-Channel Correction network (MCCNet), which can be trained to fuse the exemplar style features and input content features for efficient style transfer while naturally maintaining the coherence of input videos. Specifically, MCCNet works directly on the feature space of style and content domain where it learns to rearrange and fuse style features based on their similarity with content features. The outputs generated by MCC are features containing the desired style patterns which can further be decoded into images with vivid style textures. Moreover, MCCNet is also designed to explicitly align the features to input which ensures the output maintains the content structures as well as the temporal continuity. To further improve the performance of MCCNet under complex light conditions, we also introduce the illumination loss during training. Qualitative and quantitative evaluations demonstrate that MCCNet performs well in both arbitrary video and image style transfer tasks

Vagus nerve stimulation for refractory posttraumatic epilepsy: Efficacy and predictors of seizure outcome

BackgroundTraumatic brain injury (TBI) has been recognized as an important and common cause of epilepsy since antiquity. Posttraumatic epilepsy (PTE) is usually associated with drug resistance and poor surgical outcomes, thereby increasing the burden of the illness on patients and their families. Vagus nerve stimulation (VNS) is an adjunctive treatment for medically refractory epilepsy. This study aimed to determine the efficacy of VNS for refractory PTE and to initially evaluate the potential predictors of efficacy.MethodsWe retrospectively collected the outcomes of VNS with at least a 1-year follow-up in all patients with refractory PTE. Subgroups were classified as responders and non-responders according to the efficacy of VNS (≥50% or <50% reduction in seizure frequency). Preoperative data were analyzed to screen for potential predictors of VNS efficacy.ResultsIn total, forty-five patients with refractory PTE who underwent VNS therapy were enrolled. Responders were found in 64.4% of patients, and 15.6% of patients achieved seizure freedom at the last follow-up. In addition, the responder rate increased over time, with 37.8, 44.4, 60, and 67.6% at the 3-, 6-, 12-, and 24-month follow-ups, respectively. After multivariate analysis, generalized interictal epileptic discharges (IEDs) were found to be a negative predictor (OR: 4.861, 95% CI: 1.145–20.632) of VNS efficacy.ConclusionThe results indicated that VNS therapy was effective in refractory PTE patients and was well tolerated over a 1-year follow-up period. Patients with focal or multifocal IEDs were recognized to have better efficacy after VNS therapy

Vagus nerve stimulation for pharmacoresistant epilepsy secondary to encephalomalacia: A single-center retrospective study

ObjectiveVagus nerve stimulation (VNS) is an adjunctive treatment for pharmacoresistant epilepsy. Encephalomalacia is one of the most common MRI findings in the preoperative evaluation of patients with pharmacoresistant epilepsy. This is the first study that aimed to determine the effectiveness of VNS for pharmacoresistant epilepsy secondary to encephalomalacia and evaluate the potential predictors of VNS effectiveness.MethodsWe retrospectively analyzed the seizure outcomes of VNS with at least 1 year of follow-up in all patients with pharmacoresistant epilepsy secondary to encephalomalacia. Based on the effectiveness of VNS (≥50% or <50% reduction in seizure frequency), patients were divided into two subgroups: responders and non-responders. Preoperative data were analyzed to screen for potential predictors of VNS effectiveness.ResultsA total of 93 patients with epilepsy secondary to encephalomalacia who underwent VNS therapy were recruited. Responders were found in 64.5% of patients, and 16.1% of patients achieved seizure freedom at the last follow-up. In addition, the responder rate increased over time, with 36.6, 50.5, 64.5, and 65.4% at the 3-, 6-, 12-, and 24-month follow-ups, respectively. After multivariate analysis, seizure onset in adults (>18 years old) (OR: 0.236, 95%CI: 0.059–0.949) was found to be a positive predictor, and the bilateral interictal epileptic discharges (IEDs) (OR: 3.397, 95%CI: 1.148–10.054) and the bilateral encephalomalacia on MRI (OR: 3.193, 95%CI: 1.217–8.381) were found to be negative predictors of VNS effectiveness.ConclusionThe results demonstrated the effectiveness and safety of VNS therapy in patients with pharmacoresistant epilepsy secondary to encephalomalacia. Patients with seizure onset in adults (>18 years old), unilateral IEDs, or unilateral encephalomalacia on MRI were found to have better seizure outcomes after VNS therapy

Comprehensive analysis based on DNA methylation and RNA-seq reveals hypermethylation of the up-regulated WT1 gene with potential mechanisms in PAM50 subtypes of breast cancer

Background Breast cancer (BC), one of the most widespread cancers worldwide, caused the deaths of more than 600,000 women in 2018, accounting for about 15% of all cancer-associated deaths in women that year. In this study, we aimed to discover potential prognostic biomarkers and explore their molecular mechanisms in different BC subtypes using DNA methylation and RNA-seq. Methods We downloaded the DNA methylation datasets and the RNA expression profiles of primary tissues of the four BC molecular subtypes (luminal A, luminal B, basal-like, and HER2-enriched), as well as the survival information from The Cancer Genome Atlas (TCGA). The highly expressed and hypermethylated genes across all the four subtypes were screened. We examined the methylation sites and the downstream co-expressed genes of the selected genes and validated their prognostic value using a different dataset (GSE20685). For selected transcription factors, the downstream genes were predicted based on the Gene Transcription Regulation Database (GTRD). The tumor microenvironment was also evaluated based on the TCGA dataset. Results We found that Wilms tumor gene 1 (WT1), a transcription factor, was highly expressed and hypermethylated in all the four BC subtypes. All the WT1 methylation sites exhibited hypermethylation. The methylation levels of the TSS200 and 1stExon regions were negatively correlated with WT1 expression in two BC subtypes, while that of the gene body region was positively associated with WT1 expression in three BC subtypes. Patients with low WT1 expression had better overall survival (OS). Five genes including COL11A1, GFAP, FGF5, CD300LG, and IGFL2 were predicted as the downstream genes of WT1. Those five genes were dysregulated in the four BC subtypes. Patients with a favorable 6-gene signature (low expression of WT1 and its five predicted downstream genes) exhibited better OS than that with an unfavorable 6-gene signature. We also found a correlation between WT1 and tamoxifen using STITCH. Higher infiltration rates of CD8 T cells, plasma cells, and monocytes were found in the lower quartile WT1 group and the favorable 6-gene signature group. In conclusion, we demonstrated that WT1 is hypermethylated and up-regulated in the four BC molecular subtypes and a 6-gene signature may predict BC prognosis