TorchDEQ: A Library for Deep Equilibrium Models

Deep Equilibrium (DEQ) Models, an emerging class of implicit models that maps inputs to fixed points of neural networks, are of growing interest in the deep learning community. However, training and applying DEQ models is currently done in an ad-hoc fashion, with various techniques spread across the literature. In this work, we systematically revisit DEQs and present TorchDEQ, an out-of-the-box PyTorch-based library that allows users to define, train, and infer using DEQs over multiple domains with minimal code and best practices. Using TorchDEQ, we build a ``DEQ Zoo'' that supports six published implicit models across different domains. By developing a joint framework that incorporates the best practices across all models, we have substantially improved the performance, training stability, and efficiency of DEQs on ten datasets across all six projects in the DEQ Zoo. TorchDEQ and DEQ Zoo are released as \href{https://github.com/locuslab/torchdeq}{open source}

Eliminating Gradient Conflict in Reference-based Line-Art Colorization

Reference-based line-art colorization is a challenging task in computer vision. The color, texture, and shading are rendered based on an abstract sketch, which heavily relies on the precise long-range dependency modeling between the sketch and reference. Popular techniques to bridge the cross-modal information and model the long-range dependency employ the attention mechanism. However, in the context of reference-based line-art colorization, several techniques would intensify the existing training difficulty of attention, for instance, self-supervised training protocol and GAN-based losses. To understand the instability in training, we detect the gradient flow of attention and observe gradient conflict among attention branches. This phenomenon motivates us to alleviate the gradient issue by preserving the dominant gradient branch while removing the conflict ones. We propose a novel attention mechanism using this training strategy, Stop-Gradient Attention (SGA), outperforming the attention baseline by a large margin with better training stability. Compared with state-of-the-art modules in line-art colorization, our approach demonstrates significant improvements in Fr\'echet Inception Distance (FID, up to 27.21%) and structural similarity index measure (SSIM, up to 25.67%) on several benchmarks. The code of SGA is available at https://github.com/kunkun0w0/SGA .Comment: Accepted by ECCV202

Nε-Carboxymethyl-Lysine Negatively Regulates Foam Cell Migration via the Vav1/Rac1 Pathway

Background. Macrophage-derived foam cells play a central role in atherosclerosis, and their ultimate fate includes apoptosis, promotion of vascular inflammation, or migration to other tissues. Nε-Carboxymethyl-lysine (CML), the key active component of advanced glycation end products, induced foam cell formation and apoptosis. Previous studies have shown that the Vav1/Rac1 pathway affects the macrophage cytoskeleton and cell migration, but its role in the pathogenesis of diabetic atherosclerosis is unknown. Methods and Results. In this study, we used anterior tibiofibular vascular samples from diabetic foot amputation patients and accident amputation patients, and histological and cytological tests were performed using a diabetic ApoE-/- mouse model and primary peritoneal macrophages, respectively. The results showed that the atherosclerotic plaques of diabetic foot amputation patients and diabetic ApoE-/- mice were larger than those of the control group. Inhibition of the Vav1/Rac1 pathway reduced vascular plaques and promoted the migration of macrophages to lymph nodes. Transwell and wound healing assays showed that the migratory ability of macrophage-derived foam cells was inhibited by CML. Cytoskeletal staining showed that advanced glycation end products inhibited the formation of lamellipodia in foam cells, and inhibition of the Vav1/Rac1 pathway restored the formation of lamellipodia. Conclusion. CML inhibits the migration of foam cells from blood vessels via the Vav1/Rac1 pathway, and this process affects the formation of lamellipodia

Advanced Glycation End Products Induce Vascular Smooth Muscle Cell-Derived Foam Cell Formation and Transdifferentiate to a Macrophage-Like State

Background. Advanced glycation end products play an important role in diabetic atherosclerosis. The effects of advanced glycation end products (AGEs) on vascular smooth muscle cell- (VSMC-) derived foam cell formation and phenotypic transformation are unknown. Methods. Serological and histological samples were obtained from diabetic amputation patients and accident amputation patients from the Affiliated Hospital of Jiangsu University. CD68/Actin Alpha 2 (ACTA2) coimmunofluorescence sections were used to quantify the number of VSMCs with macrophage-like phenotypes. Western blotting was used to detect the expression of the receptor of advanced glycation end products in vascular samples. Enzyme-linked immunosorbent assay (ELISA) was used to evaluate the level of serum Nε-carboxymethyl-lysine (CML). In vitro oil red O staining was used to examine lipid accumulation in VSMCs stimulated by CML. The expression of VSMCs and macrophage markers was measured by western blotting and quantitative real-time PCR. Furthermore, changes in VSMC migration and secretion were detected by the Transwell assay and ELISA. Results. In the arterial plaque sections of diabetic patients, VSMCs transformed to a macrophage-like phenotype. The serum CML and RAGE levels in the plaques were significantly higher in the diabetes group than those in the healthy control group and were significantly related to the number of macrophage-like VSMCs. CML stimulation promoted intracellular lipid accumulation. However, CML stimulation decreased the expression of VSMC markers and increased the expression of macrophage phenotype markers. Finally, CML promoted smooth muscle cell migration and the secretion of proinflammatory-related factors. Conclusions. CML induces VSMC-derived foam cell formation, and VSMCs transdifferentiate to a macrophage-like state, which may be mediated by the activation of RAGE