SC2GAN: Rethinking Entanglement by Self-correcting Correlated GAN Space

Generative Adversarial Networks (GANs) can synthesize realistic images, with the learned latent space shown to encode rich semantic information with various interpretable directions. However, due to the unstructured nature of the learned latent space, it inherits the bias from the training data where specific groups of visual attributes that are not causally related tend to appear together, a phenomenon also known as spurious correlations, e.g., age and eyeglasses or women and lipsticks. Consequently, the learned distribution often lacks the proper modelling of the missing examples. The interpolation following editing directions for one attribute could result in entangled changes with other attributes. To address this problem, previous works typically adjust the learned directions to minimize the changes in other attributes, yet they still fail on strongly correlated features. In this work, we study the entanglement issue in both the training data and the learned latent space for the StyleGAN2-FFHQ model. We propose a novel framework SC$^2$GAN that achieves disentanglement by re-projecting low-density latent code samples in the original latent space and correcting the editing directions based on both the high-density and low-density regions. By leveraging the original meaningful directions and semantic region-specific layers, our framework interpolates the original latent codes to generate images with attribute combination that appears infrequently, then inverts these samples back to the original latent space. We apply our framework to pre-existing methods that learn meaningful latent directions and showcase its strong capability to disentangle the attributes with small amounts of low-density region samples added.Comment: Accepted to the Out Of Distribution Generalization in Computer Vision workshop at ICCV202

Efficient Algorithms for Minimizing Compositions of Convex Functions and Random Functions and Its Applications in Network Revenue Management

In this paper, we study a class of nonconvex stochastic optimization in the form of $\min_{x\in\mathcal{X}} F(x):=\mathbb{E}_\xi [f(\phi(x,\xi))]$, where the objective function $F$ is a composition of a convex function $f$ and a random function $\phi$. Leveraging an (implicit) convex reformulation via a variable transformation $u=\mathbb{E}[\phi(x,\xi)]$, we develop stochastic gradient-based algorithms and establish their sample and gradient complexities for achieving an $\epsilon$-global optimal solution. Interestingly, our proposed Mirror Stochastic Gradient (MSG) method operates only in the original $x$-space using gradient estimators of the original nonconvex objective $F$ and achieves $\tilde{\mathcal{O}}(\epsilon^{-2})$ sample and gradient complexities, which matches the lower bounds for solving stochastic convex optimization problems. Under booking limits control, we formulate the air-cargo network revenue management (NRM) problem with random two-dimensional capacity, random consumption, and routing flexibility as a special case of the stochastic nonconvex optimization, where the random function $\phi(x,\xi)=x\wedge\xi$, i.e., the random demand $\xi$ truncates the booking limit decision $x$. Extensive numerical experiments demonstrate the superior performance of our proposed MSG algorithm for booking limit control with higher revenue and lower computation cost than state-of-the-art bid-price-based control policies, especially when the variance of random capacity is large. KEYWORDS: stochastic nonconvex optimization, hidden convexity, air-cargo network revenue management, gradient-based algorithm

Deep Video Restoration for Under-Display Camera

Images or videos captured by the Under-Display Camera (UDC) suffer from severe degradation, such as saturation degeneration and color shift. While restoration for UDC has been a critical task, existing works of UDC restoration focus only on images. UDC video restoration (UDC-VR) has not been explored in the community. In this work, we first propose a GAN-based generation pipeline to simulate the realistic UDC degradation process. With the pipeline, we build the first large-scale UDC video restoration dataset called PexelsUDC, which includes two subsets named PexelsUDC-T and PexelsUDC-P corresponding to different displays for UDC. Using the proposed dataset, we conduct extensive benchmark studies on existing video restoration methods and observe their limitations on the UDC-VR task. To this end, we propose a novel transformer-based baseline method that adaptively enhances degraded videos. The key components of the method are a spatial branch with local-aware transformers, a temporal branch embedded temporal transformers, and a spatial-temporal fusion module. These components drive the model to fully exploit spatial and temporal information for UDC-VR. Extensive experiments show that our method achieves state-of-the-art performance on PexelsUDC. The benchmark and the baseline method are expected to promote the progress of UDC-VR in the community, which will be made public

Macrophage migration inhibitory factor may play a protective role in osteoarthritis

Background Osteoarthritis (OA) is the most prevalent form of arthritis and the major cause of disability and overall diminution of quality of life in the elderly population. Currently there is no cure for OA, partly due to the large gaps in our understanding of its underlying molecular and cellular mechanisms. Macrophage migration inhibitory factor (MIF) is a procytokine that mediates pleiotropic inflammatory effects in inflammatory diseases such as rheumatoid arthritis (RA) and ankylosing spondylitis (AS). However, data on the role of MIF in OA is limited with conflicting results. We undertook this study to investigate the role of MIF in OA by examining MIF genotype, mRNA expression, and protein levels in the Newfoundland Osteoarthritis Study. Methods One hundred nineteen end-stage knee/hip OA patients, 16 RA patients, and 113 healthy controls were included in the study. Two polymorphisms in the MIF gene, rs755622, and -794 CATT5-8, were genotyped using polymerase chain reaction–restriction fragment length polymorphism (PCR-RFLP) and PCR followed by automated capillary electrophoresis, respectively. MIF mRNA levels in articular cartilage and subchondral bone were measured by quantitative polymerase chain reaction. Plasma concentrations of MIF, tumor necrosis factor-alpha (TNF-α), interleukin-6 (IL-6), and interleukin-1 beta (IL-1β) were measured by enzyme-linked immunosorbent assay. Results rs755622 and -794 CATT5-8 genotypes were not associated with MIF mRNA or protein levels or OA (all p ≥ 0.19). MIF mRNA level in cartilage was lower in OA patients than in controls (p = 0.028) and RA patients (p = 0.004), while the levels in bone were comparable between OA patients and controls (p = 0.165). MIF protein level in plasma was lower in OA patients than in controls (p = 3.01 × 10−10), while the levels of TNF-α, IL-6 and IL-1β in plasma were all significantly higher in OA patients than in controls (all p ≤ 0.0007). Multivariable logistic regression showed lower MIF and higher IL-1β protein levels in plasma were independently associated with OA (OR per SD increase = 0.10 and 8.08; 95% CI = 0.04–0.19 and 4.42–16.82, respectively), but TNF-α and IL-6 became non-significant. Conclusions Reduced MIF mRNA and protein expression in OA patients suggested MIF might have a protective role in OA and could serve as a biomarker to differentiate OA from other joint disorders