Learning the Degradation Distribution for Blind Image Super-Resolution

Synthetic high-resolution (HR) \& low-resolution (LR) pairs are widely used in existing super-resolution (SR) methods. To avoid the domain gap between synthetic and test images, most previous methods try to adaptively learn the synthesizing (degrading) process via a deterministic model. However, some degradations in real scenarios are stochastic and cannot be determined by the content of the image. These deterministic models may fail to model the random factors and content-independent parts of degradations, which will limit the performance of the following SR models. In this paper, we propose a probabilistic degradation model (PDM), which studies the degradation $\mathbf{D}$ as a random variable, and learns its distribution by modeling the mapping from a priori random variable $\mathbf{z}$ to $\mathbf{D}$. Compared with previous deterministic degradation models, PDM could model more diverse degradations and generate HR-LR pairs that may better cover the various degradations of test images, and thus prevent the SR model from over-fitting to specific ones. Extensive experiments have demonstrated that our degradation model can help the SR model achieve better performance on different datasets. The source codes are released at \url{[email protected]:greatlog/UnpairedSR.git}.Comment: Accepted to CVRP202

End-to-end Alternating Optimization for Real-World Blind Super Resolution

Blind Super-Resolution (SR) usually involves two sub-problems: 1) estimating the degradation of the given low-resolution (LR) image; 2) super-resolving the LR image to its high-resolution (HR) counterpart. Both problems are ill-posed due to the information loss in the degrading process. Most previous methods try to solve the two problems independently, but often fall into a dilemma: a good super-resolved HR result requires an accurate degradation estimation, which however, is difficult to be obtained without the help of original HR information. To address this issue, instead of considering these two problems independently, we adopt an alternating optimization algorithm, which can estimate the degradation and restore the SR image in a single model. Specifically, we design two convolutional neural modules, namely \textit{Restorer} and \textit{Estimator}. \textit{Restorer} restores the SR image based on the estimated degradation, and \textit{Estimator} estimates the degradation with the help of the restored SR image. We alternate these two modules repeatedly and unfold this process to form an end-to-end trainable network. In this way, both \textit{Restorer} and \textit{Estimator} could get benefited from the intermediate results of each other, and make each sub-problem easier. Moreover, \textit{Restorer} and \textit{Estimator} are optimized in an end-to-end manner, thus they could get more tolerant of the estimation deviations of each other and cooperate better to achieve more robust and accurate final results. Extensive experiments on both synthetic datasets and real-world images show that the proposed method can largely outperform state-of-the-art methods and produce more visually favorable results. The codes are rleased at \url{https://github.com/greatlog/RealDAN.git}.Comment: Extension of our previous NeurIPS paper. Accepted to IJC

VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation

A diffusion probabilistic model (DPM), which constructs a forward diffusion process by gradually adding noise to data points and learns the reverse denoising process to generate new samples, has been shown to handle complex data distribution. Despite its recent success in image synthesis, applying DPMs to video generation is still challenging due to high-dimensional data spaces. Previous methods usually adopt a standard diffusion process, where frames in the same video clip are destroyed with independent noises, ignoring the content redundancy and temporal correlation. This work presents a decomposed diffusion process via resolving the per-frame noise into a base noise that is shared among all frames and a residual noise that varies along the time axis. The denoising pipeline employs two jointly-learned networks to match the noise decomposition accordingly. Experiments on various datasets confirm that our approach, termed as VideoFusion, surpasses both GAN-based and diffusion-based alternatives in high-quality video generation. We further show that our decomposed formulation can benefit from pre-trained image diffusion models and well-support text-conditioned video creation.Comment: Accepted to CVPR202

Fast Meta Failure Recovery for Federated Meta-Learning

In recent years, the field of distributed deep learning within the Internet of Things (IoT) or the edge has experienced exponential growth. Federated meta-learning has emerged as a significant advancement, enabling collaborative learning among source nodes to establish a global model initialization. This approach allows for optimal performance while necessitating minimal data samples for updating model parameters at the target node. Federated meta-learning has gained increased attention due to its capacity to provide real-time edge intelligence. However, a critical aspect that remains inadequately explored is the recovery of interim meta knowledge\u27s failure, which constitutes a pivotal key for adapting to new tasks. In this paper, we introduce FMRec, a novel platform designed to offer a fast and flexible recovery mechanism for failed interim meta knowledge in various federated meta-learning scenarios. FMRec serves as a complementary system compatible with different types of federated models and is adaptable to diverse tasks. We present a demonstration of its design and assess its efficiency and reliability through real-world applications

Strain Rate Effect on the Thermomechanical Behavior of NiTi Shape Memory Alloys: A Literature Review

A review of experiments and models for the strain rate effect of NiTi Shape Memory Alloys (SMAs) is presented in this paper. Experimental observations on the rate-dependent properties, such as stress responses, temperature evolutions, and phase nucleation and propagation, under uniaxial loads are classified and summarized based on the strain rate values. The strain rates are divided into five ranges and in each range the deformation mechanism is unique. For comparison, results under other loading modes are also reviewed; however, these are shorter in length due to a limited number of experiments. A brief discussion on the influences of the microstructure on the strain-rate responses is followed. Modeling the rate-dependent behaviors of NiTi SMAs focuses on incorporating the physical origins in the constitutive relationship. Thermal source models are the key rate-dependent constitutive models under quasi-static loading to account for the self-heating mechanism. Thermal kinetic models, evolving from thermal source models, address the kinetic relationship in dynamic deformation