Flow Guidance Deformable Compensation Network for Video Frame Interpolation

Motion-based video frame interpolation (VFI) methods have made remarkable progress with the development of deep convolutional networks over the past years. While their performance is often jeopardized by the inaccuracy of flow map estimation, especially in the case of large motion and occlusion. In this paper, we propose a flow guidance deformable compensation network (FGDCN) to overcome the drawbacks of existing motion-based methods. FGDCN decomposes the frame sampling process into two steps: a flow step and a deformation step. Specifically, the flow step utilizes a coarse-to-fine flow estimation network to directly estimate the intermediate flows and synthesizes an anchor frame simultaneously. To ensure the accuracy of the estimated flow, a distillation loss and a task-oriented loss are jointly employed in this step. Under the guidance of the flow priors learned in step one, the deformation step designs a pyramid deformable compensation network to compensate for the missing details of the flow step. In addition, a pyramid loss is proposed to supervise the model in both the image and frequency domain. Experimental results show that the proposed algorithm achieves excellent performance on various datasets with fewer parameters

The Adversarial Attack and Detection under the Fisher Information Metric

Many deep learning models are vulnerable to the adversarial attack, i.e., imperceptible but intentionally-designed perturbations to the input can cause incorrect output of the networks. In this paper, using information geometry, we provide a reasonable explanation for the vulnerability of deep learning models. By considering the data space as a non-linear space with the Fisher information metric induced from a neural network, we first propose an adversarial attack algorithm termed one-step spectral attack (OSSA). The method is described by a constrained quadratic form of the Fisher information matrix, where the optimal adversarial perturbation is given by the first eigenvector, and the model vulnerability is reflected by the eigenvalues. The larger an eigenvalue is, the more vulnerable the model is to be attacked by the corresponding eigenvector. Taking advantage of the property, we also propose an adversarial detection method with the eigenvalues serving as characteristics. Both our attack and detection algorithms are numerically optimized to work efficiently on large datasets. Our evaluations show superior performance compared with other methods, implying that the Fisher information is a promising approach to investigate the adversarial attacks and defenses.Comment: Accepted as an AAAI-2019 oral pape

Deep Unfolding Convolutional Dictionary Model for Multi-Contrast MRI Super-resolution and Reconstruction

Magnetic resonance imaging (MRI) tasks often involve multiple contrasts. Recently, numerous deep learning-based multi-contrast MRI super-resolution (SR) and reconstruction methods have been proposed to explore the complementary information from the multi-contrast images. However, these methods either construct parameter-sharing networks or manually design fusion rules, failing to accurately model the correlations between multi-contrast images and lacking certain interpretations. In this paper, we propose a multi-contrast convolutional dictionary (MC-CDic) model under the guidance of the optimization algorithm with a well-designed data fidelity term. Specifically, we bulid an observation model for the multi-contrast MR images to explicitly model the multi-contrast images as common features and unique features. In this way, only the useful information in the reference image can be transferred to the target image, while the inconsistent information will be ignored. We employ the proximal gradient algorithm to optimize the model and unroll the iterative steps into a deep CDic model. Especially, the proximal operators are replaced by learnable ResNet. In addition, multi-scale dictionaries are introduced to further improve the model performance. We test our MC-CDic model on multi-contrast MRI SR and reconstruction tasks. Experimental results demonstrate the superior performance of the proposed MC-CDic model against existing SOTA methods. Code is available at https://github.com/lpcccc-cv/MC-CDic.Comment: Accepted to IJCAI202

A Pan-Sharpening Method Based on Evolutionary Optimization and IHS Transformation

In many remote sensing applications, users usually prefer a multispectral image with both high spectral and high spatial information. This high quality image could be obtained by pan-sharpening techniques which fuse a high resolution panchromatic (PAN) image and a low resolution multispectral (MS) image. In this paper, we propose a new technique to do so based on the adaptive intensity-hue-saturation (IHS) transformation model and evolutionary optimization. The basic idea is to reconstruct the target image through a parameterized adaptive IHS transformation. An optimization objective is thus introduced by considering the relations between the fused image and the original PAN and MS images. The control parameters are optimized by an evolutionary algorithm. Experimental results show that our new approach is practical and performs much better than some state-of-the-art techniques according to the performance metrics

Nonnegative Matrix Factorization Numerical Method for Integrated Photonic Cavity Based Spectroscopy

Nonnegative matrix factorization numerical method has been used to improve the spectral resolution of integrated photonic cavity based spectroscopy. Based on the experimental results for integrated photonic cavity device on Optics Letters 32, 632 (2007), the theoretical results show that the spectral resolution can be improved more than 3 times from 5.5 nm to 1.8 nm. It is a promising way to release the difficulty of fabricating high-resolution devices

A PDE patch-based spectral method for progressive mesh compression and mesh denoising

The development of the patchwise partial differential equation (PDE) framework a few years ago has paved the way for the PDE method to be used in mesh signal processing. In this paper, we, for the first time, extend the use of the PDE method to progressive mesh compression and mesh denoising. We, meanwhile, upgrade the existing patchwise PDE method in patch merging, mesh partitioning, and boundary extraction to accommodate mesh signal processing. In our new method, an arbitrary mesh model is partitioned into patches, each of which can be represented by a small set of coefficients of its PDE spectral solution. Since low-frequency components contribute more to the reconstructed mesh than high-frequency ones, we can achieve progressive mesh compression and mesh denoising by manipulating the frequency terms of the PDE solution. Experimental results demonstrate the feasibility of our method in both progressive mesh compression and mesh denoising