985 research outputs found
Texture Mixer: A Network for Controllable Synthesis and Interpolation of Texture
This paper addresses the problem of interpolating visual textures. We
formulate this problem by requiring (1) by-example controllability and (2)
realistic and smooth interpolation among an arbitrary number of texture
samples. To solve it we propose a neural network trained simultaneously on a
reconstruction task and a generation task, which can project texture examples
onto a latent space where they can be linearly interpolated and projected back
onto the image domain, thus ensuring both intuitive control and realistic
results. We show our method outperforms a number of baselines according to a
comprehensive suite of metrics as well as a user study. We further show several
applications based on our technique, which include texture brush, texture
dissolve, and animal hybridization.Comment: Accepted to CVPR'1
Continuous Facial Motion Deblurring
We introduce a novel framework for continuous facial motion deblurring that
restores the continuous sharp moment latent in a single motion-blurred face
image via a moment control factor. Although a motion-blurred image is the
accumulated signal of continuous sharp moments during the exposure time, most
existing single image deblurring approaches aim to restore a fixed number of
frames using multiple networks and training stages. To address this problem, we
propose a continuous facial motion deblurring network based on GAN (CFMD-GAN),
which is a novel framework for restoring the continuous moment latent in a
single motion-blurred face image with a single network and a single training
stage. To stabilize the network training, we train the generator to restore
continuous moments in the order determined by our facial motion-based
reordering process (FMR) utilizing domain-specific knowledge of the face.
Moreover, we propose an auxiliary regressor that helps our generator produce
more accurate images by estimating continuous sharp moments. Furthermore, we
introduce a control-adaptive (ContAda) block that performs spatially deformable
convolution and channel-wise attention as a function of the control factor.
Extensive experiments on the 300VW datasets demonstrate that the proposed
framework generates a various number of continuous output frames by varying the
moment control factor. Compared with the recent single-to-single image
deblurring networks trained with the same 300VW training set, the proposed
method show the superior performance in restoring the central sharp frame in
terms of perceptual metrics, including LPIPS, FID and Arcface identity
distance. The proposed method outperforms the existing single-to-video
deblurring method for both qualitative and quantitative comparisons
JSI-GAN: GAN-Based Joint Super-Resolution and Inverse Tone-Mapping with Pixel-Wise Task-Specific Filters for UHD HDR Video
Joint learning of super-resolution (SR) and inverse tone-mapping (ITM) has
been explored recently, to convert legacy low resolution (LR) standard dynamic
range (SDR) videos to high resolution (HR) high dynamic range (HDR) videos for
the growing need of UHD HDR TV/broadcasting applications. However, previous
CNN-based methods directly reconstruct the HR HDR frames from LR SDR frames,
and are only trained with a simple L2 loss. In this paper, we take a
divide-and-conquer approach in designing a novel GAN-based joint SR-ITM
network, called JSI-GAN, which is composed of three task-specific subnets: an
image reconstruction subnet, a detail restoration (DR) subnet and a local
contrast enhancement (LCE) subnet. We delicately design these subnets so that
they are appropriately trained for the intended purpose, learning a pair of
pixel-wise 1D separable filters via the DR subnet for detail restoration and a
pixel-wise 2D local filter by the LCE subnet for contrast enhancement.
Moreover, to train the JSI-GAN effectively, we propose a novel detail GAN loss
alongside the conventional GAN loss, which helps enhancing both local details
and contrasts to reconstruct high quality HR HDR results. When all subnets are
jointly trained well, the predicted HR HDR results of higher quality are
obtained with at least 0.41 dB gain in PSNR over those generated by the
previous methods.Comment: The first two authors contributed equally to this work. Accepted at
AAAI 2020. (Camera-ready version
- …