57 research outputs found
Learning Deep CNN Denoiser Prior for Image Restoration
Model-based optimization methods and discriminative learning methods have
been the two dominant strategies for solving various inverse problems in
low-level vision. Typically, those two kinds of methods have their respective
merits and drawbacks, e.g., model-based optimization methods are flexible for
handling different inverse problems but are usually time-consuming with
sophisticated priors for the purpose of good performance; in the meanwhile,
discriminative learning methods have fast testing speed but their application
range is greatly restricted by the specialized task. Recent works have revealed
that, with the aid of variable splitting techniques, denoiser prior can be
plugged in as a modular part of model-based optimization methods to solve other
inverse problems (e.g., deblurring). Such an integration induces considerable
advantage when the denoiser is obtained via discriminative learning. However,
the study of integration with fast discriminative denoiser prior is still
lacking. To this end, this paper aims to train a set of fast and effective CNN
(convolutional neural network) denoisers and integrate them into model-based
optimization method to solve other inverse problems. Experimental results
demonstrate that the learned set of denoisers not only achieve promising
Gaussian denoising results but also can be used as prior to deliver good
performance for various low-level vision applications.Comment: Accepted to CVPR 2017. Code: https://github.com/cszn/ircn
Learning Convolutional Networks for Content-weighted Image Compression
Lossy image compression is generally formulated as a joint rate-distortion
optimization to learn encoder, quantizer, and decoder. However, the quantizer
is non-differentiable, and discrete entropy estimation usually is required for
rate control. These make it very challenging to develop a convolutional network
(CNN)-based image compression system. In this paper, motivated by that the
local information content is spatially variant in an image, we suggest that the
bit rate of the different parts of the image should be adapted to local
content. And the content aware bit rate is allocated under the guidance of a
content-weighted importance map. Thus, the sum of the importance map can serve
as a continuous alternative of discrete entropy estimation to control
compression rate. And binarizer is adopted to quantize the output of encoder
due to the binarization scheme is also directly defined by the importance map.
Furthermore, a proxy function is introduced for binary operation in backward
propagation to make it differentiable. Therefore, the encoder, decoder,
binarizer and importance map can be jointly optimized in an end-to-end manner
by using a subset of the ImageNet database. In low bit rate image compression,
experiments show that our system significantly outperforms JPEG and JPEG 2000
by structural similarity (SSIM) index, and can produce the much better visual
result with sharp edges, rich textures, and fewer artifacts
Weighted Schatten -Norm Minimization for Image Denoising and Background Subtraction
Low rank matrix approximation (LRMA), which aims to recover the underlying
low rank matrix from its degraded observation, has a wide range of applications
in computer vision. The latest LRMA methods resort to using the nuclear norm
minimization (NNM) as a convex relaxation of the nonconvex rank minimization.
However, NNM tends to over-shrink the rank components and treats the different
rank components equally, limiting its flexibility in practical applications. We
propose a more flexible model, namely the Weighted Schatten -Norm
Minimization (WSNM), to generalize the NNM to the Schatten -norm
minimization with weights assigned to different singular values. The proposed
WSNM not only gives better approximation to the original low-rank assumption,
but also considers the importance of different rank components. We analyze the
solution of WSNM and prove that, under certain weights permutation, WSNM can be
equivalently transformed into independent non-convex -norm subproblems,
whose global optimum can be efficiently solved by generalized iterated
shrinkage algorithm. We apply WSNM to typical low-level vision problems, e.g.,
image denoising and background subtraction. Extensive experimental results
show, both qualitatively and quantitatively, that the proposed WSNM can more
effectively remove noise, and model complex and dynamic scenes compared with
state-of-the-art methods.Comment: 13 pages, 11 figure
Learning Motion Refinement for Unsupervised Face Animation
Unsupervised face animation aims to generate a human face video based on the
appearance of a source image, mimicking the motion from a driving video.
Existing methods typically adopted a prior-based motion model (e.g., the local
affine motion model or the local thin-plate-spline motion model). While it is
able to capture the coarse facial motion, artifacts can often be observed
around the tiny motion in local areas (e.g., lips and eyes), due to the limited
ability of these methods to model the finer facial motions. In this work, we
design a new unsupervised face animation approach to learn simultaneously the
coarse and finer motions. In particular, while exploiting the local affine
motion model to learn the global coarse facial motion, we design a novel motion
refinement module to compensate for the local affine motion model for modeling
finer face motions in local areas. The motion refinement is learned from the
dense correlation between the source and driving images. Specifically, we first
construct a structure correlation volume based on the keypoint features of the
source and driving images. Then, we train a model to generate the tiny facial
motions iteratively from low to high resolution. The learned motion refinements
are combined with the coarse motion to generate the new image. Extensive
experiments on widely used benchmarks demonstrate that our method achieves the
best results among state-of-the-art baselines.Comment: NeurIPS 202
- …