122 research outputs found
Learning Convolutional Networks for Content-weighted Image Compression
Lossy image compression is generally formulated as a joint rate-distortion
optimization to learn encoder, quantizer, and decoder. However, the quantizer
is non-differentiable, and discrete entropy estimation usually is required for
rate control. These make it very challenging to develop a convolutional network
(CNN)-based image compression system. In this paper, motivated by that the
local information content is spatially variant in an image, we suggest that the
bit rate of the different parts of the image should be adapted to local
content. And the content aware bit rate is allocated under the guidance of a
content-weighted importance map. Thus, the sum of the importance map can serve
as a continuous alternative of discrete entropy estimation to control
compression rate. And binarizer is adopted to quantize the output of encoder
due to the binarization scheme is also directly defined by the importance map.
Furthermore, a proxy function is introduced for binary operation in backward
propagation to make it differentiable. Therefore, the encoder, decoder,
binarizer and importance map can be jointly optimized in an end-to-end manner
by using a subset of the ImageNet database. In low bit rate image compression,
experiments show that our system significantly outperforms JPEG and JPEG 2000
by structural similarity (SSIM) index, and can produce the much better visual
result with sharp edges, rich textures, and fewer artifacts
The Preventive and Therapeutic Effect of Caloric Restriction Therapy on Type 2 Diabetes Mellitus
Learning Motion Refinement for Unsupervised Face Animation
Unsupervised face animation aims to generate a human face video based on the
appearance of a source image, mimicking the motion from a driving video.
Existing methods typically adopted a prior-based motion model (e.g., the local
affine motion model or the local thin-plate-spline motion model). While it is
able to capture the coarse facial motion, artifacts can often be observed
around the tiny motion in local areas (e.g., lips and eyes), due to the limited
ability of these methods to model the finer facial motions. In this work, we
design a new unsupervised face animation approach to learn simultaneously the
coarse and finer motions. In particular, while exploiting the local affine
motion model to learn the global coarse facial motion, we design a novel motion
refinement module to compensate for the local affine motion model for modeling
finer face motions in local areas. The motion refinement is learned from the
dense correlation between the source and driving images. Specifically, we first
construct a structure correlation volume based on the keypoint features of the
source and driving images. Then, we train a model to generate the tiny facial
motions iteratively from low to high resolution. The learned motion refinements
are combined with the coarse motion to generate the new image. Extensive
experiments on widely used benchmarks demonstrate that our method achieves the
best results among state-of-the-art baselines.Comment: NeurIPS 202
- …