Deep networks have achieved great success in image rescaling (IR) task that
seeks to learn the optimal downscaled representations, i.e., low-resolution
(LR) images, to reconstruct the original high-resolution (HR) images. Compared
with super-resolution methods that consider a fixed downscaling scheme, e.g.,
bicubic, IR often achieves significantly better reconstruction performance
thanks to the learned downscaled representations. This highlights the
importance of a good downscaled representation in image reconstruction tasks.
Existing IR methods mainly learn the downscaled representation by jointly
optimizing the downscaling and upscaling models. Unlike them, we seek to
improve the downscaled representation through a different and more direct way:
optimizing the downscaled image itself instead of the down-/upscaling models.
Specifically, we propose a collaborative downscaling scheme that directly
generates the collaborative LR examples by descending the gradient w.r.t. the
reconstruction loss on them to benefit the IR process. Furthermore, since LR
images are downscaled from the corresponding HR images, one can also improve
the downscaled representation if we have a better representation in the HR
domain. Inspired by this, we propose a Hierarchical Collaborative Downscaling
(HCD) method that performs gradient descent in both HR and LR domains to
improve the downscaled representations. Extensive experiments show that our HCD
significantly improves the reconstruction performance both quantitatively and
qualitatively. Moreover, we also highlight the flexibility of our HCD since it
can generalize well across diverse IR models.Comment: 11 pages, 8 figure