2 research outputs found
Deep Exemplar-based Video Colorization
This paper presents the first end-to-end network for exemplar-based video
colorization. The main challenge is to achieve temporal consistency while
remaining faithful to the reference style. To address this issue, we introduce
a recurrent framework that unifies the semantic correspondence and color
propagation steps. Both steps allow a provided reference image to guide the
colorization of every frame, thus reducing accumulated propagation errors.
Video frames are colorized in sequence based on the colorization history, and
its coherency is further enforced by the temporal consistency loss. All of
these components, learned end-to-end, help produce realistic videos with good
temporal stability. Experiments show our result is superior to the
state-of-the-art methods both quantitatively and qualitatively
SCGAN: Saliency Map-guided Colorization with Generative Adversarial Network
Given a grayscale photograph, the colorization system estimates a visually
plausible colorful image. Conventional methods often use semantics to colorize
grayscale images. However, in these methods, only classification semantic
information is embedded, resulting in semantic confusion and color bleeding in
the final colorized image. To address these issues, we propose a fully
automatic Saliency Map-guided Colorization with Generative Adversarial Network
(SCGAN) framework. It jointly predicts the colorization and saliency map to
minimize semantic confusion and color bleeding in the colorized image. Since
the global features from pre-trained VGG-16-Gray network are embedded to the
colorization encoder, the proposed SCGAN can be trained with much less data
than state-of-the-art methods to achieve perceptually reasonable colorization.
In addition, we propose a novel saliency map-based guidance method. Branches of
the colorization decoder are used to predict the saliency map as a proxy
target. Moreover, two hierarchical discriminators are utilized for the
generated colorization and saliency map, respectively, in order to strengthen
visual perception performance. The proposed system is evaluated on ImageNet
validation set. Experimental results show that SCGAN can generate more
reasonable colorized images than state-of-the-art techniques.Comment: accepted by IEEE Transactions on Circuits and Systems for Video
Technolog