74 research outputs found
Semantic-Sparse Colorization Network for Deep Exemplar-based Colorization
Exemplar-based colorization approaches rely on reference image to provide
plausible colors for target gray-scale image. The key and difficulty of
exemplar-based colorization is to establish an accurate correspondence between
these two images. Previous approaches have attempted to construct such a
correspondence but are faced with two obstacles. First, using luminance
channels for the calculation of correspondence is inaccurate. Second, the dense
correspondence they built introduces wrong matching results and increases the
computation burden. To address these two problems, we propose Semantic-Sparse
Colorization Network (SSCN) to transfer both the global image style and
detailed semantic-related colors to the gray-scale image in a coarse-to-fine
manner. Our network can perfectly balance the global and local colors while
alleviating the ambiguous matching problem. Experiments show that our method
outperforms existing methods in both quantitative and qualitative evaluation
and achieves state-of-the-art performance.Comment: Accepted by ECCV2022; 14 pages, 10 figure
Eliminating Gradient Conflict in Reference-based Line-Art Colorization
Reference-based line-art colorization is a challenging task in computer
vision. The color, texture, and shading are rendered based on an abstract
sketch, which heavily relies on the precise long-range dependency modeling
between the sketch and reference. Popular techniques to bridge the cross-modal
information and model the long-range dependency employ the attention mechanism.
However, in the context of reference-based line-art colorization, several
techniques would intensify the existing training difficulty of attention, for
instance, self-supervised training protocol and GAN-based losses. To understand
the instability in training, we detect the gradient flow of attention and
observe gradient conflict among attention branches. This phenomenon motivates
us to alleviate the gradient issue by preserving the dominant gradient branch
while removing the conflict ones. We propose a novel attention mechanism using
this training strategy, Stop-Gradient Attention (SGA), outperforming the
attention baseline by a large margin with better training stability. Compared
with state-of-the-art modules in line-art colorization, our approach
demonstrates significant improvements in Fr\'echet Inception Distance (FID, up
to 27.21%) and structural similarity index measure (SSIM, up to 25.67%) on
several benchmarks. The code of SGA is available at
https://github.com/kunkun0w0/SGA .Comment: Accepted by ECCV202
A review of image and video colorization: From analogies to deep learning
Image colorization is a classic and important topic in computer graphics, where the aim is to add color to a monochromatic input image to produce a colorful result. In this survey, we present the history of colorization research in chronological order and summarize popular algorithms in this field. Early works on colorization mostly focused on developing techniques to improve the colorization quality. In the last few years, researchers have considered more possibilities such as combining colorization with NLP (natural language processing) and focused more on industrial applications. To better control the color, various types of color control are designed, such as providing reference images or color-scribbles. We have created a taxonomy of the colorization methods according to the input type, divided into grayscale, sketch-based and hybrid. The pros and cons are discussed for each algorithm, and they are compared according to their main characteristics. Finally, we discuss how deep learning, and in particular Generative Adversarial Networks (GANs), has changed this field
- …