260 research outputs found
Joint CNN and Variational Model for Fully-automatic Image Colorization
International audienceThis paper aims to couple the powerful prediction of the convolutional neural network (CNN) to the accuracy at pixel scale of the variational methods. In this work, the limitations of the CNN-based image colorization approaches are described. We then focus on a CNN which is able to compute a statistical distribution of the colors for each pixel of the image based on a learning over a large color image database. After describing its limitation, the variational method of [17] is briefly recalled. This method is able to select a color candidate among a given set while performing a regularization of the result. By combining this approach with a CNN, we designed a fully automatic image coloriza-tion framework with an improved accuracy in comparison with CNN alone. Some numerical experiments demonstrate the increased accuracy performed by our method
PixColor: Pixel Recursive Colorization
We propose a novel approach to automatically produce multiple colorized
versions of a grayscale image. Our method results from the observation that the
task of automated colorization is relatively easy given a low-resolution
version of the color image. We first train a conditional PixelCNN to generate a
low resolution color for a given grayscale image. Then, given the generated
low-resolution color image and the original grayscale image as inputs, we train
a second CNN to generate a high-resolution colorization of an image. We
demonstrate that our approach produces more diverse and plausible colorizations
than existing methods, as judged by human raters in a "Visual Turing Test"
Learning Diverse Image Colorization
Colorization is an ambiguous problem, with multiple viable colorizations for
a single grey-level image. However, previous methods only produce the single
most probable colorization. Our goal is to model the diversity intrinsic to the
problem of colorization and produce multiple colorizations that display
long-scale spatial co-ordination. We learn a low dimensional embedding of color
fields using a variational autoencoder (VAE). We construct loss terms for the
VAE decoder that avoid blurry outputs and take into account the uneven
distribution of pixel colors. Finally, we build a conditional model for the
multi-modal distribution between grey-level image and the color field
embeddings. Samples from this conditional model result in diverse colorization.
We demonstrate that our method obtains better diverse colorizations than a
standard conditional variational autoencoder (CVAE) model, as well as a
recently proposed conditional generative adversarial network (cGAN).Comment: This revision to appear in CVPR1
Pixelated Semantic Colorization
While many image colorization algorithms have recently shown the capability
of producing plausible color versions from gray-scale photographs, they still
suffer from limited semantic understanding. To address this shortcoming, we
propose to exploit pixelated object semantics to guide image colorization. The
rationale is that human beings perceive and distinguish colors based on the
semantic categories of objects. Starting from an autoregressive model, we
generate image color distributions, from which diverse colored results are
sampled. We propose two ways to incorporate object semantics into the
colorization model: through a pixelated semantic embedding and a pixelated
semantic generator. Specifically, the proposed convolutional neural network
includes two branches. One branch learns what the object is, while the other
branch learns the object colors. The network jointly optimizes a color
embedding loss, a semantic segmentation loss and a color generation loss, in an
end-to-end fashion. Experiments on PASCAL VOC2012 and COCO-stuff reveal that
our network, when trained with semantic segmentation labels, produces more
realistic and finer results compared to the colorization state-of-the-art
Inner Space Preserving Generative Pose Machine
Image-based generative methods, such as generative adversarial networks
(GANs) have already been able to generate realistic images with much context
control, specially when they are conditioned. However, most successful
frameworks share a common procedure which performs an image-to-image
translation with pose of figures in the image untouched. When the objective is
reposing a figure in an image while preserving the rest of the image, the
state-of-the-art mainly assumes a single rigid body with simple background and
limited pose shift, which can hardly be extended to the images under normal
settings. In this paper, we introduce an image "inner space" preserving model
that assigns an interpretable low-dimensional pose descriptor (LDPD) to an
articulated figure in the image. Figure reposing is then generated by passing
the LDPD and the original image through multi-stage augmented hourglass
networks in a conditional GAN structure, called inner space preserving
generative pose machine (ISP-GPM). We evaluated ISP-GPM on reposing human
figures, which are highly articulated with versatile variations. Test of a
state-of-the-art pose estimator on our reposed dataset gave an accuracy over
80% on PCK0.5 metric. The results also elucidated that our ISP-GPM is able to
preserve the background with high accuracy while reasonably recovering the area
blocked by the figure to be reposed.Comment: http://www.northeastern.edu/ostadabbas/2018/07/23/inner-space-preserving-generative-pose-machine
ChromaGAN: Adversarial Picture Colorization with Semantic Class Distribution
The colorization of grayscale images is an ill-posed problem, with multiple
correct solutions. In this paper, we propose an adversarial learning
colorization approach coupled with semantic information. A generative network
is used to infer the chromaticity of a given grayscale image conditioned to
semantic clues. This network is framed in an adversarial model that learns to
colorize by incorporating perceptual and semantic understanding of color and
class distributions. The model is trained via a fully self-supervised strategy.
Qualitative and quantitative results show the capacity of the proposed method
to colorize images in a realistic way achieving state-of-the-art results.Comment: 8 pages + reference
- …