Estimation of Scribble Placement for Painting Colorization

Image colorization has been a topic of interest since the mid 70’s and several algorithms have been proposed that given a grayscale image and color scribbles (hints) produce a colorized image. Recently, this approach has been introduced in the field of art conservation and cultural heritage, where B&W photographs of paintings at previous stages have been colorized. However, the questions of what is the minimum number of scribbles necessary and where they should be placed in an image remain unexplored. Here we address this limitation using an iterative algorithm that provides insights as to the relationship between locally vs. globally important scribbles. Given a color image we randomly select scribbles and we attempt to color the grayscale version of the original.We define a scribble contribution measure based on the reconstruction error. We demonstrate our approach using a widely used colorization algorithm and images from a Picasso painting and the peppers test image. We show that areas isolated by thick brushstrokes or areas with high textural variation are locally important but contribute very little to the overall representation accuracy. We also find that for the case of Picasso on average 10% of scribble coverage is enough and that flat areas can be presented by few scribbles. The proposed method can be used verbatim to test any colorization algorithm

PixColor: Pixel Recursive Colorization

We propose a novel approach to automatically produce multiple colorized versions of a grayscale image. Our method results from the observation that the task of automated colorization is relatively easy given a low-resolution version of the color image. We first train a conditional PixelCNN to generate a low resolution color for a given grayscale image. Then, given the generated low-resolution color image and the original grayscale image as inputs, we train a second CNN to generate a high-resolution colorization of an image. We demonstrate that our approach produces more diverse and plausible colorizations than existing methods, as judged by human raters in a "Visual Turing Test"

Deep Video Color Propagation

Traditional approaches for color propagation in videos rely on some form of matching between consecutive video frames. Using appearance descriptors, colors are then propagated both spatially and temporally. These methods, however, are computationally expensive and do not take advantage of semantic information of the scene. In this work we propose a deep learning framework for color propagation that combines a local strategy, to propagate colors frame-by-frame ensuring temporal stability, and a global strategy, using semantics for color propagation within a longer range. Our evaluation shows the superiority of our strategy over existing video and image color propagation methods as well as neural photo-realistic style transfer approaches.Comment: BMVC 201

Piecewise constant reconstruction of damaged color images

A variational model for reconstruction of damaged color images is studied, in particular in the case where only finitely many colors are admissible for the reconstructed image. An existence result and regularity properties of minimizers are presented

A FULLY AUTOMATIC GENETIC APPROACH FOR GRAYSCALE IMAGE COLORIZATION

Colorization is a computer assisted process of adding color to a monochrome (grayscale) image or movie. The early published methods to perform the image colorizing rely on heuristic techniques for choosing RGB colors from a global palette and applying them to regions of the target grayscaled image. The main improvement of the proposed technique is the adoption in a fully automatic way the genetic algorithm as an efficient search method to find best match for each pixel in the target image. The proposed genetic algorithm evolves a population of randomly selected individuals (that represents a possible color setting for target image using a reference colored source image toward solution that could resemble natural or real colors to the objects of the target scene). Moreover this study proposes new crossover operator, called Spread out Uniform Crossover (SUX) that turns the recombination scheme of uniform crossover over spreading vital genes at the expense of lethal genes rather than exchanging genes between mating parents to the generated offspring. The results of the proposed colorization techniques are good and plausible

Example-based image colorization using locality consistent sparse representation

—Image colorization aims to produce a natural looking color image from a given grayscale image, which remains a challenging problem. In this paper, we propose a novel examplebased image colorization method exploiting a new locality consistent sparse representation. Given a single reference color image, our method automatically colorizes the target grayscale image by sparse pursuit. For efficiency and robustness, our method operates at the superpixel level. We extract low-level intensity features, mid-level texture features and high-level semantic features for each superpixel, which are then concatenated to form its descriptor. The collection of feature vectors for all the superpixels from the reference image composes the dictionary. We formulate colorization of target superpixels as a dictionary-based sparse reconstruction problem. Inspired by the observation that superpixels with similar spatial location and/or feature representation are likely to match spatially close regions from the reference image, we further introduce a locality promoting regularization term into the energy formulation which substantially improves the matching consistency and subsequent colorization results. Target superpixels are colorized based on the chrominance information from the dominant reference superpixels. Finally, to further improve coherence while preserving sharpness, we develop a new edge-preserving filter for chrominance channels with the guidance from the target grayscale image. To the best of our knowledge, this is the first work on sparse pursuit image colorization from single reference images. Experimental results demonstrate that our colorization method outperforms state-ofthe-art methods, both visually and quantitatively using a user stud

Variational Approach for the Reconstruction of Damaged Optical Satellite Images Through Their Co-Registration with Synthetic Aperture Radar

In this paper the problem of reconstruction of damaged multi-band opticalimages is studied in the case where we have no information about brightness of suchimages in the damage region. Mostly motivated by the crop field monitoring problem,we propose a new variational approach for exact reconstruction of damaged multi-bandimages using results of their co-registration with Synthetic Aperture Radar (SAR) imagesof the same regions. We discuss the consistency of the proposed problem, give the schemefor its regularization, derive the corresponding optimality system, and describe in detailthe algorithm for the practical implementation of the reconstruction procedure.In this paper the problem of reconstruction of damaged multi-band opticalimages is studied in the case where we have no information about brightness of suchimages in the damage region. Mostly motivated by the crop field monitoring problem,we propose a new variational approach for exact reconstruction of damaged multi-bandimages using results of their co-registration with Synthetic Aperture Radar (SAR) imagesof the same regions. We discuss the consistency of the proposed problem, give the schemefor its regularization, derive the corresponding optimality system, and describe in detailthe algorithm for the practical implementation of the reconstruction procedure