1,798 research outputs found
High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks
Synthesizing face sketches from real photos and its inverse have many
applications. However, photo/sketch synthesis remains a challenging problem due
to the fact that photo and sketch have different characteristics. In this work,
we consider this task as an image-to-image translation problem and explore the
recently popular generative models (GANs) to generate high-quality realistic
photos from sketches and sketches from photos. Recent GAN-based methods have
shown promising results on image-to-image translation problems and
photo-to-sketch synthesis in particular, however, they are known to have
limited abilities in generating high-resolution realistic images. To this end,
we propose a novel synthesis framework called Photo-Sketch Synthesis using
Multi-Adversarial Networks, (PS2-MAN) that iteratively generates low resolution
to high resolution images in an adversarial way. The hidden layers of the
generator are supervised to first generate lower resolution images followed by
implicit refinement in the network to generate higher resolution images.
Furthermore, since photo-sketch synthesis is a coupled/paired translation
problem, we leverage the pair information using CycleGAN framework. Both Image
Quality Assessment (IQA) and Photo-Sketch Matching experiments are conducted to
demonstrate the superior performance of our framework in comparison to existing
state-of-the-art solutions. Code available at:
https://github.com/lidan1/PhotoSketchMAN.Comment: Accepted by 2018 13th IEEE International Conference on Automatic Face
& Gesture Recognition (FG 2018)(Oral
Fast Preprocessing for Robust Face Sketch Synthesis
Exemplar-based face sketch synthesis methods usually meet the challenging
problem that input photos are captured in different lighting conditions from
training photos. The critical step causing the failure is the search of similar
patch candidates for an input photo patch. Conventional illumination invariant
patch distances are adopted rather than directly relying on pixel intensity
difference, but they will fail when local contrast within a patch changes. In
this paper, we propose a fast preprocessing method named Bidirectional
Luminance Remapping (BLR), which interactively adjust the lighting of training
and input photos. Our method can be directly integrated into state-of-the-art
exemplar-based methods to improve their robustness with ignorable computational
cost.Comment: IJCAI 2017. Project page:
http://www.cs.cityu.edu.hk/~yibisong/ijcai17_sketch/index.htm
Content-Adaptive Sketch Portrait Generation by Decompositional Representation Learning
Sketch portrait generation benefits a wide range of applications such as
digital entertainment and law enforcement. Although plenty of efforts have been
dedicated to this task, several issues still remain unsolved for generating
vivid and detail-preserving personal sketch portraits. For example, quite a few
artifacts may exist in synthesizing hairpins and glasses, and textural details
may be lost in the regions of hair or mustache. Moreover, the generalization
ability of current systems is somewhat limited since they usually require
elaborately collecting a dictionary of examples or carefully tuning
features/components. In this paper, we present a novel representation learning
framework that generates an end-to-end photo-sketch mapping through structure
and texture decomposition. In the training stage, we first decompose the input
face photo into different components according to their representational
contents (i.e., structural and textural parts) by using a pre-trained
Convolutional Neural Network (CNN). Then, we utilize a Branched Fully
Convolutional Neural Network (BFCN) for learning structural and textural
representations, respectively. In addition, we design a Sorted Matching Mean
Square Error (SM-MSE) metric to measure texture patterns in the loss function.
In the stage of sketch rendering, our approach automatically generates
structural and textural representations for the input photo and produces the
final result via a probabilistic fusion scheme. Extensive experiments on
several challenging benchmarks suggest that our approach outperforms
example-based synthesis algorithms in terms of both perceptual and objective
metrics. In addition, the proposed method also has better generalization
ability across dataset without additional training.Comment: Published in TIP 201
- …