Stylizing Face Images via Multiple Exemplars

We address the problem of transferring the style of a headshot photo to face images. Existing methods using a single exemplar lead to inaccurate results when the exemplar does not contain sufficient stylized facial components for a given photo. In this work, we propose an algorithm to stylize face images using multiple exemplars containing different subjects in the same style. Patch correspondences between an input photo and multiple exemplars are established using a Markov Random Field (MRF), which enables accurate local energy transfer via Laplacian stacks. As image patches from multiple exemplars are used, the boundaries of facial components on the target image are inevitably inconsistent. The artifacts are removed by a post-processing step using an edge-preserving filter. Experimental results show that the proposed algorithm consistently produces visually pleasing results.Comment: In CVIU 2017. Project Page: http://www.cs.cityu.edu.hk/~yibisong/cviu17/index.htm

Improved Sketch-to-Photo Generation Using Filter Aided Generative Adversarial Network

Generating a photographic face image from given input sketch is most challenging task in computer vision. Mainly the sketches drawn by sketch artist used in human identification. Sketch to photo synthesis is very important applications in law enforcement as well as character design, educational training. In recent years Generative Adversarial Network (GAN) shows excellent performance on sketch to photo synthesis problem.  Quality of hand drawn sketches affects the quality generated photo. It might be possible that while handling the hand drawn sketches, accidently by touching the user hand on pencil sketch or similar activities causes noise in given sketch. Likewise different styles like shading, darkness of pencil used by sketch artist may cause unnecessary noise in sketches. In recent year many sketches to photo synthesis methods are proposed, but they are mainly focused on network architecture to get better performance. In this paper we proposed Filter-aided GAN framework to remove such noise while synthesizing photo images from hand drawn sketches. Here we implement and compare different filtering methods with GAN.  Quantitative and qualitative result shows that proposed Filter-aided GAN generate the photo images which are visually pleasant and closer to ground truth image

Joint Face Hallucination and Deblurring via Structure Generation and Detail Enhancement

We address the problem of restoring a high-resolution face image from a blurry low-resolution input. This problem is difficult as super-resolution and deblurring need to be tackled simultaneously. Moreover, existing algorithms cannot handle face images well as low-resolution face images do not have much texture which is especially critical for deblurring. In this paper, we propose an effective algorithm by utilizing the domain-specific knowledge of human faces to recover high-quality faces. We first propose a facial component guided deep Convolutional Neural Network (CNN) to restore a coarse face image, which is denoted as the base image where the facial component is automatically generated from the input face image. However, the CNN based method cannot handle image details well. We further develop a novel exemplar-based detail enhancement algorithm via facial component matching. Extensive experiments show that the proposed method outperforms the state-of-the-art algorithms both quantitatively and qualitatively.Comment: In IJCV 201

HairBrush for Immersive Data-Driven Hair Modeling

International audienceWhile hair is an essential component of virtual humans, it is also one of the most challenging digital assets to create. Existing automatic techniques lack the generality and flexibility to create rich hair variations, while manual authoring interfaces often require considerable artistic skills and efforts, especially for intricate 3D hair structures that can be difficult to navigate. We propose an interactive hair modeling system that can help create complex hairstyles in minutes or hours that would otherwise take much longer with existing tools. Modelers, including novice users, can focus on the overall hairstyles and local hair deformations, as our system intelligently suggests the desired hair parts. Our method combines the flexibility of manual authoring and the convenience of data-driven automation. Since hair contains intricate 3D structures such as buns, knots, and strands, they are inherently challenging to create using traditional 2D interfaces. Our system provides a new 3D hair author-ing interface for immersive interaction in virtual reality (VR). Users can draw high-level guide strips, from which our system predicts the most plausible hairstyles via a deep neural network trained from a professionally curated dataset. Each hairstyle in our dataset is composed of multiple variations, serving as blend-shapes to fit the user drawings via global blending and local deformation. The fitted hair models are visualized as interactive suggestions that the user can select, modify, or ignore. We conducted a user study to confirm that our system can significantly reduce manual labor while improve the output quality for modeling a variety of head and facial hairstyles that are challenging to create via existing techniques