225 research outputs found
Mean value coordinates–based caricature and expression synthesis
We present a novel method for caricature synthesis based on mean value coordinates (MVC). Our method can be applied to any single frontal face image to learn a specified caricature face pair for frontal and 3D caricature synthesis. This technique only requires one or a small number of exemplar pairs and a natural frontal face image training set, while the system can transfer the style of the exemplar pair across individuals. Further exaggeration can be fulfilled in a controllable way. Our method is further applied to facial expression transfer, interpolation, and exaggeration, which are applications of expression editing. Additionally, we have extended our approach to 3D caricature synthesis based on the 3D version of MVC. With experiments we demonstrate that the transferred expressions are credible and the resulting caricatures can be characterized and recognized
HeadOn: Real-time Reenactment of Human Portrait Videos
We propose HeadOn, the first real-time source-to-target reenactment approach
for complete human portrait videos that enables transfer of torso and head
motion, face expression, and eye gaze. Given a short RGB-D video of the target
actor, we automatically construct a personalized geometry proxy that embeds a
parametric head, eye, and kinematic torso model. A novel real-time reenactment
algorithm employs this proxy to photo-realistically map the captured motion
from the source actor to the target actor. On top of the coarse geometric
proxy, we propose a video-based rendering technique that composites the
modified target portrait video via view- and pose-dependent texturing, and
creates photo-realistic imagery of the target actor under novel torso and head
poses, facial expressions, and gaze directions. To this end, we propose a
robust tracking of the face and torso of the source actor. We extensively
evaluate our approach and show significant improvements in enabling much
greater flexibility in creating realistic reenacted output videos.Comment: Video: https://www.youtube.com/watch?v=7Dg49wv2c_g Presented at
Siggraph'1
Deep Manifold Traversal: Changing Labels with Convolutional Features
Many tasks in computer vision can be cast as a "label changing" problem, where the goal is to make a semantic change to the appearance of an image or some subject in an image in order to alter the class membership. Although successful task-specific methods have been developed for some label changing applications, to date no general purpose method exists. Motivated by this we propose deep manifold traversal, a method that addresses the problem in its most general form: it first approximates the manifold of natural images then morphs a test image along a traversal path away from a source class and towards a target class while staying near the manifold throughout. The resulting algorithm is surprisingly effective and versatile. It is completely data driven, requiring only an example set of images from the desired source and target domains. We demonstrate deep manifold traversal on highly diverse label changing tasks: changing an individual's appearance (age and hair color), changing the season of an outdoor image, and transforming a city skyline towards nighttime
StyleDomain: Efficient and Lightweight Parameterizations of StyleGAN for One-shot and Few-shot Domain Adaptation
Domain adaptation of GANs is a problem of fine-tuning the state-of-the-art
GAN models (e.g. StyleGAN) pretrained on a large dataset to a specific domain
with few samples (e.g. painting faces, sketches, etc.). While there are a great
number of methods that tackle this problem in different ways, there are still
many important questions that remain unanswered.
In this paper, we provide a systematic and in-depth analysis of the domain
adaptation problem of GANs, focusing on the StyleGAN model. First, we perform a
detailed exploration of the most important parts of StyleGAN that are
responsible for adapting the generator to a new domain depending on the
similarity between the source and target domains. As a result of this in-depth
study, we propose new efficient and lightweight parameterizations of StyleGAN
for domain adaptation. Particularly, we show there exist directions in
StyleSpace (StyleDomain directions) that are sufficient for adapting to similar
domains and they can be reduced further. For dissimilar domains, we propose
Affine and AffineLight parameterizations that allows us to outperform
existing baselines in few-shot adaptation with low data regime. Finally, we
examine StyleDomain directions and discover their many surprising properties
that we apply for domain mixing and cross-domain image morphing.Comment: Preprin
- …