6,005 research outputs found
Instance-level Facial Attributes Transfer with Geometry-Aware Flow
We address the problem of instance-level facial attribute transfer without
paired training data, e.g. faithfully transferring the exact mustache from a
source face to a target face. This is a more challenging task than the
conventional semantic-level attribute transfer, which only preserves the
generic attribute style instead of instance-level traits. We propose the use of
geometry-aware flow, which serves as a well-suited representation for modeling
the transformation between instance-level facial attributes. Specifically, we
leverage the facial landmarks as the geometric guidance to learn the
differentiable flows automatically, despite of the large pose gap existed.
Geometry-aware flow is able to warp the source face attribute into the target
face context and generate a warp-and-blend result. To compensate for the
potential appearance gap between source and target faces, we propose a
hallucination sub-network that produces an appearance residual to further
refine the warp-and-blend result. Finally, a cycle-consistency framework
consisting of both attribute transfer module and attribute removal module is
designed, so that abundant unpaired face images can be used as training data.
Extensive evaluations validate the capability of our approach in transferring
instance-level facial attributes faithfully across large pose and appearance
gaps. Thanks to the flow representation, our approach can readily be applied to
generate realistic details on high-resolution images.Comment: To appear in AAAI 2019. Code and models are available at:
https://github.com/wdyin/GeoGA
Neural Face Editing with Intrinsic Image Disentangling
Traditional face editing methods often require a number of sophisticated and
task specific algorithms to be applied one after the other --- a process that
is tedious, fragile, and computationally intensive. In this paper, we propose
an end-to-end generative adversarial network that infers a face-specific
disentangled representation of intrinsic face properties, including shape (i.e.
normals), albedo, and lighting, and an alpha matte. We show that this network
can be trained on "in-the-wild" images by incorporating an in-network
physically-based image formation module and appropriate loss functions. Our
disentangling latent representation allows for semantically relevant edits,
where one aspect of facial appearance can be manipulated while keeping
orthogonal properties fixed, and we demonstrate its use for a number of facial
editing applications.Comment: CVPR 2017 ora
Calipso: Physics-based Image and Video Editing through CAD Model Proxies
We present Calipso, an interactive method for editing images and videos in a
physically-coherent manner. Our main idea is to realize physics-based
manipulations by running a full physics simulation on proxy geometries given by
non-rigidly aligned CAD models. Running these simulations allows us to apply
new, unseen forces to move or deform selected objects, change physical
parameters such as mass or elasticity, or even add entire new objects that
interact with the rest of the underlying scene. In Calipso, the user makes
edits directly in 3D; these edits are processed by the simulation and then
transfered to the target 2D content using shape-to-image correspondences in a
photo-realistic rendering process. To align the CAD models, we introduce an
efficient CAD-to-image alignment procedure that jointly minimizes for rigid and
non-rigid alignment while preserving the high-level structure of the input
shape. Moreover, the user can choose to exploit image flow to estimate scene
motion, producing coherent physical behavior with ambient dynamics. We
demonstrate Calipso's physics-based editing on a wide range of examples
producing myriad physical behavior while preserving geometric and visual
consistency.Comment: 11 page
- …