213 research outputs found
Fine-Grained Face Swapping via Regional GAN Inversion
We present a novel paradigm for high-fidelity face swapping that faithfully
preserves the desired subtle geometry and texture details. We rethink face
swapping from the perspective of fine-grained face editing, \textit{i.e.,
``editing for swapping'' (E4S)}, and propose a framework that is based on the
explicit disentanglement of the shape and texture of facial components.
Following the E4S principle, our framework enables both global and local
swapping of facial features, as well as controlling the amount of partial
swapping specified by the user. Furthermore, the E4S paradigm is inherently
capable of handling facial occlusions by means of facial masks. At the core of
our system lies a novel Regional GAN Inversion (RGI) method, which allows the
explicit disentanglement of shape and texture. It also allows face swapping to
be performed in the latent space of StyleGAN. Specifically, we design a
multi-scale mask-guided encoder to project the texture of each facial component
into regional style codes. We also design a mask-guided injection module to
manipulate the feature maps with the style codes. Based on the disentanglement,
face swapping is reformulated as a simplified problem of style and mask
swapping. Extensive experiments and comparisons with current state-of-the-art
methods demonstrate the superiority of our approach in preserving texture and
shape details, as well as working with high resolution images at
10241024.Comment: Project page, see http://e4s2022.github.i
- …