In this paper, we propose a diffusion-based face swapping framework for the
first time, called DiffFace, composed of training ID conditional DDPM, sampling
with facial guidance, and a target-preserving blending. In specific, in the
training process, the ID conditional DDPM is trained to generate face images
with the desired identity. In the sampling process, we use the off-the-shelf
facial expert models to make the model transfer source identity while
preserving target attributes faithfully. During this process, to preserve the
background of the target image and obtain the desired face swapping result, we
additionally propose a target-preserving blending strategy. It helps our model
to keep the attributes of the target face from noise while transferring the
source facial identity. In addition, without any re-training, our model can
flexibly apply additional facial guidance and adaptively control the
ID-attributes trade-off to achieve the desired results. To the best of our
knowledge, this is the first approach that applies the diffusion model in face
swapping task. Compared with previous GAN-based approaches, by taking advantage
of the diffusion model for the face swapping task, DiffFace achieves better
benefits such as training stability, high fidelity, diversity of the samples,
and controllability. Extensive experiments show that our DiffFace is comparable
or superior to the state-of-the-art methods on several standard face swapping
benchmarks.Comment: Project Page: https://hxngiee.github.io/DiffFac