Image-based head swapping task aims to stitch a source head to another source
body flawlessly. This seldom-studied task faces two major challenges: 1)
Preserving the head and body from various sources while generating a seamless
transition region. 2) No paired head swapping dataset and benchmark so far. In
this paper, we propose an image-based head swapping framework (HS-Diffusion)
which consists of a semantic-guided latent diffusion model (SG-LDM) and a
semantic layout generator. We blend the semantic layouts of source head and
source body, and then inpaint the transition region by the semantic layout
generator, achieving a coarse-grained head swapping. SG-LDM can further
implement fine-grained head swapping with the blended layout as condition by a
progressive fusion process, while preserving source head and source body with
high-quality reconstruction. To this end, we design a head-cover augmentation
strategy for training and a neck alignment trick for geometric realism.
Importantly, we construct a new image-based head swapping benchmark and propose
two tailor-designed metrics (Mask-FID and Focal-FID). Extensive experiments
demonstrate the superiority of our framework. The code will be available:
https://github.com/qinghew/HS-Diffusion