Image harmonization aims to produce visually harmonious composite images by
adjusting the foreground appearance to be compatible with the background. When
the composite image has photographic foreground and painterly background, the
task is called painterly image harmonization. There are only few works on this
task, which are either time-consuming or weak in generating well-harmonized
results. In this work, we propose a novel painterly harmonization network
consisting of a dual-domain generator and a dual-domain discriminator, which
harmonizes the composite image in both spatial domain and frequency domain. The
dual-domain generator performs harmonization by using AdaIN modules in the
spatial domain and our proposed ResFFT modules in the frequency domain. The
dual-domain discriminator attempts to distinguish the inharmonious patches
based on the spatial feature and frequency feature of each patch, which can
enhance the ability of generator in an adversarial manner. Extensive
experiments on the benchmark dataset show the effectiveness of our method. Our
code and model are available at
https://github.com/bcmi/PHDNet-Painterly-Image-Harmonization.Comment: Accepted by AAAI202