3 research outputs found
Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning
In this work, we tackle the challenging problem of arbitrary image style
transfer using a novel style feature representation learning method. A suitable
style representation, as a key component in image stylization tasks, is
essential to achieve satisfactory results. Existing deep neural network based
approaches achieve reasonable results with the guidance from second-order
statistics such as Gram matrix of content features. However, they do not
leverage sufficient style information, which results in artifacts such as local
distortions and style inconsistency. To address these issues, we propose to
learn style representation directly from image features instead of their
second-order statistics, by analyzing the similarities and differences between
multiple styles and considering the style distribution. Specifically, we
present Contrastive Arbitrary Style Transfer (CAST), which is a new style
representation learning and style transfer method via contrastive learning. Our
framework consists of three key components, i.e., a multi-layer style projector
for style code encoding, a domain enhancement module for effective learning of
style distribution, and a generative network for image style transfer. We
conduct qualitative and quantitative evaluations comprehensively to demonstrate
that our approach achieves significantly better results compared to those
obtained via state-of-the-art methods. Code and models are available at
https://github.com/zyxElsa/CAST_pytorchComment: Accepted by SIGGRAPH 202