6 research outputs found
Multi-Channel Attention Selection GANs for Guided Image-to-Image Translation
We propose a novel model named Multi-Channel Attention Selection Generative
Adversarial Network (SelectionGAN) for guided image-to-image translation, where
we translate an input image into another while respecting an external semantic
guidance. The proposed SelectionGAN explicitly utilizes the semantic guidance
information and consists of two stages. In the first stage, the input image and
the conditional semantic guidance are fed into a cycled semantic-guided
generation network to produce initial coarse results. In the second stage, we
refine the initial results by using the proposed multi-scale spatial pooling \&
channel selection module and the multi-channel attention selection module.
Moreover, uncertainty maps automatically learned from attention maps are used
to guide the pixel loss for better network optimization. Exhaustive experiments
on four challenging guided image-to-image translation tasks (face, hand, body
and street view) demonstrate that our SelectionGAN is able to generate
significantly better results than the state-of-the-art methods. Meanwhile, the
proposed framework and modules are unified solutions and can be applied to
solve other generation tasks, such as semantic image synthesis. The code is
available at https://github.com/Ha0Tang/SelectionGAN.Comment: An extended version of a paper published in CVPR2019. arXiv admin
note: substantial text overlap with arXiv:1904.0680
Model-based occlusion disentanglement for image-to-image translation
Image-to-image translation is affected by entanglement phenomena, which may
occur in case of target data encompassing occlusions such as raindrops, dirt,
etc. Our unsupervised model-based learning disentangles scene and occlusions,
while benefiting from an adversarial pipeline to regress physical parameters of
the occlusion model. The experiments demonstrate our method is able to handle
varying types of occlusions and generate highly realistic translations,
qualitatively and quantitatively outperforming the state-of-the-art on multiple
datasets.Comment: ECCV 202
SAR-to-Optical Image Translation via Thermodynamics-inspired Network
Synthetic aperture radar (SAR) is prevalent in the remote sensing field but
is difficult to interpret in human visual perception. Recently, SAR-to-optical
(S2O) image conversion methods have provided a prospective solution for
interpretation. However, since there is a huge domain difference between
optical and SAR images, they suffer from low image quality and geometric
distortion in the produced optical images. Motivated by the analogy between
pixels during the S2O image translation and molecules in a heat field,
Thermodynamics-inspired Network for SAR-to-Optical Image Translation (S2O-TDN)
is proposed in this paper. Specifically, we design a Third-order Finite
Difference (TFD) residual structure in light of the TFD equation of
thermodynamics, which allows us to efficiently extract inter-domain invariant
features and facilitate the learning of the nonlinear translation mapping. In
addition, we exploit the first law of thermodynamics (FLT) to devise an
FLT-guided branch that promotes the state transition of the feature values from
the unstable diffusion state to the stable one, aiming to regularize the
feature diffusion and preserve image structures during S2O image translation.
S2O-TDN follows an explicit design principle derived from thermodynamic theory
and enjoys the advantage of explainability. Experiments on the public SEN1-2
dataset show the advantages of the proposed S2O-TDN over the current methods
with more delicate textures and higher quantitative results
Multi-channel attention selection GANs for guided image-to-image translation
We propose a novel model named Multi-Channel Attention Selection Generative Adversarial Network (SelectionGAN) for guided image-to-image translation, where we translate an input image into another while respecting an external semantic guidance. The proposed SelectionGAN explicitly utilizes the semantic guidance information and consists of two stages. In the first stage, the input image and the conditional semantic guidance are fed into a cycled semantic-guided generation network to produce initial coarse results. In the second stage, we refine the initial results by using the proposed multi-scale spatial pooling &amp; channel selection module and the multi-channel attention selection module. Moreover, uncertainty maps automatically learned from attention maps are used to guide the pixel loss for better network optimization. Exhaustive experiments on four challenging guided image-to-image translation tasks (face, hand, body, and street view) demonstrate that our SelectionGAN is able to generate significantly better results than the state-of-the-art methods. Meanwhile, the proposed framework and modules are unified solutions and can be applied to solve other generation tasks such as semantic image synthesis. The code is available at&#x00A0;<uri>https://github.com/Ha0Tang/SelectionGAN</uri>