444 research outputs found
Multi-crop Contrastive Learning for Unsupervised Image-to-Image Translation
Recently, image-to-image translation methods based on contrastive learning
achieved state-of-the-art results in many tasks. However, the negatives are
sampled from the input feature spaces in the previous work, which makes the
negatives lack diversity. Moreover, in the latent space of the embedings,the
previous methods ignore domain consistency between the generated image and the
real images of target domain. In this paper, we propose a novel contrastive
learning framework for unpaired image-to-image translation, called MCCUT. We
utilize the multi-crop views to generate the negatives via the center-crop and
the random-crop, which can improve the diversity of negatives and meanwhile
increase the quality of negatives. To constrain the embedings in the deep
feature space,, we formulate a new domain consistency loss function, which
encourages the generated images to be close to the real images in the embedding
space of same domain. Furthermore, we present a dual coordinate channel
attention network by embedding positional information into SENet, which called
DCSE module. We employ the DCSE module in the design of generator, which makes
the generator pays more attention to channels with greater weight. In many
image-to-image translation tasks, our method achieves state-of-the-art results,
and the advantages of our method have been proved through extensive comparison
experiments and ablation research
Unsupervised Joint Image Transfer and Uncertainty Quantification Using Patch Invariant Networks
Unsupervised image transfer enables intra- and inter-modality image
translation in applications where a large amount of paired training data is not
abundant. To ensure a structure-preserving mapping from the input to the target
domain, existing methods for unpaired image transfer are commonly based on
cycle-consistency, causing additional computational resources and instability
due to the learning of an inverse mapping. This paper presents a novel method
for uni-directional domain mapping that does not rely on any paired training
data. A proper transfer is achieved by using a GAN architecture and a novel
generator loss based on patch invariance. To be more specific, the generator
outputs are evaluated and compared at different scales, also leading to an
increased focus on high-frequency details as well as an implicit data
augmentation. This novel patch loss also offers the possibility to accurately
predict aleatoric uncertainty by modeling an input-dependent scale map for the
patch residuals. The proposed method is comprehensively evaluated on three
well-established medical databases. As compared to four state-of-the-art
methods, we observe significantly higher accuracy on these datasets, indicating
great potential of the proposed method for unpaired image transfer with
uncertainty taken into account. Implementation of the proposed framework is
released here:
\url{https://github.com/anger-man/unsupervised-image-transfer-and-uq}.Comment: Accepted to ECCV 2022 Workshop on Uncertainty Quantification for
Computer Vision (UNCV 2022
Place Recognition under Occlusion and Changing Appearance via Disentangled Representations
Place recognition is a critical and challenging task for mobile robots,
aiming to retrieve an image captured at the same place as a query image from a
database. Existing methods tend to fail while robots move autonomously under
occlusion (e.g., car, bus, truck) and changing appearance (e.g., illumination
changes, seasonal variation). Because they encode the image into only one code,
entangling place features with appearance and occlusion features. To overcome
this limitation, we propose PROCA, an unsupervised approach to decompose the
image representation into three codes: a place code used as a descriptor to
retrieve images, an appearance code that captures appearance properties, and an
occlusion code that encodes occlusion content. Extensive experiments show that
our model outperforms the state-of-the-art methods. Our code and data are
available at https://github.com/rover-xingyu/PROCA
- …