10 research outputs found
Shape Consistent 2D Keypoint Estimation under Domain Shift
Recent unsupervised domain adaptation methods based on deep architectures
have shown remarkable performance not only in traditional classification tasks
but also in more complex problems involving structured predictions (e.g.
semantic segmentation, depth estimation). Following this trend, in this paper
we present a novel deep adaptation framework for estimating keypoints under
domain shift}, i.e. when the training (source) and the test (target) images
significantly differ in terms of visual appearance. Our method seamlessly
combines three different components: feature alignment, adversarial training
and self-supervision. Specifically, our deep architecture leverages from
domain-specific distribution alignment layers to perform target adaptation at
the feature level. Furthermore, a novel loss is proposed which combines an
adversarial term for ensuring aligned predictions in the output space and a
geometric consistency term which guarantees coherent predictions between a
target sample and its perturbed version. Our extensive experimental evaluation
conducted on three publicly available benchmarks shows that our approach
outperforms state-of-the-art domain adaptation methods in the 2D keypoint
prediction task
An Investigation into Whitening Loss for Self-supervised Learning
A desirable objective in self-supervised learning (SSL) is to avoid feature
collapse. Whitening loss guarantees collapse avoidance by minimizing the
distance between embeddings of positive pairs under the conditioning that the
embeddings from different views are whitened. In this paper, we propose a
framework with an informative indicator to analyze whitening loss, which
provides a clue to demystify several interesting phenomena as well as a
pivoting point connecting to other SSL methods. We reveal that batch whitening
(BW) based methods do not impose whitening constraints on the embedding, but
they only require the embedding to be full-rank. This full-rank constraint is
also sufficient to avoid dimensional collapse. Based on our analysis, we
propose channel whitening with random group partition (CW-RGP), which exploits
the advantages of BW-based methods in preventing collapse and avoids their
disadvantages requiring large batch size. Experimental results on ImageNet
classification and COCO object detection reveal that the proposed CW-RGP
possesses a promising potential for learning good representations. The code is
available at https://github.com/winci-ai/CW-RGP.Comment: Accepted at NeurIPS 2022. The Code is available at:
https://github.com/winci-ai/CW-RG
Whitening and coloring batch transform for GANS
Batch Normalization (BN) is a common technique used to speed-up and stabilize training. On the other hand, the learnable parameters of BN are commonly used in conditional Generative Adversarial Networks (cGANs) for representing class-specific information using conditional Batch Normalization (cBN). In this paper we propose to generalize both BN and cBN using a Whitening and Coloring based batch normalization. We show that our conditional Coloring can represent categorical conditioning information which largely helps the cGAN qualitative results. Moreover, we show that full-feature whitening is important in a general GAN scenario in which the training process is known to be highly unstable. We test our approach on different datasets and using different GAN networks and training protocols, showing a consistent improvement in all the tested frameworks. Our CIFAR-10 conditioned results are higher than all previous works on this dataset