Equivariance and Invariance for Robust Unsupervised and Semi-Supervised Learning

Although there is a great success of applying deep learning on a wide variety of tasks, it heavily relies on a large amount of labeled training data, which could be hard to obtain in many real scenarios. To address this problem, unsupervised and semi-supervised learning emerge to take advantage of the plenty of cheap unlabeled data to improve the model generalization. In this dissertation, we claim that equivariant and invariance are two critical criteria to approach robust unsupervised and semi-supervised learning. The idea is as follows: the features of a robust model ought to be sufficiently informative and equivariant to transformations on the input data, and the classifiers should be resilient and invariant to small perturbations on the data manifold and model parameters. Specifically, features are learnt via auto-encoding the transformations on the input data, and models are regularized through minimizing the effects of perturbations on features or model parameters. Experiments on several benchmarks show the proposed methods outperform many state-of-the-art approaches on unsupervised and semi-supervised learning, proving importance of the equivariance and invariance rules for robust feature representation learning

SSL++: Improving Self-supervised Learning by Mitigating the Proxy Task-Specificity Problem

The success of deep convolutional networks (ConvNets) generally relies on a massive amount of well-labeled data, which is labor-intensive and time-consuming to collect and annotate in many scenarios. To eliminate such limitation, self-supervised learning (SSL) is recently proposed. Specifically, by solving a pre-designed proxy task, SSL is capable of capturing general-purpose features without requiring human supervision. Existing efforts focus obsessively on designing a particular proxy task but ignore the semanticity of samples that are advantageous to downstream tasks, resulting in the inherent limitation that the learned features are specific to the proxy task, namely the proxy task-specificity of features. In this work, to improve the generalizability of features learned by existing SSL methods, we present a novel self-supervised framework SSL++ to incorporate the proxy task-independent semanticity of samples into the representation learning process. Technically, SSL++ aims to leverage the complementarity, between the low-level generic features learned by a proxy task and the high-level semantic features newly learned by the generated semantic pseudo-labels, to mitigate the task-specificity and improve the generalizability of features. Extensive experiments show that SSL++ performs favorably against the state-of-the-art approaches on the established and latest SSL benchmarks. The code will be available to the public