20 research outputs found
How Useful is Self-Supervised Pretraining for Visual Tasks?
Recent advances have spurred incredible progress in self-supervised
pretraining for vision. We investigate what factors may play a role in the
utility of these pretraining methods for practitioners. To do this, we evaluate
various self-supervised algorithms across a comprehensive array of synthetic
datasets and downstream tasks. We prepare a suite of synthetic data that
enables an endless supply of annotated images as well as full control over
dataset difficulty. Our experiments offer insights into how the utility of
self-supervision changes as the number of available labels grows as well as how
the utility changes as a function of the downstream task and the properties of
the training data. We also find that linear evaluation does not correlate with
finetuning performance. Code and data is available at
\href{https://www.github.com/princeton-vl/selfstudy}{github.com/princeton-vl/selfstudy}.Comment: To appear in CVPR 202
Weakly Supervised Semantic Segmentation for Large-Scale Point Cloud
Existing methods for large-scale point cloud semantic segmentation require
expensive, tedious and error-prone manual point-wise annotations. Intuitively,
weakly supervised training is a direct solution to reduce the cost of labeling.
However, for weakly supervised large-scale point cloud semantic segmentation,
too few annotations will inevitably lead to ineffective learning of network. We
propose an effective weakly supervised method containing two components to
solve the above problem. Firstly, we construct a pretext task, \textit{i.e.,}
point cloud colorization, with a self-supervised learning to transfer the
learned prior knowledge from a large amount of unlabeled point cloud to a
weakly supervised network. In this way, the representation capability of the
weakly supervised network can be improved by the guidance from a heterogeneous
task. Besides, to generate pseudo label for unlabeled data, a sparse label
propagation mechanism is proposed with the help of generated class prototypes,
which is used to measure the classification confidence of unlabeled point. Our
method is evaluated on large-scale point cloud datasets with different
scenarios including indoor and outdoor. The experimental results show the large
gain against existing weakly supervised and comparable results to fully
supervised methods\footnote{Code based on mindspore:
https://github.com/dmcv-ecnu/MindSpore\_ModelZoo/tree/main/WS3\_MindSpore}
Siamese-Network Based Signature Verification using Self Supervised Learning
The use of signatures is often encountered in various public documents ranging from academic documents to business documents that are a sign that the existence of signatures is crucial in various administrative processes. The frequent use of signatures does not mean a procedure without loopholes, but we must remain vigilant against signature falsification carried out with various motives behind it. Therefore, in this study, a signature verification system was developed that could prevent the falsification of signatures in public documents by using digital imagery of existing signatures. This study used neural networks with siamese network-based architectures that also empower self-supervised learning techniques to improve accuracy in the realm of limited data. The final evaluation of the machine learning method used gets a maximum accuracy of 83% and this result is better than the machine learning model that does not involve self-supervised learning methods
Benchmarking Representation Learning for Natural World Image Collections
Recent progress in self-supervised learning has resulted in models that are
capable of extracting rich representations from image collections without
requiring any explicit label supervision. However, to date the vast majority of
these approaches have restricted themselves to training on standard benchmark
datasets such as ImageNet. We argue that fine-grained visual categorization
problems, such as plant and animal species classification, provide an
informative testbed for self-supervised learning. In order to facilitate
progress in this area we present two new natural world visual classification
datasets, iNat2021 and NeWT. The former consists of 2.7M images from 10k
different species uploaded by users of the citizen science application
iNaturalist. We designed the latter, NeWT, in collaboration with domain experts
with the aim of benchmarking the performance of representation learning
algorithms on a suite of challenging natural world binary classification tasks
that go beyond standard species classification. These two new datasets allow us
to explore questions related to large-scale representation and transfer
learning in the context of fine-grained categories. We provide a comprehensive
analysis of feature extractors trained with and without supervision on ImageNet
and iNat2021, shedding light on the strengths and weaknesses of different
learned features across a diverse set of tasks. We find that features produced
by standard supervised methods still outperform those produced by
self-supervised approaches such as SimCLR. However, improved self-supervised
learning methods are constantly being released and the iNat2021 and NeWT
datasets are a valuable resource for tracking their progress.Comment: CVPR 202
Learning Visual Representations for Transfer Learning by Suppressing Texture
Recent literature has shown that features obtained from supervised training
of CNNs may over-emphasize texture rather than encoding high-level information.
In self-supervised learning in particular, texture as a low-level cue may
provide shortcuts that prevent the network from learning higher level
representations. To address these problems we propose to use classic methods
based on anisotropic diffusion to augment training using images with suppressed
texture. This simple method helps retain important edge information and
suppress texture at the same time. We empirically show that our method achieves
state-of-the-art results on object detection and image classification with
eight diverse datasets in either supervised or self-supervised learning tasks
such as MoCoV2 and Jigsaw. Our method is particularly effective for transfer
learning tasks and we observed improved performance on five standard transfer
learning datasets. The large improvements (up to 11.49\%) on the
Sketch-ImageNet dataset, DTD dataset and additional visual analyses with
saliency maps suggest that our approach helps in learning better
representations that better transfer
Overwriting Pretrained Bias with Finetuning Data
Transfer learning is beneficial by allowing the expressive features of models
pretrained on large-scale datasets to be finetuned for the target task of
smaller, more domain-specific datasets. However, there is a concern that these
pretrained models may come with their own biases which would propagate into the
finetuned model. In this work, we investigate bias when conceptualized as both
spurious correlations between the target task and a sensitive attribute as well
as underrepresentation of a particular group in the dataset. Under both notions
of bias, we find that (1) models finetuned on top of pretrained models can
indeed inherit their biases, but (2) this bias can be corrected for through
relatively minor interventions to the finetuning dataset, and often with a
negligible impact to performance. Our findings imply that careful curation of
the finetuning dataset is important for reducing biases on a downstream task,
and doing so can even compensate for bias in the pretrained model.Comment: ICCV 2023 Ora