20 research outputs found
XCon: Learning with Experts for Fine-grained Category Discovery
We address the problem of generalized category discovery (GCD) in this paper,
i.e. clustering the unlabeled images leveraging the information from a set of
seen classes, where the unlabeled images could contain both seen classes and
unseen classes. The seen classes can be seen as an implicit criterion of
classes, which makes this setting different from unsupervised clustering where
the cluster criteria may be ambiguous. We mainly concern the problem of
discovering categories within a fine-grained dataset since it is one of the
most direct applications of category discovery, i.e. helping experts discover
novel concepts within an unlabeled dataset using the implicit criterion set
forth by the seen classes. State-of-the-art methods for generalized category
discovery leverage contrastive learning to learn the representations, but the
large inter-class similarity and intra-class variance pose a challenge for the
methods because the negative examples may contain irrelevant cues for
recognizing a category so the algorithms may converge to a local-minima. We
present a novel method called Expert-Contrastive Learning (XCon) to help the
model to mine useful information from the images by first partitioning the
dataset into sub-datasets using k-means clustering and then performing
contrastive learning on each of the sub-datasets to learn fine-grained
discriminative features. Experiments on fine-grained datasets show a clear
improved performance over the previous best methods, indicating the
effectiveness of our method
Automatically Discovering and Learning New Visual Categories with Ranking Statistics
We tackle the problem of discovering novel classes in an image collection
given labelled examples of other classes. This setting is similar to
semi-supervised learning, but significantly harder because there are no
labelled examples for the new classes. The challenge, then, is to leverage the
information contained in the labelled images in order to learn a
general-purpose clustering model and use the latter to identify the new classes
in the unlabelled data. In this work we address this problem by combining three
ideas: (1) we suggest that the common approach of bootstrapping an image
representation using the labeled data only introduces an unwanted bias, and
that this can be avoided by using self-supervised learning to train the
representation from scratch on the union of labelled and unlabelled data; (2)
we use rank statistics to transfer the model's knowledge of the labelled
classes to the problem of clustering the unlabelled images; and, (3) we train
the data representation by optimizing a joint objective function on the
labelled and unlabelled subsets of the data, improving both the supervised
classification of the labelled data, and the clustering of the unlabelled data.
We evaluate our approach on standard classification benchmarks and outperform
current methods for novel category discovery by a significant margin.Comment: ICLR 2020, code: http://www.robots.ox.ac.uk/~vgg/research/auto_nove
Automatically Discovering Novel Visual Categories with Self-supervised Prototype Learning
This paper tackles the problem of novel category discovery (NCD), which aims
to discriminate unknown categories in large-scale image collections. The NCD
task is challenging due to the closeness to the real-world scenarios, where we
have only encountered some partial classes and images. Unlike other works on
the NCD, we leverage the prototypes to emphasize the importance of category
discrimination and alleviate the issue of missing annotations of novel classes.
Concretely, we propose a novel adaptive prototype learning method consisting of
two main stages: prototypical representation learning and prototypical
self-training. In the first stage, we obtain a robust feature extractor, which
could serve for all images with base and novel categories. This ability of
instance and category discrimination of the feature extractor is boosted by
self-supervised learning and adaptive prototypes. In the second stage, we
utilize the prototypes again to rectify offline pseudo labels and train a final
parametric classifier for category clustering. We conduct extensive experiments
on four benchmark datasets and demonstrate the effectiveness and robustness of
the proposed method with state-of-the-art performance.Comment: In Submissio
Demystifying Assumptions in Learning to Discover Novel Classes
In learning to discover novel classes (L2DNC), we are given labeled data from
seen classes and unlabeled data from unseen classes, and we train clustering
models for the unseen classes. However, the rigorous definition of L2DNC is
unexplored, which results in that its implicit assumptions are still unclear.
In this paper, we demystify assumptions behind L2DNC and find that high-level
semantic features should be shared among the seen and unseen classes. This
naturally motivates us to link L2DNC to meta-learning that has exactly the same
assumption as L2DNC. Based on this finding, L2DNC is not only theoretically
solvable, but can also be empirically solved by meta-learning algorithms after
slight modifications. This L2DNC methodology significantly reduces the amount
of unlabeled data needed for training and makes it more practical, as
demonstrated in experiments. The use of very limited data is also justified by
the application scenario of L2DNC: since it is unnatural to label only
seen-class data, L2DNC is sampling instead of labeling in causality. Therefore,
unseen-class data should be collected on the way of collecting seen-class data,
which is why they are novel and first need to be clustered
ClusterNet: A Perception-Based Clustering Model for Scattered Data
Visualizations for scattered data are used to make users understand certain
attributes of their data by solving different tasks, e.g. correlation
estimation, outlier detection, cluster separation. In this paper, we focus on
the later task, and develop a technique that is aligned to human perception,
that can be used to understand how human subjects perceive clusterings in
scattered data and possibly optimize for better understanding. Cluster
separation in scatterplots is a task that is typically tackled by widely used
clustering techniques, such as for instance k-means or DBSCAN. However, as
these algorithms are based on non-perceptual metrics, we can show in our
experiments, that their output do not reflect human cluster perception. We
propose a learning strategy which directly operates on scattered data. To learn
perceptual cluster separation on this data, we crowdsourced a large scale
dataset, consisting of 7,320 point-wise cluster affiliations for bivariate
data, which has been labeled by 384 human crowd workers. Based on this data, we
were able to train ClusterNet, a point-based deep learning model, trained to
reflect human perception of cluster separability. In order to train ClusterNet
on human annotated data, we use a PointNet++ architecture enabling inference on
point clouds directly. In this work, we provide details on how we collected our
dataset, report statistics of the resulting annotations, and investigate
perceptual agreement of cluster separation for real-world data. We further
report the training and evaluation protocol of ClusterNet and introduce a novel
metric, that measures the accuracy between a clustering technique and a group
of human annotators. Finally, we compare our approach against existing
state-of-the-art clustering techniques and can show, that ClusterNet is able to
generalize to unseen and out of scope data.Comment: Currently, this manuscript is under revision at TVC