Although a number of studies are devoted to novel category discovery, most of
them assume a static setting where both labeled and unlabeled data are given at
once for finding new categories. In this work, we focus on the application
scenarios where unlabeled data are continuously fed into the category discovery
system. We refer to it as the {\bf Continuous Category Discovery} ({\bf CCD})
problem, which is significantly more challenging than the static setting. A
common challenge faced by novel category discovery is that different sets of
features are needed for classification and category discovery: class
discriminative features are preferred for classification, while rich and
diverse features are more suitable for new category mining. This challenge
becomes more severe for dynamic setting as the system is asked to deliver good
performance for known classes over time, and at the same time continuously
discover new classes from unlabeled data. To address this challenge, we develop
a framework of {\bf Grow and Merge} ({\bf GM}) that works by alternating
between a growing phase and a merging phase: in the growing phase, it increases
the diversity of features through a continuous self-supervised learning for
effective category mining, and in the merging phase, it merges the grown model
with a static one to ensure satisfying performance for known classes. Our
extensive studies verify that the proposed GM framework is significantly more
effective than the state-of-the-art approaches for continuous category
discovery.Comment: This paper has already been accepted by 36th Conference on Neural
Information Processing Systems (NeurIPS 2022