537 research outputs found
Initial Classifier Weights Replay for Memoryless Class Incremental Learning
Incremental Learning (IL) is useful when artificial systems need to deal with
streams of data and do not have access to all data at all times. The most
challenging setting requires a constant complexity of the deep model and an
incremental model update without access to a bounded memory of past data. Then,
the representations of past classes are strongly affected by catastrophic
forgetting. To mitigate its negative effect, an adapted fine tuning which
includes knowledge distillation is usually deployed. We propose a different
approach based on a vanilla fine tuning backbone. It leverages initial
classifier weights which provide a strong representation of past classes
because they are trained with all class data. However, the magnitude of
classifiers learned in different states varies and normalization is needed for
a fair handling of all classes. Normalization is performed by standardizing the
initial classifier weights, which are assumed to be normally distributed. In
addition, a calibration of prediction scores is done by using state level
statistics to further improve classification fairness. We conduct a thorough
evaluation with four public datasets in a memoryless incremental learning
setting. Results show that our method outperforms existing techniques by a
large margin for large-scale datasets.Comment: Accepted in BMVC202
Consistency Regularization with High-dimensional Non-adversarial Source-guided Perturbation for Unsupervised Domain Adaptation in Segmentation
Unsupervised domain adaptation for semantic segmentation has been intensively
studied due to the low cost of the pixel-level annotation for synthetic data.
The most common approaches try to generate images or features mimicking the
distribution in the target domain while preserving the semantic contents in the
source domain so that a model can be trained with annotations from the latter.
However, such methods highly rely on an image translator or feature extractor
trained in an elaborated mechanism including adversarial training, which brings
in extra complexity and instability in the adaptation process. Furthermore,
these methods mainly focus on taking advantage of the labeled source dataset,
leaving the unlabeled target dataset not fully utilized. In this paper, we
propose a bidirectional style-induced domain adaptation method, called BiSIDA,
that employs consistency regularization to efficiently exploit information from
the unlabeled target domain dataset, requiring only a simple neural style
transfer model. BiSIDA aligns domains by not only transferring source images
into the style of target images but also transferring target images into the
style of source images to perform high-dimensional perturbation on the
unlabeled target images, which is crucial to the success in applying
consistency regularization in segmentation tasks. Extensive experiments show
that our BiSIDA achieves new state-of-the-art on two commonly-used
synthetic-to-real domain adaptation benchmarks: GTA5-to-CityScapes and
SYNTHIA-to-CityScapes
- …