Currently intelligent diagnosis systems lack the ability of continually
learning to diagnose new diseases once deployed, under the condition of
preserving old disease knowledge. In particular, updating an intelligent
diagnosis system with training data of new diseases would cause catastrophic
forgetting of old disease knowledge. To address the catastrophic forgetting
issue, a novel adapter-based strategy is proposed to help effectively learn a
set of new diseases at each round (or task) of continual learning, without
changing the shared feature extractor. The learnable lightweight task-specific
adapter(s) can be flexibly designed (e.g., two convolutional layers) and then
added to the pretrained and fixed feature extractor. Together with a specially
designed task-specific head which absorbs all previously learned old diseases
as a single 'out-of-distribution' category, task-specific adapter(s) can help
the pretrained feature extractor more effectively extract discriminative
features between diseases. In addition, a simple yet effective fine-tuning is
applied to collaboratively fine-tune multiple task-specific heads such that
outputs from different heads are comparable and consequently the appropriate
classifier head can be more accurately selected during model inference.
Extensive empirical evaluations on three image datasets demonstrate the
superior performance of the proposed method in continual learning of new
diseases. The source code will be released publicly.Comment: 10 page