2 research outputs found
Prompt-based Tuning of Transformer Models for Multi-Center Medical Image Segmentation
Medical image segmentation is a vital healthcare endeavor requiring precise
and efficient models for appropriate diagnosis and treatment. Vision
transformer-based segmentation models have shown great performance in
accomplishing this task. However, to build a powerful backbone, the
self-attention block of ViT requires large-scale pre-training data. The present
method of modifying pre-trained models entails updating all or some of the
backbone parameters. This paper proposes a novel fine-tuning strategy for
adapting a pretrained transformer-based segmentation model on data from a new
medical center. This method introduces a small number of learnable parameters,
termed prompts, into the input space (less than 1\% of model parameters) while
keeping the rest of the model parameters frozen. Extensive studies employing
data from new unseen medical centers show that prompts-based fine-tuning of
medical segmentation models provides excellent performance on the new center
data with a negligible drop on the old centers. Additionally, our strategy
delivers great accuracy with minimum re-training on new center data,
significantly decreasing the computational and time costs of fine-tuning
pre-trained models