Skin cancer is a major concern to public health, accounting for one-third of
the reported cancers. If not detected early, the cancer has the potential for
severe consequences. Recognizing the critical need for effective skin cancer
classification, we address the limitations of existing models, which are often
too large to deploy in areas with limited computational resources. In response,
we present a knowledge distillation based approach for creating a lightweight
yet high-performing classifier. The proposed solution involves fusing three
models, namely ResNet152V2, ConvNeXtBase, and ViT Base, to create an effective
teacher model. The teacher model is then employed to guide a lightweight
student model of size 2.03 MB. This student model is further compressed to
469.77 KB using 16-bit quantization, enabling smooth incorporation into edge
devices. With six-stage image preprocessing, data augmentation, and a rigorous
ablation study, the model achieves an impressive accuracy of 98.75% on the
HAM10000 dataset and 98.94% on the Kaggle dataset in classifying benign and
malignant skin cancers. With its high accuracy and compact size, our model
appears to be a potential choice for accurate skin cancer classification,
particularly in resource-constrained settings
