CS-REG-NET: Termal ve optik görüntülerde çapraz-spektral çakıştırma için görsel-durum uzayı tabanlı özgözetimli ögrenmeli mimari

Abstract

Modern deep models for multispectral image matching typically rely on large, supervised datasets, which can be prohibitively expensive. To overcome this challenge, we introduce CS-REG-NET, a self-supervised, detector-based framework that requires no external labels. Instead, it uses RIFT2 detector to generate pseudo-ground-truth keypoints. A VMamba encoder, pre-trained on a segmentation task, processes image pairs, while two output heads learn feature heatmaps and descriptors. CSREG-NET significantly outperforms existing methods, delivering superior keypoint detection and homography estimation. This real-time framework thus provides a robust, extensible solution for multispectral image matching.TÜBİTA