1 research outputs found
Multi-modal Fusion for Diabetes Mellitus and Impaired Glucose Regulation Detection
Effective and accurate diagnosis of Diabetes Mellitus (DM), as well as its
early stage Impaired Glucose Regulation (IGR), has attracted much attention
recently. Traditional Chinese Medicine (TCM) [3], [5] etc. has proved that
tongue, face and sublingual diagnosis as a noninvasive method is a reasonable
way for disease detection. However, most previous works only focus on a single
modality (tongue, face or sublingual) for diagnosis, although different
modalities may provide complementary information for the diagnosis of DM and
IGR. In this paper, we propose a novel multi-modal classification method to
discriminate between DM (or IGR) and healthy controls. Specially, the tongue,
facial and sublingual images are first collected by using a non-invasive
capture device. The color, texture and geometry features of these three types
of images are then extracted, respectively. Finally, our so-called multi-modal
similar and specific learning (MMSSL) approach is proposed to combine features
of tongue, face and sublingual, which not only exploits the correlation but
also extracts individual components among them. Experimental results on a
dataset consisting of 192 Healthy, 198 DM and 114 IGR samples (all samples were
obtained from Guangdong Provincial Hospital of Traditional Chinese Medicine)
substantiate the effectiveness and superiority of our proposed method for the
diagnosis of DM and IGR, compared to the case of using a single modality.Comment: 9 pages, 8 figures, 30 conferenc