Raman spectroscopy (RS) has been widely used for disease diagnosis, e.g.,
cardiovascular disease (CVD), owing to its efficiency and component-specific
testing capabilities. A series of popular deep learning methods have recently
been introduced to learn nuance features from RS for binary classifications and
achieved outstanding performance than conventional machine learning methods.
However, these existing deep learning methods still confront some challenges in
classifying subtypes of CVD. For example, the nuance between subtypes is quite
hard to capture and represent by intelligent models due to the chillingly
similar shape of RS sequences. Moreover, medical history information is an
essential resource for distinguishing subtypes, but they are underutilized. In
light of this, we propose a multi-modality multi-scale model called M3S, which
is a novel deep learning method with two core modules to address these issues.
First, we convert RS data to various resolution images by the Gramian angular
field (GAF) to enlarge nuance, and a two-branch structure is leveraged to get
embeddings for distinction in the multi-scale feature extraction module.
Second, a probability matrix and a weight matrix are used to enhance the
classification capacity by combining the RS and medical history data in the
multi-modality data fusion module. We perform extensive evaluations of M3S and
found its outstanding performance on our in-house dataset, with accuracy,
precision, recall, specificity, and F1 score of 0.9330, 0.9379, 0.9291, 0.9752,
and 0.9334, respectively. These results demonstrate that the M3S has high
performance and robustness compared with popular methods in diagnosing CVD
subtypes