Uncovering convolutional neural network decisions for diagnosing multiple sclerosis on conventional MRI using layer-wise relevance propagation

Machine learning-based imaging diagnostics has recently reached or even superseded the level of clinical experts in several clinical domains. However, classification decisions of a trained machine learning system are typically non-transparent, a major hindrance for clinical integration, error tracking or knowledge discovery. In this study, we present a transparent deep learning framework relying on convolutional neural networks (CNNs) and layer-wise relevance propagation (LRP) for diagnosing multiple sclerosis (MS). MS is commonly diagnosed utilizing a combination of clinical presentation and conventional magnetic resonance imaging (MRI), specifically the occurrence and presentation of white matter lesions in T2-weighted images. We hypothesized that using LRP in a naive predictive model would enable us to uncover relevant image features that a trained CNN uses for decision-making. Since imaging markers in MS are well-established this would enable us to validate the respective CNN model. First, we pre-trained a CNN on MRI data from the Alzheimer's Disease Neuroimaging Initiative (n = 921), afterwards specializing the CNN to discriminate between MS patients and healthy controls (n = 147). Using LRP, we then produced a heatmap for each subject in the holdout set depicting the voxel-wise relevance for a particular classification decision. The resulting CNN model resulted in a balanced accuracy of 87.04% and an area under the curve of 96.08% in a receiver operating characteristic curve. The subsequent LRP visualization revealed that the CNN model focuses indeed on individual lesions, but also incorporates additional information such as lesion location, non-lesional white matter or gray matter areas such as the thalamus, which are established conventional and advanced MRI markers in MS. We conclude that LRP and the proposed framework have the capability to make diagnostic decisions of..

Personalized Prediction of Future Lesion Activity and Treatment Effect in Multiple Sclerosis from Baseline MRI

Precision medicine for chronic diseases such as multiple sclerosis (MS) involves choosing a treatment which best balances efficacy and side effects/preferences for individual patients. Making this choice as early as possible is important, as delays in finding an effective therapy can lead to irreversible disability accrual. To this end, we present the first deep neural network model for individualized treatment decisions from baseline magnetic resonance imaging (MRI) (with clinical information if available) for MS patients. Our model (a) predicts future new and enlarging T2 weighted (NE-T2) lesion counts on follow-up MRI on multiple treatments and (b) estimates the conditional average treatment effect (CATE), as defined by the predicted future suppression of NE-T2 lesions, between different treatment options relative to placebo. Our model is validated on a proprietary federated dataset of 1817 multi-sequence MRIs acquired from MS patients during four multi-centre randomized clinical trials. Our framework achieves high average precision in the binarized regression of future NE-T2 lesions on five different treatments, identifies heterogeneous treatment effects, and provides a personalized treatment recommendation that accounts for treatment-associated risk (e.g. side effects, patient preference, administration difficulties).Comment: Accepted to MIDL 202

Vitamin D and Disease Severity in Multiple Sclerosis-Baseline Data From the Randomized Controlled Trial (EVIDIMS)

Objective: To investigate the associations between hypovitaminosis D and disease activity in a cohort of relapsing remitting multiple sclerosis (RRMS) and clinically isolated syndrome (CIS) patients. Methods: In 51 RRMS and 2 CIS patients on stable interferon-β-1b (IFN-β-1b) treatment recruited to the EVIDIMS study (Efficacy of Vitamin D Supplementation in Multiple Sclerosis (NCT01440062) baseline serum vitamin D levels were evaluated. Patients were dichotomized based on the definition of vitamin D deficiency which is reflected by a < 30 vs. ≥ 30 ng/ml level of 25-hydroxyvitamin D (25(OH)D). Possible associations between vitamin D deficiency and both clinical and MRI features of the disease were analyzed. Results: Median (25, 75% quartiles, Q) 25(OH)D level was 18 ng/ml (12, 24). Forty eight out of 53 (91%) patients had 25(OH)D levels < 30 ng/ml (p < 0.001). Patients with 25(OH)D ≥ 30 ng/ml had lower median (25, 75% Q) T2-weighted lesion counts [25 (24, 33)] compared to patients with 25(OH)D < 30 ng/ml [60 (36, 84), p = 0.03; adjusted for age, gender and disease duration: p < 0.001]. Expanded disability status scale (EDSS) score was negatively associated with serum 25(OH)D levels in a multiple linear regression, including age, sex, and disease duration (adjusted: p < 0.001). Interpretation: Most patients recruited in the EVIDIMS study were vitamin D deficient. Higher 25(OH)D levels were associated with reduced T2 weighted lesion count and lower EDSS scores

Exploring RCNN for the automated analysis of paramagnetic rim lesions in Multiple Sclerosis MRI

In multiple sclerosis, lesions with a peripheral paramagnetic rim is a negative prognostic imaging biomarker and represents a potential outcome measure in MRI-based clinical trials. Nowadays, the presence or absence of paramagnetic rims is determined through visual inspection by medical experts, which is tedious, time consuming and prone to observer variability. So far, few solutions to the automated classification of rims problem have been proposed. These studies present limitations that represent an obstacle to full automation of the rim analysis process and its large-scale application. Our goal is to implement and assess a fully automated algorithm capable of identifying rim lesions in MRI. In this work, we explore a Region-proposal CNN deep learning approach to solve the fully automated rim lesions classification problem that perform instance segmentation by object detection and have shown promising results in recent challenges, particularly in medical imaging. After different approaches focus on implifying the task, Mask RCNN with MobileNet v2 as backbone using attention gaussian filtering to the input images showed better performance than the rest with rates of 0.42 TPR and 0.61 FPR for the test set. However, the achieved results reveal the weaknesses of our approach and the difficulty of our classification problem

White matter hyperintensity and stroke lesion segmentation and differentiation using convolutional neural networks

The accurate assessment of White matter hyperintensities (WMH) burden is of crucial importance for epidemiological studies to determine association between WMHs, cognitive and clinical data. The manual delineation of WMHs is tedious, costly and time consuming. This is further complicated by the fact that other pathological features (i.e. stroke lesions) often also appear as hyperintense. Several automated methods aiming to tackle the challenges of WMH segmentation have been proposed, however cannot differentiate between WMH and strokes. Other methods, capable of distinguishing between different pathologies in brain MRI, are not designed with simultaneous WMH and stroke segmentation in mind. In this work we propose to use a convolutional neural network (CNN) that is able to segment hyperintensities and differentiate between WMHs and stroke lesions. Specifically, we aim to distinguish between WMH pathologies from those caused by stroke lesions due to either cortical, large or small subcortical infarcts. As far as we know, this is the first time such differentiation task has explicitly been proposed. The proposed fully convolutional CNN architecture, is comprised of an analysis path, that gradually learns low and high level features, followed by a synthesis path, that gradually combines and up-samples the low and high level features into a class likelihood semantic segmentation. Quantitatively, the proposed CNN architecture is shown to outperform other well established and state-of-the-art algorithms in terms of overlap with manual expert annotations. Clinically, the extracted WMH volumes were found to correlate better with the Fazekas visual rating score. Additionally, a comparison of the associations found between clinical risk-factors and the WMH volumes generated by the proposed method, were found to be in line with the associations found with the expert-annotated volumes

Boosting multiple sclerosis lesion segmentation through attention mechanism

Magnetic resonance imaging is a fundamental tool to reach a diagnosis of multiple sclerosis and monitoring its progression. Although several attempts have been made to segment multiple sclerosis lesions using artificial intelligence, fully automated analysis is not yet available. State-of-the-art methods rely on slight variations in segmentation architectures (e.g. U-Net, etc.). However, recent research has demonstrated how exploiting temporal-aware features and attention mechanisms can provide a significant boost to traditional architectures. This paper proposes a framework that exploits an augmented U-Net architecture with a convolutional long short-term memory layer and attention mechanism which is able to segment and quantify multiple sclerosis lesions detected in magnetic resonance images. Quantitative and qualitative evaluation on challenging examples demonstrated how the method outperforms previous state-of-the-art approaches, reporting an overall Dice score of 89% and also demonstrating robustness and generalization ability on never seen new test samples of a new dedicated under construction dataset

Detection of lesions in the optic nerve with magnetic resonance imaging using a 3D convolutional neural network

Deep learning; Multiple sclerosis, Optic nerveAprendizaje profundo; Esclerosis múltiple; Nervio ópticoAprenentatge profund; Esclerosi múltiple; Nervi òpticBackground Optic neuritis (ON) is one of the first manifestations of multiple sclerosis, a disabling disease with rising prevalence. Detecting optic nerve lesions could be a relevant diagnostic marker in patients with multiple sclerosis. Objectives We aim to create an automated, interpretable method for optic nerve lesion detection from MRI scans. Materials and Methods We present a 3D convolutional neural network (CNN) model that learns to detect optic nerve lesions based on T2-weighted fat-saturated MRI scans. We validated our system on two different datasets (N = 107 and 62) and interpreted the behaviour of the model using saliency maps. Results The model showed good performance (68.11% balanced accuracy) that generalizes to unseen data (64.11%). The developed network focuses its attention to the areas that correspond to lesions in the optic nerve. Conclusions The method shows robustness and, when using only a single imaging sequence, its performance is not far from diagnosis by trained radiologists with the same constraint. Given its speed and performance, the developed methodology could serve as a first step to develop methods that could be translated into a clinical setting.This project was developed as a part of Gerard Martí-Juan ECTRIMS Research Fellowship Program 2021–2022. This study was partially supported by the Projects (PI18/00823, PI19/00950), from the Fondo de Investigación Sanitaria (FIS), Instituto de Salud Carlos III