Uncovering convolutional neural network decisions for diagnosing multiple sclerosis on conventional MRI using layer-wise relevance propagation

Machine learning-based imaging diagnostics has recently reached or even superseded the level of clinical experts in several clinical domains. However, classification decisions of a trained machine learning system are typically non-transparent, a major hindrance for clinical integration, error tracking or knowledge discovery. In this study, we present a transparent deep learning framework relying on convolutional neural networks (CNNs) and layer-wise relevance propagation (LRP) for diagnosing multiple sclerosis (MS). MS is commonly diagnosed utilizing a combination of clinical presentation and conventional magnetic resonance imaging (MRI), specifically the occurrence and presentation of white matter lesions in T2-weighted images. We hypothesized that using LRP in a naive predictive model would enable us to uncover relevant image features that a trained CNN uses for decision-making. Since imaging markers in MS are well-established this would enable us to validate the respective CNN model. First, we pre-trained a CNN on MRI data from the Alzheimer's Disease Neuroimaging Initiative (n = 921), afterwards specializing the CNN to discriminate between MS patients and healthy controls (n = 147). Using LRP, we then produced a heatmap for each subject in the holdout set depicting the voxel-wise relevance for a particular classification decision. The resulting CNN model resulted in a balanced accuracy of 87.04% and an area under the curve of 96.08% in a receiver operating characteristic curve. The subsequent LRP visualization revealed that the CNN model focuses indeed on individual lesions, but also incorporates additional information such as lesion location, non-lesional white matter or gray matter areas such as the thalamus, which are established conventional and advanced MRI markers in MS. We conclude that LRP and the proposed framework have the capability to make diagnostic decisions of..

Applications of Deep Learning Techniques for Automated Multiple Sclerosis Detection Using Magnetic Resonance Imaging: A Review

Multiple Sclerosis (MS) is a type of brain disease which causes visual, sensory, and motor problems for people with a detrimental effect on the functioning of the nervous system. In order to diagnose MS, multiple screening methods have been proposed so far; among them, magnetic resonance imaging (MRI) has received considerable attention among physicians. MRI modalities provide physicians with fundamental information about the structure and function of the brain, which is crucial for the rapid diagnosis of MS lesions. Diagnosing MS using MRI is time-consuming, tedious, and prone to manual errors. Research on the implementation of computer aided diagnosis system (CADS) based on artificial intelligence (AI) to diagnose MS involves conventional machine learning and deep learning (DL) methods. In conventional machine learning, feature extraction, feature selection, and classification steps are carried out by using trial and error; on the contrary, these steps in DL are based on deep layers whose values are automatically learn. In this paper, a complete review of automated MS diagnosis methods performed using DL techniques with MRI neuroimaging modalities is provided. Initially, the steps involved in various CADS proposed using MRI modalities and DL techniques for MS diagnosis are investigated. The important preprocessing techniques employed in various works are analyzed. Most of the published papers on MS diagnosis using MRI modalities and DL are presented. The most significant challenges facing and future direction of automated diagnosis of MS using MRI modalities and DL techniques are also provided

The macular retinal ganglion cell layer as a biomarker for diagnosis and prognosis in multiple sclerosis: A deep learning approach

AbstractPurposeThe macular ganglion cell layer (mGCL) is a strong potential biomarker of axonal degeneration in multiple sclerosis (MS). For this reason, this study aims to develop a computer‐aided method to facilitate diagnosis and prognosis in MS.MethodsThis paper combines a cross‐sectional study of 72 MS patients and 30 healthy control subjects for diagnosis and a 10‐year longitudinal study of the same MS patients for the prediction of disability progression, during which the mGCL was measured using optical coherence tomography (OCT). Deep neural networks were used as an automatic classifier.ResultsFor MS diagnosis, greatest accuracy (90.3%) was achieved using 17 features as inputs. The neural network architecture comprised the input layer, two hidden layers and the output layer with softmax activation. For the prediction of disability progression 8 years later, accuracy of 81.9% was achieved with a neural network comprising two hidden layers and 400 epochs.ConclusionWe present evidence that by applying deep learning techniques to clinical and mGCL thickness data it is possible to identify MS and predict the course of the disease. This approach potentially constitutes a non‐invasive, low‐cost, easy‐to‐implement and effective method

Considering patient clinical history impacts performance of machine learning models in predicting course of multiple sclerosis

Multiple Sclerosis (MS) progresses at an unpredictable rate, but predictions on the disease course in each patient would be extremely useful to tailor therapy to the individual needs. We explore different machine learning (ML) approaches to predict whether a patient will shift from the initial Relapsing-Remitting (RR) to the Secondary Progressive (SP) form of the disease, using only “real world” data available in clinical routine. The clinical records of 1624 outpatients (207 in the SP phase) attending the MS service of Sant'Andrea hospital, Rome, Italy, were used. Predictions at 180, 360 or 720 days from the last visit were obtained considering either the data of the last available visit (Visit-Oriented setting), comparing four classical ML methods (Random Forest, Support Vector Machine, K-Nearest Neighbours and AdaBoost) or the whole clinical history of each patient (History-Oriented setting), using a Recurrent Neural Network model, specifically designed for historical data. Missing values were handled by removing either all clinical records presenting at least one missing parameter (Feature-saving approach) or the 3 clinical parameters which contained missing values (Record-saving approach). The performances of the classifiers were rated using common indicators, such as Recall (or Sensitivity) and Precision (or Positive predictive value). In the visit-oriented setting, the Record-saving approach yielded Recall values from 70% to 100%, but low Precision (5% to 10%), which however increased to 50% when considering only predictions for which the model returned a probability above a given “confidence threshold”. For the History-oriented setting, both indicators increased as prediction time lengthened, reaching values of 67% (Recall) and 42% (Precision) at 720 days. We show how “real world” data can be effectively used to forecast the evolution of MS, leading to high Recall values and propose innovative approaches to improve Precision towards clinically useful values

Uncovering convolutional neural network decisions for diagnosing multiple sclerosis on conventional MRI using layer-wise relevance propagation

Machine learning-based imaging diagnostics has recently reached or even surpassed the level of clinical experts in several clinical domains. However, classification decisions of a trained machine learning system are typically non-transparent, a major hindrance for clinical integration, error tracking or knowledge discovery. In this study, we present a transparent deep learning framework relying on 3D convolutional neural networks (CNNs) and layer-wise relevance propagation (LRP) for diagnosing multiple sclerosis (MS), the most widespread autoimmune neuroinflammatory disease. MS is commonly diagnosed utilizing a combination of clinical presentation and conventional magnetic resonance imaging (MRI), specifically the occurrence and presentation of white matter lesions in T2-weighted images. We hypothesized that using LRP in a naive predictive model would enable us to uncover relevant image features that a trained CNN uses for decision-making. Since imaging markers in MS are well-established this would enable us to validate the respective CNN model. First, we pre-trained a CNN on MRI data from the Alzheimer's Disease Neuroimaging Initiative (n = 921), afterwards specializing the CNN to discriminate between MS patients (n = 76) and healthy controls (n = 71). Using LRP, we then produced a heatmap for each subject in the holdout set depicting the voxel-wise relevance for a particular classification decision. The resulting CNN model resulted in a balanced accuracy of 87.04% and an area under the curve of 96.08% in a receiver operating characteristic curve. The subsequent LRP visualization revealed that the CNN model focuses indeed on individual lesions, but also incorporates additional information such as lesion location, non-lesional white matter or gray matter areas such as the thalamus, which are established conventional and advanced MRI markers in MS. We conclude that LRP and the proposed framework have the capability to make diagnostic decisions of CNN models transparent, which could serve to justify classification decisions for clinical review, verify diagnosis-relevant features and potentially gather new disease knowledge

Supervised machine learning in multiple sclerosis: applications to clinically isolated syndromes

Multiple sclerosis (MS) is an inflammatory, demyelinating disease that can cause various neurological symptoms. The first episode of this disease is called a clinically isolated syndrome (CIS) and leads to the diagnosis of MS in the majority of patients in the long-term. Fast conversion from CIS to MS is associated with higher disability and more severe disease progression so that it is of high clinical interest to identify risk patients that will convert to MS within a short time. Several risk factors for conversion have been identified but they can only be applied on cohort levels. In this thesis we provide an overview of supervised machine learning approaches that can be used to distinguish individual CIS-stable patients from those who will experience a second attack within one to five years and consequently will be diagnosed with clinically definite MS. This classification is based on information available at baseline derived from routine MRI scans and complemented by clinical information such as lesion masks, age, gender, disability and CIS type of onset. We introduce the classification landscape, an overview of supervised classification studies with respect to their method and task complexity, and show that our experiments cover a large range of feature complexities in this landscape for the rather complex task of outcome prediction in CIS patients. We show that low-level voxel-based information such as tissue density of grey and white matter are not informative and lead to inconclusive results, whereas the introduction of high-level features such as lesion load, age, gender or disability improves accuracies to 71.4 % and 68 % at one- and three-year follow-up respectively in a single-centre data set. Finally, we propose a recursive feature elimination method that is able to identify specific regions that are relevant with respect to disease progression in MS and achieves accuracies of 73.9 % and 74.3 % at one- and three-year follow-up respectively even in a multi-centre setting

Brain imaging biomarkers in Multiple Sclerosis

Background: Iron rim lesions (IRLs), white matter lesions (WMLs) accumulation and linear brain atrophy measurements have been suggested to be important imaging biomarkers in multiple sclerosis (MS). The extent to which these markers are related to MS diagnosis and predict disease prognosis remains unclear. Furthermore, research Magnetic Resonance Imaging (MRI) findings need validation in clinical settings before they can be incorporated into clinical practice. Methods: I conducted two reviews one was a mapping review on IRLs and the other was a meta-analysis on WMLs in MS. I then tested the diagnostic and prognostic usefulness of the IRL in two studies: (1) a large, cross-sectional, multi-centre study of patients with MS and mimicking disorders using 3T MRI, (2) a retrospective single-centre study of patients with first presentation of a clinically isolated syndrome (CIS) or at the early stage of the disease using 7T MRI. I also explored the utility of routine, non-standardised MRI scans measuring WMLs number, volume and linear measures of atrophy at the early stage of the disease and examined their role in predicting long-term disability. Results: The IRLs achieved high specificity (up to 99%) in diagnosing MS compared to MS-mimics but low sensitivity of 24%. All patients with IRLs showing a central vein sign (CVS) had MS or CIS, giving a diagnostic specificity of 100% but equally low sensitivity of 21%. Moreover, the presence of IRLs was also a predictor of long-term disability, especially in patients with ≥4 IRLs. IRLs had a greater impact on disability compared to the WMLs number and volume. Linear brain atrophy of Inter-Caudate Distance (ICD) and Third Ventricle Width (TVW) had a significant impact in predicting disability after 10 years. Conclusions: The perilesional IRLs may reduce diagnostic uncertainty in MS by being a highly specific imaging diagnostic biomarker, especially when used in conjunction with the CVS. Also, the presence and number of IRLs hold prognostic value for long-term physical disability in MS. Simple and reliable assessment of brain atrophy remains challenging in clinical practice

Translation of quantitative MRI analysis tools for clinical neuroradiology application

Quantification of imaging features can assist radiologists by reducing subjectivity, aiding detection of subtle pathology, and increasing reporting consistency. Translation of quantitative image analysis techniques to clinical use is currently uncommon and challenging. This thesis explores translation of quantitative imaging support tools for clinical neuroradiology use. I have proposed a translational framework for development of quantitative imaging tools, using dementia as an exemplar application. This framework emphasises the importance of clinical validation, which is not currently prioritised. Aspects of the framework were then applied to four disease areas: hippocampal sclerosis (HS) as a cause of epilepsy; dementia; multiple sclerosis (MS) and gliomas. A clinical validation study for an HS quantitative report showed that when image interpreters used the report, they were more accurate and confident in their assessments, particularly for challenging bilateral cases. A similar clinical validation study for a dementia reporting tool found improved sensitivity for all image interpreters and increased assessment accuracy for consultant radiologists. These studies indicated benefits from quantitative reports that contextualise a patient’s results with appropriate normative reference data. For MS, I addressed a technical translational challenge by applying lesion and brain quantification tools to standard clinical image acquisitions which do not include a conventional T1-weighted sequence. Results were consistent with those from conventional sequence inputs and therefore I pursued this concept to establish a clinically applicable normative reference dataset for development of a quantitative reporting tool for clinical use. I focused on current radiology reporting of gliomas to establish which features are commonly missed and may be important for clinical management decisions. This informs both the potential utility of a quantitative report for gliomas and its design and content. I have identified numerous translational challenges for quantitative reporting and explored aspects of how to address these for several applications across clinical neuroradiology

Brain imaging biomarkers in Multiple Sclerosis

Background: Iron rim lesions (IRLs), white matter lesions (WMLs) accumulation and linear brain atrophy measurements have been suggested to be important imaging biomarkers in multiple sclerosis (MS). The extent to which these markers are related to MS diagnosis and predict disease prognosis remains unclear. Furthermore, research Magnetic Resonance Imaging (MRI) findings need validation in clinical settings before they can be incorporated into clinical practice. Methods: I conducted two reviews one was a mapping review on IRLs and the other was a meta-analysis on WMLs in MS. I then tested the diagnostic and prognostic usefulness of the IRL in two studies: (1) a large, cross-sectional, multi-centre study of patients with MS and mimicking disorders using 3T MRI, (2) a retrospective single-centre study of patients with first presentation of a clinically isolated syndrome (CIS) or at the early stage of the disease using 7T MRI. I also explored the utility of routine, non-standardised MRI scans measuring WMLs number, volume and linear measures of atrophy at the early stage of the disease and examined their role in predicting long-term disability. Results: The IRLs achieved high specificity (up to 99%) in diagnosing MS compared to MS-mimics but low sensitivity of 24%. All patients with IRLs showing a central vein sign (CVS) had MS or CIS, giving a diagnostic specificity of 100% but equally low sensitivity of 21%. Moreover, the presence of IRLs was also a predictor of long-term disability, especially in patients with ≥4 IRLs. IRLs had a greater impact on disability compared to the WMLs number and volume. Linear brain atrophy of Inter-Caudate Distance (ICD) and Third Ventricle Width (TVW) had a significant impact in predicting disability after 10 years. Conclusions: The perilesional IRLs may reduce diagnostic uncertainty in MS by being a highly specific imaging diagnostic biomarker, especially when used in conjunction with the CVS. Also, the presence and number of IRLs hold prognostic value for long-term physical disability in MS. Simple and reliable assessment of brain atrophy remains challenging in clinical practice

Personalized Diagnosis and Therapy for Multiple Sclerosis

We all agree that people with MS need to be cared in a profoundly personalized way. The care of the patient with MS is still based on the presence of relapses, so their successful diagnosis and treatment is fundamental and will condition the therapeutic strategies to follow with the patient. The treatment strategies are a highly controversial topic of debate that is increasingly supported by robust objective biological markers of response and that also increasingly take into account the dynamics and predictors of cognitive impairment along the disease course, which includes the adoption of new trends in the field of machine learning techniques. However, we all know that patient care goes beyond being treated with drugs and we cannot overlook reminding patients of the importance of their lifestyle behaviors that vary according to the MS phenotype, in order to improve their quality of life. Teleconsultation is a new care strategy proved to be feasible and well-received by patients with MS that will undoubtedly become reinforced because it will allow a closer follow-up of the patient without the need for displacement