Relating multi-sequence longitudinal intensity profiles and clinical covariates in new multiple sclerosis lesions

Structural magnetic resonance imaging (MRI) can be used to detect lesions in the brains of multiple sclerosis (MS) patients. The formation of these lesions is a complex process involving inflammation, tissue damage, and tissue repair, all of which are visible on MRI. Here we characterize the lesion formation process on longitudinal, multi-sequence structural MRI from 34 MS patients and relate the longitudinal changes we observe within lesions to therapeutic interventions. In this article, we first outline a pipeline to extract voxel level, multi-sequence longitudinal profiles from four MRI sequences within lesion tissue. We then propose two models to relate clinical covariates to the longitudinal profiles. The first model is a principal component analysis (PCA) regression model, which collapses the information from all four profiles into a scalar value. We find that the score on the first PC identifies areas of slow, long-term intensity changes within the lesion at a voxel level, as validated by two experienced clinicians, a neuroradiologist and a neurologist. On a quality scale of 1 to 4 (4 being the highest) the neuroradiologist gave the score on the first PC a median rating of 4 (95% CI: [4,4]), and the neurologist gave it a median rating of 3 (95% CI: [3,3]). In the PCA regression model, we find that treatment with disease modifying therapies (p-value < 0.01), steroids (p-value < 0.01), and being closer to the boundary of abnormal signal intensity (p-value < 0.01) are associated with a return of a voxel to intensity values closer to that of normal-appearing tissue. The second model is a function-on-scalar regression, which allows for assessment of the individual time points at which the covariates are associated with the profiles. In the function-on-scalar regression both age and distance to the boundary were found to have a statistically significant association with the profiles

Multi-branch Convolutional Neural Network for Multiple Sclerosis Lesion Segmentation

In this paper, we present an automated approach for segmenting multiple sclerosis (MS) lesions from multi-modal brain magnetic resonance images. Our method is based on a deep end-to-end 2D convolutional neural network (CNN) for slice-based segmentation of 3D volumetric data. The proposed CNN includes a multi-branch downsampling path, which enables the network to encode information from multiple modalities separately. Multi-scale feature fusion blocks are proposed to combine feature maps from different modalities at different stages of the network. Then, multi-scale feature upsampling blocks are introduced to upsize combined feature maps to leverage information from lesion shape and location. We trained and tested the proposed model using orthogonal plane orientations of each 3D modality to exploit the contextual information in all directions. The proposed pipeline is evaluated on two different datasets: a private dataset including 37 MS patients and a publicly available dataset known as the ISBI 2015 longitudinal MS lesion segmentation challenge dataset, consisting of 14 MS patients. Considering the ISBI challenge, at the time of submission, our method was amongst the top performing solutions. On the private dataset, using the same array of performance metrics as in the ISBI challenge, the proposed approach shows high improvements in MS lesion segmentation compared with other publicly available tools.Comment: This paper has been accepted for publication in NeuroImag

Simultaneous lesion and neuroanatomy segmentation in Multiple Sclerosis using deep neural networks

Segmentation of both white matter lesions and deep grey matter structures is an important task in the quantification of magnetic resonance imaging in multiple sclerosis. Typically these tasks are performed separately: in this paper we present a single segmentation solution based on convolutional neural networks (CNNs) for providing fast, reliable segmentations of multimodal magnetic resonance images into lesion classes and normal-appearing grey- and white-matter structures. We show substantial, statistically significant improvements in both Dice coefficient and in lesion-wise specificity and sensitivity, compared to previous approaches, and agreement with individual human raters in the range of human inter-rater variability. The method is trained on data gathered from a single centre: nonetheless, it performs well on data from centres, scanners and field-strengths not represented in the training dataset. A retrospective study found that the classifier successfully identified lesions missed by the human raters. Lesion labels were provided by human raters, while weak labels for other brain structures (including CSF, cortical grey matter, cortical white matter, cerebellum, amygdala, hippocampus, subcortical GM structures and choroid plexus) were provided by Freesurfer 5.3. The segmentations of these structures compared well, not only with Freesurfer 5.3, but also with FSL-First and Freesurfer 6.0

Multi-contrast, isotropic, single-slab 3D MR imaging in multiple sclerosis

To describe signal and contrast properties of an isotropic, single-slab 3D dataset [double inversion- recovery (DIR), fluid-attenuated inversion recovery (FLAIR), T2, and T1-weighted magnetization prepared rapid acquisition gradient-echo (MPRAGE)] and to evaluate its performance in detecting multiple sclerosis (MS) brain lesions compared to 2D T2-weighted spin-echo (T2SE). All single-slab 3D sequences and 2DT2SE were acquired in 16 MS patients and 9 age-matched healthy controls. Lesions were scored independently by two raters and characterized anatomically. Two-tailed Bonferroni-corrected Student’s t-tests were used to detect differences in lesion detection between the various sequences per anatomical area after logtransformation. In general, signal and contrast properties of the 3D sequences enabled improved detection of MS brain lesions compared to 2DT2SE. Specifically, 3D-DIR showed the highest detection of intracortical and mixed WM-GM lesions, whereas 3D-FLAIR showed the highest total number of WM lesions. Both 3D-DIR and 3D-FLAIR showed the highest number of infratentorial lesions. 3DT2 and 3D-MPRAGE did not improve lesion detection compared to 2DT2SE. Multi-contrast, isotropic, single-slab 3D MRI allowed an improved detection of both GM and WM lesions compared to 2D-T2SE. A selection of single-slab 3D contrasts, for example, 3D-FLAIR and 3D-DIR, could replace 2D sequences in the radiological practice

Uncovering convolutional neural network decisions for diagnosing multiple sclerosis on conventional MRI using layer-wise relevance propagation

Machine learning-based imaging diagnostics has recently reached or even superseded the level of clinical experts in several clinical domains. However, classification decisions of a trained machine learning system are typically non-transparent, a major hindrance for clinical integration, error tracking or knowledge discovery. In this study, we present a transparent deep learning framework relying on convolutional neural networks (CNNs) and layer-wise relevance propagation (LRP) for diagnosing multiple sclerosis (MS). MS is commonly diagnosed utilizing a combination of clinical presentation and conventional magnetic resonance imaging (MRI), specifically the occurrence and presentation of white matter lesions in T2-weighted images. We hypothesized that using LRP in a naive predictive model would enable us to uncover relevant image features that a trained CNN uses for decision-making. Since imaging markers in MS are well-established this would enable us to validate the respective CNN model. First, we pre-trained a CNN on MRI data from the Alzheimer's Disease Neuroimaging Initiative (n = 921), afterwards specializing the CNN to discriminate between MS patients and healthy controls (n = 147). Using LRP, we then produced a heatmap for each subject in the holdout set depicting the voxel-wise relevance for a particular classification decision. The resulting CNN model resulted in a balanced accuracy of 87.04% and an area under the curve of 96.08% in a receiver operating characteristic curve. The subsequent LRP visualization revealed that the CNN model focuses indeed on individual lesions, but also incorporates additional information such as lesion location, non-lesional white matter or gray matter areas such as the thalamus, which are established conventional and advanced MRI markers in MS. We conclude that LRP and the proposed framework have the capability to make diagnostic decisions of..

Improving longitudinal spinal cord atrophy measurements for clinical trials in multiple sclerosis by using the generalised boundary shift integral (GBSI)

Spinal cord atrophy is a common and clinically relevant feature of multiple sclerosis (MS), and can be used to monitor disease progression and as an outcome measure in clinical trials. Spinal cord atrophy is conventionally estimated with segmentation-based methods (e.g., cross-sectional spinal cord area (CSA)), where spinal cord change is calculated indirectly by numerical difference between timepoints. In this thesis, I validated the generalised boundary shift integral (GBSI), as the first registration-based method for longitudinal spinal cord atrophy measurement. The GBSI registers the baseline and follow-up spinal cord scans in a common half-way space, to directly determine atrophy on the cord edges. First, on a test dataset (9 MS patients and 9 controls), I have found that GBSI presented with lower random measurement error, than CSA, reflected by lower standard deviation, coefficient of variation and median absolute deviation. Then, on multi-centre, multi-manufacturer, and multi–field‐strength scans (282 MS patients and 82 controls), I confirmed that GBSI provided lower measurement variability in all MS subtypes and controls, than CSA, resulting into better separation between MS patients and controls, improved statistical power, and reduced sample size estimates. Finally, on a phase 2 clinical trial (220 primary-progressive MS patients), I demonstrated that spinal cord atrophy measurements on GBSI could be obtained from brain scans, considering their quality and association with corresponding spinal cord MRI-derived measurements. Not least, 1-year spinal cord atrophy measurements on GBSI, but not CSA, were associated with upper and lower limb motor function. In conclusion, spinal cord atrophy on the GBSI had higher measurement precision and stronger clinical correlates, than the segmentation method, and could be derived from high-quality brain acquisitions. Longitudinal spinal cord atrophy on GBSI could become a gold standard for clinical trials including spinal cord atrophy as an outcome measure, but should remain a secondary outcome measure, until further advancements increase the ease of acquisition and processing

TEXTURAL CLASSIFICATION OF MULTIPLE SCLEROSISLESIONS IN MULTIMODAL MRI VOLUMES

Background and objectives:Multiple Sclerosis is a common relapsing demyelinating diseasecausing the significant degradation of cognitive and motor skills and contributes towards areduced life expectancy of 5 to 10 years. The identification of Multiple Sclerosis Lesionsat early stages of a patient’s life can play a significant role in the diagnosis, treatment andprognosis for that individual. In recent years the process of disease detection has been aidedthrough the implementation of radiomic pipelines for texture extraction and classificationutilising Computer Vision and Machine Learning techniques. Eight Multiple Sclerosis Patient datasets have been supplied, each containing one standardclinical T2 MRI sequence and four diffusion-weighted sequences (T2, FA, ADC, AD, RD).This work proposes a Multimodal Multiple Sclerosis Lesion segmentation methodology util-ising supervised texture analysis, feature selection and classification. Three Machine Learningmodels were applied to Multimodal MRI data and tested using unseen patient datasets to eval-uate the classification performance of various extracted features, feature selection algorithmsand classifiers to MRI volumes uncommonly applied to MS Lesion detection. Method: First Order Statistics, Haralick Texture Features, Gray-Level Run-Lengths, His-togram of Oriented Gradients and Local Binary Patterns were extracted from MRI volumeswhich were minimally pre-processed using a skull stripping and background removal algorithm.mRMR and LASSO feature selection algorithms were applied to identify a subset of rankingsfor use in Machine Learning using Support Vector Machine, Random Forests and ExtremeLearning Machine classification. Results: ELM achieved a top slice classification accuracy of 85% while SVM achieved 79%and RF 78%. It was found that combining information from all MRI sequences increased theclassification performance when analysing unseen T2 scans in almost all cases. LASSO andmRMR feature selection methods failed to increase accuracy, and the highest-scoring groupof features were Haralick Texture Features, derived from Grey-Level Co-occurrence matrices