Uncovering convolutional neural network decisions for diagnosing multiple sclerosis on conventional MRI using layer-wise relevance propagation

Machine learning-based imaging diagnostics has recently reached or even superseded the level of clinical experts in several clinical domains. However, classification decisions of a trained machine learning system are typically non-transparent, a major hindrance for clinical integration, error tracking or knowledge discovery. In this study, we present a transparent deep learning framework relying on convolutional neural networks (CNNs) and layer-wise relevance propagation (LRP) for diagnosing multiple sclerosis (MS). MS is commonly diagnosed utilizing a combination of clinical presentation and conventional magnetic resonance imaging (MRI), specifically the occurrence and presentation of white matter lesions in T2-weighted images. We hypothesized that using LRP in a naive predictive model would enable us to uncover relevant image features that a trained CNN uses for decision-making. Since imaging markers in MS are well-established this would enable us to validate the respective CNN model. First, we pre-trained a CNN on MRI data from the Alzheimer's Disease Neuroimaging Initiative (n = 921), afterwards specializing the CNN to discriminate between MS patients and healthy controls (n = 147). Using LRP, we then produced a heatmap for each subject in the holdout set depicting the voxel-wise relevance for a particular classification decision. The resulting CNN model resulted in a balanced accuracy of 87.04% and an area under the curve of 96.08% in a receiver operating characteristic curve. The subsequent LRP visualization revealed that the CNN model focuses indeed on individual lesions, but also incorporates additional information such as lesion location, non-lesional white matter or gray matter areas such as the thalamus, which are established conventional and advanced MRI markers in MS. We conclude that LRP and the proposed framework have the capability to make diagnostic decisions of..

Patch individual filter layers in CNNs to harness the spatial homogeneity of neuroimaging data

Convolutional neural networks (CNNs)-as a type of deep learning-have been specifically designed for highly heterogeneous data, such as natural images. Neuroimaging data, however, is comparably homogeneous due to (1) the uniform structure of the brain and (2) additional efforts to spatially normalize the data to a standard template using linear and non-linear transformations. To harness spatial homogeneity of neuroimaging data, we suggest here a new CNN architecture that combines the idea of hierarchical abstraction in CNNs with a prior on the spatial homogeneity of neuroimaging data. Whereas early layers are trained globally using standard convolutional layers, we introduce patch individual filters (PIF) for higher, more abstract layers. By learning filters in individual latent space patches without sharing weights, PIF layers can learn abstract features faster and specific to regions. We thoroughly evaluated PIF layers for three different tasks and data sets, namely sex classification on UK Biobank data, Alzheimer's disease detection on ADNI data and multiple sclerosis detection on private hospital data, and compared it with two baseline models, a standard CNN and a patch-based CNN. We obtained two main results: First, CNNs using PIF layers converge consistently faster, measured in run time in seconds and number of iterations than both baseline models. Second, both the standard CNN and the PIF model outperformed the patch-based CNN in terms of balanced accuracy and receiver operating characteristic area under the curve (ROC AUC) with a maximal balanced accuracy (ROC AUC) of 94.21% (99.10%) for the sex classification task (PIF model), and 81.24% and 80.48% (88.89% and 87.35%) respectively for the Alzheimer's disease and multiple sclerosis detection tasks (standard CNN model). In conclusion, we demonstrated that CNNs using PIF layers result in faster convergence while obtaining the same predictive performance as a standard CNN. To the best of our knowledge, this is the first study that introduces a prior in form of an inductive bias to harness spatial homogeneity of neuroimaging data

Computational approaches to predicting treatment response to obesity using neuroimaging

Obesity is a worldwide disease associated with multiple severe adverse consequences and comorbid conditions. While an increased body weight is the defining feature in obesity, etiologies, clinical phenotypes and treatment responses vary between patients. These variations can be observed within individual treatment options which comprise lifestyle interventions, pharmacological treatment, and bariatric surgery. Bariatric surgery can be regarded as the most effective treatment method. However, long-term weight regain is comparably frequent even for this treatment and its application is not without risk. A prognostic tool that would help predict the effectivity of the individual treatment methods in the long term would be essential in a personalized medicine approach. In line with this objective, an increasing number of studies have combined neuroimaging and computational modeling to predict treatment outcome in obesity. In our review, we begin by outlining the central nervous mechanisms measured with neuroimaging in these studies. The mechanisms are primarily related to reward-processing and include "incentive salience" and psychobehavioral control. We then present the diverse neuroimaging methods and computational prediction techniques applied. The studies included in this review provide consistent support for the importance of incentive salience and psychobehavioral control for treatment outcome in obesity. Nevertheless, further studies comprising larger sample sizes and rigorous validation processes are necessary to answer the question of whether or not the approach is sufficiently accurate for clinical real-world application

Neural Processes of Psychological Stress and Relaxation Predict the Future Evolution of Quality of Life in Multiple Sclerosis

Health-related quality of life (HRQoL) is an essential complementary parameter in the assessment of disease burden and treatment outcome in multiple sclerosis (MS) and can be affected by neuropsychiatric symptoms, which in turn are sensitive to psychological stress. However, until now, the impact of neurobiological stress and relaxation on HRQoL in MS has not been investigated. We thus evaluated whether the activity of neural networks triggered by mild psychological stress (elicited in an fMRI task comprising mental arithmetic with feedback) or by stress termination (i.e., relaxation) at baseline (T0) predicts HRQoL variations occurring between T0 and a follow-up visit (T1) in 28 patients using a robust regression and permutation testing. The median delay between T0 and T1 was 902 (range: 363-1,169) days. We assessed HRQoL based on the Hamburg Quality of Life Questionnaire in MS (HAQUAMS) and accounted for the impact of established HRQoL predictors and the cognitive performance of the participants. Relaxation-triggered activity of a widespread neural network predicted future variations in overall HRQoL (t = 3.68, p(family-wise error [FWE])-corrected = 0.008). Complementary analyses showed that relaxation-triggered activity of the same network at baseline was associated with variations in the HAQUAMS mood subscale on an alpha(FWE) = 0.1 level (t = 3.37, p(FWE) = 0.087). Finally, stress-induced activity of a prefronto-limbic network predicted future variations in the HAQUAMS lower limb mobility subscale (t = -3.62, p(FWE) = 0.020). Functional neural network measures of psychological stress and relaxation contain prognostic information for future HRQoL evolution in MS independent of clinical predictors

Uncovering convolutional neural network decisions for diagnosing multiple sclerosis on conventional MRI using layer-wise relevance propagation

Machine learning-based imaging diagnostics has recently reached or even surpassed the level of clinical experts in several clinical domains. However, classification decisions of a trained machine learning system are typically non-transparent, a major hindrance for clinical integration, error tracking or knowledge discovery. In this study, we present a transparent deep learning framework relying on 3D convolutional neural networks (CNNs) and layer-wise relevance propagation (LRP) for diagnosing multiple sclerosis (MS), the most widespread autoimmune neuroinflammatory disease. MS is commonly diagnosed utilizing a combination of clinical presentation and conventional magnetic resonance imaging (MRI), specifically the occurrence and presentation of white matter lesions in T2-weighted images. We hypothesized that using LRP in a naive predictive model would enable us to uncover relevant image features that a trained CNN uses for decision-making. Since imaging markers in MS are well-established this would enable us to validate the respective CNN model. First, we pre-trained a CNN on MRI data from the Alzheimer's Disease Neuroimaging Initiative (n = 921), afterwards specializing the CNN to discriminate between MS patients (n = 76) and healthy controls (n = 71). Using LRP, we then produced a heatmap for each subject in the holdout set depicting the voxel-wise relevance for a particular classification decision. The resulting CNN model resulted in a balanced accuracy of 87.04% and an area under the curve of 96.08% in a receiver operating characteristic curve. The subsequent LRP visualization revealed that the CNN model focuses indeed on individual lesions, but also incorporates additional information such as lesion location, non-lesional white matter or gray matter areas such as the thalamus, which are established conventional and advanced MRI markers in MS. We conclude that LRP and the proposed framework have the capability to make diagnostic decisions of CNN models transparent, which could serve to justify classification decisions for clinical review, verify diagnosis-relevant features and potentially gather new disease knowledge

Altered Coupling of Psychological Relaxation and Regional Volume of Brain Reward Areas in Multiple Sclerosis

Background:Psychological stress can influence the severity of multiple sclerosis (MS), but little is known about neurobiological factors potentially counteracting these effects. Objective:To identify gray matter (GM) brain regions related to relaxation after stress exposure in persons with MS (PwMS). Methods:36 PwMS and 21 healthy controls (HCs) reported their feeling of relaxation during a mild stress task. These markers were related to regional GM volumes, heart rate, and depressive symptoms. Results:Relaxation was differentially linked to heart rate in both groups (t= 2.20,p= 0.017), i.e., both markers were only related in HCs. Relaxation was positively linked to depressive symptoms across all participants (t= 1.99,p= 0.045) although this link differed weakly between groups (t= 1.62,p= 0.108). Primarily, the volume in medial temporal gyrus was negatively linked to relaxation in PwMS (t= -5.55, p(family-wise-error(FWE)corrected)= 0.018). A group-specific coupling of relaxation and GM volume was found in ventromedial prefrontal cortex (VMPFC) (t= -4.89, p(FWE)= 0.039). Conclusion:PwMS appear unable to integrate peripheral stress signals into their perception of relaxation. Together with the group-specific coupling of relaxation and VMPFC volume, a key area of the brain reward system for valuation of affectively relevant stimuli, this finding suggests a clinically relevant misinterpretation of stress-related affective stimuli in MS

Similar neural pathways link psychological stress and brain-age in health and multiple sclerosis

Clinical and neuroscientific studies suggest a link between psychological stress and reduced brain health in health and neurological disease but it is unclear whether mediating pathways are similar. Consequently, we applied an arterial-spin-labeling MRI stress task in 42 healthy persons and 56 with multiple sclerosis, and investigated regional neural stress responses, associations between functional connectivity of stress-responsive regions and the brain-age prediction error, a highly sensitive machine learning brain health biomarker, and regional brain-age constituents in both groups. Stress responsivity did not differ between groups. Although elevated brain-age prediction errors indicated worse brain health in patients, anterior insula–occipital cortex (healthy persons: occipital pole; patients: fusiform gyrus) functional connectivity correlated with brain-age prediction errors in both groups. Finally, also gray matter contributed similarly to regional brain-age across groups. These findings might suggest a common stress–brain health pathway whose impact is amplified in multiple sclerosis by disease-specific vulnerability factors

MRI Pattern Recognition in Multiple Sclerosis Normal-Appearing Brain Areas

Objective Here, we use pattern-classification to investigate diagnostic information for multiple sclerosis (MS; relapsing­remitting type) in lesioned areas, areas of normal­appearing grey matter (NAGM), and normal-appearing white matter (NAWM) as measured by standard MR techniques. Methods A lesion mapping was carried out by an experienced neurologist for Turbo Inversion Recovery Magnitude (TIRM) images of individual subjects. Combining this mapping with templates from a neuroanatomic atlas, the TIRM images were segmented into three areas of homogenous tissue types (Lesions, NAGM, and NAWM) after spatial standardization. For each area, a linear Support Vector Machine algorithm was used in multiple local classification analyses to determine the diagnostic accuracy in separating MS patients from healthy controls based on voxel tissue intensity patterns extracted from small spherical subregions of these larger areas. To control for covariates, we also excluded group-specific biases in deformation fields as a potential source of information. Results Among regions containing lesions a posterior parietal WM area was maximally informative about the clinical status (96% accuracy, p<10−13). Cerebellar regions were maximally informative among NAGM areas (84% accuracy, p<10−7). A posterior brain region was maximally informative among NAWM areas (91% accuracy, p<10−10). Interpretation We identified regions indicating MS in lesioned, but also NAGM, and NAWM areas. This complements the current perception that standard MR techniques mainly capture macroscopic tissue variations due to focal lesion processes. Compared to current diagnostic guidelines for MS that define areas of diagnostic information with moderate spatial specificity, we identified hotspots of MS associated tissue alterations with high specificity defined on a millimeter scale

Prefrontal-amygdala emotion regulation and depression in multiple sclerosis

Depression is among the most common comorbidities in multiple sclerosis and has severe psychosocial consequences. Alterations in neural emotion regulation in amygdala and prefrontal cortex have been recognized as key mechanism of depression but never been investigated in multiple sclerosis depression. In this cross-sectional observational study, we employed a functional MRI task investigating neural emotion regulation by contrasting regulated versus unregulated negative stimulus perception in 16 persons with multiple sclerosis and depression (47.9 ± 11.8 years; 14 female) and 26 persons with multiple sclerosis but without depression (47.3 ± 11.7 years; 14 female). We tested the impact of depression and its interaction with lesions in amygdala-prefrontal fibre tracts on brain activity reflecting emotion regulation. A potential impact of sex, age, information processing speed, disease duration, overall lesion load, grey matter fraction, and treatment was taken into account in these analyses. Patients with depression were less able (i) to downregulate negative emotions than those without (t = -2.25, P = 0.012, β = -0.33) on a behavioural level according to self-report data and (ii) to downregulate activity in a left amygdala coordinate (t = 3.03, P(Family-wise error [FWE]-corrected) = 0.017, β = 0.39). Moreover, (iii) an interdependent effect of depression and lesions in amygdala-prefrontal tracts on activity was found in two left amygdala coordinates (t = 3.53, p(FWE9 = 0.007, β = 0.48; t = 3.21, p(FWE) = 0.0158, β = 0.49) and one right amygdala coordinate (t = 3.41, p(FWE) = 0.009, β = 0.51). Compatible with key elements of the cognitive depression theory formulated for idiopathic depression, our study demonstrates that depression in multiple sclerosis is characterized by impaired neurobehavioural emotion regulation. Complementing these findings, it shows that the relation between neural emotion regulation and depression is affected by lesion load, a key pathological feature of multiple sclerosis, located in amygdala-prefrontal tracts

Central stress processing, T-cell responsivity to stress hormones, and disease severity in multiple sclerosis

Epidemiological, clinical and neuroscientific studies support a link between psychobiological stress and multiple sclerosis. Neuroimaging suggests that blunted central stress processing goes along with higher multiple sclerosis severity, neuroendocrine studies suggest that blunted immune system sensitivity to stress hormones is linked to stronger neuroinflammation. Until now, however, no effort has been made to elucidate whether central stress processing and immune system sensitivity to stress hormones are related in a disease-specific fashion, and if so, whether this relation is clinically meaningful. Consequently, we conducted two functional MRI analyses based on a total of 39 persons with multiple sclerosis and 25 healthy persons. Motivated by findings of an altered interplay between neuroendocrine stress processing and T-cell glucocorticoid sensitivity in multiple sclerosis, we searched for neural networks whose stress task-evoked activity is differentially linked to peripheral T-cell glucocorticoid signalling in patients versus healthy persons as a potential indicator of disease-specific CNS–immune crosstalk. Subsequently, we tested whether this activity is simultaneously related to disease severity. We found that activity of a network comprising right anterior insula, right fusiform gyrus, left midcingulate and lingual gyrus was differentially coupled to T-cell glucocorticoid signalling across groups. This network’s activity was simultaneously linked to patients’ lesion volume, clinical disability and information-processing speed. Complementary analyses revealed that T-cell glucocorticoid signalling was not directly linked to disease severity. Our findings show that alterations in the coupling between central stress processing and T-cell stress hormone sensitivity are related to key severity measures of multiple sclerosis