Layer-Wise Relevance Propagation for Explaining Deep Neural Network Decisions in MRI-Based Alzheimer's Disease Classification

Deep neural networks have led to state-of-the-art results in many medical imaging tasks including Alzheimer’s disease (AD) detection based on structural magnetic resonance imaging (MRI) data. However, the network decisions are often perceived as being highly non-transparent, making it difficult to apply these algorithms in clinical routine. In this study, we propose using layer-wise relevance propagation (LRP) to visualize convolutional neural network decisions for AD based on MRI data. Similarly to other visualization methods, LRP produces a heatmap in the input space indicating the importance/relevance of each voxel contributing to the final classification outcome. In contrast to susceptibility maps produced by guided backpropagation (“Which change in voxels would change the outcome most?”), the LRP method is able to directly highlight positive contributions to the network classification in the input space. In particular, we show that (1) the LRP method is very specific for individuals (“Why does this person have AD?”) with high inter-patient variability, (2) there is very little relevance for AD in healthy controls and (3) areas that exhibit a lot of relevance correlate well with what is known from literature. To quantify the latter, we compute size-corrected metrics of the summed relevance per brain area, e.g., relevance density or relevance gain. Although these metrics produce very individual “fingerprints” of relevance patterns for AD patients, a lot of importance is put on areas in the temporal lobe including the hippocampus. After discussing several limitations such as sensitivity toward the underlying model and computation parameters, we conclude that LRP might have a high potential to assist clinicians in explaining neural network decisions for diagnosing AD (and potentially other diseases) based on structural MRI data

Harnessing spatial homogeneity of neuroimaging data: patch individual filter layers for CNNs

Neuroimaging data, e.g. obtained from magnetic resonance imaging (MRI), is comparably homogeneous due to (1) the uniform structure of the brain and (2) additional efforts to spatially normalize the data to a standard template using linear and non-linear transformations. Convolutional neural networks (CNNs), in contrast, have been specifically designed for highly heterogeneous data, such as natural images, by sliding convolutional filters over different positions in an image. Here, we suggest a new CNN architecture that combines the idea of hierarchical abstraction in neural networks with a prior on the spatial homogeneity of neuroimaging data: Whereas early layers are trained globally using standard convolutional layers, we introduce for higher, more abstract layers patch individual filters (PIF). By learning filters in individual image regions (patches) without sharing weights, PIF layers can learn abstract features faster and with fewer samples. We thoroughly evaluated PIF layers for three different tasks and data sets, namely sex classification on UK Biobank data, Alzheimer's disease detection on ADNI data and multiple sclerosis detection on private hospital data. We demonstrate that CNNs using PIF layers result in higher accuracies, especially in low sample size settings, and need fewer training epochs for convergence. To the best of our knowledge, this is the first study which introduces a prior on brain MRI for CNN learning

Multimodal prediction of conversion to Alzheimer's disease based on incomplete biomarkers.

BACKGROUND: This study investigates the prediction of mild cognitive impairment-to-Alzheimer's disease (MCI-to-AD) conversion based on extensive multimodal data with varying degrees of missing values. METHODS: Based on Alzheimer's Disease Neuroimaging Initiative data from MCI-patients including all available modalities, we predicted the conversion to AD within 3 years. Different ways of replacing missing data in combination with different classification algorithms are compared. The performance was evaluated on features prioritized by experts and automatically selected features. RESULTS: The conversion to AD could be predicted with a maximal accuracy of 73% using support vector machines and features chosen by experts. Among data modalities, neuropsychological, magnetic resonance imaging, and positron emission tomography data were most informative. The best single feature was the functional activities questionnaire. CONCLUSION: Extensive multimodal and incomplete data can be adequately handled by a combination of missing data substitution, feature selection, and classification

Uncovering convolutional neural network decisions for diagnosing multiple sclerosis on conventional MRI using layer-wise relevance propagation

Machine learning-based imaging diagnostics has recently reached or even superseded the level of clinical experts in several clinical domains. However, classification decisions of a trained machine learning system are typically non-transparent, a major hindrance for clinical integration, error tracking or knowledge discovery. In this study, we present a transparent deep learning framework relying on convolutional neural networks (CNNs) and layer-wise relevance propagation (LRP) for diagnosing multiple sclerosis (MS). MS is commonly diagnosed utilizing a combination of clinical presentation and conventional magnetic resonance imaging (MRI), specifically the occurrence and presentation of white matter lesions in T2-weighted images. We hypothesized that using LRP in a naive predictive model would enable us to uncover relevant image features that a trained CNN uses for decision-making. Since imaging markers in MS are well-established this would enable us to validate the respective CNN model. First, we pre-trained a CNN on MRI data from the Alzheimer's Disease Neuroimaging Initiative (n = 921), afterwards specializing the CNN to discriminate between MS patients and healthy controls (n = 147). Using LRP, we then produced a heatmap for each subject in the holdout set depicting the voxel-wise relevance for a particular classification decision. The resulting CNN model resulted in a balanced accuracy of 87.04% and an area under the curve of 96.08% in a receiver operating characteristic curve. The subsequent LRP visualization revealed that the CNN model focuses indeed on individual lesions, but also incorporates additional information such as lesion location, non-lesional white matter or gray matter areas such as the thalamus, which are established conventional and advanced MRI markers in MS. We conclude that LRP and the proposed framework have the capability to make diagnostic decisions of..

Association Between Fatigue and Motor Exertion in Patients With Multiple Sclerosis - a Prospective Study

Background: Fatigue in multiple sclerosis (MS) is conceived as a multidimensional construct. Objectives: This study aims to describe the changes of balance and gait parameters after 6 min of walking (6 MW) as potential quantitative markers for perceptions of state fatigue and trait fatigue in MS. Methods: A total of 19 patients with MS (17 with fatigue) and 24 healthy subjects underwent static posturography, gait analysis, and ratings of perceived exertion before and after 6 MW. Results: 6 MW was perceived as exhaustive, but both groups featured more dynamic comfortable speed walking after 6 MW. Shorter stride length at maximum speed and increased postural sway after 6 MW indicated fatigability of balance and gait in MS group only. While most changes were related to higher levels of perceived exertion after 6 MW (state fatigue), higher fatigue ratings (trait fatigue) were only associated with less increase in arm swing at comfortable speed. Further analysis revealed different associations of trait fatigue and performance fatigability with disability and motor functions. Performance fatigability was most closely related to the Expanded Disability Status Scale, while for trait fatigue, the strongest correlations were seen with balance function and handgrip strength. Conclusions: Fatigability of performance was closely related to perceptions of exertion after 6 MW (state fatigue) and disability in MS but distinct from fatigue ratings, conceived as trait fatigue. Our study identified postural sway, arm swing during gait, and hand grip strength as unexpected potential motor indicators of fatigue ratings in MS

Neural Processes of Psychological Stress and Relaxation Predict the Future Evolution of Quality of Life in Multiple Sclerosis

Health-related quality of life (HRQoL) is an essential complementary parameter in the assessment of disease burden and treatment outcome in multiple sclerosis (MS) and can be affected by neuropsychiatric symptoms, which in turn are sensitive to psychological stress. However, until now, the impact of neurobiological stress and relaxation on HRQoL in MS has not been investigated. We thus evaluated whether the activity of neural networks triggered by mild psychological stress (elicited in an fMRI task comprising mental arithmetic with feedback) or by stress termination (i.e., relaxation) at baseline (T0) predicts HRQoL variations occurring between T0 and a follow-up visit (T1) in 28 patients using a robust regression and permutation testing. The median delay between T0 and T1 was 902 (range: 363-1,169) days. We assessed HRQoL based on the Hamburg Quality of Life Questionnaire in MS (HAQUAMS) and accounted for the impact of established HRQoL predictors and the cognitive performance of the participants. Relaxation-triggered activity of a widespread neural network predicted future variations in overall HRQoL (t = 3.68, p(family-wise error [FWE])-corrected = 0.008). Complementary analyses showed that relaxation-triggered activity of the same network at baseline was associated with variations in the HAQUAMS mood subscale on an alpha(FWE) = 0.1 level (t = 3.37, p(FWE) = 0.087). Finally, stress-induced activity of a prefronto-limbic network predicted future variations in the HAQUAMS lower limb mobility subscale (t = -3.62, p(FWE) = 0.020). Functional neural network measures of psychological stress and relaxation contain prognostic information for future HRQoL evolution in MS independent of clinical predictors

Altered Coupling of Psychological Relaxation and Regional Volume of Brain Reward Areas in Multiple Sclerosis

Background:Psychological stress can influence the severity of multiple sclerosis (MS), but little is known about neurobiological factors potentially counteracting these effects. Objective:To identify gray matter (GM) brain regions related to relaxation after stress exposure in persons with MS (PwMS). Methods:36 PwMS and 21 healthy controls (HCs) reported their feeling of relaxation during a mild stress task. These markers were related to regional GM volumes, heart rate, and depressive symptoms. Results:Relaxation was differentially linked to heart rate in both groups (t= 2.20,p= 0.017), i.e., both markers were only related in HCs. Relaxation was positively linked to depressive symptoms across all participants (t= 1.99,p= 0.045) although this link differed weakly between groups (t= 1.62,p= 0.108). Primarily, the volume in medial temporal gyrus was negatively linked to relaxation in PwMS (t= -5.55, p(family-wise-error(FWE)corrected)= 0.018). A group-specific coupling of relaxation and GM volume was found in ventromedial prefrontal cortex (VMPFC) (t= -4.89, p(FWE)= 0.039). Conclusion:PwMS appear unable to integrate peripheral stress signals into their perception of relaxation. Together with the group-specific coupling of relaxation and VMPFC volume, a key area of the brain reward system for valuation of affectively relevant stimuli, this finding suggests a clinically relevant misinterpretation of stress-related affective stimuli in MS

Delayed access to conscious processing in multiple sclerosis: reduced cortical activation and impaired structural connectivity

Although multiple sclerosis (MS) is frequently accompanied by visuo-cognitive impairment, especially functional brain mechanisms underlying this impairment are still not well understood. Consequently, we used a functional MRI (fMRI) backward masking task to study visual information processing stratifying unconscious and conscious in MS. Specifically, 30 persons with MS (pwMS) and 34 healthy controls (HC) were shown target stimuli followed by a mask presented 8-150 ms later and had to compare the target to a reference stimulus. Retinal integrity (via optical coherence tomography), optic tract integrity (visual evoked potential; VEP) and whole brain structural connectivity (probabilistic tractography) were assessed as complementary structural brain integrity markers. On a psychophysical level, pwMS reached conscious access later than HC (50 vs. 16 ms, p < .001). The delay increased with disease duration (p < .001, β = .37) and disability (p < .001, β = .24), but did not correlate with conscious information processing speed (Symbol digit modality test, β = .07, p = .817). No association was found for VEP and retinal integrity markers. Moreover, pwMS were characterized by decreased brain activation during unconscious processing compared with HC. No group differences were found during conscious processing. Finally, a complementary structural brain integrity analysis showed that a reduced fractional anisotropy in corpus callosum and an impaired connection between right insula and primary visual areas was related to delayed conscious access in pwMS. Our study revealed slowed conscious access to visual stimulus material in MS and a complex pattern of functional and structural alterations coupled to unconscious processing of/delayed conscious access to visual stimulus material in MS

Experience of Subjective Symptoms in Euthymic Patients with Bipolar Disorder

Bipolar patients often experience subjective symptoms even if they do not have active psychotic symptoms in their euthymic state. Most studies about subjective symptoms are conducted in schizophrenia, and there are few studies involving bipolar patients. We examined the nature of the subjective symptoms of bipolar patients in their euthymic state, and we also compared it to that of schizophrenia and normal control. Thirty bipolar patients, 25 patients with schizophrenia, and 21 normal control subjects were included. Subjective symptoms were assessed using the Korean version of the Frankfurter Beschwerde Fragebogen (K-FBF) and the Symptom Check List 90-R (SCL90-R). Euthymic state was confirmed by assessing objective psychopathology with the Positive and Negative Syndrome scale of Schizophrenia (PANSS), the Young Mania Rating Scale (YMRS), and the Montgomery Asberg Depression Rating Scale (MADRS). K-FBF score was significantly higher in bipolar patients than in normal controls, but similar to that in schizophrenia patients (F=5.86, p=0.004, R2=2033.6). In contrast, SCL90-R scores did not differ significantly among the three groups. Euthymic bipolar patients experience subjective symptoms that are more confined to cognitive domain. This finding supports the hypothesis that subtle cognitive impairments persists in euthymic bipolar patients

MRI Pattern Recognition in Multiple Sclerosis Normal-Appearing Brain Areas

Objective Here, we use pattern-classification to investigate diagnostic information for multiple sclerosis (MS; relapsing­remitting type) in lesioned areas, areas of normal­appearing grey matter (NAGM), and normal-appearing white matter (NAWM) as measured by standard MR techniques. Methods A lesion mapping was carried out by an experienced neurologist for Turbo Inversion Recovery Magnitude (TIRM) images of individual subjects. Combining this mapping with templates from a neuroanatomic atlas, the TIRM images were segmented into three areas of homogenous tissue types (Lesions, NAGM, and NAWM) after spatial standardization. For each area, a linear Support Vector Machine algorithm was used in multiple local classification analyses to determine the diagnostic accuracy in separating MS patients from healthy controls based on voxel tissue intensity patterns extracted from small spherical subregions of these larger areas. To control for covariates, we also excluded group-specific biases in deformation fields as a potential source of information. Results Among regions containing lesions a posterior parietal WM area was maximally informative about the clinical status (96% accuracy, p<10−13). Cerebellar regions were maximally informative among NAGM areas (84% accuracy, p<10−7). A posterior brain region was maximally informative among NAWM areas (91% accuracy, p<10−10). Interpretation We identified regions indicating MS in lesioned, but also NAGM, and NAWM areas. This complements the current perception that standard MR techniques mainly capture macroscopic tissue variations due to focal lesion processes. Compared to current diagnostic guidelines for MS that define areas of diagnostic information with moderate spatial specificity, we identified hotspots of MS associated tissue alterations with high specificity defined on a millimeter scale