Explanations of Machine Learning Models in Repeated Nested Cross-Validation: An Application in Age Prediction Using Brain Complexity Features

SHAP (Shapley additive explanations) is a framework for explainable AI that makes explanations locally and globally. In this work, we propose a general method to obtain representative SHAP values within a repeated nested cross-validation procedure and separately for the training and test sets of the different cross-validation rounds to assess the real generalization abilities of the explanations. We applied this method to predict individual age using brain complexity features extracted from MRI scans of 159 healthy subjects. In particular, we used four implementations of the fractal dimension (FD) of the cerebral cortex—a measurement of brain complexity. Representative SHAP values highlighted that the most recent implementation of the FD had the highest impact over the others and was among the top-ranking features for predicting age. SHAP rankings were not the same in the training and test sets, but the top-ranking features were consistent. In conclusion, we propose a method—and share all the source code—that allows a rigorous assessment of the SHAP explanations of a trained model in a repeated nested cross-validation setting

Efficacy of MRI data harmonization in the age of machine learning. A multicenter study across 36 datasets

Pooling publicly-available MRI data from multiple sites allows to assemble extensive groups of subjects, increase statistical power, and promote data reuse with machine learning techniques. The harmonization of multicenter data is necessary to reduce the confounding effect associated with non-biological sources of variability in the data. However, when applied to the entire dataset before machine learning, the harmonization leads to data leakage, because information outside the training set may affect model building, and potentially falsely overestimate performance. We propose a 1) measurement of the efficacy of data harmonization; 2) harmonizer transformer, i.e., an implementation of the ComBat harmonization allowing its encapsulation among the preprocessing steps of a machine learning pipeline, avoiding data leakage. We tested these tools using brain T1-weighted MRI data from 1740 healthy subjects acquired at 36 sites. After harmonization, the site effect was removed or reduced, and we measured the data leakage effect in predicting individual age from MRI data, highlighting that introducing the harmonizer transformer into a machine learning pipeline allows for avoiding data leakage

Prediction of Impaired Performance in Trail Making Test in MCI Patients With Small Vessel Disease Using DTI Data

Mild cognitive impairment (MCI) is a common condition in patients with diffuse hyperintensities of cerebral white matter (WM) in T2-weighted magnetic resonance images and cerebral small vessel disease (SVD). In MCI due to SVD, the most prominent feature of cognitive impairment lies in degradation of executive functions, i.e., of processes that supervise the organization and execution of complex behavior. The trail making test is a widely employed test sensitive to cognitive processing speed and executive functioning. MCI due to SVD has been hypothesized to be the effect of WM damage, and diffusion tensor imaging (DTI) is a well-established technique for in vivo characterization of WM. We propose a machine learning scheme tailored to 1) predicting the impairment in executive functions in patients with MCI and SVD, and 2) examining the brain substrates of this impairment. We employed data from 40 MCI patients with SVD and created feature vectors by averaging mean diffusivity (MD) and fractional anisotropy maps within 50 WM regions of interest. We trained support vector machines (SVMs) with polynomial as well as radial basis function kernels using different DTI-derived features while simultaneously optimizing parameters in leave-one-out nested cross validation. The best performance was obtained using MD features only and linear kernel SVMs, which were able to distinguish an impaired performance with high sensitivity (72.7%-89.5%), specificity (71.4%-83.3%), and accuracy (77.5%-80.0%). While brain substrates of executive functions are still debated, feature ranking confirm that MD in several WM regions, not limited to the frontal lobes, are truly predictive of executive functions

3D Convolutional Neural Networks for Diagnosis of Alzheimer’s Disease via structural MRI

Alzheimer’s Disease (AD) is a widespread neurodegenerative disease caused by structural changes in the brain and leads to deterioration of cognitive functions. Patients usually experience diagnostic symptoms at later stages after irreversible neural damage occurs. Early detection of AD is crucial in maximizing patients' quality of life and to start treatments to decelerate the progress of the disease. Early detection may be possible via computer-assisted systems using neuroimaging data. Among all, deep learning utilizing magnetic resonance imaging (MRI) have become a prominent tool due to its capability to extract high-level features through local connectivity, weight sharing, and spatial invariance. This paper describes our investigation of the classification accuracy based on two publicly available data sets, namely, ADNI and OASIS, by building a 3D VGG variant convolutional network (CNN). We used 3D models to avoid information loss, which occurs during the process of slicing 3D MRI into 2D images and analyzing them by 2D convolutional filters. We also conducted a pre-processing of the data to enhance the effectiveness and classification performance of the model. The proposed model achieved 73.4% classification accuracy on ADNI and 69.9% on OASIS dataset with 5-fold cross-validation (CV). These results are comparable to other studies using various convolutional models. However, our subject-based divided dataset has only one MRI of a single patient to prevent possible data leakage whereas some other studies have different screenings of the same patients "over a time period'" in their datasets

Fractal dimension of the cortical gray matter outweighs other brain MRI features as a predictor of transition to dementia in patients with mild cognitive impairment and leukoaraiosis

BackgroundThe relative contribution of changes in the cerebral white matter (WM) and cortical gray matter (GM) to the transition to dementia in patients with mild cognitive impairment (MCI) is not yet established. In this longitudinal study, we aimed to analyze MRI features that may predict the transition to dementia in patients with MCI and T2 hyperintensities in the cerebral WM, also known as leukoaraiosis.MethodsSixty-four participants with MCI and moderate to severe leukoaraiosis underwent baseline MRI examinations and annual neuropsychological testing over a 2 year period. The diagnosis of dementia was based on established criteria. We evaluated demographic, neuropsychological, and several MRI features at baseline as predictors of the clinical transition. The MRI features included visually assessed MRI features, such as the number of lacunes, microbleeds, and dilated perivascular spaces, and quantitative MRI features, such as volumes of the cortical GM, hippocampus, T2 hyperintensities, and diffusion indices of the cerebral WM. Additionally, we examined advanced quantitative features such as the fractal dimension (FD) of cortical GM and WM, which represents an index of tissue structural complexity derived from 3D-T1 weighted images. To assess the prediction of transition to dementia, we employed an XGBoost-based machine learning system using SHapley Additive exPlanations (SHAP) values to provide explainability to the machine learning model.ResultsAfter 2 years, 18 (28.1%) participants had transitioned from MCI to dementia. The area under the receiving operator characteristic curve was 0.69 (0.53, 0.85) [mean (90% confidence interval)]. The cortical GM-FD emerged as the top-ranking predictive feature of transition. Furthermore, aggregated quantitative neuroimaging features outperformed visually assessed MRI features in predicting conversion to dementia.DiscussionOur findings confirm the complementary roles of cortical GM and WM changes as underlying factors in the development of dementia in subjects with MCI and leukoaraiosis. FD appears to be a biomarker potentially more sensitive than other brain features

Assessment of spontaneous cardiovascular oscillations in Parkinson's disease

Parkinson's disease (PD) has been reported to involve postganglionic sympathetic failure and a wide spectrum of autonomic dysfunctions including cardiovascular, sexual, bladder, gastrointestinal and sudo-motor abnormalities. While these symptoms may have a significant impact on daily activities, as well as quality of life, the evaluation of autonomic nervous system (ANS) dysfunctions relies on a large and expensive battery of autonomic tests only accessible in highly specialized laboratories. In this paper we aim to devise a comprehensive computational assessment of disease-related heartbeat dynamics based on instantaneous, time-varying estimates of spontaneous (resting state) cardiovascular oscillations in PD. To this end, we combine standard ANS-related heart rate variability (HRV) metrics with measures of instantaneous complexity (dominant Lyapunov exponent and entropy) and higher-order statistics (bispectra). Such measures are computed over 600-s recordings acquired at rest in 29 healthy subjects and 30 PD patients. The only significant group-wise differences were found in the variability of the dominant Lyapunov exponent. Also, the best PD vs. healthy controls classification performance (balanced accuracy: 73.47%) was achieved only when retaining the time-varying, non-stationary structure of the dynamical features, whereas classification performance dropped significantly (balanced accuracy: 61.91%) when excluding variability-related features. Additionally, both linear and nonlinear model features correlated with both clinical and neuropsychological assessments of the considered patient population. Our results demonstrate the added value and potential of instantaneous measures of heartbeat dynamics and its variability in characterizing PD-related disabilities in motor and cognitive domains

Artificial intelligence-based models for reconstructing the critical current and index-value surfaces of HTS tapes

For modelling superconductors, interpolation and analytical formulas are commonly used to consider the relationship between the critical current density and other electromagnetic and physical quantities. However, look-up tables are not available in all modelling and coding environments, and interpolation methods must be manually implemented. Moreover, analytical formulas only approximate real physics of superconductors and, in many cases, lack a high level of accuracy. In this paper, we propose a new approach for addressing this problem involving artificial intelligence (AI) techniques for reconstructing the critical surface of high temperature superconducting (HTS) tapes and predicting their index value known as n-value. Different AI models were proposed and implemented, relying on a public experimental database for electromagnetic specifications of HTS tapes, including artificial neural networks (ANN), eXtreme Gradient Boosting (XGBoost), and kernel ridge regressor (KRR). The ANN model was the most accurate in predicting the critical current of HTS materials, performing goodness of fit very close to 1 and extremely low root mean squared error. The XGBoost model proved to be the fastest method, with training computational times under 1 s; whilst KRR could be used as an alternative solution with intermediate performance

Fractal dimension of cerebral white matter : A consistent feature for prediction of the cognitive performance in patients with small vessel disease and mild cognitive impairment

Patients with cerebral small vessel disease (SVD) frequently show decline in cognitive performance. However, neuroimaging in SVD patients discloses a wide range of brain lesions and alterations so that it is often difficult to understand which of these changes are the most relevant for cognitive decline. It has also become evident that visually-rated alterations do not fully explain the neuroimaging correlates of cognitive decline in SVD. Fractal dimension (FD), a unitless feature of structural complexity that can be computed from high-resolution T1-weighted images, has been recently applied to the neuroimaging evaluation of the human brain. Indeed, white matter (WM) and cortical gray matter (GM) exhibit an inherent structural complexity that can be measured through the FD. In our study, we included 64 patients (mean age \ub1 standard deviation, 74.6 \ub1 6.9, education 7.9 \ub1 4.2 years, 53% males) with SVD and mild cognitive impairment (MCI), and a control group of 24 healthy subjects (mean age \ub1 standard deviation, 72.3 \ub1 4.4 years, 50% males). With the aim of assessing whether the FD values of cerebral WM (WM FD) and cortical GM (GM FD) could be valuable structural predictors of cognitive performance in patients with SVD and MCI, we employed a machine learning strategy based on LASSO (least absolute shrinkage and selection operator) regression applied on a set of standard and advanced neuroimaging features in a nested cross-validation (CV) loop. This approach was aimed at 1) choosing the best predictive models, able to reliably predict the individual neuropsychological scores sensitive to attention and executive dysfunctions (prominent features of subcortical vascular cognitive impairment) and 2) identifying a features ranking according to their importance in the model through the assessment of the out-of-sample error. For each neuropsychological test, using 1000 repetitions of LASSO regression and 5000 random permutations, we found that the statistically significant models were those for the Montreal Cognitive Assessment scores (p-value =.039), Symbol Digit Modalities Test scores (p-value =.039), and Trail Making Test Part A scores (p-value =.025). Significant prediction of these scores was obtained using different sets of neuroimaging features in which the WM FD was the most frequently selected feature. In conclusion, we showed that a machine learning approach could be useful in SVD research field using standard and advanced neuroimaging features. Our study results raise the possibility that FD may represent a consistent feature in predicting cognitive decline in SVD that can complement standard imaging