Assessing and tuning brain decoders: cross-validation, caveats, and guidelines

International audienceDecoding, ie prediction from brain images or signals, calls for empirical evaluation of its predictive power. Such evaluation is achieved via cross-validation, a method also used to tune decoders' hyper-parameters. This paper is a review on cross-validation procedures for decoding in neuroimaging. It includes a didactic overview of the relevant theoretical considerations. Practical aspects are highlighted with an extensive empirical study of the common decoders in within-and across-subject predictions, on multiple datasets –anatomical and functional MRI and MEG– and simulations. Theory and experiments outline that the popular " leave-one-out " strategy leads to unstable and biased estimates, and a repeated random splits method should be preferred. Experiments outline the large error bars of cross-validation in neuroimaging settings: typical confidence intervals of 10%. Nested cross-validation can tune decoders' parameters while avoiding circularity bias. However we find that it can be more favorable to use sane defaults, in particular for non-sparse decoders

Cortical Thickness Estimation in Individuals With Cerebral Small Vessel Disease, Focal Atrophy, and Chronic Stroke Lesions

Background: Regional changes to cortical thickness in individuals with neurodegenerative and cerebrovascular diseases (CVD) can be estimated using specialized neuroimaging software. However, the presence of cerebral small vessel disease, focal atrophy, and cortico-subcortical stroke lesions, pose significant challenges that increase the likelihood of misclassification errors and segmentation failures. Purpose: The main goal of this study was to examine a correction procedure developed for enhancing FreeSurfer’s (FS’s) cortical thickness estimation tool, particularly when applied to the most challenging MRI obtained from participants with chronic stroke and CVD, with varying degrees of neurovascular lesions and brain atrophy. Methods: In 155 CVD participants enrolled in the Ontario Neurodegenerative Disease Research Initiative (ONDRI), FS outputs were compared between a fully automated, unmodified procedure and a corrected procedure that accounted for potential sources of error due to atrophy and neurovascular lesions. Quality control (QC) measures were obtained from both procedures. Association between cortical thickness and global cognitive status as assessed by the Montreal Cognitive Assessment (MoCA) score was also investigated from both procedures. Results: Corrected procedures increased “Acceptable” QC ratings from 18 to 76% for the cortical ribbon and from 38 to 92% for tissue segmentation. Corrected procedures reduced “Fail” ratings from 11 to 0% for the cortical ribbon and 62 to 8% for tissue segmentation. FS-based segmentation of T1-weighted white matter hypointensities were significantly greater in the corrected procedure (5.8 mL vs. 15.9 mL, p \u3c 0.001). The unmodified procedure yielded no significant associations with global cognitive status, whereas the corrected procedure yielded positive associations between MoCA total score and clusters of cortical thickness in the left superior parietal (p = 0.018) and left insula (p = 0.04) regions. Further analyses with the corrected cortical thickness results and MoCA subscores showed a positive association between left superior parietal cortical thickness and Attention (p \u3c 0.001). Conclusion: These findings suggest that correction procedures which account for brain atrophy and neurovascular lesions can significantly improve FS’s segmentation results and reduce failure rates, thus maximizing power by preventing the loss of our important study participants. Future work will examine relationships between cortical thickness, cerebral small vessel disease, and cognitive dysfunction due to neurodegenerative disease in the ONDRI study

The past, present, and future of the Brain Imaging Data Structure (BIDS)

The Brain Imaging Data Structure (BIDS) is a community-driven standard for the organization of data and metadata from a growing range of neuroscience modalities. This paper is meant as a history of how the standard has developed and grown over time. We outline the principles behind the project, the mechanisms by which it has been extended, and some of the challenges being addressed as it evolves. We also discuss the lessons learned through the project, with the aim of enabling researchers in other domains to learn from the success of BIDS

graynet: single subject morphometric networks

Single-subject networks from structural shape and morphological features (thickness, gray matter densit etc

Histogram-weighted Networks for Feature Extraction, Connectivity and Advanced Analysis in Neuroscience

Network-level analysis of various features, esp. if it can be individualized for a single-subject is proving to be a valuable tool in many applications. Ability to extract the networks for a given subject individually on its own, would allow for feature extraction conducive to predictive modeling, unlike group-wise networks which can only be used for descriptive and explanatory purposes. This package extracts single-subject (individualized, or intrinsic) networks from node-wise data by computing the edge weights based on histogram distance between the distributions of values within each node. Individual nodes could be an ROI or a patch or a cube, or any other unit of relevance in your application. This is a great way to take advantage of the full distribution of values available within each node, relative to the simpler use of averages (or another summary statistic) to compare two nodes/ROIs within a given subject

Impact of spatial scale in thickness networks is not significant in predicting Alzheimer’s and Autism

Impact of spatial scale in thickness networks is not significant in predicting Alzheimer’s and Autism <br

Multi-class Differential Diagnosis among Alzheimer’s, Parkinson’s, Corticobasal syndrome and Progressive Supranuclear Palsy

Multi-class Differential Diagnosis among Alzheimer’s, Parkinson’s, Corticobasal syndrome and Progressive Supranuclear Pals

Three-Class Differential Diagnosis among Alzheimer Disease, Frontotemporal Dementia, and Controls.

Biomarkers derived from brain magnetic resonance (MR) imaging have promise in being able to assist in the clinical diagnosis of brain pathologies. These have been used in many studies in which the goal has been to distinguish between pathologies such as Alzheimer's disease and healthy aging. However, other dementias, in particular, frontotemporal dementia, also present overlapping pathological brain morphometry patterns. Hence, a classifier that can discriminate morphometric features from a brain MRI from the three classes of normal aging, Alzheimer's disease (AD), and frontotemporal dementia (FTD) would offer considerable utility in aiding in correct group identification. Compared to the conventional use of multiple pair-wise binary classifiers that learn to discriminate between two classes at each stage, we propose a single three-way classification system that can discriminate between three classes at the same time. We present a novel classifier that is able to perform a three-class discrimination test for discriminating among AD, FTD, and normal controls (NC) using volumes, shape invariants, and local displacements (three features) of hippocampi and lateral ventricles (two structures times two hemispheres individually) obtained from brain MR images. In order to quantify its utility in correct discrimination, we optimize the three-class classifier on a training set and evaluate its performance using a separate test set. This is a novel, first-of-its-kind comparative study of multiple individual biomarkers in a three-class setting. Our results demonstrate that local atrophy features in lateral ventricles offer the potential to be a biomarker in discriminating among AD, FTD, and NC in a three-class setting for individual patient classification