Standardized Assessment of Automatic Segmentation of White Matter Hyperintensities and Results of the WMH Segmentation Challenge

Quantification of cerebral white matter hyperintensities (WMH) of presumed vascular origin is of key importance in many neurological research studies. Currently, measurements are often still obtained from manual segmentations on brain MR images, which is a laborious procedure. The automatic WMH segmentation methods exist, but a standardized comparison of the performance of such methods is lacking. We organized a scientific challenge, in which developers could evaluate their methods on a standardized multi-center/-scanner image dataset, giving an objective comparison: the WMH Segmentation Challenge. Sixty T1 + FLAIR images from three MR scanners were released with the manual WMH segmentations for training. A test set of 110 images from five MR scanners was used for evaluation. The segmentation methods had to be containerized and submitted to the challenge organizers. Five evaluation metrics were used to rank the methods: 1) Dice similarity coefficient; 2) modified Hausdorff distance (95th percentile); 3) absolute log-transformed volume difference; 4) sensitivity for detecting individual lesions; and 5) F1-score for individual lesions. In addition, the methods were ranked on their inter-scanner robustness; 20 participants submitted their methods for evaluation. This paper provides a detailed analysis of the results. In brief, there is a cluster of four methods that rank significantly better than the other methods, with one clear winner. The inter-scanner robustness ranking shows that not all the methods generalize to unseen scanners. The challenge remains open for future submissions and provides a public platform for method evaluation

Network impact score is an independent predictor of post-stroke cognitive impairment: A multicenter cohort study in 2341 patients with acute ischemic stroke

BACKGROUND: Post-stroke cognitive impairment (PSCI) is a common consequence of stroke. Accurate prediction of PSCI risk is challenging. The recently developed network impact score, which integrates information on infarct location and size with brain network topology, may improve PSCI risk prediction. AIMS: To determine if the network impact score is an independent predictor of PSCI, and of cognitive recovery or decline. METHODS: We pooled data from patients with acute ischemic stroke from 12 cohorts through the Meta VCI Map consortium. PSCI was defined as impairment in ≥ 1 cognitive domain on neuropsychological examination, or abnormal Montreal Cognitive Assessment. Cognitive recovery was defined as conversion from PSCI 24 months) and cognitive recovery or decline using logistic regression. Models were adjusted for age, sex, education, prior stroke, infarct volume, and study site. RESULTS: We included 2341 patients with 4657 cognitive assessments. PSCI was present in 398/844 patients (47%) 24 months. Cognitive recovery occurred in 64/181 (35%) patients and cognitive decline in 26/287 (9%). The network impact score predicted PSCI in the univariable (OR 1.50, 95%CI 1.34-1.68) and multivariable (OR 1.27, 95%CI 1.10-1.46) GEE model, with similar ORs in the logistic regression models for specified post-stroke intervals. The network impact score was not associated with cognitive recovery or decline. CONCLUSIONS: The network impact score is an independent predictor of PSCI. As such, the network impact score may contribute to a more precise and individualized cognitive prognostication in patients with ischemic stroke. Future studies should address if multimodal prediction models, combining the network impact score with demographics, clinical characteristics and other advanced brain imaging biomarkers, will provide accurate individualized prediction of PSCI. A tool for calculating the network impact score is freely available at https://metavcimap.org/features/software-tools/lsm-viewer/

Quality control for more reliable integration of deep learning-based image segmentation into medical workflows

Machine learning algorithms underpin modern diagnostic-aiding software, whichhas proved valuable in clinical practice, particularly in radiology. However,inaccuracies, mainly due to the limited availability of clinical samples fortraining these algorithms, hamper their wider applicability, acceptance, andrecognition amongst clinicians. We present an analysis of state-of-the-artautomatic quality control (QC) approaches that can be implemented within thesealgorithms to estimate the certainty of their outputs. We validated the mostpromising approaches on a brain image segmentation task identifying whitematter hyperintensities (WMH) in magnetic resonance imaging data. WMH are acorrelate of small vessel disease common in mid-to-late adulthood and areparticularly challenging to segment due to their varied size, anddistributional patterns. Our results show that the aggregation of uncertaintyand Dice prediction were most effective in failure detection for this task.Both methods independently improved mean Dice from 0.82 to 0.84. Our workreveals how QC methods can help to detect failed segmentation cases andtherefore make automatic segmentation more reliable and suitable for clinicalpractice.<br

Spatial distributions of white matter hyperintensities on brain MRI: A pooled analysis of individual participant data from 11 memory clinic cohorts

Introduction: The spatial distribution of white matter hyperintensities (WMH) on MRI is often considered in the diagnostic evaluation of patients with cognitive problems. In some patients, clinicians may classify WMH patterns as unusual, but this is largely based on expert opinion, because detailed quantitative information about WMH distribution frequencies in a memory clinic setting is lacking. Here we report voxel wise 3D WMH distribution frequencies in a large multicenter dataset and also aimed to identify individuals with unusual WMH patterns. Methods: Individual participant data (N = 3525, including 777 participants with subjective cognitive decline, 1389 participants with mild cognitive impairment and 1359 patients with dementia) from eleven memory clinic cohorts, recruited through the Meta VCI Map Consortium, were used. WMH segmentations were provided by participating centers or performed in Utrecht and registered to the Montreal Neurological Institute (MNI)-152 brain template for spatial normalization. To determine WMH distribution frequencies, we calculated WMH probability maps at voxel level. To identify individuals with unusual WMH patterns, region-of-interest (ROI) based WMH probability maps, rule-based scores, and a machine learning method (Local Outlier Factor (LOF)), were implemented. Results: WMH occurred in 82% of voxels from the white matter template with large variation between subjects. Only a small proportion of the white matter (1.7%), mainly in the periventricular areas, was affected by WMH in at least 20% of participants. A large portion of the total white matter was affected infrequently. Nevertheless, 93.8% of individual participants had lesions in voxels that were affected in less than 2% of the population, mainly located in subcortical areas. Only the machine learning method effectively identified individuals with unusual patterns, in particular subjects with asymmetric WMH distribution or with WMH at relatively rarely affected locations despite common locations not being affected. Discussion: Aggregating data from several memory clinic cohorts, we provide a detailed 3D map of WMH lesion distribution frequencies, that informs on common as well as rare localizations. The use of data-driven analysis with LOF can be used to identify unusual patterns, which might serve as an alert that rare causes of WMH should be considered