Accuracy and reproducibility of automated white matter hyperintensities segmentation with lesion segmentation tool: A European multi-site 3T study

Brain vascular damage accumulate in aging and often manifest as white matter hyperintensities (WMHs) on MRI. Despite increased interest in automated methods to segment WMHs, a gold standard has not been achieved and their longitudinal reproducibility has been poorly investigated. The aim of present work is to evaluate accuracy and reproducibility of two freely available segmentation algorithms. A harmonized MRI protocol was implemented in 3T-scanners across 13 European sites, each scanning five volunteers twice (test-retest) using 2D-FLAIR. Automated segmentation was performed using Lesion segmentation tool algorithms (LST): the Lesion growth algorithm (LGA) in SPM8 and 12 and the Lesion prediction algorithm (LPA). To assess reproducibility, we applied the LST longitudinal pipeline to the LGA and LPA outputs for both the test and retest scans. We evaluated volumetric and spatial accuracy comparing LGA and LPA with manual tracing, and for reproducibility the test versus retest. Median volume difference between automated WMH and manual segmentations (mL) was −0.22[IQR = 0.50] for LGA-SPM8, −0.12[0.57] for LGA-SPM12, −0.09[0.53] for LPA, while the spatial accuracy (Dice Coefficient) was 0.29[0.31], 0.33[0.26] and 0.41[0.23], respectively. The reproducibility analysis showed a median reproducibility error of 20%[IQR = 41] for LGA-SPM8, 14% [31] for LGA-SPM12 and 10% [27] with the LPA cross-sectional pipeline. Applying the LST longitudinal pipeline, the reproducibility errors were considerably reduced (LGA: 0%[IQR = 0], p < 0.001; LPA: 0% [3], p < 0.001) compared to those derived using the cross-sectional algorithms. The DC using the longitudinal pipeline was excellent (median = 1) for LGA [IQR = 0] and LPA [0.02]. LST algorithms showed moderate accuracy and good reproducibility. Therefore, it can be used as a reliable cross-sectional and longitudinal tool in multi-site studies

Functional connectivity and cognitive changes after donepezil treatment in healthy participants

International audienc

Brain morphometry reproducibility in multi-center 3T MRI studies: a comparison of cross-sectional and longitudinal segmentations

International audienc

Amygdalar nuclei and hippocampal subfields on MRI: Test-retest reliability of automated volumetry across different MRI sites and vendors

BACKGROUND: The amygdala and the hippocampus are two limbic structures that play a critical role in cognition and behavior, however their manual segmentation and that of their smaller nuclei/subfields in multicenter datasets is time consuming and difficult due to the low contrast of standard MRI. Here, we assessed the reliability of the automated segmentation of amygdalar nuclei and hippocampal subfields across sites and vendors using FreeSurfer in two independent cohorts of older and younger healthy adults. METHODS: Sixty-five healthy older (cohort 1) and 68 younger subjects (cohort 2), from the PharmaCog and CoRR consortia, underwent repeated 3D-T1 MRI (interval 1 to 90 days). Segmentation was performed using FreeSurfer v6.0. Reliability was assessed using volume reproducibility error (ε) and spatial overlapping coefficient (DICE) between test and retest session. RESULTS: Significant MRI site and vendor effects (p .43, p 0.80). CONCLUSION: Our results support the use of volumetric measures of larger amygdalar nuclei and hippocampal subfields in multisite MRI studies. These measures could be useful for disease tracking and assessment of efficacy in drug trials

Amygdalar nuclei and hippocampal subfields on MRI: Test-retest reliability of automated volumetry across different MRI sites and vendors

Background: The amygdala and the hippocampus are two limbic structures that play a critical role in cognition and behavior, however their manual segmentation and that of their smaller nuclei/subfields in multicenter datasets is time consuming and difficult due to the low contrast of standard MRI. Here, we assessed the reliability of the automated segmentation of amygdalar nuclei and hippocampal subfields across sites and vendors using FreeSurfer in two independent cohorts of older and younger healthy adults. Methods: Sixty-five healthy older (cohort 1) and 68 younger subjects (cohort 2), from the PharmaCog and CoRR consortia, underwent repeated 3D-T1 MRI (interval 1\u201390 days). Segmentation was performed using FreeSurfer v6.0. Reliability was assessed using volume reproducibility error (\u3b5) and spatial overlapping coefficient (DICE) between test and retest session. Results: Significant MRI site and vendor effects (p \u200b&lt; \u200b.05) were found in a few subfields/nuclei for the \u3b5, while extensive effects were found for the DICE score of most subfields/nuclei. Reliability was strongly influenced by volume, as \u3b5 correlated negatively and DICE correlated positively with volume size of structures (absolute value of Spearman's r correlations &gt;0.43, p \u200b&lt; \u200b1.39E-36). In particular, volumes larger than 200 \u200bmm3 (for amygdalar nuclei) and 300 \u200bmm3 (for hippocampal subfields, except for molecular layer) had the best test-retest reproducibility (\u3b5 \u200b&lt; \u200b5% and DICE \u200b&gt; \u200b0.80). Conclusion: Our results support the use of volumetric measures of larger amygdalar nuclei and hippocampal subfields in multisite MRI studies. These measures could be useful for disease tracking and assessment of efficacy in drug trials

Amygdalar nuclei and hippocampal subfields on MRI:Test-retest reliability of automated volumetry across different MRI sites and vendors

Background: The amygdala and the hippocampus are two limbic structures that play a critical role in cognition and behavior, however their manual segmentation and that of their smaller nuclei/subfields in multicenter datasets is time consuming and difficult due to the low contrast of standard MRI. Here, we assessed the reliability of the automated segmentation of amygdalar nuclei and hippocampal subfields across sites and vendors using FreeSurfer in two independent cohorts of older and younger healthy adults. Methods: Sixty-five healthy older (cohort 1) and 68 younger subjects (cohort 2), from the PharmaCog and CoRR consortia, underwent repeated 3D-T1 MRI (interval 1 to 90 days). Segmentation was performed using FreeSurfer v6.0. Reliability was assessed using volume reproducibility error (\u3b5) and spatial overlapping coefficient (DICE) between test and retest session. Results: Significant MRI site and vendor effects (p &lt; .05) were found in a few subfields/nuclei for the \u3b5, while extensive effects were found for the DICE score of most subfields/nuclei. Reliability was strongly influenced by volume, as \u3b5 correlated negatively and DICE correlated positively with volume size of structures (absolute value of Spearman's r correlations &gt;.43, p &lt; 1.39E-36). In particular, volumes larger than 200 mm3 (for amygdalar nuclei) and 300 mm3 (for hippocampal subfields, except for molecular layer) had the best test-retest reproducibility (\u3b5 &lt; 5% and DICE &gt; 0.80). Conclusion: Our results support the use of volumetric measures of larger amygdalar nuclei and hippocampal subfields in multisite MRI studies. These measures could be useful for disease tracking and assessment of efficacy in drug trials

Multisite longitudinal reliability of tract-based spatial statistics in diffusion tensor imaging of healthy elderly subjects

Large-scale longitudinal neuroimaging studies with diffusion imaging techniques are necessary to test and validate models of white matter neurophysiological processes that change in time, both in healthy and diseased brains. The predictive power of such longitudinal models will always be limited by the reproducibility of repeated measures acquired during different sessions. At present, there is limited quantitative knowledge about the across-session reproducibility of standard diffusion metrics in 3T multi-centric studies on subjects in stable conditions, in particular when using tract based spatial statistics and with elderly people. In this study we implemented a multi-site brain diffusion protocol in 10 clinical 3T MRI sites distributed across 4 countries in Europe (Italy, Germany, France and Greece) using vendor provided sequences from Siemens (Allegra, Trio Tim, Verio, Skyra, Biograph mMR), Philips (Achieva) and GE (HDxt) scanners. We acquired DTI data (2 × 2 × 2 mm(3), b = 700 s/mm(2), 5 b0 and 30 diffusion weighted volumes) of a group of healthy stable elderly subjects (5 subjects per site) in two separate sessions at least a week apart. For each subject and session four scalar diffusion metrics were considered: fractional anisotropy (FA), mean diffusivity (MD), radial diffusivity (RD) and axial (AD) diffusivity. The diffusion metrics from multiple subjects and sessions at each site were aligned to their common white matter skeleton using tract-based spatial statistics. The reproducibility at each MRI site was examined by looking at group averages of absolute changes relative to the mean (%) on various parameters: i) reproducibility of the signal-to-noise ratio (SNR) of the b0 images in centrum semiovale, ii) full brain test-retest differences of the diffusion metric maps on the white matter skeleton, iii) reproducibility of the diffusion metrics on atlas-based white matter ROIs on the white matter skeleton. Despite the differences of MRI scanner configurations across sites (vendors, models, RF coils and acquisition sequences) we found good and consistent test-retest reproducibility. White matter b0 SNR reproducibility was on average 7 ± 1% with no significant MRI site effects. Whole brain analysis resulted in no significant test-retest differences at any of the sites with any of the DTI metrics. The atlas-based ROI analysis showed that the mean reproducibility errors largely remained in the 2-4% range for FA and AD and 2-6% for MD and RD, averaged across ROIs. Our results show reproducibility values comparable to those reported in studies using a smaller number of MRI scanners, slightly different DTI protocols and mostly younger populations. We therefore show that the acquisition and analysis protocols used are appropriate for multi-site experimental scenarios

Longitudinal reproducibility of automatically segmented hippocampal subfields: A multisite European 3T study on healthy elderly

Recently, there has been an increased interest in the use of automatically segmented subfields of the human hippocampal formation derived from magnetic resonance imaging (MRI). However, little is known about the test-retest reproducibility of such measures, particularly in the context of multisite studies. Here, we report the reproducibility of automated Freesurfer hippocampal subfields segmentations in 65 healthy elderly enrolled in a consortium of 13 3T MRI sites (five subjects per site). Participants were scanned in two sessions (test and retest) at least one week apart. Each session included two anatomical 3D T1 MRI acquisitions harmonized in the consortium. We evaluated the test-retest reproducibility of subfields segmentation (i) to assess the effects of averaging two within-session T1 images and (ii) to compare subfields with whole hippocampus volume and spatial reliability. We found that within-session averaging of two T1 images significantly improved the reproducibility of all hippocampal subfields but not that of the whole hippocampus. Volumetric and spatial reproducibility across MRI sites were very good for the whole hippocampus, CA2-3, CA4-dentate gyrus (DG), subiculum (reproducibility error∼2% and DICE > 0.90), good for CA1 and presubiculum (reproducibility error ∼ 5% and DICE ∼ 0.90), and poorer for fimbria and hippocampal fissure (reproducibility error ∼ 15% and DICE < 0.80). Spearman's correlations confirmed that test-retest reproducibility improved with volume size. Despite considerable differences of MRI scanner configurations, we found consistent hippocampal subfields volumes estimation. CA2-3, CA4-DG, and sub-CA1 (subiculum, presubiculum, and CA1 pooled together) gave test-retest reproducibility similar to the whole hippocampus. Our findings suggest that the larger hippocampal subfields volume may be reliable longitudinal markers in multisite studies. Hum Brain Mapp, 2015. © 2015 Wiley Periodicals, Inc

Accuracy and reproducibility of automated white matter hyperintensities segmentation with lesion segmentation tool: A European multi-site 3T study

Brain vascular damage accumulate in aging and often manifest as white matter hyperintensities (WMHs) on MRI. Despite increased interest in automated methods to segment WMHs, a gold standard has not been achieved and their longitudinal reproducibility has been poorly investigated. The aim of present work is to evaluate accuracy and reproducibility of two freely available segmentation algorithms. A harmonized MRI protocol was implemented in 3T-scanners across 13 European sites, each scanning five volunteers twice (test-retest) using 2D-FLAIR. Automated segmentation was performed using Lesion segmentation tool algorithms (LST): the Lesion growth algorithm (LGA) in SPM8 and 12 and the Lesion prediction algorithm (LPA). To assess reproducibility, we applied the LST longitudinal pipeline to the LGA and LPA outputs for both the test and retest scans. We evaluated volumetric and spatial accuracy comparing LGA and LPA with manual tracing, and for reproducibility the test versus retest. Median volume difference between automated WMH and manual segmentations (mL) was −0.22[IQR = 0.50] for LGA-SPM8, −0.12[0.57] for LGA-SPM12, −0.09[0.53] for LPA, while the spatial accuracy (Dice Coefficient) was 0.29[0.31], 0.33[0.26] and 0.41[0.23], respectively. The reproducibility analysis showed a median reproducibility error of 20%[IQR = 41] for LGA-SPM8, 14% [31] for LGA-SPM12 and 10% [27] with the LPA cross-sectional pipeline. Applying the LST longitudinal pipeline, the reproducibility errors were considerably reduced (LGA: 0%[IQR = 0], p &lt; 0.001; LPA: 0% [3], p &lt; 0.001) compared to those derived using the cross-sectional algorithms. The DC using the longitudinal pipeline was excellent (median = 1) for LGA [IQR = 0] and LPA [0.02]. LST algorithms showed moderate accuracy and good reproducibility. Therefore, it can be used as a reliable cross-sectional and longitudinal tool in multi-site studies