DELMEP: a deep learning algorithm for automated annotation of motor evoked potential latencies

Abstract The analysis of motor evoked potentials (MEPs) generated by transcranial magnetic stimulation (TMS) is crucial in research and clinical medical practice. MEPs are characterized by their latency and the treatment of a single patient may require the characterization of thousands of MEPs. Given the difficulty of developing reliable and accurate algorithms, currently the assessment of MEPs is performed with visual inspection and manual annotation by a medical expert; making it a time-consuming, inaccurate, and error-prone process. In this study, we developed DELMEP, a deep learning-based algorithm to automate the estimation of MEP latency. Our algorithm resulted in a mean absolute error of about 0.5 ms and an accuracy that was practically independent of the MEP amplitude. The low computational cost of the DELMEP algorithm allows employing it in on-the-fly characterization of MEPs for brain-state-dependent and closed-loop brain stimulation protocols. Moreover, its learning ability makes it a particularly promising option for artificial-intelligence-based personalized clinical applications

Tractography reproducibility challenge with empirical data (TraCED): The 2017 ISMRM diffusion study group challenge

Background: Fiber tracking with diffusion-weighted MRI has become an essential tool for estimating in vivo brain white matter architecture. Fiber tracking results are sensitive to the choice of processing method and tracking criteria. Purpose: To assess the variability for an algorithm in group studies reproducibility is of critical context. However, reproducibility does not assess the validity of the brain connections. Phantom studies provide concrete quantitative comparisons of methods relative to absolute ground truths, yet do no capture variabilities because of in vivo physiological factors. The ISMRM 2017 TraCED challenge was created to fulfill the gap. Study Type: A systematic review of algorithms and tract reproducibility studies. Subjects: Single healthy volunteers. Field Strength/Sequence: 3.0T, two different scanners by the same manufacturer. The multishell acquisition included b-values of 1000, 2000, and 3000 s/mm 2 with 20, 45, and 64 diffusion gradient directions per shell, respectively. Assessment: Nine international groups submitted 46 tractography algorithm entries each consisting 16 tracts per scan. The algorithms were assessed using intraclass correlation (ICC) and the Dice similarity measure. Statistical Tests: Containment analysis was performed to assess if the submitted algorithms had containment within tracts of larger volume submissions. This also serves the purpose to detect if spurious submissions had been made. Results: The top five submissions had high ICC and Dice >0.88. Reproducibility was high within the top five submissions when assessed across sessions or across scanners: 0.87–0.97. Containment analysis shows that the top five submissions are contained within larger volume submissions. From the total of 16 tracts as an outcome relatively the number of tracts with high, moderate, and low reproducibility were 8, 4, and 4. Data Conclusion: The different methods clearly result in fundamentally different tract structures at the more conservative specificity choices. Data and challenge infrastructure remain available for continued analysis and provide a platform for comparison. Level of Evidence: 5. Technical Efficacy Stage: 1 J. Magn. Reson. Imaging 2020;51:234–249

Limits to anatomical accuracy of diffusion tractography using modern approaches

Diffusion MRI fiber tractography is widely used to probe the structural connectivity of the brain, with a range of applications in both clinical and basic neuroscience. Despite widespread use, tractography has well-known pitfalls that limits the anatomical accuracy of this technique. Numerous modern methods have been developed to address these shortcomings through advances in acquisition, modeling, and computation. To test whether these advances improve tractography accuracy, we organized the 3-D Validation of Tractography with Experimental MRI (3D-VoTEM) challenge at the ISBI 2018 conference. We made available three unique independent tractography validation datasets – a physical phantom and two ex vivo brain specimens - resulting in 176 distinct submissions from 9 research groups. By comparing results over a wide range of fiber complexities and algorithmic strategies, this challenge provides a more comprehensive assessment of tractography's inherent limitations than has been reported previously. The central results were consistent across all sub-challenges in that, despite advances in tractography methods, the anatomical accuracy of tractography has not dramatically improved in recent years. Taken together, our results independently confirm findings from decades of tractography validation studies, demonstrate inherent limitations in reconstructing white matter pathways using diffusion MRI data alone, and highlight the need for alternative or combinatorial strategies to accurately map the fiber pathways of the brain

Insights from the IronTract challenge: Optimal methods for mapping brain pathways from multi-shell diffusion MRI

Limitations in the accuracy of brain pathways reconstructed by diffusion MRI (dMRI) tractography have received considerable attention. While the technical advances spearheaded by the Human Connectome Project (HCP) led to significant improvements in dMRI data quality, it remains unclear how these data should be analyzed to maximize tractography accuracy. Over a period of two years, we have engaged the dMRI community in the IronTract Challenge, which aims to answer this question by leveraging a unique dataset. Macaque brains that have received both tracer injections and ex vivo dMRI at high spatial and angular resolution allow a comprehensive, quantitative assessment of tractography accuracy on state-of-the-art dMRI acquisition schemes. We find that, when analysis methods are carefully optimized, the HCP scheme can achieve similar accuracy as a more timeconsuming, Cartesian-grid scheme. Importantly, we show that simple pre-and post-processing strategies can improve the accuracy and robustness of many tractography methods. Finally, we find that fiber configurations that go beyond crossing (e.g., fanning, branching) are the most challenging for tractography. The IronTract Challenge remains open and we hope that it can serve as a valuable validation tool for both users and developers of dMRI analysis methods

Tractography passes the test : Results from the diffusion-simulated connectivity (disco) challenge

Estimating structural connectivity from diffusion-weighted magnetic resonance imaging is a challenging task, partly due to the presence of false-positive connections and the misestimation of connection weights. Building on previous efforts, the MICCAI-CDMRI Diffusion-Simulated Connectivity (DiSCo) challenge was carried out to evaluate state-of-the-art connectivity methods using novel large-scale numerical phantoms. The diffusion signal for the phantoms was obtained from Monte Carlo simulations. The results of the challenge suggest that methods selected by the 14 teams participating in the challenge can provide high correlations between estimated and ground-truth connectivity weights, in complex numerical environments. Additionally, the methods used by the participating teams were able to accurately identify the binary connectivity of the numerical dataset. However, specific false positive and false negative connections were consistently estimated across all methods. Although the challenge dataset doesn't capture the complexity of a real brain, it provided unique data with known macrostructure and microstructure ground-truth properties to facilitate the development of connectivity estimation methods.Peer reviewe

Tractography dissection variability: What happens when 42 groups dissect 14 white matter bundles on the same dataset?

White matter bundle segmentation using diffusion MRI fiber tractography has become the method of choice to identify white matter fiber pathways in vivo in human brains. However, like other analyses of complex data, there is considerable variability in segmentation protocols and techniques. This can result in different reconstructions of the same intended white matter pathways, which directly affects tractography results, quantification, and interpretation. In this study, we aim to evaluate and quantify the variability that arises from different protocols for bundle segmentation. Through an open call to users of fiber tractography, including anatomists, clinicians, and algorithm developers, 42 independent teams were given processed sets of human whole-brain streamlines and asked to segment 14 white matter fascicles on six subjects. In total, we received 57 different bundle segmentation protocols, which enabled detailed volume-based and streamline-based analyses of agreement and disagreement among protocols for each fiber pathway. Results show that even when given the exact same sets of underlying streamlines, the variability across protocols for bundle segmentation is greater than all other sources of variability in the virtual dissection process, including variability within protocols and variability across subjects. In order to foster the use of tractography bundle dissection in routine clinical settings, and as a fundamental analytical tool, future endeavors must aim to resolve and reduce this heterogeneity. Although external validation is needed to verify the anatomical accuracy of bundle dissections, reducing heterogeneity is a step towards reproducible research and may be achieved through the use of standard nomenclature and definitions of white matter bundles and well-chosen constraints and decisions in the dissection process

Tractography dissection variability

Funding Information: This work was conducted in part using the resources of the Advanced Computing Center for Research and Education at Vanderbilt University, Nashville, TN. KS, BL, CH were supported by the National Institutes of Health under award numbers R01EB017230, and T32EB001628, and in part by ViSE/VICTR VR3029 and the National Center for Research Resources, Grant UL1 RR024975-01. This work was also possible thanks to the support of the Institutional Research Chair in NeuroInformatics of Université de Sherbrooke, NSERC and Compute Canada (MD, FR). MP received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 754462. The Wisconsin group acknowledges the support from a core grant to the Waisman Center from the National Institute of Child Health and Human Development (IDDRC U54 HD090256). NSF OAC-1916518, NSF IIS-1912270, NSF IIS-1636893, NSF BCS-1734853, NIH NIBIB 1R01EB029272-01, and a Microsoft Faculty Fellowship to F.P. LF acknowledges the support of the Cluster of Excellence Matters of Activity. Image Space Material funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany´s Excellence Strategy – EXC 2025. SW is supported by a Medical Research Council PhD Studentship UK [MR/N013913/1]. The Nottingham group's processing was performed using the University of Nottingham's Augusta HPC service and the Precision Imaging Beacon Cluster. JPA, MA and SMS acknowledges the support of FCT - Fundação para a Ciência e a Tecnologia within CINTESIS, R&D Unit (reference UID/IC/4255/2013). MM was funded by the Wellcome Trust through a Sir Henry Wellcome Postdoctoral Fellowship [213722/Z/18/Z]. EJC-R is supported by the Swiss National Science Foundation (SNSF, Ambizione grant PZ00P2 185814/1). CMWT is supported by a Sir Henry Wellcome Fellowship (215944/Z/19/Z) and a Veni grant from the Dutch Research Council (NWO) (17331). FC acknowledges the support of the National Health and Medical Research Council ofAustralia (APP1091593 and APP1117724) and the Australian Research Council (DP170101815). NSF OAC-1916518, NSF IIS-1912270, NSF IIS-1636893, NSF BCS-1734853, Microsoft Faculty Fellowship to F.P. D.B. was partially supported by NIH NIMH T32-MH103213 to William Hetrick (Indiana University). CL is partly supported by NIH grants P41 EB027061 and P30 NS076408 “Institutional Center Cores for Advanced Neuroimaging. JYMY received positional funding from the Royal Children's Hospital Foundation (RCH 1000). JYMY, JC, and CEK acknowledge the support of the Royal Children's Hospital Foundation, Murdoch Children's Research Institute, The University of Melbourne Department of Paediatrics, and the Victorian Government's Operational Infrastructure Support Program. C-HY is grateful to the Ministry of Science and Technology of Taiwan (MOST 109-2222-E-182-001-MY3) for the support. LC acknowledges support from CONACYT and UNAM. ARM acknowledges support from CONACYT. LJO, YR, and FZ were supported by NIH P41EB015902 and R01MH119222. AJG was supported by P41EB015898. NM was supported by R01MH119222, K24MH116366, and R01MH111917. This project has received funding from the European Union's Horizon 2020 Research and Innovation Programme under Grant Agreement No. 785907 & 945539 (HBP SGA2 & SGA3), and from the ANR IFOPASUBA- 19-CE45-0022-01. PG, CR, NL and AV were partially supported by ANID-Basal FB0008 and ANID-FONDECYT 1190701 grants. We would like to acknowledge John C Gore, Hiromasa Takemura, Anastasia Yendiki, and Riccardo Galbusera for their helplful suggestions regarding the analysis, figures, and discussions. Funding Information: This work was conducted in part using the resources of the Advanced Computing Center for Research and Education at Vanderbilt University, Nashville, TN. KS, BL, CH were supported by the National Institutes of Health under award numbers R01EB017230, and T32EB001628, and in part by ViSE/VICTR VR3029 and the National Center for Research Resources, Grant UL1 RR024975-01. This work was also possible thanks to the support of the Institutional Research Chair in NeuroInformatics of Universit? de Sherbrooke, NSERC and Compute Canada (MD, FR). MP received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sk?odowska-Curie grant agreement No 754462. The Wisconsin group acknowledges the support from a core grant to the Waisman Center from the National Institute of Child Health and Human Development (IDDRC U54 HD090256). NSF OAC-1916518, NSF IIS-1912270, NSF IIS-1636893, NSF BCS-1734853, NIH NIBIB 1R01EB029272-01, and a Microsoft Faculty Fellowship to F.P. LF acknowledges the support of the Cluster of Excellence Matters of Activity. Image Space Material funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany?s Excellence Strategy ? EXC 2025. SW is supported by a Medical Research Council PhD Studentship UK [MR/N013913/1]. The Nottingham group's processing was performed using the University of Nottingham's Augusta HPC service and the Precision Imaging Beacon Cluster. JPA, MA and SMS acknowledges the support of FCT - Funda??o para a Ci?ncia e a Tecnologia within CINTESIS, R&D Unit (reference UID/IC/4255/2013). MM was funded by the Wellcome Trust through a Sir Henry Wellcome Postdoctoral Fellowship [213722/Z/18/Z]. EJC-R is supported by the Swiss National Science Foundation (SNSF, Ambizione grant PZ00P2 185814/1). CMWT is supported by a Sir Henry Wellcome Fellowship (215944/Z/19/Z) and a Veni grant from the Dutch Research Council (NWO) (17331). FC acknowledges the support of the National Health and Medical Research Council of Australia (APP1091593 and APP1117724) and the Australian Research Council (DP170101815). NSF OAC-1916518, NSF IIS-1912270, NSF IIS-1636893, NSF BCS-1734853, Microsoft Faculty Fellowship to F.P. D.B. was partially supported by NIH NIMH T32-MH103213 to William Hetrick (Indiana University). CL is partly supported by NIH grants P41 EB027061 and P30 NS076408 ?Institutional Center Cores for Advanced Neuroimaging. JYMY received positional funding from the Royal Children's Hospital Foundation (RCH 1000). JYMY, JC, and CEK acknowledge the support of the Royal Children's Hospital Foundation, Murdoch Children's Research Institute, The University of Melbourne Department of Paediatrics, and the Victorian Government's Operational Infrastructure Support Program. C-HY is grateful to the Ministry of Science and Technology of Taiwan (MOST 109-2222-E-182-001-MY3) for the support. LC acknowledges support from CONACYT and UNAM. ARM acknowledges support from CONACYT. LJO, YR, and FZ were supported by NIH P41EB015902 and R01MH119222. AJG was supported by P41EB015898. NM was supported by R01MH119222, K24MH116366, and R01MH111917. This project has received funding from the European Union's Horizon 2020 Research and Innovation Programme under Grant Agreement No. 785907 & 945539 (HBP SGA2 & SGA3), and from the ANR IFOPASUBA- 19-CE45-0022-01. PG, CR, NL and AV were partially supported by ANID-Basal FB0008 and ANID-FONDECYT 1190701 grants. We would like to acknowledge John C Gore, Hiromasa Takemura, Anastasia Yendiki, and Riccardo Galbusera for their helplful suggestions regarding the analysis, figures, and discussions. Publisher Copyright: © 2021White matter bundle segmentation using diffusion MRI fiber tractography has become the method of choice to identify white matter fiber pathways in vivo in human brains. However, like other analyses of complex data, there is considerable variability in segmentation protocols and techniques. This can result in different reconstructions of the same intended white matter pathways, which directly affects tractography results, quantification, and interpretation. In this study, we aim to evaluate and quantify the variability that arises from different protocols for bundle segmentation. Through an open call to users of fiber tractography, including anatomists, clinicians, and algorithm developers, 42 independent teams were given processed sets of human whole-brain streamlines and asked to segment 14 white matter fascicles on six subjects. In total, we received 57 different bundle segmentation protocols, which enabled detailed volume-based and streamline-based analyses of agreement and disagreement among protocols foreach fiber pathway. Results show that even when given the exact same sets of underlying streamlines, the variability across protocols for bundle segmentation is greater than all other sources of variability in the virtual dissection process, including variability within protocols and variability across subjects. In order to foster the use of tractography bundle dissection in routine clinical settings, and as a fundamental analytical tool, future endeavors must aim to resolve and reduce this heterogeneity. Although external validation is needed to verify the anatomical accuracy of bundle dissections, reducing heterogeneity is a step towards reproducible research and may be achieved through the use of standard nomenclature and definitions of white matter bundles and well-chosen constraints and decisions in the dissection process.Peer reviewe

Tractography dissection variability: What happens when 42 groups dissect 14 white matter bundles on the same dataset?

White matter bundle segmentation using diffusion MRI fiber tractography has become the method of choice to identify white matter fiber pathways in vivo in human brains. However, like other analyses of complex data, there is considerable variability in segmentation protocols and techniques. This can result in different reconstructions of the same intended white matter pathways, which directly affects tractography results, quantification, and interpretation. In this study, we aim to evaluate and quantify the variability that arises from different protocols for bundle segmentation. Through an open call to users of fiber tractography, including anatomists, clinicians, and algorithm developers, 42 independent teams were given processed sets of human whole-brain streamlines and asked to segment 14 white matter fascicles on six subjects. In total, we received 57 different bundle segmentation protocols, which enabled detailed volume-based and streamline-based analyses of agreement and disagreement among protocols for each fiber pathway. Results show that even when given the exact same sets of underlying streamlines, the variability across protocols for bundle segmentation is greater than all other sources of variability in the virtual dissection process, including variability within protocols and variability across subjects. In order to foster the use of tractography bundle dissection in routine clinical settings, and as a fundamental analytical tool, future endeavors must aim to resolve and reduce this heterogeneity. Although external validation is needed to verify the anatomical accuracy of bundle dissections, reducing heterogeneity is a step towards reproducible research and may be achieved through the use of standard nomenclature and definitions of white matter bundles and well-chosen constraints and decisions in the dissection processDepto. de Psicobiología y Metodología en Ciencias del ComportamientoFac. de PsicologíaTRUEpu