Spatio-temporal motion correction and iterative reconstruction of in-utero fetal fMRI

Resting-state functional Magnetic Resonance Imaging (fMRI) is a powerful imaging technique for studying functional development of the brain in utero. However, unpredictable and excessive movement of fetuses have limited its clinical applicability. Previous studies have focused primarily on the accurate estimation of the motion parameters employing a single step 3D interpolation at each individual time frame to recover a motion-free 4D fMRI image. Using only information from a 3D spatial neighborhood neglects the temporal structure of fMRI and useful information from neighboring timepoints. Here, we propose a novel technique based on four dimensional iterative reconstruction of the motion scattered fMRI slices. Quantitative evaluation of the proposed method on a cohort of real clinical fetal fMRI data indicates improvement of reconstruction quality compared to the conventional 3D interpolation approaches.Comment: Accepted by MICCAI 202

Spatiotemporal alignment of in utero BOLD-MRI series

Purpose: To present a method for spatiotemporal alignment of in-utero magnetic resonance imaging (MRI) time series acquired during maternal hyperoxia for enabling improved quantitative tracking of blood oxygen level-dependent (BOLD) signal changes that characterize oxygen transport through the placenta to fetal organs. Materials and Methods: The proposed pipeline for spatiotemporal alignment of images acquired with a single-shot gradient echo echo-planar imaging includes 1) signal nonuniformity correction, 2) intravolume motion correction based on nonrigid registration, 3) correction of motion and nonrigid deformations across volumes, and 4) detection of the outlier volumes to be discarded from subsequent analysis. BOLD MRI time series collected from 10 pregnant women during 3T scans were analyzed using this pipeline. To assess pipeline performance, signal fluctuations between consecutive timepoints were examined. In addition, volume overlap and distance between manual region of interest (ROI) delineations in a subset of frames and the delineations obtained through propagation of the ROIs from the reference frame were used to quantify alignment accuracy. A previously demonstrated rigid registration approach was used for comparison. Results: The proposed pipeline improved anatomical alignment of placenta and fetal organs over the state-of-the-art rigid motion correction methods. In particular, unexpected temporal signal fluctuations during the first normoxia period were significantly decreased (P < 0.01) and volume overlap and distance between region boundaries measures were significantly improved (P < 0.01). Conclusion: The proposed approach to align MRI time series enables more accurate quantitative studies of placental function by improving spatiotemporal alignment across placenta and fetal organs.National Institutes of Health (NIH) . Grant Numbers: U01 HD087211 , R01 EB017337 Consejeria de Educacion, Juventud y Deporte de la Comunidad de Madrid (Spain) through the Madrid-MIT M+Vision Consortium

A Survey on Deep Learning in Medical Image Analysis

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks and provide concise overviews of studies per application area. Open challenges and directions for future research are discussed.Comment: Revised survey includes expanded discussion section and reworked introductory section on common deep architectures. Added missed papers from before Feb 1st 201

Motion robust acquisition and reconstruction of quantitative T2* maps in the developing brain

The goal of the research presented in this thesis was to develop methods for quantitative T2* mapping of the developing brain. Brain maturation in the early period of life involves complex structural and physiological changes caused by synaptogenesis, myelination and growth of cells. Molecular structures and biological processes give rise to varying levels of T2* relaxation time, which is an inherent contrast mechanism in magnetic resonance imaging. The knowledge of T2* relaxation times in the brain can thus help with evaluation of pathology by establishing its normative values in the key areas of the brain. T2* relaxation values are a valuable biomarker for myelin microstructure and iron concentration, as well as an important guide towards achievement of optimal fMRI contrast. However, fetal MR imaging is a significant step up from neonatal or adult MR imaging due to the complexity of the acquisition and reconstruction techniques that are required to provide high quality artifact-free images in the presence of maternal respiration and unpredictable fetal motion. The first contribution of this thesis, described in Chapter 4, presents a novel acquisition method for measurement of fetal brain T2* values. At the time of publication, this was the first study of fetal brain T2* values. Single shot multi-echo gradient echo EPI was proposed as a rapid method for measuring fetal T2* values by effectively freezing intra-slice motion. The study concluded that fetal T2* values are higher than those previously reported for pre-term neonates and decline with a consistent trend across gestational age. The data also suggested that longer than usual echo times or direct T2* measurement should be considered when performing fetal fMRI in order to reach optimal BOLD sensitivity. For the second contribution, described in Chapter 5, measurements were extended to a higher field strength of 3T and reported, for the first time, both for fetal and neonatal subjects at this field strength. The technical contribution of this work is a fully automatic segmentation framework that propagates brain tissue labels onto the acquired T2* maps without the need for manual intervention. The third contribution, described in Chapter 6, proposed a new method for performing 3D fetal brain reconstruction where the available data is sparse and is therefore limited in the use of current state of the art techniques for 3D brain reconstruction in the presence of motion. To enable a high resolution reconstruction, a generative adversarial network was trained to perform image to image translation between T2 weighted and T2* weighted data. Translated images could then be served as a prior for slice alignment and super resolution reconstruction of 3D brain image.Open Acces

SVRDA: A Web-based Dataset Annotation Tool for Slice-to-Volume Registration

Background and Objective: The lack of benchmark datasets has impeded the development of slice-to-volume registration algorithms. Such datasets are difficult to annotate, primarily due to the dimensional difference within data and the dearth of task-specific software. We aim to develop a user-friendly tool to streamline dataset annotation for slice-to-volume registration. Methods: The proposed tool, named SVRDA, is an installation-free web application for platform-agnostic collaborative dataset annotation. It enables efficient transformation manipulation via keyboard shortcuts and smooth case transitions with auto-saving. SVRDA supports configuration-based data loading and adheres to the separation of concerns, offering great flexibility and extensibility for future research. Various supplementary features have been implemented to facilitate slice-to-volume registration. Results: We validated the effectiveness of SVRDA by indirectly evaluating the post-registration segmentation quality on UK Biobank data, observing a dramatic overall improvement (24.02% in the Dice Similarity Coefficient and 48.93% in the 95th percentile Hausdorff distance, respectively) supported by highly statistically significant evidence ($p<0.001$).We further showcased the clinical usage of SVRDA by integrating it into test-retest T1 quantification on in-house magnetic resonance images, leading to more consistent results after registration. Conclusions: SVRDA can facilitate collaborative annotation of benchmark datasets while being potentially applicable to other pipelines incorporating slice-to-volume registration. Full source code and documentation are available at https://github.com/Roldbach/SVRDAComment: 18 pages, 11 figures, In submission to Computer Methods and Programs in Biomedicin

An ode to fetal, infant, and toddler neuroimaging: chronicling early clinical to research applications with MRI, and an introduction to an academic society connecting the field

Fetal, infant, and toddler neuroimaging is commonly thought of as a development of modern times (last two decades). Yet, this field mobilized shortly after the discovery and implementation of MRI technology. Here, we provide a review of the parallel advancements in the fields of fetal, infant, and toddler neuroimaging, noting the shifts from clinical to research use, and the ongoing challenges in this fast-growing field. We chronicle the pioneering science of fetal, infant, and toddler neuroimaging, highlighting the early studies that set the stage for modern advances in imaging during this developmental period, and the large-scale multi-site efforts which ultimately led to the explosion of interest in the field today. Lastly, we consider the growing pains of the community and the need for an academic society that bridges expertise in developmental neuroscience, clinical science, as well as computational and biomedical engineering, to ensure special consideration of the vulnerable mother-offspring dyad (especially during pregnancy), data quality, and image processing tools that are created, rather than adapted, for the young brain.UL1 TR001863 - NCATS NIH HHS; R01 MH117983 - NIMH NIH HHS; K24 MH127381 - NIMH NIH HHS; UL1 TR001873 - NCATS NIH HHS; TL1 TR001875 - NCATS NIH HHS; T32 MH018268 - NIMH NIH HHS; ZIA MH002782 - Intramural NIH HHS; UL1 TR003015 - NCATS NIH HHS; KL2 TR003016 - NCATS NIH HHS; R01 HD065762 - NICHD NIH HHS; R03 EB022754 - NIBIB NIH HHS; R21 HD095338 - NICHD NIH HHS; R01 HD093578 - NICHD NIH HHS; R01 HD099846 - NICHD NIH HHS; R01 HD100560 - NICHD NIH HHSPublished versio

Recalage déformable a l'aide de graphes de coupes 2D et de volumes 3D

Deformable image registration plays a fundamental role in many clinical applications. In this paper we investigate the use of graphical models in the context of a particular type of image registration problem, known as slice-to-volume registration. We introduce a scalable, modular and flexible formulation that can accommodate low-rank and high order terms, that simultaneously selects the plane and estimates the in-plane deformation through a single shot optimization approach. The proposed framework is instantiated into different variants seeking either a compromise between computational efficiency (soft plane selection constraints and approximate definition of the data similarity terms through pair-wise components) or exact definition of the data terms and the constraints on the plane selection. Simulated and real-data in the context of ultrasound and magnetic resonance registration (where both framework instantiations as well as different optimization strategies are considered) demonstrate the potentials of our method.Le recalage d'images déformable est un élément essentiel dans de nombreuses applications cliniques. Dans ce rapport, nous nous intéressons aux modèles graphiques utilisés dans un type de recalage particulier : volume 3D et coupe 2D. Nous établissons un modèle modulaire, flexible et de taille variable qui intègre les potentiels d'ordres supérieurs et résoud simultanément la sélection de plan et l'estimation des transformations intra-plan, en une seule et même optimisation. Le cadre proposé peut être modifié selon plusieurs variantes cherchant soit un compromis entre l'efficacité de calcul (contraintes douces de sélection du plan et calcul approché du terme d'attache aux données par un potentiel à deux nœuds) ou une définition exacte du terme d'attache aux données et des contraintes de la sélection de plan. Nos expériences sur des données simulées et réelles pour des images ultrasons et des IRM (où différentes instanciations et méthodes d'optimisation ont été considérées) prouvent le potentiel de notre méthode

Volumetric MRI Reconstruction from 2D Slices in the Presence of Motion

Despite recent advances in acquisition techniques and reconstruction algorithms, magnetic resonance imaging (MRI) remains challenging in the presence of motion. To mitigate this, ultra-fast two-dimensional (2D) MRI sequences are often used in clinical practice to acquire thick, low-resolution (LR) 2D slices to reduce in-plane motion. The resulting stacks of thick 2D slices typically provide high-quality visualizations when viewed in the in-plane direction. However, the low spatial resolution in the through-plane direction in combination with motion commonly occurring between individual slice acquisitions gives rise to stacks with overall limited geometric integrity. In further consequence, an accurate and reliable diagnosis may be compromised when using such motion-corrupted, thick-slice MRI data. This thesis presents methods to volumetrically reconstruct geometrically consistent, high-resolution (HR) three-dimensional (3D) images from motion-corrupted, possibly sparse, low-resolution 2D MR slices. It focuses on volumetric reconstructions techniques using inverse problem formulations applicable to a broad field of clinical applications in which associated motion patterns are inherently different, but the use of thick-slice MR data is current clinical practice. In particular, volumetric reconstruction frameworks are developed based on slice-to-volume registration with inter-slice transformation regularization and robust, complete-outlier rejection for the reconstruction step that can either avoid or efficiently deal with potential slice-misregistrations. Additionally, this thesis describes efficient Forward-Backward Splitting schemes for image registration for any combination of differentiable (not necessarily convex) similarity measure and convex (not necessarily smooth) regularization with a tractable proximal operator. Experiments are performed on fetal and upper abdominal MRI, and on historical, printed brain MR films associated with a uniquely long-term study dating back to the 1980s. The results demonstrate the broad applicability of the presented frameworks to achieve robust reconstructions with the potential to improve disease diagnosis and patient management in clinical practice

Recalage déformable à base de graphes : mise en correspondance coupe-vers-volume et méthodes contextuelles

Image registration methods, which aim at aligning two or more images into one coordinate system, are among the oldest and most widely used algorithms in computer vision. Registration methods serve to establish correspondence relationships among images (captured at different times, from different sensors or from different viewpoints) which are not obvious for the human eye. A particular type of registration algorithm, known as graph-based deformable registration methods, has become popular during the last decade given its robustness, scalability, efficiency and theoretical simplicity. The range of problems to which it can be adapted is particularly broad. In this thesis, we propose several extensions to the graph-based deformable registration theory, by exploring new application scenarios and developing novel methodological contributions.Our first contribution is an extension of the graph-based deformable registration framework, dealing with the challenging slice-to-volume registration problem. Slice-to-volume registration aims at registering a 2D image within a 3D volume, i.e. we seek a mapping function which optimally maps a tomographic slice to the 3D coordinate space of a given volume. We introduce a scalable, modular and flexible formulation accommodating low-rank and high order terms, which simultaneously selects the plane and estimates the in-plane deformation through a single shot optimization approach. The proposed framework is instantiated into different variants based on different graph topology, label space definition and energy construction. Simulated and real-data in the context of ultrasound and magnetic resonance registration (where both framework instantiations as well as different optimization strategies are considered) demonstrate the potentials of our method.The other two contributions included in this thesis are related to how semantic information can be encompassed within the registration process (independently of the dimensionality of the images). Currently, most of the methods rely on a single metric function explaining the similarity between the source and target images. We argue that incorporating semantic information to guide the registration process will further improve the accuracy of the results, particularly in the presence of semantic labels making the registration a domain specific problem.We consider a first scenario where we are given a classifier inferring probability maps for different anatomical structures in the input images. Our method seeks to simultaneously register and segment a set of input images, incorporating this information within the energy formulation. The main idea is to use these estimated maps of semantic labels (provided by an arbitrary classifier) as a surrogate for unlabeled data, and combine them with population deformable registration to improve both alignment and segmentation.Our last contribution also aims at incorporating semantic information to the registration process, but in a different scenario. In this case, instead of supposing that we have pre-trained arbitrary classifiers at our disposal, we are given a set of accurate ground truth annotations for a variety of anatomical structures. We present a methodological contribution that aims at learning context specific matching criteria as an aggregation of standard similarity measures from the aforementioned annotated data, using an adapted version of the latent structured support vector machine (LSSVM) framework.Les méthodes de recalage d’images, qui ont pour but l’alignement de deux ou plusieurs images dans un même système de coordonnées, sont parmi les algorithmes les plus anciens et les plus utilisés en vision par ordinateur. Les méthodes de recalage servent à établir des correspondances entre des images (prises à des moments différents, par différents senseurs ou avec différentes perspectives), lesquelles ne sont pas évidentes pour l’œil humain. Un type particulier d’algorithme de recalage, connu comme « les méthodes de recalage déformables à l’aide de modèles graphiques » est devenu de plus en plus populaire ces dernières années, grâce à sa robustesse, sa scalabilité, son efficacité et sa simplicité théorique. La gamme des problèmes auxquels ce type d’algorithme peut être adapté est particulièrement vaste. Dans ce travail de thèse, nous proposons plusieurs extensions à la théorie de recalage déformable à l’aide de modèles graphiques, en explorant de nouvelles applications et en développant des contributions méthodologiques originales.Notre première contribution est une extension du cadre du recalage à l’aide de graphes, en abordant le problème très complexe du recalage d’une tranche avec un volume. Le recalage d’une tranche avec un volume est le recalage 2D dans un volume 3D, comme par exemple le mapping d’une tranche tomographique dans un système de coordonnées 3D d’un volume en particulier. Nos avons proposé une formulation scalable, modulaire et flexible pour accommoder des termes d'ordre élevé et de rang bas, qui peut sélectionner le plan et estimer la déformation dans le plan de manière simultanée par une seule approche d'optimisation. Le cadre proposé est instancié en différentes variantes, basés sur différentes topologies du graph, définitions de l'espace des étiquettes et constructions de l'énergie. Le potentiel de notre méthode a été démontré sur des données réelles ainsi que des données simulées dans le cadre d’une résonance magnétique d’ultrason (où le cadre d’installation et les stratégies d’optimisation ont été considérés).Les deux autres contributions inclues dans ce travail de thèse, sont liées au problème de l’intégration de l’information sémantique dans la procédure de recalage (indépendamment de la dimensionnalité des images). Actuellement, la plupart des méthodes comprennent une seule fonction métrique pour expliquer la similarité entre l’image source et l’image cible. Nous soutenons que l'intégration des informations sémantiques pour guider la procédure de recalage pourra encore améliorer la précision des résultats, en particulier en présence d'étiquettes sémantiques faisant du recalage un problème spécifique adapté à chaque domaine.Nous considérons un premier scénario en proposant un classificateur pour inférer des cartes de probabilité pour les différentes structures anatomiques dans les images d'entrée. Notre méthode vise à recaler et segmenter un ensemble d'images d'entrée simultanément, en intégrant cette information dans la formulation de l'énergie. L'idée principale est d'utiliser ces cartes estimées des étiquettes sémantiques (fournie par un classificateur arbitraire) comme un substitut pour les données non-étiquettées, et les combiner avec le recalage déformable pour améliorer l'alignement ainsi que la segmentation.Notre dernière contribution vise également à intégrer l'information sémantique pour la procédure de recalage, mais dans un scénario différent. Dans ce cas, au lieu de supposer que nous avons des classificateurs arbitraires pré-entraînés à notre disposition, nous considérons un ensemble d’annotations précis (vérité terrain) pour une variété de structures anatomiques. Nous présentons une contribution méthodologique qui vise à l'apprentissage des critères correspondants au contexte spécifique comme une agrégation des mesures de similarité standard à partir des données annotées, en utilisant une adaptation de l’algorithme « Latent Structured Support Vector Machine »