A Survey on Deep Learning in Medical Image Registration: New Technologies, Uncertainty, Evaluation Metrics, and Beyond

Over the past decade, deep learning technologies have greatly advanced the field of medical image registration. The initial developments, such as ResNet-based and U-Net-based networks, laid the groundwork for deep learning-driven image registration. Subsequent progress has been made in various aspects of deep learning-based registration, including similarity measures, deformation regularizations, and uncertainty estimation. These advancements have not only enriched the field of deformable image registration but have also facilitated its application in a wide range of tasks, including atlas construction, multi-atlas segmentation, motion estimation, and 2D-3D registration. In this paper, we present a comprehensive overview of the most recent advancements in deep learning-based image registration. We begin with a concise introduction to the core concepts of deep learning-based image registration. Then, we delve into innovative network architectures, loss functions specific to registration, and methods for estimating registration uncertainty. Additionally, this paper explores appropriate evaluation metrics for assessing the performance of deep learning models in registration tasks. Finally, we highlight the practical applications of these novel techniques in medical imaging and discuss the future prospects of deep learning-based image registration

Multi-Contrast Computed Tomography Atlas of Healthy Pancreas

With the substantial diversity in population demographics, such as differences in age and body composition, the volumetric morphology of pancreas varies greatly, resulting in distinctive variations in shape and appearance. Such variations increase the difficulty at generalizing population-wide pancreas features. A volumetric spatial reference is needed to adapt the morphological variability for organ-specific analysis. Here, we proposed a high-resolution computed tomography (CT) atlas framework specifically optimized for the pancreas organ across multi-contrast CT. We introduce a deep learning-based pre-processing technique to extract the abdominal region of interests (ROIs) and leverage a hierarchical registration pipeline to align the pancreas anatomy across populations. Briefly, DEEDs affine and non-rigid registration are performed to transfer patient abdominal volumes to a fixed high-resolution atlas template. To generate and evaluate the pancreas atlas template, multi-contrast modality CT scans of 443 subjects (without reported history of pancreatic disease, age: 15-50 years old) are processed. Comparing with different registration state-of-the-art tools, the combination of DEEDs affine and non-rigid registration achieves the best performance for the pancreas label transfer across all contrast phases. We further perform external evaluation with another research cohort of 100 de-identified portal venous scans with 13 organs labeled, having the best label transfer performance of 0.504 Dice score in unsupervised setting. The qualitative representation (e.g., average mapping) of each phase creates a clear boundary of pancreas and its distinctive contrast appearance. The deformation surface renderings across scales (e.g., small to large volume) further illustrate the generalizability of the proposed atlas template

Atlas construction and spatial normalisation to facilitate radiation-induced late effects research in childhood cancer

Reducing radiation-induced side effects is one of the most important challenges in paediatric cancer treatment. Recently, there has been growing interest in using spatial normalisation to enable voxel-based analysis of radiation-induced toxicities in a variety of patient groups. The need to consider three-dimensional distribution of doses, rather than dose-volume histograms, is desirable but not yet explored in paediatric populations. In this paper, we investigate the feasibility of atlas construction and spatial normalisation in paediatric radiotherapy. We used planning computed tomography (CT) scans from twenty paediatric patients historically treated with craniospinal irradiation to generate a template CT that is suitable for spatial normalisation. This childhood cancer population representative template was constructed using groupwise image registration. An independent set of 53 subjects from a variety of childhood malignancies was then used to assess the quality of the propagation of new subjects to this common reference space using deformable image registration (i.e., spatial normalisation). The method was evaluated in terms of overall image similarity metrics, contour similarity and preservation of dose-volume properties. After spatial normalisation, we report a dice similarity coefficient of 0.95±0.05, 0.85±0.04, 0.96±0.01, 0.91±0.03, 0.83±0.06 and 0.65±0.16 for brain and spinal canal, ocular globes, lungs, liver, kidneys and bladder. We then demonstrated the potential advantages of an atlas-based approach to study the risk of second malignant neoplasms after radiotherapy. Our findings indicate satisfactory mapping between a heterogeneous group of patients and the template CT. The poorest performance was for organs in the abdominal and pelvic region, likely due to respiratory and physiological motion and to the highly deformable nature of abdominal organs. More specialised algorithms should be explored in the future to improve mapping in these regions. This study is the first step toward voxel-based analysis in radiation-induced toxicities following paediatric radiotherapy

Groupwise registration with global-local graph shrinkage in atlas construction

Graph-based groupwise registration methods are widely used in atlas construction. Given a group of images, a graph is built whose nodes represent the images, and whose edges represent a geodesic path between two nodes. The distribution of images on an image manifold is explored through edge traversal in a graph. The final atlas is a mean image at the population center of the distribution on the manifold. The procedure of warping all images to the mean image turns to dynamic graph shrinkage in which nodes become closer to each other. Most conventional groupwise registration frameworks construct and shrink a graph without considering the local distribution of images on the dataset manifold and the local structure variations between image pairs. Neglecting the local information fundamentally decrease the accuracy and efficiency when population atlases are built for organs with large inter-subject anatomical variabilities. To overcome the problem, this paper proposes a global-local graph shrinkage approach that can generate accurate atlas. A connected graph is constructed automatically based on global similarities across the images to explore the global distribution. A local image distribution obtained by image clustering is used to simplify the edges of the constructed graph. Subsequently, local image similarities refine the deformation estimated through global image similarity for each image warping along the graph edges. Through the image warping, the overall simplified graph shrinks gradually to yield the atlas with respecting both global and local features. The proposed method is evaluated on 61 synthetic and 20 clinical liver datasets, and the results are compared with those of six state-of-the-art groupwise registration methods. The experimental results show that the proposed method outperforms non-global-local method approaches in terms of accuracy

Doctor of Philosophy

dissertationThe statistical study of anatomy is one of the primary focuses of medical image analysis. It is well-established that the appropriate mathematical settings for such analyses are Riemannian manifolds and Lie group actions. Statistically defined atlases, in which a mean anatomical image is computed from a collection of static three-dimensional (3D) scans, have become commonplace. Within the past few decades, these efforts, which constitute the field of computational anatomy, have seen great success in enabling quantitative analysis. However, most of the analysis within computational anatomy has focused on collections of static images in population studies. The recent emergence of large-scale longitudinal imaging studies and four-dimensional (4D) imaging technology presents new opportunities for studying dynamic anatomical processes such as motion, growth, and degeneration. In order to make use of this new data, it is imperative that computational anatomy be extended with methods for the statistical analysis of longitudinal and dynamic medical imaging. In this dissertation, the deformable template framework is used for the development of 4D statistical shape analysis, with applications in motion analysis for individualized medicine and the study of growth and disease progression. A new method for estimating organ motion directly from raw imaging data is introduced and tested extensively. Polynomial regression, the staple of curve regression in Euclidean spaces, is extended to the setting of Riemannian manifolds. This polynomial regression framework enables rigorous statistical analysis of longitudinal imaging data. Finally, a new diffeomorphic model of irrotational shape change is presented. This new model presents striking practical advantages over standard diffeomorphic methods, while the study of this new space promises to illuminate aspects of the structure of the diffeomorphism group

Proceedings of the Second International Workshop on Mathematical Foundations of Computational Anatomy (MFCA'08) - Geometrical and Statistical Methods for Modelling Biological Shape Variability

International audienceThe goal of computational anatomy is to analyze and to statistically model the anatomy of organs in different subjects. Computational anatomic methods are generally based on the extraction of anatomical features or manifolds which are then statistically analyzed, often through a non-linear registration. There are nowadays a growing number of methods that can faithfully deal with the underlying biomechanical behavior of intra-subject deformations. However, it is more difficult to relate the anatomies of different subjects. In the absence of any justified physical model, diffeomorphisms provide a general mathematical framework that enforce topological consistency. Working with such infinite dimensional space raises some deep computational and mathematical problems, in particular for doing statistics. Likewise, modeling the variability of surfaces leads to rely on shape spaces that are much more complex than for curves. To cope with these, different methodological and computational frameworks have been proposed (e.g. smooth left-invariant metrics, focus on well-behaved subspaces of diffeomorphisms, modeling surfaces using courants, etc.) The goal of the Mathematical Foundations of Computational Anatomy (MFCA) workshop is to foster the interactions between the mathematical community around shapes and the MICCAI community around computational anatomy applications. It targets more particularly researchers investigating the combination of statistical and geometrical aspects in the modeling of the variability of biological shapes. The workshop aims at being a forum for the exchange of the theoretical ideas and a source of inspiration for new methodological developments in computational anatomy. A special emphasis is put on theoretical developments, applications and results being welcomed as illustrations. Following the very successful first edition of this workshop in 2006 (see http://www.inria.fr/sophia/asclepios/events/MFCA06/), the second edition was held in New-York on September 6, in conjunction with MICCAI 2008. Contributions were solicited in Riemannian and group theoretical methods, Geometric measurements of the anatomy, Advanced statistics on deformations and shapes, Metrics for computational anatomy, Statistics of surfaces. 34 submissions were received, among which 9 were accepted to MICCAI and had to be withdrawn from the workshop. Each of the remaining 25 paper was reviewed by three members of the program committee. To guaranty a high level program, 16 papers only were selected

Identification et caractérisation des conditions aux limites pour des simulations biomécaniques patient-spécifiques

The purpose of the work is to find a way to estimate the boundary conditions of the liver. They play an essential role in forming the predictive capacity of the biomechanical model, but are presented mainly by ligaments, vessels, and surrounding organs, the properties of which are "patient specific" and cannot be measured reliably. We propose to present the boundary conditions as nonlinear springs and estimate their parameters. Firstly, we create a generalized initial approximation using the constitutive law available in the literature and a statistical atlas, obtained from a set of models with segmented ligaments. Then, we correct the approximation based on the nonlinear Kalman filtering approach, which assimilates data obtained from a modality during surgical intervention. To assess the approach, we performed experiments for both synthetic and real data. The results show a certain improvement in simulation accuracy for the cases with estimated boundaries.L'objectif de ce travail est trouvé un moyen d'estimer les conditions aux limites du foie. Elles jouent un rôle essentiel dans la capacité de prédiction du modèle biomécanique, mais sont principalement présentées par les ligaments, les vaisseaux et les organes environnants, dont les propriétés sont "spécifiques au patient" et ne peuvent être mesurées fidèlement. Nous proposons de présenter ces conditions comme des ressorts non linéaires et d'estimer ses paramètres. D’abord, nous créons une approximation initiale en utilisant la loi constitutive disponible dans la littérature et un atlas statistique obtenu à partir des modèles avec des ligaments segmentés. Après, nous la corrigeons basée sur le filtrage de Kalman non linéaire, qui assimile les données acquises d'une modalité pendant la chirurgie. Pour évaluation, nous avons réalisé des expériences avec des données synthétiques et réelles. Les résultats montrent une amélioration de la précision pour les cas avec des limites estimées

Evaluating and Improving 4D-CT Image Segmentation for Lung Cancer Radiotherapy

Lung cancer is a high-incidence disease with low survival despite surgical advances and concurrent chemo-radiotherapy strategies. Image-guided radiotherapy provides for treatment measures, however, significant challenges exist for imaging, treatment planning, and delivery of radiation due to the influence of respiratory motion. 4D-CT imaging is capable of improving image quality of thoracic target volumes influenced by respiratory motion. 4D-CT-based treatment planning strategies requires highly accurate anatomical segmentation of tumour volumes for radiotherapy treatment plan optimization. Variable segmentation of tumour volumes significantly contributes to uncertainty in radiotherapy planning due to a lack of knowledge regarding the exact shape of the lesion and difficulty in quantifying variability. As image-segmentation is one of the earliest tasks in the radiotherapy process, inherent geometric uncertainties affect subsequent stages, potentially jeopardizing patient outcomes. Thus, this work assesses and suggests strategies for mitigation of segmentation-related geometric uncertainties in 4D-CT-based lung cancer radiotherapy at pre- and post-treatment planning stages