2D-3D Rigid-Body Registration of X-Ray Fluoroscopy and CT Images

The registration of pre-operative volumetric datasets to intra- operative two-dimensional images provides an improved way of verifying patient position and medical instrument loca- tion. In applications from orthopedics to neurosurgery, it has a great value in maintaining up-to-date information about changes due to intervention. We propose a mutual information- based registration algorithm to establish the proper align- ment. For optimization purposes, we compare the perfor- mance of the non-gradient Powell method and two slightly di erent versions of a stochastic gradient ascent strategy: one using a sparsely sampled histogramming approach and the other Parzen windowing to carry out probability density approximation. Our main contribution lies in adopting the stochastic ap- proximation scheme successfully applied in 3D-3D registra- tion problems to the 2D-3D scenario, which obviates the need for the generation of full DRRs at each iteration of pose op- timization. This facilitates a considerable savings in compu- tation expense. We also introduce a new probability density estimator for image intensities via sparse histogramming, de- rive gradient estimates for the density measures required by the maximization procedure and introduce the framework for a multiresolution strategy to the problem. Registration results are presented on uoroscopy and CT datasets of a plastic pelvis and a real skull, and on a high-resolution CT- derived simulated dataset of a real skull, a plastic skull, a plastic pelvis and a plastic lumbar spine segment

Synthèse d’images tomodensitométriques à partir d’IRM par des réseaux adverses génératifs pour le recalage 3D/2D de la colonne vertébrale

L’information structurelle tridimensionnelle apporte une aide précieuse aux procédures or-thopédiques qui, le plus souvent, n’ont à portée de main que des modalités d’imagerie bi-dimensionnelles pour se guider. Non seulement cela aide-t-il à améliorer la précision des manoeuvres, mais cela permet aussi, dans plusieurs cas, de rendre les procédures moins in-vasives. La modalité d’imagerie bidimensionnelle sur laquelle se concentre ce mémoire est la fluoroscopie de bras en C. Celle-ci est très répandue dans les salles opératoires et permet une acquisition rapide et versatile pour guider les procédure orthopédiques. Cette modalité est le plus souvent fusionnée avec la tomodensitométrie par un recalage 2D/3D. Ces deux modalités reposent sur le principe d’absorption de rayons X, ce qui fait que l’on retrouve des similarités dans la géométrie et dans les intensités d’une modalité à l’autre. Cela simplifie le problème. Toutefois, la tomodensitométrie n’o˙re pas les meilleurs contrastes pour visualiser les organes, les nerfs et les tissus mous ; d’autant plus qu’elle cause une irradiation non né-gligeable au patient. En revanche, l’IRM se prête mieux à la visualisation des organes et des tissus mous. Elle a aussi l’avantage de ne pas irradier le patient. Le contraste de celle-ci est très di˙érent de la tomodensitométrie, surtout au niveau de la colonne vertébrale ; ce qui rend le problème de recalage 2D/3D avec la fluoroscopie de bras en C plus diÿcile. Les dernières avancées en apprentissage profond montrent des résultats très prometteurs pour la tache de translation d’image. Ceci est applicable à la génération de tomodensitométrie synthétique à partir d’IRM pour un recalage 2D/3D subséquent. Nous proposons une méthode de recalage 2D/3D par Digitally Reconstructed Radiograph (DRR) entre l’imagerie par résonance magnétique et la fluoroscopie de bras en C, basée sur la synthèse de tomodensitométrie par des méthodes d’apprentissage profond. En premier lieu, nous explorons plusieurs architectures de réseaux adverses génératifs. Ces architectures-là ont montré d’impressionnants résultats pour la translation d’images non médicales. Nous expérimentons avec un ensemble de données public constitué de 18 volumes d’IRM et de CT. Nous constatons que l’architecture CycleGAN généralise mieux à des données non-observées que l’architecture cGAN, et que celle-ci tend à sur-apprendre. Nous introduisons aussi deux nouvelles composantes à l’architecture CycleGAN pour améliorer la résolution tridimension-nelle ainsi que la distribution des intensités pour les CT synthétiques. Enfin, nous e˙ectuons un recalage 2D/3D par DRR en utilisant les tomodensitométries synthétiques, avec une er-reur de recalage de 2.1 ± 0.2mm pour valider notre méthode. La méthode proposée, de par ses composantes que par son application, présente un fort potentiel tant pour la synthèse d’images médicales que pour le recalage multimodal.----------ABSTRACT The structural information in three dimensions brings valuable insight and added precision to orthopedic interventions, which may otherwise only rely on bidimensional imaging modalities. Not only does 3D imaging help improve surgical accuracy, but it also helps reduce invasive-ness. The bidimensional imaging modality on which we focus in this work is the C-Arm fluroscopy. The latter is very common in operating theaters and allows for real-time versatile image acquisition to help guide interventions. That modality is often fused with CT scans to bring the added precision from the third dimension and lift the projective uncertainty on depth. CT and C-Arm fluoroscopy both rely on the physical principle of X-Ray absorption, which allows for their respective geometry and intensity distributions to be strongly corre-lated, and makes the registration problem relatively easier. However, computed tomography does not have the best contrast to visualize organs, nerves and soft tissue. It also involves a non-negligeable radiation dose. On the other hand, MRI allows itself to a better visualization of those organs and of soft tissue. It also has the advantage of not exposing the patient to ionizing radiation. The MRI contrast being very di˙erent than that of the CT, especially for the spine, makes the 2D/3D registration problem much harder. The latest advances in deep learning show promising results for the task of image translation, which is applicable to the generation of a synthetic CT from an MRI, for a subsequent 2D/3D registration to C-Arm fluoroscopy through Digitally Reconstructed Radiographs (DRR). We propose such a method for 2D/3D registration between magnetic resonance imaging and C-Arm fluoroscopy, based on synthetic CT generation using deep learning methods. First, we explore numerous generative adversarial network architectures. Those architectures have shown impressive results for non-medical image translation. We experiment with a public dataset of 18 MRI and CT volumes. We notice that the CycleGAN architecture generalizes better to unseen data than the cGAN architecture does. The latter tends to overfit. We also introduce two new components to the CycleGAN architecture, which improves the tridimen-sional resolution as well as the voxel intensity distribution of the synthetic data. Finally, we perform DRR-based 2D/3D registration using the synthetic CT, and validate our method with a registration error of 2.1 ± 0.2mm. The proposed method, through its components and through its application, o˙ers a strong potential for medical image synthesis as well as multimodal registration

Shape/image registration for medical imaging : novel algorithms and applications.

This dissertation looks at two different categories of the registration approaches: Shape registration, and Image registration. It also considers the applications of these approaches into the medical imaging field. Shape registration is an important problem in computer vision, computer graphics and medical imaging. It has been handled in different manners in many applications like shapebased segmentation, shape recognition, and tracking. Image registration is the process of overlaying two or more images of the same scene taken at different times, from different viewpoints, and/or by different sensors. Many image processing applications like remote sensing, fusion of medical images, and computer-aided surgery need image registration. This study deals with two different applications in the field of medical image analysis. The first one is related to shape-based segmentation of the human vertebral bodies (VBs). The vertebra consists of the VB, spinous, and other anatomical regions. Spinous pedicles, and ribs should not be included in the bone mineral density (BMD) measurements. The VB segmentation is not an easy task since the ribs have similar gray level information. This dissertation investigates two different segmentation approaches. Both of them are obeying the variational shape-based segmentation frameworks. The first approach deals with two dimensional (2D) case. This segmentation approach starts with obtaining the initial segmentation using the intensity/spatial interaction models. Then, shape model is registered to the image domain. Finally, the optimal segmentation is obtained using the optimization of an energy functional which integrating the shape model with the intensity information. The second one is a 3D simultaneous segmentation and registration approach. The information of the intensity is handled by embedding a Willmore flow into the level set segmentation framework. Then the shape variations are estimated using a new distance probabilistic model. The experimental results show that the segmentation accuracy of the framework are much higher than other alternatives. Applications on BMD measurements of vertebral body are given to illustrate the accuracy of the proposed segmentation approach. The second application is related to the field of computer-aided surgery, specifically on ankle fusion surgery. The long-term goal of this work is to apply this technique to ankle fusion surgery to determine the proper size and orientation of the screws that are used for fusing the bones together. In addition, we try to localize the best bone region to fix these screws. To achieve these goals, the 2D-3D registration is introduced. The role of 2D-3D registration is to enhance the quality of the surgical procedure in terms of time and accuracy, and would greatly reduce the need for repeated surgeries; thus, saving the patients time, expense, and trauma

Differential geometry methods for biomedical image processing : from segmentation to 2D/3D registration

This thesis establishes a biomedical image analysis framework for the advanced visualization of biological structures. It consists of two important parts: 1) the segmentation of some structures of interest in 3D medical scans, and 2) the registration of patient-specific 3D models with 2D interventional images. Segmenting biological structures results in 3D computational models that are simple to visualize and that can be analyzed quantitatively. Registering a 3D model with interventional images permits to position the 3D model within the physical world. By combining the information from a 3D model and 2D interventional images, the proposed framework can improve the guidance of surgical intervention by reducing the ambiguities inherent to the interpretation of 2D images. Two specific segmentation problems are considered: 1) the segmentation of large structures with low frequency intensity nonuniformity, and 2) the detection of fine curvilinear structures. First, we directed our attention toward the segmentation of relatively large structures with low frequency intensity nonuniformity. Such structures are important in medical imaging since they are commonly encountered in MRI. Also, the nonuniform diffusion of the contrast agent in some other modalities, such as CTA, leads to structures of nonuniform appearance. A level-set method that uses a local-linear region model is defined, and applied to the challenging problem of segmenting brain tissues in MRI. The unique characteristics of the proposed method permit to account for important image nonuniformity implicitly. To the best of our knowledge, this is the first time a region-based level-set model has been used to perform the segmentation of real world MRI brain scans with convincing results. The second segmentation problem considered is the detection of fine curvilinear structures in 3D medical images. Detecting those structures is crucial since they can represent veins, arteries, bronchi or other important tissues. Unfortunately, most currently available curvilinear structure detection filters incur significant signal lost at bifurcations of two structures. This peculiarity limits the performance of all subsequent processes, whether it be understanding an angiography acquisition, computing an accurate tractography, or automatically classifying the image voxels. This thesis presents a new curvilinear structure detection filter that is robust to the presence of X- and Y-junctions. At the same time, it is conceptually simple and deterministic, and allows for an intuitive representation of the structure’s principal directions. Once a 3D computational model is available, it can be used to enhance surgical guidance. A 2D/3D non-rigid method is proposed that brings a 3D centerline model of the coronary arteries into correspondence with bi-plane fluoroscopic angiograms. The registered model is overlaid on top of the interventional angiograms to provide surgical assistance during image-guided chronic total occlusion procedures, which reduces the uncertainty inherent in 2D interventional images. A fully non-rigid registration model is proposed and used to compensate for any local shape discrepancy. This method is based on a variational framework, and uses a simultaneous matching and reconstruction process. With a typical run time of less than 3 seconds, the algorithms are fast enough for interactive applications

Atlas-based indexing of brain sections via 2-D to 3-D image registration

IEEE Transactions on Biomedical Engineering, 55(1): pp. 147-156.A 2-D to 3-D nonlinear intensity-based registration method is proposed in which the alignment of histological brain sections with a volumetric brain atlas is performed. First, sparsely cut brain sections were linearly matched with an oblique slice automatically extracted from the atlas. Second, a planar-to-curved surface alignment was employed in order to match each section with its corresponding image overlaid on a curved-surface within the atlas. For the latter, a PDE-based registration technique was developed that is driven by a local normalized-mutual-information similarity measure. We demonstrate the method and evaluate its performance with simulated and real data experiments. An atlasguided segmentation of mouse brains’ hippocampal complex, retrieved from the Mouse Brain Library (MBL) database, is demonstrated with the proposed algorithm

Recalage et mise en correspondance d’images tomographiques et de projection: Résultats préliminaires d’une solution hybride en radiochirurgie

A new method for 2D/3D registration, applied to Magnetic Resonance Imaging (3D) and to X-Ray angiography (2D), has been adapted and used for planning treatment in radiosurgery. The imaging flow needed for planning radiosurgery is considerable and using registration technique would make lighter the imaging protocol without restricting planning. We describe the preliminary results of the evaluation giving criteria to compare registration technique and localization using stereotactic frame, which is the gold standard method. Preliminary results obtained during this first step in validating registration put forward which kind of MRI sequence are more suitable to registration.Une méthode de recalage d’images multimodales 2D/3D entre Imagerie par Résonance Magnétique (3D) et à l’angiographie par rayons X (2D) est appliquée à la planification dosimétrique en radiochirurgie. Le flux d’images nécessaires à la réalisation du traitement en radiochirurgie est considérable. La fusion de ces images multimodales dans un espace commun est requise pour la planification. Ainsi, elles nécessitent d’être acquises en utilisant un référentiel dit «stéréotaxique». Cependant, l’utilisation d’algorithmes de recalage dans la phase de planification permet de simplifier les procédures d’imagerie en diminuant l’usage du cadre sans contraindre la planification. Nous proposons ici les résultats préliminaires de l’application du recalage dans un contexte radiochirurgical par comparaison avec la méthode basée sur un repérage stéréotaxique qui constitue le gold standard. Les résultats préliminaires obtenus lors de cette première phase de validation permettent de conclure sur la compatibilité de certaines séquences d’images IRM avec le recalage d’images tomographique et de projectio

Digitally reconstructed wall radiographs

Master'sMASTER OF SCIENC