Iterative PnP and its application in 3D-2D vascular image registration for robot navigation

This paper reports on a new real-time robot-centered 3D-2D vascular image alignment algorithm, which is robust to outliers and can align nonrigid shapes. Few works have managed to achieve both real-time and accurate performance for vascular intervention robots. This work bridges high-accuracy 3D-2D registration techniques and computational efficiency requirements in intervention robot applications. We categorize centerline-based vascular 3D-2D image registration problems as an iterative Perspective-n-Point (PnP) problem and propose to use the Levenberg-Marquardt solver on the Lie manifold. Then, the recently developed Reproducing Kernel Hilbert Space (RKHS) algorithm is introduced to overcome the ``big-to-small'' problem in typical robotic scenarios. Finally, an iterative reweighted least squares is applied to solve RKHS-based formulation efficiently. Experiments indicate that the proposed algorithm processes registration over 50 Hz (rigid) and 20 Hz (nonrigid) and obtains competing registration accuracy similar to other works. Results indicate that our Iterative PnP is suitable for future vascular intervention robot applications.Comment: Submitted to ICRA 202

Coronary Artery Segmentation and Motion Modelling

Conventional coronary artery bypass surgery requires invasive sternotomy and the use of a cardiopulmonary bypass, which leads to long recovery period and has high infectious potential. Totally endoscopic coronary artery bypass (TECAB) surgery based on image guided robotic surgical approaches have been developed to allow the clinicians to conduct the bypass surgery off-pump with only three pin holes incisions in the chest cavity, through which two robotic arms and one stereo endoscopic camera are inserted. However, the restricted field of view of the stereo endoscopic images leads to possible vessel misidentification and coronary artery mis-localization. This results in 20-30% conversion rates from TECAB surgery to the conventional approach. We have constructed patient-specific 3D + time coronary artery and left ventricle motion models from preoperative 4D Computed Tomography Angiography (CTA) scans. Through temporally and spatially aligning this model with the intraoperative endoscopic views of the patient's beating heart, this work assists the surgeon to identify and locate the correct coronaries during the TECAB precedures. Thus this work has the prospect of reducing the conversion rate from TECAB to conventional coronary bypass procedures. This thesis mainly focus on designing segmentation and motion tracking methods of the coronary arteries in order to build pre-operative patient-specific motion models. Various vessel centreline extraction and lumen segmentation algorithms are presented, including intensity based approaches, geometric model matching method and morphology-based method. A probabilistic atlas of the coronary arteries is formed from a group of subjects to facilitate the vascular segmentation and registration procedures. Non-rigid registration framework based on a free-form deformation model and multi-level multi-channel large deformation diffeomorphic metric mapping are proposed to track the coronary motion. The methods are applied to 4D CTA images acquired from various groups of patients and quantitatively evaluated

Continuous roadmapping in liver TACE procedures using 2D–3D catheter-based registration

PURPOSE: Fusion of pre/perioperative images and intra-operative images may add relevant information during image-guided procedures. In abdominal procedures, respiratory motion changes the position of organs, and thus accurate image guidance requires a continuous update of the spatial alignment of the (pre/perioperative) information with the organ position during the intervention. METHODS: In this paper, we propose a method to register in real time perioperative 3D rotational angiography images (3DRA) to intra-operative single-plane 2D fluoroscopic images for improved guidance in TACE interventions. The method uses the shape of 3D vessels extracted from the 3DRA and the 2D catheter shape extracted from fluoroscopy. First, the appropriate 3D vessel is selected from the complete vascular tree using a shape similarity metric. Subsequently, the catheter is registered to this vessel, and the 3DRA is visualized based on the registration results. The method is evaluated on simulated data and clinical data. RESULTS: The first selected vessel, ranked with the shape similarity metric, is used more than 39 % in the final registration and the second more than 21 %. The median of the closest corresponding points distance between 2D angiography vessels and projected 3D vessels is 4.7–5.4 mm when using the brute force optimizer and 5.2–6.6 mm when using the Powell optimizer. CONCLUSION: We present a catheter-based registration method to continuously fuse a 3DRA roadmap arterial tree onto 2D fluoroscopic images with an efficient shape similarity

Template-based 3D-2D rigid registration of vascular structures in frequency domain from a single view

Image guided interventions in angiography are performed with a real-time X-ray sequences acquired by a C-arm device which provides the surgeon two dimensional visualization needed to guide the surgical instruments. This visualization may be augmented by registering a three dimensional preoperative volume with the interventional images to provide additional information such as depth, removal of occlusions and alternative views of vessel paths. This thesis presents two novel methods for rigid registration of vascular structures in the preoperative volume to the interventional X-ray image for enhancing visualization in Image Guided Interventions. In the first part of this thesis, estimation of rotation and translation are decoupled. Rotation is estimated by comparing rotated projections of the segmented vessels of the volume with segmented X-ray vessels in frequency domain. Translation is then estimated by minimizing the distances and maximizing the overlap ratio between segmented vessels. The registration results are reported in mean Projection Distances. The second part of the thesis adds separation of out-of-plane translation estimation to the first part and replaces segmentation by gradients. Rotation and out-of-plane translation are estimated by comparing rotational projected templates of volume with depth templates formed by scaling the X-ray image in the Fourier Magnitude Domain. The in-plane translation is then estimated by a Fourier Phase correlation. The registration results are evaluated by a Gold Standard dataset on cerebral arteries. This method is robust against occlusions and noises due to its usage of gradients and frequency domain similarity, has high capture range and fast, fixed computation times for every step due to template based framework

Semiautomated Skeletonization of the Pulmonary Arterial Tree in Micro-CT Images

We present a simple and robust approach that utilizes planar images at different angular rotations combined with unfiltered back-projection to locate the central axes of the pulmonary arterial tree. Three-dimensional points are selected interactively by the user. The computer calculates a sub- volume unfiltered back-projection orthogonal to the vector connecting the two points and centered on the first point. Because more x-rays are absorbed at the thickest portion of the vessel, in the unfiltered back-projection, the darkest pixel is assumed to be the center of the vessel. The computer replaces this point with the newly computer-calculated point. A second back-projection is calculated around the original point orthogonal to a vector connecting the newly-calculated first point and user-determined second point. The darkest pixel within the reconstruction is determined. The computer then replaces the second point with the XYZ coordinates of the darkest pixel within this second reconstruction. Following a vector based on a moving average of previously determined 3- dimensional points along the vessel\u27s axis, the computer continues this skeletonization process until stopped by the user. The computer estimates the vessel diameter along the set of previously determined points using a method similar to the full width-half max algorithm. On all subsequent vessels, the process works the same way except that at each point, distances between the current point and all previously determined points along different vessels are determined. If the difference is less than the previously estimated diameter, the vessels are assumed to branch. This user/computer interaction continues until the vascular tree has been skeletonized

3D registration of MR and X-ray spine images using an articulated model

Présentation: Cet article a été publié dans le journal : Computerised medical imaging and graphics (CMIG). Le but de cet article est de recaler les vertèbres extraites à partir d’images RM avec des vertèbres extraites à partir d’images RX pour des patients scoliotiques, en tenant compte des déformations non-rigides due au changement de posture entre ces deux modalités. À ces fins, une méthode de recalage à l’aide d’un modèle articulé est proposée. Cette méthode a été comparée avec un recalage rigide en calculant l’erreur sur des points de repère, ainsi qu’en calculant la différence entre l’angle de Cobb avant et après recalage. Une validation additionelle de la méthode de recalage présentée ici se trouve dans l’annexe A. Ce travail servira de première étape dans la fusion des images RM, RX et TP du tronc complet. Donc, cet article vérifie l’hypothèse 1 décrite dans la section 3.2.1.Abstract This paper presents a magnetic resonance image (MRI)/X-ray spine registration method that compensates for the change in the curvature of the spine between standing and prone positions for scoliotic patients. MRIs in prone position and X-rays in standing position are acquired for 14 patients with scoliosis. The 3D reconstructions of the spine are then aligned using an articulated model which calculates intervertebral transformations. Results show significant decrease in regis- tration error when the proposed articulated model is compared with rigid registration. The method can be used as a basis for full body MRI/X-ray registration incorporating soft tissues for surgical simulation.Canadian Institute of Health Research (CIHR

Automatic Estimation of Coronary Blood Flow Velocity Step 1 for Developing a Tool to Diagnose Patients With Micro-Vascular Angina Pectoris

Aim: Our aim was to automatically estimate the blood velocity in coronary arteries using cine X-ray angiographic sequence. Estimating the coronary blood velocity is a key approach in investigating patients with angina pectoris and no significant coronary artery disease. Blood velocity estimation is central in assessing coronary flow reserve. Methods and Results: A multi-step automatic method for blood flow velocity estimation based on the information extracted solely from the cine X-ray coronary angiography sequence obtained by invasive selective coronary catheterization was developed. The method includes (1) an iterative process of segmenting coronary arteries modeling and removing the heart motion using a non-rigid registration, (2) measuring the area of the segmented arteries in each frame, (3) fitting the measured sequence of areas with a 7◦ polynomial to find start and stop time of dye propagation, and (4) estimating the blood flow velocity based on the time of the dye propagation and the length of the artery-tree. To evaluate the method, coronary angiography recordings from 21 patients with no obstructive coronary artery disease were used. In addition, coronary flow velocity was measured in the same patients using a modified transthoracic Doppler assessment of the left anterior descending artery. We found a moderate but statistically significant correlation between flow velocity assessed by trans thoracic Doppler and the proposed method applying both Spearman and Pearson tests. Conclusion: Measures of coronary flow velocity using a novel fully automatic method that utilizes the information from the X-ray coronary angiographic sequence were statistically significantly correlated to measurements obtained with transthoracic Doppler recordings.publishedVersio

Computerized Analysis of Magnetic Resonance Images to Study Cerebral Anatomy in Developing Neonates

The study of cerebral anatomy in developing neonates is of great importance for the understanding of brain development during the early period of life. This dissertation therefore focuses on three challenges in the modelling of cerebral anatomy in neonates during brain development. The methods that have been developed all use Magnetic Resonance Images (MRI) as source data. To facilitate study of vascular development in the neonatal period, a set of image analysis algorithms are developed to automatically extract and model cerebral vessel trees. The whole process consists of cerebral vessel tracking from automatically placed seed points, vessel tree generation, and vasculature registration and matching. These algorithms have been tested on clinical Time-of- Flight (TOF) MR angiographic datasets. To facilitate study of the neonatal cortex a complete cerebral cortex segmentation and reconstruction pipeline has been developed. Segmentation of the neonatal cortex is not effectively done by existing algorithms designed for the adult brain because the contrast between grey and white matter is reversed. This causes pixels containing tissue mixtures to be incorrectly labelled by conventional methods. The neonatal cortical segmentation method that has been developed is based on a novel expectation-maximization (EM) method with explicit correction for mislabelled partial volume voxels. Based on the resulting cortical segmentation, an implicit surface evolution technique is adopted for the reconstruction of the cortex in neonates. The performance of the method is investigated by performing a detailed landmark study. To facilitate study of cortical development, a cortical surface registration algorithm for aligning the cortical surface is developed. The method first inflates extracted cortical surfaces and then performs a non-rigid surface registration using free-form deformations (FFDs) to remove residual alignment. Validation experiments using data labelled by an expert observer demonstrate that the method can capture local changes and follow the growth of specific sulcus

Model of a Vascular C-Arm for 3D Augmented Fluoroscopy in Interventional Radiology

International audienceThis paper deals with the modeling of a vascular C-arm to generate 3D augmented fuoroscopic images in an interventional radiology context. A methodology based on the use of a multi-image calibration is proposed to assess the physical behavior of the C-arm. From the knowledge of the main characteristics of the C-arm, realistic models of the acquisition geometry are proposed. Their accuracy was evaluated and experiments showed that the C-arm geometry can be predicted with a mean 2D reprojection error of 0.5 mm. The interest of 3D augmented uoroscopy is also assessed on a clinical case