Iterative PnP and its application in 3D-2D vascular image registration for robot navigation

This paper reports on a new real-time robot-centered 3D-2D vascular image alignment algorithm, which is robust to outliers and can align nonrigid shapes. Few works have managed to achieve both real-time and accurate performance for vascular intervention robots. This work bridges high-accuracy 3D-2D registration techniques and computational efficiency requirements in intervention robot applications. We categorize centerline-based vascular 3D-2D image registration problems as an iterative Perspective-n-Point (PnP) problem and propose to use the Levenberg-Marquardt solver on the Lie manifold. Then, the recently developed Reproducing Kernel Hilbert Space (RKHS) algorithm is introduced to overcome the ``big-to-small'' problem in typical robotic scenarios. Finally, an iterative reweighted least squares is applied to solve RKHS-based formulation efficiently. Experiments indicate that the proposed algorithm processes registration over 50 Hz (rigid) and 20 Hz (nonrigid) and obtains competing registration accuracy similar to other works. Results indicate that our Iterative PnP is suitable for future vascular intervention robot applications.Comment: Submitted to ICRA 202

A tree-topology preserving pairing for 3D/2D registration

Information Processing in Computer-Assisted Interventions (IPCAI) 2015 Special IssueInternational audiencePurpose: Fusing pre-operative and intra-operative information into a single space aims at taking advantage of two complementary modalities and necessitates a step of registration that must provide good alignment and relevant correspondences. This paper addresses both purposes in the case of 3D/2D vessel tree matching. Method: We propose a registration algorithm endorsing this vascular tree nature by providing a pairing procedure that preserves the tree topology and by integrating this pairing into an iterative algorithm maintaining pairing coherence. In addition, we define two complementary error measures quantifying the resulting alignment error and pairing error. Both are based on manual ground-truth that is independent of the type of transformation to retrieve. Results: Experiments were conducted on a database of 63 clinical cases, evaluating robustness and accuracy of our approach with respect to the iterative closest point algorithm. Conclusion: The proposed method exhibits good results both in term of pairing and alignment as well as low sensitivity to rotations to be compensated (up to 30 degrees)

Continuous roadmapping in liver TACE procedures using 2D–3D catheter-based registration

PURPOSE: Fusion of pre/perioperative images and intra-operative images may add relevant information during image-guided procedures. In abdominal procedures, respiratory motion changes the position of organs, and thus accurate image guidance requires a continuous update of the spatial alignment of the (pre/perioperative) information with the organ position during the intervention. METHODS: In this paper, we propose a method to register in real time perioperative 3D rotational angiography images (3DRA) to intra-operative single-plane 2D fluoroscopic images for improved guidance in TACE interventions. The method uses the shape of 3D vessels extracted from the 3DRA and the 2D catheter shape extracted from fluoroscopy. First, the appropriate 3D vessel is selected from the complete vascular tree using a shape similarity metric. Subsequently, the catheter is registered to this vessel, and the 3DRA is visualized based on the registration results. The method is evaluated on simulated data and clinical data. RESULTS: The first selected vessel, ranked with the shape similarity metric, is used more than 39 % in the final registration and the second more than 21 %. The median of the closest corresponding points distance between 2D angiography vessels and projected 3D vessels is 4.7–5.4 mm when using the brute force optimizer and 5.2–6.6 mm when using the Powell optimizer. CONCLUSION: We present a catheter-based registration method to continuously fuse a 3DRA roadmap arterial tree onto 2D fluoroscopic images with an efficient shape similarity

Automatic whole heart segmentation based on image registration

Whole heart segmentation can provide important morphological information of the heart, potentially enabling the development of new clinical applications and the planning and guidance of cardiac interventional procedures. This information can be extracted from medical images, such as these of magnetic resonance imaging (MRI), which is becoming a routine modality for the determination of cardiac morphology. Since manual delineation is labour intensive and subject to observer variation, it is highly desirable to develop an automatic method. However, automating the process is complicated by the large shape variation of the heart and limited quality of the data. The aim of this work is to develop an automatic and robust segmentation framework from cardiac MRI while overcoming these difficulties. The main challenge of this segmentation is initialisation of the substructures and inclusion of shape constraints. We propose the locally affine registration method (LARM) and the freeform deformations with adaptive control point status to tackle the challenge. They are applied to the atlas propagation based segmentation framework, where the multi-stage scheme is used to hierarchically increase the degree of freedom. In this segmentation framework, it is also needed to compute the inverse transformation for the LARM registration. Therefore, we propose a generic method, using Dynamic Resampling And distance Weighted interpolation (DRAW), for inverting dense displacements. The segmentation framework is validated on a clinical dataset which includes nine pathologies. To further improve the nonrigid registration against local intensity distortions in the images, we propose a generalised spatial information encoding scheme and the spatial information encoded mutual information (SIEMI) registration. SIEMI registration is applied to the segmentation framework to improve the accuracy. Furthermore, to demonstrate the general applicability of SIEMI registration, we apply it to the registration of cardiac MRI, brain MRI, and the contrast enhanced MRI of the liver. SIEMI registration is shown to perform well and achieve significantly better accuracy compared to the registration using normalised mutual information

A novel MRA-based framework for the detection of changes in cerebrovascular blood pressure.

Background: High blood pressure (HBP) affects 75 million adults and is the primary or contributing cause of mortality in 410,000 adults each year in the United States. Chronic HBP leads to cerebrovascular changes and is a significant contributor for strokes, dementia, and cognitive impairment. Non-invasive measurement of changes in cerebral vasculature and blood pressure (BP) may enable physicians to optimally treat HBP patients. This manuscript describes a method to non-invasively quantify changes in cerebral vasculature and BP using Magnetic Resonance Angiography (MRA) imaging. Methods: MRA images and BP measurements were obtained from patients (n=15, M=8, F=7, Age= 49.2 ± 7.3 years) over a span of 700 days. A novel segmentation algorithm was developed to identify brain vasculature from surrounding tissue. The data was processed to calculate the vascular probability distribution function (PDF); a measure of the vascular diameters in the brain. The initial (day 0) PDF and final (day 700) PDF were used to correlate the changes in cerebral vasculature and BP. Correlation was determined by a mixed effects linear model analysis. Results: The segmentation algorithm had a 99.9% specificity and 99.7% sensitivity in identifying and delineating cerebral vasculature. The PDFs had a statistically significant correlation to BP changes below the circle of Willis (p-value = 0.0007), but not significant (p-value = 0.53) above the circle of Willis, due to smaller blood vessels. Conclusion: Changes in cerebral vasculature and pressure can be non-invasively obtained through MRA image analysis, which may be a useful tool for clinicians to optimize medical management of HBP

Coronary Artery Segmentation and Motion Modelling

Conventional coronary artery bypass surgery requires invasive sternotomy and the use of a cardiopulmonary bypass, which leads to long recovery period and has high infectious potential. Totally endoscopic coronary artery bypass (TECAB) surgery based on image guided robotic surgical approaches have been developed to allow the clinicians to conduct the bypass surgery off-pump with only three pin holes incisions in the chest cavity, through which two robotic arms and one stereo endoscopic camera are inserted. However, the restricted field of view of the stereo endoscopic images leads to possible vessel misidentification and coronary artery mis-localization. This results in 20-30% conversion rates from TECAB surgery to the conventional approach. We have constructed patient-specific 3D + time coronary artery and left ventricle motion models from preoperative 4D Computed Tomography Angiography (CTA) scans. Through temporally and spatially aligning this model with the intraoperative endoscopic views of the patient's beating heart, this work assists the surgeon to identify and locate the correct coronaries during the TECAB precedures. Thus this work has the prospect of reducing the conversion rate from TECAB to conventional coronary bypass procedures. This thesis mainly focus on designing segmentation and motion tracking methods of the coronary arteries in order to build pre-operative patient-specific motion models. Various vessel centreline extraction and lumen segmentation algorithms are presented, including intensity based approaches, geometric model matching method and morphology-based method. A probabilistic atlas of the coronary arteries is formed from a group of subjects to facilitate the vascular segmentation and registration procedures. Non-rigid registration framework based on a free-form deformation model and multi-level multi-channel large deformation diffeomorphic metric mapping are proposed to track the coronary motion. The methods are applied to 4D CTA images acquired from various groups of patients and quantitatively evaluated

Automated Analysis of 3D Stress Echocardiography

__Abstract__ The human circulatory system consists of the heart, blood, arteries, veins and capillaries. The heart is the muscular organ which pumps the blood through the human body (Fig. 1.1,1.2). Deoxygenated blood flows through the right atrium into the right ventricle, which pumps the blood into the pulmonary arteries. The blood is carried to the lungs, where it passes through a capillary network that enables the release of carbon dioxide and the uptake of oxygen. Oxygenated blood then returns to the heart via the pulmonary veins and flows from the left atrium into the left ventricle. The left ventricle then pumps the blood through the aorta, the major artery which supplies blood to the rest of the body [Drake et a!., 2005; Guyton and Halt 1996]. Therefore, it is vital that the cardiovascular system remains healthy. Disease of the cardiovascular system, if untreated, ultimately leads to the failure of other organs and death

Combinatorial optimisation for arterial image segmentation.

Cardiovascular disease is one of the leading causes of the mortality in the western world. Many imaging modalities have been used to diagnose cardiovascular diseases. However, each has different forms of noise and artifacts that make the medical image analysis field important and challenging. This thesis is concerned with developing fully automatic segmentation methods for cross-sectional coronary arterial imaging in particular, intra-vascular ultrasound and optical coherence tomography, by incorporating prior and tracking information without any user intervention, to effectively overcome various image artifacts and occlusions. Combinatorial optimisation methods are proposed to solve the segmentation problem in polynomial time. A node-weighted directed graph is constructed so that the vessel border delineation is considered as computing a minimum closed set. A set of complementary edge and texture features is extracted. Single and double interface segmentation methods are introduced. Novel optimisation of the boundary energy function is proposed based on a supervised classification method. Shape prior model is incorporated into the segmentation framework based on global and local information through the energy function design and graph construction. A combination of cross-sectional segmentation and longitudinal tracking is proposed using the Kalman filter and the hidden Markov model. The border is parameterised using the radial basis functions. The Kalman filter is used to adapt the inter-frame constraints between every two consecutive frames to obtain coherent temporal segmentation. An HMM-based border tracking method is also proposed in which the emission probability is derived from both the classification-based cost function and the shape prior model. The optimal sequence of the hidden states is computed using the Viterbi algorithm. Both qualitative and quantitative results on thousands of images show superior performance of the proposed methods compared to a number of state-of-the-art segmentation methods