Iterative PnP and its application in 3D-2D vascular image registration for robot navigation

This paper reports on a new real-time robot-centered 3D-2D vascular image alignment algorithm, which is robust to outliers and can align nonrigid shapes. Few works have managed to achieve both real-time and accurate performance for vascular intervention robots. This work bridges high-accuracy 3D-2D registration techniques and computational efficiency requirements in intervention robot applications. We categorize centerline-based vascular 3D-2D image registration problems as an iterative Perspective-n-Point (PnP) problem and propose to use the Levenberg-Marquardt solver on the Lie manifold. Then, the recently developed Reproducing Kernel Hilbert Space (RKHS) algorithm is introduced to overcome the ``big-to-small'' problem in typical robotic scenarios. Finally, an iterative reweighted least squares is applied to solve RKHS-based formulation efficiently. Experiments indicate that the proposed algorithm processes registration over 50 Hz (rigid) and 20 Hz (nonrigid) and obtains competing registration accuracy similar to other works. Results indicate that our Iterative PnP is suitable for future vascular intervention robot applications.Comment: Submitted to ICRA 202

Three-Dimensional Iron Oxide Nanoparticle-Based Contrast-Enhanced Magnetic Resonance Imaging for Characterization of Cerebral Arteriogenesis in the Mouse Neocortex

Purpose: Subsurface blood vessels in the cerebral cortex have been identified as a bottleneck in cerebral perfusion with the potential for collateral remodeling. However, valid techniques for non-invasive, longitudinal characterization of neocortical microvessels are still lacking. In this study, we validated contrast-enhanced magnetic resonance imaging (CE-MRI) for in vivo characterization of vascular changes in a model of spontaneous collateral outgrowth following chronic cerebral hypoperfusion. Methods: C57BL/6J mice were randomly assigned to unilateral internal carotid artery occlusion or sham surgery and after 21 days, CE-MRI based on T2*-weighted imaging was performed using ultra-small superparamagnetic iron oxide nanoparticles to obtain subtraction angiographies and steady-state cerebral blood volume (ss-CBV) maps. First pass dynamic susceptibility contrast MRI (DSC-MRI) was performed for internal validation of ss-CBV. Further validation at the histological level was provided by ex vivo serial two-photon tomography (STP). Results: Qualitatively, an increase in vessel density was observed on CE-MRI subtraction angiographies following occlusion; however, a quantitative vessel tracing analysis was prone to errors in our model. Measurements of ss-CBV reliably identified an increase in cortical vasculature, validated by DSC-MRI and STP. Conclusion: Iron oxide nanoparticle-based ss-CBV serves as a robust, non-invasive imaging surrogate marker for neocortical vessels, with the potential to reduce and refine preclinical models targeting the development and outgrowth of cerebral collateralization

Computerized Analysis of Magnetic Resonance Images to Study Cerebral Anatomy in Developing Neonates

The study of cerebral anatomy in developing neonates is of great importance for the understanding of brain development during the early period of life. This dissertation therefore focuses on three challenges in the modelling of cerebral anatomy in neonates during brain development. The methods that have been developed all use Magnetic Resonance Images (MRI) as source data. To facilitate study of vascular development in the neonatal period, a set of image analysis algorithms are developed to automatically extract and model cerebral vessel trees. The whole process consists of cerebral vessel tracking from automatically placed seed points, vessel tree generation, and vasculature registration and matching. These algorithms have been tested on clinical Time-of- Flight (TOF) MR angiographic datasets. To facilitate study of the neonatal cortex a complete cerebral cortex segmentation and reconstruction pipeline has been developed. Segmentation of the neonatal cortex is not effectively done by existing algorithms designed for the adult brain because the contrast between grey and white matter is reversed. This causes pixels containing tissue mixtures to be incorrectly labelled by conventional methods. The neonatal cortical segmentation method that has been developed is based on a novel expectation-maximization (EM) method with explicit correction for mislabelled partial volume voxels. Based on the resulting cortical segmentation, an implicit surface evolution technique is adopted for the reconstruction of the cortex in neonates. The performance of the method is investigated by performing a detailed landmark study. To facilitate study of cortical development, a cortical surface registration algorithm for aligning the cortical surface is developed. The method first inflates extracted cortical surfaces and then performs a non-rigid surface registration using free-form deformations (FFDs) to remove residual alignment. Validation experiments using data labelled by an expert observer demonstrate that the method can capture local changes and follow the growth of specific sulcus

Coronary Artery Segmentation and Motion Modelling

Conventional coronary artery bypass surgery requires invasive sternotomy and the use of a cardiopulmonary bypass, which leads to long recovery period and has high infectious potential. Totally endoscopic coronary artery bypass (TECAB) surgery based on image guided robotic surgical approaches have been developed to allow the clinicians to conduct the bypass surgery off-pump with only three pin holes incisions in the chest cavity, through which two robotic arms and one stereo endoscopic camera are inserted. However, the restricted field of view of the stereo endoscopic images leads to possible vessel misidentification and coronary artery mis-localization. This results in 20-30% conversion rates from TECAB surgery to the conventional approach. We have constructed patient-specific 3D + time coronary artery and left ventricle motion models from preoperative 4D Computed Tomography Angiography (CTA) scans. Through temporally and spatially aligning this model with the intraoperative endoscopic views of the patient's beating heart, this work assists the surgeon to identify and locate the correct coronaries during the TECAB precedures. Thus this work has the prospect of reducing the conversion rate from TECAB to conventional coronary bypass procedures. This thesis mainly focus on designing segmentation and motion tracking methods of the coronary arteries in order to build pre-operative patient-specific motion models. Various vessel centreline extraction and lumen segmentation algorithms are presented, including intensity based approaches, geometric model matching method and morphology-based method. A probabilistic atlas of the coronary arteries is formed from a group of subjects to facilitate the vascular segmentation and registration procedures. Non-rigid registration framework based on a free-form deformation model and multi-level multi-channel large deformation diffeomorphic metric mapping are proposed to track the coronary motion. The methods are applied to 4D CTA images acquired from various groups of patients and quantitatively evaluated

Template-based 3D-2D rigid registration of vascular structures in frequency domain from a single view

Image guided interventions in angiography are performed with a real-time X-ray sequences acquired by a C-arm device which provides the surgeon two dimensional visualization needed to guide the surgical instruments. This visualization may be augmented by registering a three dimensional preoperative volume with the interventional images to provide additional information such as depth, removal of occlusions and alternative views of vessel paths. This thesis presents two novel methods for rigid registration of vascular structures in the preoperative volume to the interventional X-ray image for enhancing visualization in Image Guided Interventions. In the first part of this thesis, estimation of rotation and translation are decoupled. Rotation is estimated by comparing rotated projections of the segmented vessels of the volume with segmented X-ray vessels in frequency domain. Translation is then estimated by minimizing the distances and maximizing the overlap ratio between segmented vessels. The registration results are reported in mean Projection Distances. The second part of the thesis adds separation of out-of-plane translation estimation to the first part and replaces segmentation by gradients. Rotation and out-of-plane translation are estimated by comparing rotational projected templates of volume with depth templates formed by scaling the X-ray image in the Fourier Magnitude Domain. The in-plane translation is then estimated by a Fourier Phase correlation. The registration results are evaluated by a Gold Standard dataset on cerebral arteries. This method is robust against occlusions and noises due to its usage of gradients and frequency domain similarity, has high capture range and fast, fixed computation times for every step due to template based framework

Generalizable automated pixel-level structural segmentation of medical and biological data

Over the years, the rapid expansion in imaging techniques and equipments has driven the demand for more automation in handling large medical and biological data sets. A wealth of approaches have been suggested as optimal solutions for their respective imaging types. These solutions span various image resolutions, modalities and contrast (staining) mechanisms. Few approaches generalise well across multiple image types, contrasts or resolution. This thesis proposes an automated pixel-level framework that addresses 2D, 2D+t and 3D structural segmentation in a more generalizable manner, yet has enough adaptability to address a number of specific image modalities, spanning retinal funduscopy, sequential fluorescein angiography and two-photon microscopy. The pixel-level segmentation scheme involves: i ) constructing a phase-invariant orientation field of the local spatial neighbourhood; ii ) combining local feature maps with intensity-based measures in a structural patch context; iii ) using a complex supervised learning process to interpret the combination of all the elements in the patch in order to reach a classification decision. This has the advantage of transferability from retinal blood vessels in 2D to neural structures in 3D. To process the temporal components in non-standard 2D+t retinal angiography sequences, we first introduce a co-registration procedure: at the pairwise level, we combine projective RANSAC with a quadratic homography transformation to map the coordinate systems between any two frames. At the joint level, we construct a hierarchical approach in order for each individual frame to be registered to the global reference intra- and inter- sequence(s). We then take a non-training approach that searches in both the spatial neighbourhood of each pixel and the filter output across varying scales to locate and link microvascular centrelines to (sub-) pixel accuracy. In essence, this \link while extract" piece-wise segmentation approach combines the local phase-invariant orientation field information with additional local phase estimates to obtain a soft classification of the centreline (sub-) pixel locations. Unlike retinal segmentation problems where vasculature is the main focus, 3D neural segmentation requires additional exibility, allowing a variety of structures of anatomical importance yet with different geometric properties to be differentiated both from the background and against other structures. Notably, cellular structures, such as Purkinje cells, neural dendrites and interneurons, all display certain elongation along their medial axes, yet each class has a characteristic shape captured by an orientation field that distinguishes it from other structures. To take this into consideration, we introduce a 5D orientation mapping to capture these orientation properties. This mapping is incorporated into the local feature map description prior to a learning machine. Extensive performance evaluations and validation of each of the techniques presented in this thesis is carried out. For retinal fundus images, we compute Receiver Operating Characteristic (ROC) curves on existing public databases (DRIVE & STARE) to assess and compare our algorithms with other benchmark methods. For 2D+t retinal angiography sequences, we compute the error metrics ("Centreline Error") of our scheme with other benchmark methods. For microscopic cortical data stacks, we present segmentation results on both surrogate data with known ground-truth and experimental rat cerebellar cortex two-photon microscopic tissue stacks.Open Acces

Whole‐brain microcirculation detection after ischemic stroke based on swept‐source optical coherence tomography

The occurrence and development of ischemic stroke are closely related to cerebral blood flow. Real‐time monitoring of cerebral perfusion level is very useful for understanding the mechanisms of the disease. A wide field of view (FOV) is conducive to capturing lesions and observing the progression of the disease. In this paper, we attempt to monitor the whole‐brain microcirculation in middle cerebral artery occlusion (MCAO) rats over time using a wide FOV swept‐source OCT (SS‐OCT) system. A constrained image registration algorithm is used to remove motion artifacts that are prone to occur in a wide FOV angiography. During ischemia, cerebral perfusion levels in the left and right hemispheres, as well as in the whole brain were quantified and compared. Changes in the shape and location of blood vessels were also recorded. The results showed that the trend in cerebral perfusion levels of both hemispheres was highly consistent during MCAO, and the position of the blood vessels varied over time. This work will provide new insights of ischemic stroke and is helpful to assess the effectiveness of potential treatment strategies.En face maximum intensity projections (MIP) of the whole‐brain vascular networks obtained by wide field of view (FOV) swept‐source optical coherence tomography (SS‐OCT) system.Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/151895/1/jbio201900122_am.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/151895/2/jbio201900122.pd

Continuous roadmapping in liver TACE procedures using 2D–3D catheter-based registration

PURPOSE: Fusion of pre/perioperative images and intra-operative images may add relevant information during image-guided procedures. In abdominal procedures, respiratory motion changes the position of organs, and thus accurate image guidance requires a continuous update of the spatial alignment of the (pre/perioperative) information with the organ position during the intervention. METHODS: In this paper, we propose a method to register in real time perioperative 3D rotational angiography images (3DRA) to intra-operative single-plane 2D fluoroscopic images for improved guidance in TACE interventions. The method uses the shape of 3D vessels extracted from the 3DRA and the 2D catheter shape extracted from fluoroscopy. First, the appropriate 3D vessel is selected from the complete vascular tree using a shape similarity metric. Subsequently, the catheter is registered to this vessel, and the 3DRA is visualized based on the registration results. The method is evaluated on simulated data and clinical data. RESULTS: The first selected vessel, ranked with the shape similarity metric, is used more than 39 % in the final registration and the second more than 21 %. The median of the closest corresponding points distance between 2D angiography vessels and projected 3D vessels is 4.7–5.4 mm when using the brute force optimizer and 5.2–6.6 mm when using the Powell optimizer. CONCLUSION: We present a catheter-based registration method to continuously fuse a 3DRA roadmap arterial tree onto 2D fluoroscopic images with an efficient shape similarity