Fast catheter segmentation and tracking based on x-ray fluoroscopic and echocardiographic modalities for catheter-based cardiac minimally invasive interventions

X-ray fluoroscopy and echocardiography imaging (ultrasound, US) are two imaging modalities that are widely used in cardiac catheterization. For these modalities, a fast, accurate and stable algorithm for the detection and tracking of catheters is required to allow clinicians to observe the catheter location in real-time. Currently X-ray fluoroscopy is routinely used as the standard modality in catheter ablation interventions. However, it lacks the ability to visualize soft tissue and uses harmful radiation. US does not have these limitations but often contains acoustic artifacts and has a small field of view. These make the detection and tracking of the catheter in US very challenging. The first contribution in this thesis is a framework which combines Kalman filter and discrete optimization for multiple catheter segmentation and tracking in X-ray images. Kalman filter is used to identify the whole catheter from a single point detected on the catheter in the first frame of a sequence of x-ray images. An energy-based formulation is developed that can be used to track the catheters in the following frames. We also propose a discrete optimization for minimizing the energy function in each frame of the X-ray image sequence. Our approach is robust to tangential motion of the catheter and combines the tubular and salient feature measurements into a single robust and efficient framework. The second contribution is an algorithm for catheter extraction in 3D ultrasound images based on (a) the registration between the X-ray and ultrasound images and (b) the segmentation of the catheter in X-ray images. The search space for the catheter extraction in the ultrasound images is constrained to lie on or close to a curved surface in the ultrasound volume. The curved surface corresponds to the back-projection of the extracted catheter from the X-ray image to the ultrasound volume. Blob-like features are detected in the US images and organized in a graphical model. The extracted catheter is modelled as the optimal path in this graphical model. Both contributions allow the use of ultrasound imaging for the improved visualization of soft tissue. However, X-ray imaging is still required for each ultrasound frame and the amount of X-ray exposure has not been reduced. The final contribution in this thesis is a system that can track the catheter in ultrasound volumes automatically without the need for X-ray imaging during the tracking. Instead X-ray imaging is only required for the system initialization and for recovery from tracking failures. This allows a significant reduction in the amount of X-ray exposure for patient and clinicians.Open Acces

Non-Classical Nucleation Phenomena Study And Following Process Monitoring and Optimization in Solution Crystallization Process

Nucleation is a crucial step in the solution crystallization process. Despite their good development, classical nucleation theory and two-step nucleation theory cannot explain all the nucleation phenomena, especially for the non-classical nucleation phenomena which include oiling out, gelation and non-monotonic nucleation. Accordingly, for the non-classical nucleation systems, the crystallization processes are seldom designed based on the nucleation monitoring and supervision. In this thesis, crystallization process optimization was conducted to study the mechanism of non-classical nucleation phenomena and in-line process monitoring technology development. Two kinds of non-classical nucleation phenomena with non-monotonic nucleation rate and gel formation were investigated, and accordingly, two nucleation pathways that self-induced nucleation and jellylike phase mediated nucleation were proposed based on the analysis of in-line spectral monitoring and off-line sample characterizations. Results indicated the agitation level would affect the pre-nucleation clusters’ existence in the non-monotonic nucleation system, and the properties of solvent determined the formation of jellylike phase and the transformation to crystals. Motion-based objects tracking model and the state-of-the-art neural network Mask R-CNN were introduced to monitor the onset of nucleation and following the crystallization process. Combined with a cost-effective camera probe, the developed real-time tracking system can detect the nucleation onset accurately even with ultrasonic irradiation and can extract much more information during the whole crystallization process. Subsequently, ultrasonic irradiation and seeding were used to optimize a non-classical nucleation system that accompanied oiling out phenomenon. Different frequencies and intensities of ultrasonic irradiation and seeds addition time were screened to optimize the nucleation step, which proved their effectiveness of promoting nucleation and narrowing the metastable zone widths of oiling out and nucleation. A fine-tuning of nucleation step was carried out in a mixed suspension mixed product removal (MSMPR)-tubular crystallizer series. The nucleation step was optimized in the MSMPR stage with the aid of principal component analysis, which enabled the growth of crystals in the tubular crystallizer with preferred polymorphism, shape, and size. The study in this thesis provides insights into non-classical nucleation mechanism and nucleation based crystallization process design and optimization

Deep learning for cardiac image segmentation: A review

Deep learning has become the most widely used approach for cardiac image segmentation in recent years. In this paper, we provide a review of over 100 cardiac image segmentation papers using deep learning, which covers common imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound (US) and major anatomical structures of interest (ventricles, atria and vessels). In addition, a summary of publicly available cardiac image datasets and code repositories are included to provide a base for encouraging reproducible research. Finally, we discuss the challenges and limitations with current deep learning-based approaches (scarcity of labels, model generalizability across different domains, interpretability) and suggest potential directions for future research

Coronary Artery Segmentation and Motion Modelling

Conventional coronary artery bypass surgery requires invasive sternotomy and the use of a cardiopulmonary bypass, which leads to long recovery period and has high infectious potential. Totally endoscopic coronary artery bypass (TECAB) surgery based on image guided robotic surgical approaches have been developed to allow the clinicians to conduct the bypass surgery off-pump with only three pin holes incisions in the chest cavity, through which two robotic arms and one stereo endoscopic camera are inserted. However, the restricted field of view of the stereo endoscopic images leads to possible vessel misidentification and coronary artery mis-localization. This results in 20-30% conversion rates from TECAB surgery to the conventional approach. We have constructed patient-specific 3D + time coronary artery and left ventricle motion models from preoperative 4D Computed Tomography Angiography (CTA) scans. Through temporally and spatially aligning this model with the intraoperative endoscopic views of the patient's beating heart, this work assists the surgeon to identify and locate the correct coronaries during the TECAB precedures. Thus this work has the prospect of reducing the conversion rate from TECAB to conventional coronary bypass procedures. This thesis mainly focus on designing segmentation and motion tracking methods of the coronary arteries in order to build pre-operative patient-specific motion models. Various vessel centreline extraction and lumen segmentation algorithms are presented, including intensity based approaches, geometric model matching method and morphology-based method. A probabilistic atlas of the coronary arteries is formed from a group of subjects to facilitate the vascular segmentation and registration procedures. Non-rigid registration framework based on a free-form deformation model and multi-level multi-channel large deformation diffeomorphic metric mapping are proposed to track the coronary motion. The methods are applied to 4D CTA images acquired from various groups of patients and quantitatively evaluated

Fast catheter segmentation from echocardiographic sequences based on segmentation from corresponding X-ray fluoroscopy for cardiac catheterization interventions

© 2014 IEEE. Echocardiography is a potential alternative to X-ray fluoroscopy in cardiac catheterization given its richness in soft tissue information and its lack of ionizing radiation. However, its small field of view and acoustic artifacts make direct automatic segmentation of the catheters very challenging. In this study, a fast catheter segmentation framework for echocardiographic imaging guided by the segmentation of corresponding X-ray fluoroscopic imaging is proposed. The complete framework consists of: 1) catheter initialization in the first X-ray frame; 2) catheter tracking in the rest of the X-ray sequence; 3) fast registration of corresponding X-ray and ultrasound frames; and 4) catheter segmentation in ultrasound images guided by the results of both X-ray tracking and fast registration. The main contributions include: 1) a Kalman filter-based growing strategy with more clinical data evalution; 2) a SURF detector applied in a constrained search space for catheter segmentation in ultrasound images; 3) a two layer hierarchical graph model to integrate and smooth catheter fragments into a complete catheter; and 4) the integration of these components into a system for clinical applications. This framework is evaluated on five sequences of porcine data and four sequences of patient data comprising more than 3000 X-ray frames and more than 1000 ultrasound frames. The results show that our algorithm is able to track the catheter in ultrasound images at 1.3 s per frame, with an error of less than 2 mm. However, although this may satisfy the accuracy for visualization purposes and is also fast, the algorithm still needs to be further accelerated for real-time clinical applications

Towards clinical photoacoustic imaging: developing next-generation endoscopy systems and exploring new contrast agents

Photoacoustic imaging holds great clinical promise because it achieves high-resolution tomographic imaging at depths. Moreover, its strong spectroscopic imaging capability provides a wealth of molecular and functional information based on. Still, despite recent advances, existing photoacoustic systems cannot be readily applied in the clinical environment. This dissertation aims to push the frontier of clinical photoacoustic imaging from both technological and applicational perspectives. The first part of this dissertation describes the development of photoacoustic endoscopy (PAE) systems for imaging human Barrett\u27s esophagus and studying preterm birth. We have developed optical resolution-PAE, which significantly improved lateral resolutions, laparoscopic-PAE, which can guide minimally-invasive surgeries, and catheter-based-PAE, which opens up new opportunities to image the human esophagus. For each system, we tested and optimized the imaging performance in phantom and animal experiments, and then validated them in humans. The second part of the dissertation describes advanced photoacoustic imaging aided by contrast agents. Specifically, gold nanoparticles were used to quantify biological diffusion photoacoustically. In addition, ion nanosensors were applied for continuously monitoring therapeutic lithium concentration in deep tissue in vivo

The Role of Imaging in the Management of Cardiorenal Syndrome

Imaging of the kidney and the heart can provide valuable information in the diagnosis and management of cardiorenal syndromes. Ultrasound- (US-) based imaging (echocardiogram and renal US) is an essential component in the initial diagnostic workup of CRS. Echocardiography provides information on the structure and function of heart, and renal ultrasound is useful in differentiating between acute and chronic kidney disease and excluding certain causes of acute kidney injury such as obstructive uropathy. In this paper we overview the basic concepts of echocardiogram and renal ultrasound and will discuss the clinical utility of these imaging techniques in the management of cardiorenal syndromes. We will also discuss the role of other imaging modalities currently in clinical use such as computerized tomography and magnetic resonance imaging as well as novel techniques such as contrast-enhanced ultrasound imaging