Automatic segmentation of wall structures from cardiac images

One important topic in medical image analysis is segmenting wall structures from different cardiac medical imaging modalities such as computed tomography (CT) and magnetic resonance imaging (MRI). This task is typically done by radiologists either manually or semi-automatically, which is a very time-consuming process. To reduce the laborious human efforts, automatic methods have become popular in this research. In this thesis, features insensitive to data variations are explored to segment the ventricles from CT images and extract the left atrium from MR images. As applications, the segmentation results are used to facilitate cardiac disease analysis. Specifically, 1. An automatic method is proposed to extract the ventricles from CT images by integrating surface decomposition with contour evolution techniques. In particular, the ventricles are first identified on a surface extracted from patient-specific image data. Then, the contour evolution is employed to refine the identified ventricles. The proposed method is robust to variations of ventricle shapes, volume coverages, and image quality. 2. A variational region-growing method is proposed to segment the left atrium from MR images. Because of the localized property of this formulation, the proposed method is insensitive to data variabilities that are hard to handle by globalized methods. 3. In applications, a geometrical computational framework is proposed to estimate the myocardial mass at risk caused by stenoses. In addition, the segmentation of the left atrium is used to identify scars for MR images of post-ablation.PhDCommittee Chair: Yezzi, Anthony; Committee Co-Chair: Tannenbaum, Allen; Committee Member: Egerstedt, Magnus ; Committee Member: Fedele, Francesco ; Committee Member: Stillman, Arthur; Committee Member: Vela,Patrici

Medical Image Analysis on Left Atrial LGE MRI for Atrial Fibrillation Studies: A Review

Late gadolinium enhancement magnetic resonance imaging (LGE MRI) is commonly used to visualize and quantify left atrial (LA) scars. The position and extent of scars provide important information of the pathophysiology and progression of atrial fibrillation (AF). Hence, LA scar segmentation and quantification from LGE MRI can be useful in computer-assisted diagnosis and treatment stratification of AF patients. Since manual delineation can be time-consuming and subject to intra- and inter-expert variability, automating this computing is highly desired, which nevertheless is still challenging and under-researched. This paper aims to provide a systematic review on computing methods for LA cavity, wall, scar and ablation gap segmentation and quantification from LGE MRI, and the related literature for AF studies. Specifically, we first summarize AF-related imaging techniques, particularly LGE MRI. Then, we review the methodologies of the four computing tasks in detail, and summarize the validation strategies applied in each task. Finally, the possible future developments are outlined, with a brief survey on the potential clinical applications of the aforementioned methods. The review shows that the research into this topic is still in early stages. Although several methods have been proposed, especially for LA segmentation, there is still large scope for further algorithmic developments due to performance issues related to the high variability of enhancement appearance and differences in image acquisition.Comment: 23 page

심장 컴퓨터 단층촬영 영상으로부터 경사도 보조 지역 능동 윤곽 모델을 이용한 심장 영역 자동 분할 기법

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2015. 2. 신영길.The heart is one of the most important human organs, and composed of complex structures. Computed tomography angiography (CTA), magnetic resonance imaging (MRI), and single photon emission computed tomography are widely used, non-invasive cardiac imaging modalities. Compared with other modalities, CTA can provide more detailed anatomic information of the heart chambers, vessels, and coronary arteries due to its higher spatial resolution. To obtain important morphological information of the heart, whole heart segmentation is necessary and it can be used for clinical diagnosis. In this paper, we propose a novel framework to segment the four chambers of the heart automatically. First, the whole heart is coarsely extracted. This is separated into the left and right parts using a geometric analysis based on anatomical information and a subsequent power watershed. Then, the proposed gradient-assisted localized active contour model (GLACM) refines the left and right sides of the heart segmentation accurately. Finally, the left and right sides of the heart are separated into atrium and ventricle by minimizing the proposed split energy function that determines the boundary between the atrium and ventricle based on the shape and intensity of the heart. The main challenge of heart segmentation is to extract four chambers from cardiac CTA which has weak edges or separators. To enhance the accuracy of the heart segmentation, we use region-based information and edge-based information for the robustness of the accuracy in heterogeneous region. Model-based method, which requires a number of training data and proper template model, has been widely used for heat segmentation. It is difficult to model those data, since training data should describe precise heart regions and the number of data should be high in order to produce more accurate segmentation results. Besides, the training data are required to be represented with remarkable features, which are generated by manual setting, and these features must have correspondence for each other. However in our proposed methods, the training data and template model is not necessary. Instead, we use edge, intensity and shape information from cardiac CTA for each chamber segmentation. The intensity information of CTA can be substituted for the shape information of the template model. In addition, we devised adaptive radius function and Gaussian-pyramid edge map for GLACM in order to utilize the edge information effectively and improve the accuracy of segmentation comparison with original localizing region-based active contour model (LACM). Since the radius of LACM affects the overall segmentation performance, we proposed an energy function for changing radius adaptively whether homogeneous or heterogeneous region. Also we proposed split energy function in order to segment four chambers of the heart in cardiac CT images and detects the valve of atrium and ventricle. In experimental results using twenty clinical datasets, the proposed method identified the four chambers accurately and efficiently. We also demonstrated that this approach can assist the cardiologist for the clinical investigations and functional analysis.Contents Chapter 1 Introduction 1 1.1 Background and Motivation 1 1.2 Dissertation Goal 7 1.3 Main Contribtions 9 1.4 Organization of the Dissertation 10 Chapter 2 Related Works 11 2.1 Medical Image Segmentation 11 2.1.1 Classic Methods 11 2.1.2 Variational Methods 15 2.1.3 Image Features of the Curve 21 2.1.4 Combinatorial Methods 25 2.1.5 Difficulty of Segmentation 30 2.2 Heart Segmentation 33 2.2.1 Non-Model-Based Segmentation 34 2.2.2 Unstatistical Model-Based Segmentation 35 2.2.3 Statistical Model-Based Segmentation 37 Chapter 3 Gradient-assisted Localized Active Contour Model 41 3.1 LACM 41 3.2 Gaussian-pyramid Edge Map 46 3.3 Adaptive Radius Function 50 3.4 LACM with Gaussian-pyramid Edge Map and Adaptive Radius Function 52 Chapter 4 Segmentation of Four Chambers of Heart 54 4.1 Overview 54 4.2 Segmentation of Whole Heart 56 4.3 Separation of Left and Right Sides of Heart 59 4.3.1 Extraction of Candidate Regions of LV and RV 60 4.3.2 Detection of Left and Right sides of Heart 62 4.4 Segmentation of Left and Right Sides of Heart 66 4.5 Separation of Atrium and Ventricle from Heart 69 4.5.1 Calculation of Principal Axes of Left and Right Sides of Heart 69 4.5.2 Detection of Separation Plane Using Split Energy Function 70 Chapter 5 Experiments 74 5.1 Performance Evaluation 74 5.2 Comparison with Conventional Method 79 5.3 Parametric Study 84 5.4 Computational Performance 85 Chapter 6 Conclusion 86 Bibliography 89Docto

From Fully-Supervised Single-Task to Semi-Supervised Multi-Task Deep Learning Architectures for Segmentation in Medical Imaging Applications

Medical imaging is routinely performed in clinics worldwide for the diagnosis and treatment of numerous medical conditions in children and adults. With the advent of these medical imaging modalities, radiologists can visualize both the structure of the body as well as the tissues within the body. However, analyzing these high-dimensional (2D/3D/4D) images demands a significant amount of time and effort from radiologists. Hence, there is an ever-growing need for medical image computing tools to extract relevant information from the image data to help radiologists perform efficiently. Image analysis based on machine learning has pivotal potential to improve the entire medical imaging pipeline, providing support for clinical decision-making and computer-aided diagnosis. To be effective in addressing challenging image analysis tasks such as classification, detection, registration, and segmentation, specifically for medical imaging applications, deep learning approaches have shown significant improvement in performance. While deep learning has shown its potential in a variety of medical image analysis problems including segmentation, motion estimation, etc., generalizability is still an unsolved problem and many of these successes are achieved at the cost of a large pool of datasets. For most practical applications, getting access to a copious dataset can be very difficult, often impossible. Annotation is tedious and time-consuming. This cost is further amplified when annotation must be done by a clinical expert in medical imaging applications. Additionally, the applications of deep learning in the real-world clinical setting are still limited due to the lack of reliability caused by the limited prediction capabilities of some deep learning models. Moreover, while using a CNN in an automated image analysis pipeline, it’s critical to understand which segmentation results are problematic and require further manual examination. To this extent, the estimation of uncertainty calibration in a semi-supervised setting for medical image segmentation is still rarely reported. This thesis focuses on developing and evaluating optimized machine learning models for a variety of medical imaging applications, ranging from fully-supervised, single-task learning to semi-supervised, multi-task learning that makes efficient use of annotated training data. The contributions of this dissertation are as follows: (1) developing a fully-supervised, single-task transfer learning for the surgical instrument segmentation from laparoscopic images; and (2) utilizing supervised, single-task, transfer learning for segmenting and digitally removing the surgical instruments from endoscopic/laparoscopic videos to allow the visualization of the anatomy being obscured by the tool. The tool removal algorithms use a tool segmentation mask and either instrument-free reference frames or previous instrument-containing frames to fill in (inpaint) the instrument segmentation mask; (3) developing fully-supervised, single-task learning via efficient weight pruning and learned group convolution for accurate left ventricle (LV), right ventricle (RV) blood pool and myocardium localization and segmentation from 4D cine cardiac MR images; (4) demonstrating the use of our fully-supervised memory-efficient model to generate dynamic patient-specific right ventricle (RV) models from cine cardiac MRI dataset via an unsupervised learning-based deformable registration field; and (5) integrating a Monte Carlo dropout into our fully-supervised memory-efficient model with inherent uncertainty estimation, with the overall goal to estimate the uncertainty associated with the obtained segmentation and error, as a means to flag regions that feature less than optimal segmentation results; (6) developing semi-supervised, single-task learning via self-training (through meta pseudo-labeling) in concert with a Teacher network that instructs the Student network by generating pseudo-labels given unlabeled input data; (7) proposing largely-unsupervised, multi-task learning to demonstrate the power of a simple combination of a disentanglement block, variational autoencoder (VAE), generative adversarial network (GAN), and a conditioning layer-based reconstructor for performing two of the foremost critical tasks in medical imaging — segmentation of cardiac structures and reconstruction of the cine cardiac MR images; (8) demonstrating the use of 3D semi-supervised, multi-task learning for jointly learning multiple tasks in a single backbone module – uncertainty estimation, geometric shape generation, and cardiac anatomical structure segmentation of the left atrial cavity from 3D Gadolinium-enhanced magnetic resonance (GE-MR) images. This dissertation summarizes the impact of the contributions of our work in terms of demonstrating the adaptation and use of deep learning architectures featuring different levels of supervision to build a variety of image segmentation tools and techniques that can be used across a wide spectrum of medical image computing applications centered on facilitating and promoting the wide-spread computer-integrated diagnosis and therapy data science

AtrialGeneral: Domain Generalization for Left Atrial Segmentation of Multi-Center LGE MRIs

Left atrial (LA) segmentation from late gadolinium enhanced magnetic resonance imaging (LGE MRI) is a crucial step needed for planning the treatment of atrial fibrillation. However, automatic LA segmentation from LGE MRI is still challenging, due to the poor image quality, high variability in LA shapes, and unclear LA boundary. Though deep learning-based methods can provide promising LA segmentation results, they often generalize poorly to unseen domains, such as data from different scanners and/or sites. In this work, we collect 210 LGE MRIs from different centers with different levels of image quality. To evaluate the domain generalization ability of models on the LA segmentation task, we employ four commonly used semantic segmentation networks for the LA segmentation from multi-center LGE MRIs. Besides, we investigate three domain generalization strategies, i.e., histogram matching, mutual information based disentangled representation, and random style transfer, where a simple histogram matching is proved to be most effective.Comment: 10 pages, 4 figures, MICCAI202

Computer Vision Techniques for Transcatheter Intervention

Minimally invasive transcatheter technologies have demonstrated substantial promise for the diagnosis and treatment of cardiovascular diseases. For example, TAVI is an alternative to AVR for the treatment of severe aortic stenosis and TAFA is widely used for the treatment and cure of atrial fibrillation. In addition, catheter-based IVUS and OCT imaging of coronary arteries provides important information about the coronary lumen, wall and plaque characteristics. Qualitative and quantitative analysis of these cross-sectional image data will be beneficial for the evaluation and treatment of coronary artery diseases such as atherosclerosis. In all the phases (preoperative, intraoperative, and postoperative) during the transcatheter intervention procedure, computer vision techniques (e.g., image segmentation, motion tracking) have been largely applied in the field to accomplish tasks like annulus measurement, valve selection, catheter placement control, and vessel centerline extraction. This provides beneficial guidance for the clinicians in surgical planning, disease diagnosis, and treatment assessment. In this paper, we present a systematical review on these state-of-the-art methods.We aim to give a comprehensive overview for researchers in the area of computer vision on the subject of transcatheter intervention. Research in medical computing is multi-disciplinary due to its nature, and hence it is important to understand the application domain, clinical background, and imaging modality so that methods and quantitative measurements derived from analyzing the imaging data are appropriate and meaningful. We thus provide an overview on background information of transcatheter intervention procedures, as well as a review of the computer vision techniques and methodologies applied in this area

A 0.6V, 8mW 3D Vision Processor for a Navigation Device for the Visually Impaired

This paper presents an energy-efficient computer vision processor for a navigation device for the visually impaired. Utilizing a shared parallel datapath, out-of-order processing and co-optimization with hardware-oriented algorithms, the processor consumes 8mW at 0.6V while processing 30 fps input data stream in real time. The test chip fabricated in 40nm is demonstrated as a core part of a navigation device based on a ToF camera, which successfully detects safe areas and obstacles.Texas Instruments Incorporate