Computational Methods for Segmentation of Multi-Modal Multi-Dimensional Cardiac Images

Segmentation of the heart structures helps compute the cardiac contractile function quantified via the systolic and diastolic volumes, ejection fraction, and myocardial mass, representing a reliable diagnostic value. Similarly, quantification of the myocardial mechanics throughout the cardiac cycle, analysis of the activation patterns in the heart via electrocardiography (ECG) signals, serve as good cardiac diagnosis indicators. Furthermore, high quality anatomical models of the heart can be used in planning and guidance of minimally invasive interventions under the assistance of image guidance. The most crucial step for the above mentioned applications is to segment the ventricles and myocardium from the acquired cardiac image data. Although the manual delineation of the heart structures is deemed as the gold-standard approach, it requires significant time and effort, and is highly susceptible to inter- and intra-observer variability. These limitations suggest a need for fast, robust, and accurate semi- or fully-automatic segmentation algorithms. However, the complex motion and anatomy of the heart, indistinct borders due to blood flow, the presence of trabeculations, intensity inhomogeneity, and various other imaging artifacts, makes the segmentation task challenging. In this work, we present and evaluate segmentation algorithms for multi-modal, multi-dimensional cardiac image datasets. Firstly, we segment the left ventricle (LV) blood-pool from a tri-plane 2D+time trans-esophageal (TEE) ultrasound acquisition using local phase based filtering and graph-cut technique, propagate the segmentation throughout the cardiac cycle using non-rigid registration-based motion extraction, and reconstruct the 3D LV geometry. Secondly, we segment the LV blood-pool and myocardium from an open-source 4D cardiac cine Magnetic Resonance Imaging (MRI) dataset by incorporating average atlas based shape constraint into the graph-cut framework and iterative segmentation refinement. The developed fast and robust framework is further extended to perform right ventricle (RV) blood-pool segmentation from a different open-source 4D cardiac cine MRI dataset. Next, we employ convolutional neural network based multi-task learning framework to segment the myocardium and regress its area, simultaneously, and show that segmentation based computation of the myocardial area is significantly better than that regressed directly from the network, while also being more interpretable. Finally, we impose a weak shape constraint via multi-task learning framework in a fully convolutional network and show improved segmentation performance for LV, RV and myocardium across healthy and pathological cases, as well as, in the challenging apical and basal slices in two open-source 4D cardiac cine MRI datasets. We demonstrate the accuracy and robustness of the proposed segmentation methods by comparing the obtained results against the provided gold-standard manual segmentations, as well as with other competing segmentation methods

Role of deep learning techniques in non-invasive diagnosis of human diseases.

Machine learning, a sub-discipline in the domain of artificial intelligence, concentrates on algorithms able to learn and/or adapt their structure (e.g., parameters) based on a set of observed data. The adaptation is performed by optimizing over a cost function. Machine learning obtained a great attention in the biomedical community because it offers a promise for improving sensitivity and/or specificity of detection and diagnosis of diseases. It also can increase objectivity of the decision making, decrease the time and effort on health care professionals during the process of disease detection and diagnosis. The potential impact of machine learning is greater than ever due to the increase in medical data being acquired, the presence of novel modalities being developed and the complexity of medical data. In all of these scenarios, machine learning can come up with new tools for interpreting the complex datasets that confront clinicians. Much of the excitement for the application of machine learning to biomedical research comes from the development of deep learning which is modeled after computation in the brain. Deep learning can help in attaining insights that would be impossible to obtain through manual analysis. Deep learning algorithms and in particular convolutional neural networks are different from traditional machine learning approaches. Deep learning algorithms are known by their ability to learn complex representations to enhance pattern recognition from raw data. On the other hand, traditional machine learning requires human engineering and domain expertise to design feature extractors and structure data. With increasing demands upon current radiologists, there are growing needs for automating the diagnosis. This is a concern that deep learning is able to address. In this dissertation, we present four different successful applications of deep learning for diseases diagnosis. All the work presented in the dissertation utilizes medical images. In the first application, we introduce a deep-learning based computer-aided diagnostic system for the early detection of acute renal transplant rejection. The system is based on the fusion of both imaging markers (apparent diffusion coefficients derived from diffusion-weighted magnetic resonance imaging) and clinical biomarkers (creatinine clearance and serum plasma creatinine). The fused data is then used as an input to train and test a convolutional neural network based classifier. The proposed system is tested on scans collected from 56 subjects from geographically diverse populations and different scanner types/image collection protocols. The overall accuracy of the proposed system is 92.9% with 93.3% sensitivity and 92.3% specificity in distinguishing non-rejected kidney transplants from rejected ones. In the second application, we propose a novel deep learning approach for the automated segmentation and quantification of the LV from cardiac cine MR images. We aimed at achieving lower errors for the estimated heart parameters compared to the previous studies by proposing a novel deep learning segmentation method. Using fully convolutional neural networks, we proposed novel methods for the extraction of a region of interest that contains the left ventricle, and the segmentation of the left ventricle. Following myocardial segmentation, functional and mass parameters of the left ventricle are estimated. Automated Cardiac Diagnosis Challenge dataset was used to validate our framework, which gave better segmentation, accurate estimation of cardiac parameters, and produced less error compared to other methods applied on the same dataset. Furthermore, we showed that our segmentation approach generalizes well across different datasets by testing its performance on a locally acquired dataset. In the third application, we propose a novel deep learning approach for automated quantification of strain from cardiac cine MR images of mice. For strain analysis, we developed a Laplace-based approach to track the LV wall points by solving the Laplace equation between the LV contours of each two successive image frames over the cardiac cycle. Following tracking, the strain estimation is performed using the Lagrangian-based approach. This new automated system for strain analysis was validated by comparing the outcome of these analysis with the tagged MR images from the same mice. There were no significant differences between the strain data obtained from our algorithm using cine compared to tagged MR imaging. In the fourth application, we demonstrate how a deep learning approach can be utilized for the automated classification of kidney histopathological images. Our approach can classify four classes: the fat, the parenchyma, the clear cell renal cell carcinoma, and the unusual cancer which has been discovered recently, called clear cell papillary renal cell carcinoma. Our framework consists of three convolutional neural networks and the whole-slide kidney images were divided into patches with three different sizes to be inputted to the networks. Our approach can provide patch-wise and pixel-wise classification. Our approach classified the four classes accurately and surpassed other state-of-the-art methods such as ResNet (pixel accuracy: 0.89 Resnet18, 0.93 proposed). In conclusion, the results of our proposed systems demonstrate the potential of deep learning for the efficient, reproducible, fast, and affordable disease diagnosis

A Survey on Deep Learning in Medical Image Analysis

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks and provide concise overviews of studies per application area. Open challenges and directions for future research are discussed.Comment: Revised survey includes expanded discussion section and reworked introductory section on common deep architectures. Added missed papers from before Feb 1st 201

합성곱 신경망을 이용한 심폐소생술 중 심초음파 영상에서 실시간 좌심실 부피 추정

학위논문 (석사)-- 서울대학교 대학원 공과대학 협동과정 바이오엔지니어링전공, 2017. 8. 이정찬.This thesis describes the method for real-time segmentation based on echocardiography and three-dimensional transformation model for the left ventricular volume estimation during cardiopulmonary resuscitation (CPR). Because all people have a different structure of thoracic and the position of the heart, it has been required to optimize CPR by a person. As one of the improved methods, bio-signal feedback using echocardiography CPR is carried out. Echocardiography shows how the heart is compressed by chest compression, which directly shows cardiac output. There are two steps in estimating the cardiac output in echocardiography. The left ventricular segmentation from the echocardiography is needed to be segmented. After that, the three-dimensional volume is required to be estimated with two-dimensional segmented images. However, echocardiography during CPR is difficult due to the instability of contact between the transducer and the chest. Moreover, the previous models that map the segmented two-dimensional image to the left ventricular volume assume the heart is contracted isometrically, which is different from the condition of the heart during CPR. To solve these problems, the method for segmentation of the left ventricle stable during CPR and the model that can be applied to CPR conditions is suggested in this dissertation. The convolutional neural network is adopted to the left ventricular segmentation problem. Based on the structure of SegNet that is a fully convolutional network for real-time segmentation, skip connection and dice coefficient are applied to adapt the model to echocardiography domain. The former one helps the network to preserve the information of original images, and the latter one is used for stable segmentation. Moreover, Gated recurrent unit that is used for time series data analysis is applied to reflect the previous frames. The network achieves robust and accurate segmentation by referencing the previous frames in the segmentation of current frame. Comparing to Geodesic Active Contour method that shows the best performance in echocardiography, the proposed algorithm accomplishes higher accuracy and robust to unclear images. The left ventricular model is derived with applying constraints during CPR for modeling problem. The heart during CPR is not contracted. Thus, the assumption of the same surface between the diastolic heart and compressed heart is used. Moreover, the single ellipsoid model with the same length in the minor and intermediate axes is adopted. In comparison experiment to ETCO2 that affects the cardiac output during CPR, the proposed model show much greater correlation than the previous model.1. Introduction 1 1.1. Problems 1 1.2. Aims 2 1.3. Related work 2 1.4. Proposed solution 3 2. Literature review 4 2.1 Image segmentation method 4 2.2 Left ventricle modeling 11 3. Basic theory 14 4. Methods 25 4.1. The left ventricular segmentation 25 4.2. The left ventricle model 34 5. Experiment result 37 5.1. Experiment method 37 5.2. Result 41 5.3. Result analysis 42 6. Discussion 52 6.1. Left ventricle segmentation 52 6.2. Left ventricle model 53 6.3. Combining segmentation and 3D transformation 53 7. Conclusion 57Maste

Advancements and Breakthroughs in Ultrasound Imaging

Ultrasonic imaging is a powerful diagnostic tool available to medical practitioners, engineers and researchers today. Due to the relative safety, and the non-invasive nature, ultrasonic imaging has become one of the most rapidly advancing technologies. These rapid advances are directly related to the parallel advancements in electronics, computing, and transducer technology together with sophisticated signal processing techniques. This book focuses on state of the art developments in ultrasonic imaging applications and underlying technologies presented by leading practitioners and researchers from many parts of the world

Multidimensional embedded MEMS motion detectors for wearable mechanocardiography and 4D medical imaging

Background: Cardiovascular diseases are the number one cause of death. Of these deaths, almost 80% are due to coronary artery disease (CAD) and cerebrovascular disease. Multidimensional microelectromechanical systems (MEMS) sensors allow measuring the mechanical movement of the heart muscle offering an entirely new and innovative solution to evaluate cardiac rhythm and function. Recent advances in miniaturized motion sensors present an exciting opportunity to study novel device-driven and functional motion detection systems in the areas of both cardiac monitoring and biomedical imaging, for example, in computed tomography (CT) and positron emission tomography (PET). Methods: This Ph.D. work describes a new cardiac motion detection paradigm and measurement technology based on multimodal measuring tools — by tracking the heart’s kinetic activity using micro-sized MEMS sensors — and novel computational approaches — by deploying signal processing and machine learning techniques—for detecting cardiac pathological disorders. In particular, this study focuses on the capability of joint gyrocardiography (GCG) and seismocardiography (SCG) techniques that constitute the mechanocardiography (MCG) concept representing the mechanical characteristics of the cardiac precordial surface vibrations. Results: Experimental analyses showed that integrating multisource sensory data resulted in precise estimation of heart rate with an accuracy of 99% (healthy, n=29), detection of heart arrhythmia (n=435) with an accuracy of 95-97%, ischemic disease indication with approximately 75% accuracy (n=22), as well as significantly improved quality of four-dimensional (4D) cardiac PET images by eliminating motion related inaccuracies using MEMS dual gating approach. Tissue Doppler imaging (TDI) analysis of GCG (healthy, n=9) showed promising results for measuring the cardiac timing intervals and myocardial deformation changes. Conclusion: The findings of this study demonstrate clinical potential of MEMS motion sensors in cardiology that may facilitate in time diagnosis of cardiac abnormalities. Multidimensional MCG can effectively contribute to detecting atrial fibrillation (AFib), myocardial infarction (MI), and CAD. Additionally, MEMS motion sensing improves the reliability and quality of cardiac PET imaging.Moniulotteisten sulautettujen MEMS-liiketunnistimien käyttö sydänkardiografiassa sekä lääketieteellisessä 4D-kuvantamisessa Tausta: Sydän- ja verisuonitaudit ovat yleisin kuolinsyy. Näistä kuolemantapauksista lähes 80% johtuu sepelvaltimotaudista (CAD) ja aivoverenkierron häiriöistä. Moniulotteiset mikroelektromekaaniset järjestelmät (MEMS) mahdollistavat sydänlihaksen mekaanisen liikkeen mittaamisen, mikä puolestaan tarjoaa täysin uudenlaisen ja innovatiivisen ratkaisun sydämen rytmin ja toiminnan arvioimiseksi. Viimeaikaiset teknologiset edistysaskeleet mahdollistavat uusien pienikokoisten liiketunnistusjärjestelmien käyttämisen sydämen toiminnan tutkimuksessa sekä lääketieteellisen kuvantamisen, kuten esimerkiksi tietokonetomografian (CT) ja positroniemissiotomografian (PET), tarkkuuden parantamisessa. Menetelmät: Tämä väitöskirjatyö esittelee uuden sydämen kineettisen toiminnan mittaustekniikan, joka pohjautuu MEMS-anturien käyttöön. Uudet laskennalliset lähestymistavat, jotka perustuvat signaalinkäsittelyyn ja koneoppimiseen, mahdollistavat sydämen patologisten häiriöiden havaitsemisen MEMS-antureista saatavista signaaleista. Tässä tutkimuksessa keskitytään erityisesti mekanokardiografiaan (MCG), joihin kuuluvat gyrokardiografia (GCG) ja seismokardiografia (SCG). Näiden tekniikoiden avulla voidaan mitata kardiorespiratorisen järjestelmän mekaanisia ominaisuuksia. Tulokset: Kokeelliset analyysit osoittivat, että integroimalla usean sensorin dataa voidaan mitata syketiheyttä 99% (terveillä n=29) tarkkuudella, havaita sydämen rytmihäiriöt (n=435) 95-97%, tarkkuudella, sekä havaita iskeeminen sairaus noin 75% tarkkuudella (n=22). Lisäksi MEMS-kaksoistahdistuksen avulla voidaan parantaa sydämen 4D PET-kuvan laatua, kun liikeepätarkkuudet voidaan eliminoida paremmin. Doppler-kuvantamisessa (TDI, Tissue Doppler Imaging) GCG-analyysi (terveillä, n=9) osoitti lupaavia tuloksia sydänsykkeen ajoituksen ja intervallien sekä sydänlihasmuutosten mittaamisessa. Päätelmä: Tämän tutkimuksen tulokset osoittavat, että kardiologisilla MEMS-liikeantureilla on kliinistä potentiaalia sydämen toiminnallisten poikkeavuuksien diagnostisoinnissa. Moniuloitteinen MCG voi edistää eteisvärinän (AFib), sydäninfarktin (MI) ja CAD:n havaitsemista. Lisäksi MEMS-liiketunnistus parantaa sydämen PET-kuvantamisen luotettavuutta ja laatua

LV Mass Assessed by Echocardiography and CMR, Cardiovascular Outcomes, and Medical Practice

The authors investigated 3 important areas related to the clinical use of left ventricular mass (LVM): accuracy of assessments by echocardiography and cardiac magnetic resonance (CMR), the ability to predict cardiovascular outcomes, and the comparative value of different indexing methods. The recommended formula for echocardiographic estimation of LVM uses linear measurements and is based on the assumption of the left ventricle (LV) as a prolate ellipsoid of revolution. CMR permits a modeling of the LV free of cardiac geometric assumptions or acoustic window dependency, showing better accuracy and reproducibility. However, echocardiography has lower cost, easier availability, and better tolerability. From the MEDLINE database, 26 longitudinal echocardiographic studies and 5 CMR studies investigating LVM or LV hypertrophy as predictors of death or major cardiovascular outcomes were identified. LVM and LV hypertrophy were reliable cardiovascular risk predictors using both modalities. However, no study directly compared the methods for the ability to predict events, agreement in hypertrophy classification, or performance in cardiovascular risk reclassification. Indexing LVM to body surface area was the earliest normalization process used, but it seems to underestimate the prevalence of hypertrophy in obese and overweight subjects. Dividing LVM by height to the allometric power of 1.7 or 2.7 is the most promising normalization method in terms of practicality and usefulness from a clinical and scientific standpoint for scaling myocardial mass to body size. The measurement of LVM, calculation of LVM index, and classification for LV hypertrophy should be standardized by scientific societies across measurement techniques and adopted by clinicians in risk stratification and therapeutic decision making

A Survey on Deep Learning in Medical Image Registration: New Technologies, Uncertainty, Evaluation Metrics, and Beyond

Over the past decade, deep learning technologies have greatly advanced the field of medical image registration. The initial developments, such as ResNet-based and U-Net-based networks, laid the groundwork for deep learning-driven image registration. Subsequent progress has been made in various aspects of deep learning-based registration, including similarity measures, deformation regularizations, and uncertainty estimation. These advancements have not only enriched the field of deformable image registration but have also facilitated its application in a wide range of tasks, including atlas construction, multi-atlas segmentation, motion estimation, and 2D-3D registration. In this paper, we present a comprehensive overview of the most recent advancements in deep learning-based image registration. We begin with a concise introduction to the core concepts of deep learning-based image registration. Then, we delve into innovative network architectures, loss functions specific to registration, and methods for estimating registration uncertainty. Additionally, this paper explores appropriate evaluation metrics for assessing the performance of deep learning models in registration tasks. Finally, we highlight the practical applications of these novel techniques in medical imaging and discuss the future prospects of deep learning-based image registration