Deep learning for cardiac image segmentation: A review

Deep learning has become the most widely used approach for cardiac image segmentation in recent years. In this paper, we provide a review of over 100 cardiac image segmentation papers using deep learning, which covers common imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound (US) and major anatomical structures of interest (ventricles, atria and vessels). In addition, a summary of publicly available cardiac image datasets and code repositories are included to provide a base for encouraging reproducible research. Finally, we discuss the challenges and limitations with current deep learning-based approaches (scarcity of labels, model generalizability across different domains, interpretability) and suggest potential directions for future research

Development, Implementation and Pre-clinical Evaluation of Medical Image Computing Tools in Support of Computer-aided Diagnosis: Respiratory, Orthopedic and Cardiac Applications

Over the last decade, image processing tools have become crucial components of all clinical and research efforts involving medical imaging and associated applications. The imaging data available to the radiologists continue to increase their workload, raising the need for efficient identification and visualization of the required image data necessary for clinical assessment. Computer-aided diagnosis (CAD) in medical imaging has evolved in response to the need for techniques that can assist the radiologists to increase throughput while reducing human error and bias without compromising the outcome of the screening, diagnosis or disease assessment. More intelligent, but simple, consistent and less time-consuming methods will become more widespread, reducing user variability, while also revealing information in a more clear, visual way. Several routine image processing approaches, including localization, segmentation, registration, and fusion, are critical for enhancing and enabling the development of CAD techniques. However, changes in clinical workflow require significant adjustments and re-training and, despite the efforts of the academic research community to develop state-of-the-art algorithms and high-performance techniques, their footprint often hampers their clinical use. Currently, the main challenge seems to not be the lack of tools and techniques for medical image processing, analysis, and computing, but rather the lack of clinically feasible solutions that leverage the already developed and existing tools and techniques, as well as a demonstration of the potential clinical impact of such tools. Recently, more and more efforts have been dedicated to devising new algorithms for localization, segmentation or registration, while their potential and much intended clinical use and their actual utility is dwarfed by the scientific, algorithmic and developmental novelty that only result in incremental improvements over already algorithms. In this thesis, we propose and demonstrate the implementation and evaluation of several different methodological guidelines that ensure the development of image processing tools --- localization, segmentation and registration --- and illustrate their use across several medical imaging modalities --- X-ray, computed tomography, ultrasound and magnetic resonance imaging --- and several clinical applications: Lung CT image registration in support for assessment of pulmonary nodule growth rate and disease progression from thoracic CT images. Automated reconstruction of standing X-ray panoramas from multi-sector X-ray images for assessment of long limb mechanical axis and knee misalignment. Left and right ventricle localization, segmentation, reconstruction, ejection fraction measurement from cine cardiac MRI or multi-plane trans-esophageal ultrasound images for cardiac function assessment. When devising and evaluating our developed tools, we use clinical patient data to illustrate the inherent clinical challenges associated with highly variable imaging data that need to be addressed before potential pre-clinical validation and implementation. In an effort to provide plausible solutions to the selected applications, the proposed methodological guidelines ensure the development of image processing tools that help achieve sufficiently reliable solutions that not only have the potential to address the clinical needs, but are sufficiently streamlined to be potentially translated into eventual clinical tools provided proper implementation. G1: Reducing the number of degrees of freedom (DOF) of the designed tool, with a plausible example being avoiding the use of inefficient non-rigid image registration methods. This guideline addresses the risk of artificial deformation during registration and it clearly aims at reducing complexity and the number of degrees of freedom. G2: The use of shape-based features to most efficiently represent the image content, either by using edges instead of or in addition to intensities and motion, where useful. Edges capture the most useful information in the image and can be used to identify the most important image features. As a result, this guideline ensures a more robust performance when key image information is missing. G3: Efficient method of implementation. This guideline focuses on efficiency in terms of the minimum number of steps required and avoiding the recalculation of terms that only need to be calculated once in an iterative process. An efficient implementation leads to reduced computational effort and improved performance. G4: Commence the workflow by establishing an optimized initialization and gradually converge toward the final acceptable result. This guideline aims to ensure reasonable outcomes in consistent ways and it avoids convergence to local minima, while gradually ensuring convergence to the global minimum solution. These guidelines lead to the development of interactive, semi-automated or fully-automated approaches that still enable the clinicians to perform final refinements, while they reduce the overall inter- and intra-observer variability, reduce ambiguity, increase accuracy and precision, and have the potential to yield mechanisms that will aid with providing an overall more consistent diagnosis in a timely fashion

Artificial intelligence in pediatric and adult congenital cardiac MRI: an unmet clinical need.

Cardiac MRI (CMR) allows non-invasive, non-ionizing assessment of cardiac function and anatomy in patients with congenital heart disease (CHD). The utility of CMR as a non-invasive imaging tool for evaluation of CHD have been growing exponentially over the past decade. The algorithms based on artificial intelligence (AI), and in particular, deep learning, have rapidly become a methodology of choice for analyzing CMR. A wide range of applications for AI have been developed to tackle challenges in various aspects of CMR, and significant advances have also been made from image acquisition to image analysis and diagnosis. We include an overview of AI definitions, different architectures, and details on well-known methods. This paper reviews the major deep learning concepts used for analyses of patients with CHD. In the end, we have summarized a list of open challenges and concerns to be considered for future studies

Deep Learning in Cardiology

The medical field is creating large amount of data that physicians are unable to decipher and use efficiently. Moreover, rule-based expert systems are inefficient in solving complicated medical tasks or for creating insights using big data. Deep learning has emerged as a more accurate and effective technology in a wide range of medical problems such as diagnosis, prediction and intervention. Deep learning is a representation learning method that consists of layers that transform the data non-linearly, thus, revealing hierarchical relationships and structures. In this review we survey deep learning application papers that use structured data, signal and imaging modalities from cardiology. We discuss the advantages and limitations of applying deep learning in cardiology that also apply in medicine in general, while proposing certain directions as the most viable for clinical use.Comment: 27 pages, 2 figures, 10 table

Computational Methods for Segmentation of Multi-Modal Multi-Dimensional Cardiac Images

Segmentation of the heart structures helps compute the cardiac contractile function quantified via the systolic and diastolic volumes, ejection fraction, and myocardial mass, representing a reliable diagnostic value. Similarly, quantification of the myocardial mechanics throughout the cardiac cycle, analysis of the activation patterns in the heart via electrocardiography (ECG) signals, serve as good cardiac diagnosis indicators. Furthermore, high quality anatomical models of the heart can be used in planning and guidance of minimally invasive interventions under the assistance of image guidance. The most crucial step for the above mentioned applications is to segment the ventricles and myocardium from the acquired cardiac image data. Although the manual delineation of the heart structures is deemed as the gold-standard approach, it requires significant time and effort, and is highly susceptible to inter- and intra-observer variability. These limitations suggest a need for fast, robust, and accurate semi- or fully-automatic segmentation algorithms. However, the complex motion and anatomy of the heart, indistinct borders due to blood flow, the presence of trabeculations, intensity inhomogeneity, and various other imaging artifacts, makes the segmentation task challenging. In this work, we present and evaluate segmentation algorithms for multi-modal, multi-dimensional cardiac image datasets. Firstly, we segment the left ventricle (LV) blood-pool from a tri-plane 2D+time trans-esophageal (TEE) ultrasound acquisition using local phase based filtering and graph-cut technique, propagate the segmentation throughout the cardiac cycle using non-rigid registration-based motion extraction, and reconstruct the 3D LV geometry. Secondly, we segment the LV blood-pool and myocardium from an open-source 4D cardiac cine Magnetic Resonance Imaging (MRI) dataset by incorporating average atlas based shape constraint into the graph-cut framework and iterative segmentation refinement. The developed fast and robust framework is further extended to perform right ventricle (RV) blood-pool segmentation from a different open-source 4D cardiac cine MRI dataset. Next, we employ convolutional neural network based multi-task learning framework to segment the myocardium and regress its area, simultaneously, and show that segmentation based computation of the myocardial area is significantly better than that regressed directly from the network, while also being more interpretable. Finally, we impose a weak shape constraint via multi-task learning framework in a fully convolutional network and show improved segmentation performance for LV, RV and myocardium across healthy and pathological cases, as well as, in the challenging apical and basal slices in two open-source 4D cardiac cine MRI datasets. We demonstrate the accuracy and robustness of the proposed segmentation methods by comparing the obtained results against the provided gold-standard manual segmentations, as well as with other competing segmentation methods

Fast fully automatic myocardial segmentation in 4D cine cardiac magnetic resonance datasets

Dissertação de mestrado integrado em Engenharia BiomédicaCardiovascular diseases (CVDs) are the leading cause of death in the world, representing 30% of all global deaths. Among others, assessment of the left ventricular (LV) morphology and global function using non-invasive cardiac imaging is an interesting technique for diagnosis and treatment follow-up of patients with CVDs. Nowadays, cardiac magnetic resonance (CMR) imaging is the gold-standard technique for the quantification of LV volumes, mass and ejection fraction, requiring the delineation of endocardial and epicardial contours of the left ventricle from cine MR images. In clinical practice, the physicians perform this segmentation manually, being a tedious, time consuming and unpractical task. Even though several (semi-)automated methods have been presented for LV CMR segmentation, fast, automatic and optimal boundaries assessment is still lacking, usually requiring the physician to manually correct the contours. In the present work, we propose a novel fast fully automatic 3D+time LV segmentation framework for CMR datasets. The proposed framework presents three conceptual blocks: 1) an automatic 2D mid-ventricular initialization and segmentation; 2) an automatic stack initialization followed by a 3D segmentation at the end-diastolic phase; and 3) a tracking procedure to delineate both endo and epicardial contours throughout the cardiac cycle. In each block, specific CMR-targeted algorithms are proposed for the different steps required. Hereto, we propose automatic and feasible initialization procedures. Moreover, we adapt the recent B-spline Explicit Active Surfaces (BEAS) framework to the properties of CMR image segmentation by integrating dedicated energy terms and making use of a cylindrical coordinate system that better fits the topology of CMR data. At last, two tracking methods are presented and compared. The proposed framework has been validated on 45 4D CMR datasets from a publicly available database and on a large database from an ongoing multi-center clinical trial with 318 4D datasets. In the technical validation, the framework showed competitive results against the state-of-the-art methods, presenting leading results in both accuracy and average computational time in the common database used for comparative purposes. Moreover, the results in the large scale clinical validation confirmed the high feasibility and robustness of the proposed framework for accurate LV morphology and global function assessment. In combination with the low computational burden of the method, the present methodology seems promising to be used in daily clinical practice.As doenças cardiovasculares (DCVs) são a principal causa de morte no mundo, representando 30% destas a nível global. Na prática clínica, uma técnica empregue no diagnóstico de pacientes com DCVs é a avaliação da morfologia e da função global do ventrículo esquerdo (VE), através de técnicas de imagiologia não-invasivas. Atualmente, a ressonância magnética cardíaca (RMC) é a modalidade de referência na quantificação dos volumes, massa e fração de ejeção do VE, exigindo a delimitação dos contornos do endocárdio e epicárdio a partir de imagens dinâmicas de RMC. Na prática clínica diária, o método preferencial é a segmentação manual. No entanto, esta é uma tarefa demorada, sujeita a erro humano e pouco prática. Apesar de até à data diversos métodos (semi)-automáticos terem sido apresentados para a segmentação do VE em imagens de RMC, ainda não existe um método capaz de avaliar idealmente os contornos de uma forma automática, rápida e precisa, levando a que geralmente o médico necessite de corrigir manualmente os contornos. No presente trabalho é proposta uma nova framework para a segmentação automática do VE em imagens 3D+tempo de RMC. O algoritmo apresenta três blocos principais: 1) uma inicialização e segmentação automática 2D num corte medial do ventrículo; 2) uma inicialização e segmentação tridimensional no volume correspondente ao final da diástole; e 3) um algoritmo de tracking para obter os contornos ao longo de todo o ciclo cardíaco. Neste sentido, são propostos procedimentos de inicialização automática com elevada robustez. Mais ainda, é proposta uma adaptação da recente framework “B-spline Explicit Active Surfaces” (BEAS) com a integração de uma energia específica para as imagens de RMC e utilizando uma formulação cilíndrica para tirar partido da topologia destas imagens. Por último, são apresentados e comparados dois algoritmos de tracking para a obtenção dos contornos ao longo do tempo. A framework proposta foi validada em 45 datasets de RMC provenientes de uma base de dados disponível ao público, bem como numa extensa base de dados com 318 datasets para uma validação clínica. Na avaliação técnica, a framework proposta obteve resultados competitivos quando comparada com outros métodos do estado da arte, tendo alcançado resultados de precisão e tempo computacional superiores a estes. Na validação clínica em larga escala, a framework provou apresentar elevada viabilidade e robustez na avaliação da morfologia e função global do VE. Em combinação com o baixo custo computacional do algoritmo, a presente metodologia apresenta uma perspetiva promissora para a sua aplicação na prática clínica diária