Deep learning for cardiac image segmentation: A review

Deep learning has become the most widely used approach for cardiac image segmentation in recent years. In this paper, we provide a review of over 100 cardiac image segmentation papers using deep learning, which covers common imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound (US) and major anatomical structures of interest (ventricles, atria and vessels). In addition, a summary of publicly available cardiac image datasets and code repositories are included to provide a base for encouraging reproducible research. Finally, we discuss the challenges and limitations with current deep learning-based approaches (scarcity of labels, model generalizability across different domains, interpretability) and suggest potential directions for future research

MyoPS A Benchmark of Myocardial Pathology Segmentation Combining Three-Sequence Cardiac Magnetic Resonance Images

Assessment of myocardial viability is essential in diagnosis and treatment management of patients suffering from myocardial infarction, and classification of pathology on myocardium is the key to this assessment. This work defines a new task of medical image analysis, i.e., to perform myocardial pathology segmentation (MyoPS) combining three-sequence cardiac magnetic resonance (CMR) images, which was first proposed in the MyoPS challenge, in conjunction with MICCAI 2020. The challenge provided 45 paired and pre-aligned CMR images, allowing algorithms to combine the complementary information from the three CMR sequences for pathology segmentation. In this article, we provide details of the challenge, survey the works from fifteen participants and interpret their methods according to five aspects, i.e., preprocessing, data augmentation, learning strategy, model architecture and post-processing. In addition, we analyze the results with respect to different factors, in order to examine the key obstacles and explore potential of solutions, as well as to provide a benchmark for future research. We conclude that while promising results have been reported, the research is still in the early stage, and more in-depth exploration is needed before a successful application to the clinics. Note that MyoPS data and evaluation tool continue to be publicly available upon registration via its homepage (www.sdspeople.fudan.edu.cn/zhuangxiahai/0/myops20/)

Changes and classification in myocardial contractile function in the left ventricle following acute myocardial infarction

In this research, we hypothesized that novel biomechanical parameters are discriminative in patients following acute ST-segment elevation myocardial infarction (STEMI). To identify these biomechanical biomarkers and bring computational biomechanics ‘closer to the clinic’, we applied state-of-the-art multiphysics cardiac modelling combined with advanced machine learning and multivariate statistical inference to a clinical database of myocardial infarction. We obtained data from 11 STEMI patients (ClinicalTrials.gov NCT01717573) and 27 healthy volunteers, and developed personalized mathematical models for the left ventricle (LV) using an immersed boundary method. Subject-specific constitutive parameters were achieved by matching to clinical measurements. We have shown, for the first time, that compared with healthy controls, patients with STEMI exhibited increased LV wall active tension when normalized by systolic blood pressure, which suggests an increased demand on the contractile reserve of remote functional myocardium. The statistical analysis reveals that the required patient-specific contractility, normalized active tension and the systolic myofilament kinematics have the strongest explanatory power for identifying the myocardial function changes post-MI. We further observed a strong correlation between two biomarkers and the changes in LV ejection fraction at six months from baseline (the required contractility (r = − 0.79, p < 0.01) and the systolic myofilament kinematics (r = 0.70, p = 0.02)). The clinical and prognostic significance of these biomechanical parameters merits further scrutinization

Artificial intelligence and cardiovascular magnetic resonance imaging in myocardial infarction patients.

Cardiovascular magnetic resonance (CMR) is an important cardiac imaging tool for assessing the prognostic extent of myocardial injury after myocardial infarction (MI). Within the context of clinical trials, CMR is also useful for assessing the efficacy of potential cardioprotective therapies in reducing MI size and preventing adverse left ventricular (LV) remodelling in reperfused MI. However, manual contouring and analysis can be time-consuming with interobserver and intraobserver variability, which can in turn lead to reduction in accuracy and precision of analysis. There is thus a need to automate CMR scan analysis in MI patients to save time, increase accuracy, increase reproducibility and increase precision. In this regard, automated imaging analysis techniques based on artificial intelligence (AI) that are developed with machine learning (ML), and more specifically deep learning (DL) strategies, can enable efficient, robust, accurate and clinician-friendly tools to be built so as to try and improve both clinician productivity and quality of patient care. In this review, we discuss basic concepts of ML in CMR, important prognostic CMR imaging biomarkers in MI and the utility of current ML applications in their analysis as assessed in research studies. We highlight potential barriers to the mainstream implementation of these automated strategies and discuss related governance and quality control issues. Lastly, we discuss the future role of ML applications in clinical trials and the need for global collaboration in growing this field

A Novel Approach Based on Spatio-temporal Features and Random Forest for Scar Detection Using Cine Cardiac Magnetic Resonance Images

Aim. To identify the presence of scar tissue in the left ventricle from Gadolinium (Gd)-free magnetic resonance cine sequences using a learning-based approach relying on spatio-temporal features. Methods. The spatial and temporal features were extracted using local binary patterns from (i) cine end-diastolic frame and (ii) two parametric images of amplitude and phase wall motion, respectively, and classified with Random Forest. Results. When tested on 328 cine sequences from 40 patients, a recall of 70% was achieved, improving significantly the classification resulting from spatial and temporal features processed separately. Conclusions. The proposed approach showed promising results, paving the way for scar identification from Gd-free images

Improving the domain generalization and robustness of neural networks for medical imaging

Deep neural networks are powerful tools to process medical images, with great potential to accelerate clinical workflows and facilitate large-scale studies. However, in order to achieve satisfactory performance at deployment, these networks generally require massive labeled data collected from various domains (e.g., hospitals, scanners), which is rarely available in practice. The main goal of this work is to improve the domain generalization and robustness of neural networks for medical imaging when labeled data is limited. First, we develop multi-task learning methods to exploit auxiliary data to enhance networks. We first present a multi-task U-net that performs image classification and MR atrial segmentation simultaneously. We then present a shape-aware multi-view autoencoder together with a multi-view U-net, which enables extracting useful shape priors from complementary long-axis views and short-axis views in order to assist the left ventricular myocardium segmentation task on the short-axis MR images. Experimental results show that the proposed networks successfully leverage complementary information from auxiliary tasks to improve model generalization on the main segmentation task. Second, we consider utilizing unlabeled data. We first present an adversarial data augmentation method with bias fields to improve semi-supervised learning for general medical image segmentation tasks. We further explore a more challenging setting where the source and the target images are from different data distributions. We demonstrate that an unsupervised image style transfer method can bridge the domain gap, successfully transferring the knowledge learned from labeled balanced Steady-State Free Precession (bSSFP) images to unlabeled Late Gadolinium Enhancement (LGE) images, achieving state-of-the-art performance on a public multi-sequence cardiac MR segmentation challenge. For scenarios with limited training data from a single domain, we first propose a general training and testing pipeline to improve cardiac image segmentation across various unseen domains. We then present a latent space data augmentation method with a cooperative training framework to further enhance model robustness against unseen domains and imaging artifacts.Open Acces

Automated Diagnosis of Cardiovascular Diseases from Cardiac Magnetic Resonance Imaging Using Deep Learning Models: A Review

In recent years, cardiovascular diseases (CVDs) have become one of the leading causes of mortality globally. CVDs appear with minor symptoms and progressively get worse. The majority of people experience symptoms such as exhaustion, shortness of breath, ankle swelling, fluid retention, and other symptoms when starting CVD. Coronary artery disease (CAD), arrhythmia, cardiomyopathy, congenital heart defect (CHD), mitral regurgitation, and angina are the most common CVDs. Clinical methods such as blood tests, electrocardiography (ECG) signals, and medical imaging are the most effective methods used for the detection of CVDs. Among the diagnostic methods, cardiac magnetic resonance imaging (CMR) is increasingly used to diagnose, monitor the disease, plan treatment and predict CVDs. Coupled with all the advantages of CMR data, CVDs diagnosis is challenging for physicians due to many slices of data, low contrast, etc. To address these issues, deep learning (DL) techniques have been employed to the diagnosis of CVDs using CMR data, and much research is currently being conducted in this field. This review provides an overview of the studies performed in CVDs detection using CMR images and DL techniques. The introduction section examined CVDs types, diagnostic methods, and the most important medical imaging techniques. In the following, investigations to detect CVDs using CMR images and the most significant DL methods are presented. Another section discussed the challenges in diagnosing CVDs from CMR data. Next, the discussion section discusses the results of this review, and future work in CVDs diagnosis from CMR images and DL techniques are outlined. The most important findings of this study are presented in the conclusion section

Comparison of T1-maps and late gadolinium enhancement images in the detection of Myocardial Fibrosis in Hypertrophic Cardiomyopathy

Tese de Mestrado Integrado, Engenharia Biomédica e Biofísica, 2021, Universidade de Lisboa, Faculdade de CiênciasHypertrophic Cardiomyopathy (HCM) is characterized as an abnormal and heterogeneous thickening of the Left Ventricle (LV) wall. HCM is the leading cause of sudden cardiac death in children and young people, with an estimated prevalence of 1:500 in the general population. Myocardial fibrosis is the key histopathological hallmark in HCM and is presented in different patterns: interstitial diffuse fibrosis which, if not treated, evolves to replacement fibrosis. Cardiac Magnetic Resonance (CMR) imaging has been used for the detection and quantification of myocardial fibrosis. The Late Gadolinium Enhancement (LGE) technique is the primary tool for non-invasive tissue characterization, particularly for replacement fibrosis. Conversely, T1 mapping is commonly used for the detection of diffuse interstitial fibrosis, frequently missed using LGE. The clear disadvantage of LGE relies on the need to inject contrast agents that, despite being considered safe, may accumulate in the body for years and potentially cause nephrogenic systemic fibrosis in end-stage chronic kidney disease patients. The capability of native T1 mapping identifying not only diffuse interstitial but also replacement fibrosis would play a pivotal role in HCM diagnosis. The potential of native T1 mapping for a cheaper and non-contrast HCM assessment needs to be further studied. A database of 15 HCM patients, without and with fibrosis, was acquired at Hospital da Luz, Lisboa. In this project, (1) an extensive image preprocessing pipeline was applied to aim for the best possible spatial alignment of the myocardium between the two modalities (native T1 mapping and LGE); (2) the mean native T1 values of individuals without and with the presence of scarred tissue were examined; (3) a pixel-by-pixel analysis was performed to investigate if there is a correlation between fibrotic tissue in LGE and hyperintense regions in native T1 mapping; (4) a Texture Analysis (TA) was performed to study if texture information of native T1 mapping could provide differential diagnosis or prognostic information beyond mean T1 values. The first step was the most longstanding and challenging process. The registration of T1 and LGE images is difficult due to the different intensity profiles. The registration of the myocardial masks using a model with rigid, affine, and free-form deformation transformations revealed to be the best methodology. Mean native T1 values were not increased in patients with scarred tissue. Regarding the third aim, no clear intensity correlation between techniques was observed, which suggests the need for the TA. Seven features (in a total of 350) were selected to distinguish between cardiac segments without and with fibrotic tissue using a ML (Machine Learning) algorithm that finds the features that most contribute to distinguish the two groups. Four first-order features distinguish the cohorts due to the presence of scarred tissue - hyperintense zones - and three texture features suggest that the fibrotic remodeling in the myocardium of HCM patients might be associated with a more heterogeneous tissue texture. A Receiver Operating Characteristics (ROC) analysis was performed and revealed that the Cluster Prominence is the feature that best distinguishes sections without and with fibrotic tissue (accuracy of 70%) but with low sensitivity (65%) and low specifity (64%). A model with the 90th Percentile feature revealed an accuracy of 64%, sensitivity of 71% and specificity of 57%. Studying the Variance feature, the achieved accuracy was 63%, with 66% of sensitivity and 60% of specificity. The remaining features yielded lower accuracy values than the ones previously mentioned, but all of them higher than 50%. The low sensitivity and specificity of the best three models suggest that analysing these values considering these features may help cardiologists to identify focal fibrosis regions and avoid contrast injection methods but may not provide an accurate diagnosis of the presence of fibrotic tissue alone. Further research on the correlation of native T1 mapping and LGE cardiac images is highly recommended to develop a contrast-agent-free technology to replace LGE.A Cardiomiopatia Hipertrófica (do inglês, HCM) é descrita por um espessamento anormal e heterogéneo da parede do ventrículo esquerdo (do inglês, LV). A HCM é a principal causa de morte súbita cardíaca em crianças e jovens, com uma prevalência estimada de 1:500 na população em geral. Esta doença é, na sua maioria, hereditária, e causada por variantes nos genes da proteína do sarcómero (predominantemente MYH7 e MYBPC3). A fibrose do miocárdio é a principal marca histopatológica da HCM e apresenta-se em diferentes padrões: fibrose intersticial difusa que, se não tratada, evolui para fibrose focal. A fibrose é caracterizada por um aumento da deposição de colagénio, que afeta a viabilidade do miocárdio. A imagem de Ressonância Magnética Cardíaca (do inglês, CMR) tem sido usada para a deteção e quantificação de fibrose do miocárdio. A técnica de Realce Tardio (do inglês, LGE) é a principal ferramenta para caracterização não invasiva de tecidos, particularmente de fibrose focal. Em contrapartida, o mapeamento T1 é a técnica mais utilizada para deteção de fibrose intersticial difusa, frequentemente não detetada usando LGE. A clara desvantagem do LGE reside na necessidade de injeção de agentes de contraste. Apesar destes agentes serem considerados seguros, frequentemente causam alergias, podem-se acumular no corpo, por anos, e podem causar fibrose sistémica nefrogénica em pacientes com doença renal crónica terminal. A capacidade do mapeamento T1 nativo identificar, não só a fibrose intersticial difusa mas também a fibrose focal, desempenharia um papel fundamental no diagnóstico da HCM. Consequentemente, é de extrema importância estudar o potencial do mapeamento T1 nativo para uma avaliação desta patologia sem contraste e, desta forma, eliminar os riscos associados à injeção de contraste e reduzir os custos e tempo de preparação associados à utilização de gadolínio. Uma base de dados de 15 pacientes com HCM, com e sem fibrose, previamente adquirida no Hospital da Luz, Lisboa, foi analisada. Neste projeto, (1) aplicou-se um extenso conjunto de passos de pré-processamento de imagem para alcançar a melhor técnica possível de alinhamento espacial do miocárdio entre as duas modalidades (mapeamento T1 nativo e Realce Tardio); (2) após a divisão do miocárdio em 6 secções, como sugerido pela American Heart Association, examinaram-se os valores médios de T1, para cada secção, de indivíduos sem e com presença de tecido cicatricial; (3) realizou-se uma análise pixel a pixel para investigar se existe uma correlação entre o tecido fibrótico em LGE e as regiões hiperintensas no mapeamento T1 nativo; (4) realizou-se uma análise de textura para estudar se a informação de textura do mapeamento T1 nativo poderia fornecer um diagnóstico diferencial ou informação prognóstica além dos valores médios de T1 nativo. A primeira etapa revelou ser o processo mais demorado e desafiante. O batimento cardíaco e o ciclo respiratório representam dois desafios no registo de imagens cardíacas. Para além dos comuns desafios em alinhamento de imagens cardíacas da mesma modalidade, alinhar imagens de diferentes modalidades torna-se um processo mais complexo. Em primeiro lugar, o registo de imagens T1 e de LGE é dificultado pelos distintos perfis de intensidade das duas modalidades. Em segundo lugar, a aquisição de imagens de Realce Tardio ocorre cerca de 7 minutos após a aquisição do mapeamento T1, e o movimento dos pacientes durante este intervalo de tempo é uma fonte adicional de erro. Diferentes softwares foram utilizados, e uma imagem sintética ponderada em T1 foi criada, com o intuito de apresentar intensidades mais similares à imagem a ser alinhada (imagem de LGE). O registo das máscaras miocárdicas por meio de um modelo com transformações rígida, afim e deformações livres mostrou ser a melhor metodologia a aplicar. Os valores médios de T1 nativo não aumentaram significativamente em pacientes com tecido cicatricial, apesar de haver um aumento dos valores de T1 nativo em determinadas secções, em cortes basais e intermédios. Relativamente ao terceiro objetivo abordado, não foi observada uma clara correlação de intensidades entre as técnicas, o que reforçou a necessidade de uma análise de textura (do inglês, TA). Esta análise revelou as sete melhores características (num total de 350) que distinguem segmentos cardíacos sem e com tecido fibrótico, aplicando um método de Machine Learning (do inglês, ML) que identificou, sequencialmente, as features que adicionavam mais informação ao modelo que distinguia os dois grupos de segmentos. Quatro características de primeira ordem distinguem os segmentos devido à presença de tecido cicatricial - zonas hiperintensas - e três características de textura sugerem que a remodelação fibrótica no miocárdio de pacientes com HCM pode estar associada a uma textura mais heterogénea. Foi implementada uma análise ao desempenho de modelos com as features selecionadas, que revelou que a Cluster Prominence é a característica que melhor distingue secções sem e com tecido fibrótico, apesar de com baixa sensibilidade (65%) e baixa especificidade (64%). Um modelo que analisa o Percentil 90 revelou uma precisão de 64%, sensibilidade de 71% e especificidade de 57%. No estudo da Variância, a precisão foi de 63%, a sensibilidade 66% e a especificidade 60%. As restantes features apresentaram valores de precisão inferiores aos mencionados mas acima de 50%. Um modelo com a combinação das sete features selecionadas não melhorou a performance do modelo (precisão de 62%, sensibilidade de 75% e 49% de especificidade). A baixa sensibilidade e especificidade sugerem que a análise desses valores nessas características pode ajudar os cardiologistas a identificar regiões focais de fibrose e evitar métodos de injeção de contraste, mas pode não fornecer um diagnóstico preciso da presença de tecido fibrótico por si só. Em futuras aquisições, encontrar valores semelhantes nas features acima mencionadas, principalmente na Cluster Prominence, em novos dados, poderia ajudar os cardiologistas a identificar regiões de fibrose focal. Desta forma, não seria necessário analisar imagens de Realce Tardio, o que se traduziria na eliminação de injeção de agentes de contraste. Pesquisas adicionais focadas na correlação do mapeamento T1 nativo e imagens cardíacas de LGE são de extrema importância para desenvolver uma tecnologia independente da injeção de agentes de contraste, que substitua o Realce Tardio