Automated Diagnosis of Cardiovascular Diseases from Cardiac Magnetic Resonance Imaging Using Deep Learning Models: A Review

In recent years, cardiovascular diseases (CVDs) have become one of the leading causes of mortality globally. CVDs appear with minor symptoms and progressively get worse. The majority of people experience symptoms such as exhaustion, shortness of breath, ankle swelling, fluid retention, and other symptoms when starting CVD. Coronary artery disease (CAD), arrhythmia, cardiomyopathy, congenital heart defect (CHD), mitral regurgitation, and angina are the most common CVDs. Clinical methods such as blood tests, electrocardiography (ECG) signals, and medical imaging are the most effective methods used for the detection of CVDs. Among the diagnostic methods, cardiac magnetic resonance imaging (CMR) is increasingly used to diagnose, monitor the disease, plan treatment and predict CVDs. Coupled with all the advantages of CMR data, CVDs diagnosis is challenging for physicians due to many slices of data, low contrast, etc. To address these issues, deep learning (DL) techniques have been employed to the diagnosis of CVDs using CMR data, and much research is currently being conducted in this field. This review provides an overview of the studies performed in CVDs detection using CMR images and DL techniques. The introduction section examined CVDs types, diagnostic methods, and the most important medical imaging techniques. In the following, investigations to detect CVDs using CMR images and the most significant DL methods are presented. Another section discussed the challenges in diagnosing CVDs from CMR data. Next, the discussion section discusses the results of this review, and future work in CVDs diagnosis from CMR images and DL techniques are outlined. The most important findings of this study are presented in the conclusion section

From Fully-Supervised Single-Task to Semi-Supervised Multi-Task Deep Learning Architectures for Segmentation in Medical Imaging Applications

Medical imaging is routinely performed in clinics worldwide for the diagnosis and treatment of numerous medical conditions in children and adults. With the advent of these medical imaging modalities, radiologists can visualize both the structure of the body as well as the tissues within the body. However, analyzing these high-dimensional (2D/3D/4D) images demands a significant amount of time and effort from radiologists. Hence, there is an ever-growing need for medical image computing tools to extract relevant information from the image data to help radiologists perform efficiently. Image analysis based on machine learning has pivotal potential to improve the entire medical imaging pipeline, providing support for clinical decision-making and computer-aided diagnosis. To be effective in addressing challenging image analysis tasks such as classification, detection, registration, and segmentation, specifically for medical imaging applications, deep learning approaches have shown significant improvement in performance. While deep learning has shown its potential in a variety of medical image analysis problems including segmentation, motion estimation, etc., generalizability is still an unsolved problem and many of these successes are achieved at the cost of a large pool of datasets. For most practical applications, getting access to a copious dataset can be very difficult, often impossible. Annotation is tedious and time-consuming. This cost is further amplified when annotation must be done by a clinical expert in medical imaging applications. Additionally, the applications of deep learning in the real-world clinical setting are still limited due to the lack of reliability caused by the limited prediction capabilities of some deep learning models. Moreover, while using a CNN in an automated image analysis pipeline, it’s critical to understand which segmentation results are problematic and require further manual examination. To this extent, the estimation of uncertainty calibration in a semi-supervised setting for medical image segmentation is still rarely reported. This thesis focuses on developing and evaluating optimized machine learning models for a variety of medical imaging applications, ranging from fully-supervised, single-task learning to semi-supervised, multi-task learning that makes efficient use of annotated training data. The contributions of this dissertation are as follows: (1) developing a fully-supervised, single-task transfer learning for the surgical instrument segmentation from laparoscopic images; and (2) utilizing supervised, single-task, transfer learning for segmenting and digitally removing the surgical instruments from endoscopic/laparoscopic videos to allow the visualization of the anatomy being obscured by the tool. The tool removal algorithms use a tool segmentation mask and either instrument-free reference frames or previous instrument-containing frames to fill in (inpaint) the instrument segmentation mask; (3) developing fully-supervised, single-task learning via efficient weight pruning and learned group convolution for accurate left ventricle (LV), right ventricle (RV) blood pool and myocardium localization and segmentation from 4D cine cardiac MR images; (4) demonstrating the use of our fully-supervised memory-efficient model to generate dynamic patient-specific right ventricle (RV) models from cine cardiac MRI dataset via an unsupervised learning-based deformable registration field; and (5) integrating a Monte Carlo dropout into our fully-supervised memory-efficient model with inherent uncertainty estimation, with the overall goal to estimate the uncertainty associated with the obtained segmentation and error, as a means to flag regions that feature less than optimal segmentation results; (6) developing semi-supervised, single-task learning via self-training (through meta pseudo-labeling) in concert with a Teacher network that instructs the Student network by generating pseudo-labels given unlabeled input data; (7) proposing largely-unsupervised, multi-task learning to demonstrate the power of a simple combination of a disentanglement block, variational autoencoder (VAE), generative adversarial network (GAN), and a conditioning layer-based reconstructor for performing two of the foremost critical tasks in medical imaging — segmentation of cardiac structures and reconstruction of the cine cardiac MR images; (8) demonstrating the use of 3D semi-supervised, multi-task learning for jointly learning multiple tasks in a single backbone module – uncertainty estimation, geometric shape generation, and cardiac anatomical structure segmentation of the left atrial cavity from 3D Gadolinium-enhanced magnetic resonance (GE-MR) images. This dissertation summarizes the impact of the contributions of our work in terms of demonstrating the adaptation and use of deep learning architectures featuring different levels of supervision to build a variety of image segmentation tools and techniques that can be used across a wide spectrum of medical image computing applications centered on facilitating and promoting the wide-spread computer-integrated diagnosis and therapy data science

Cardiac motion and deformation estimation in tagged magnetic resonance imaging

Dissertação de mestrado integrado em Engenharia Biomédica (área de especialização em Electrónica Médica)Cardiovascular diseases are the main cause of death in Europe, with an estimate of 4.3 million deaths each year. The assessment of the regional wall deformation is a relevant clinical indicator, and can be used to detect several cardiac lesions. Nowadays, this study can be performed using several image modalities. In the current thesis, we focus on tagged Magnetic Resonance imaging (t-MRI) technique. Such technique allows acquiring images with tags on the myocardium, which deform with the muscle. The present thesis intends to assess the left ventricle (LV) deformation using radial and circumferential strain. To compute such strain values, both endo- and epicardial contours of the LV are required. As such, a new framework to automatically assess the LV function is proposed. This framework presents: (i) an automatic segmentation technique, based on a tag suppression strategy followed by an active contour segmentation method, and (ii) a tracking approach to extract myocardial deformation, based on a non-rigid registration method. The automatic segmentation uses the B-spline Explicit Active Surface framework, which was previously applied in ultra-sound and cine-MRI images. In both cases, a real-time and accurate contour was achieved. Regarding the registration step, starting from a state-of-art approach, termed sequential 2D, we suggest a new method (termed sequential 2D+t), where the temporal information is included on the model. The tracking methods were first tested on synthetic data to study the registration parameters influence. Furthermore, the proposed and original methods were applied on porcine data with myocardial ischemia. Both methods were able to detect dysfunctional regions. A comparison between the strain curve in the sequential 2D and sequential 2D+t strategies was also shown. As conclusion, a smoothing effect in the strain curve was detected in the sequential 2D+t strategy. The validation of the segmentation approach uses a human dataset. A comparison between the manual contour and the proposed segmentation method results was performed. The results, suggest that proposed method has an acceptable performance, removing the tedious task related with manual segmentation and the intra-observer variability. Finally, a comparison between the proposed framework and the currently available commercial software was performed. The commercial software results were obtained from core-lab analysis. An acceptable result (r = 0.601) was achieved when comparing the strain peak values. Importantly, the proposed framework appears to present a more acceptable result.As doenças cardiovasculares são a principal causa de morte na Europa, com aproximadamente 4.7 milhões de mortes por ano. A avaliação da deformação do miocárdio a um nível local é um importante indicador clínico e pode ser usado para a deteção de lesões cardíacas. Este estudo é normalmente realizado usando várias modalidades de imagem médica. Nesta tese, a Resonância Magnética (RM) marcada foi a técnica selecionada. Estas imagens têm marcadores no músculo cardíaco, os quais se deformam com o miocárdio e podem ser usados para o estudo da deformação cardíaca. Nesta tese, pretende-se estudar a deformação radial e circunferencial do ventrículo esquerdo (VE). Assim, um contorno do endo- e epicárdio no VE é essencial. Desta forma, uma ferramenta para o estudo da deformação do VE foi desenvolvida. Esta possui: (i) um método de segmentação automático, usando uma estratégia de supressão dos marcadores, seguido de uma segmentação c um contorno ativo, e (ii) um método de tracking para determinação da deformação cardíaca, baseado em registo não rígido. A segmentação automática utiliza a ferramenta B-spline Explicit Active Surface, que foi previamente aplicada em imagens de ultrassons e cine-RM. Em ambos os casos, uma segmentação em tempo real e com elevada exatidão foi alcançada. Vários esquemas de registo foram apresentados. Neste ponto, começando com uma técnica do estado da arte (designada de sequencial 2D), uma nova metodologia foi proposta (sequencial 2D+t), onde a informação temporal é incorporada no modelo. De forma a analisar a influência dos parâmetros do registo, estes foram estudados num dataset sintético. De seguida, os diferentes esquemas de registo foram testados num dataset suíno com isquemia. Ambos os métodos foram capazes de detetar as regiões disfuncionais. De igual forma, utilizando as curvas de deformação obtidas para cada um dos métodos propostos, foi possível observar uma suavização na direção temporal para o método sequencial 2D+t. Relativamente à segmentação, esta foi validada com um dataset humano. Um contorno manual foi comparado com o obtido pelo método proposto. Os resultados sugerem que a nova estratégia é aceitável, sendo mais rápida do que a realização de um contorno manual e eliminando a variabilidade entre observadores. Por fim, realizou-se uma comparação entre a ferramenta proposta e um software comercial (com análise de core-lab). A comparação entre os valores de pico da deformação exibe uma correlação plausível (r=0.601). Contudo, é importante notar, que a nova ferramenta tende a apresentar um resultado mais aceitável

Automatic segmentation in CMR - Development and validation of algorithms for left ventricular function, myocardium at risk and myocardial infarction

In this thesis four new algorithms are presented for automatic segmentation in cardiovascular magnetic resonance (CMR); automatic segmentation of the left ventricle, myocardial infarction, and myocardium at risk in two different image types. All four algorithms were implemented in freely available software for image analysis and were validated against reference delineations with a low bias and high regional agreement. CMR is the most accurate and reproducible method for assessment of left ventricular mass and volumes and reference standard for assessment of myocardial infarction. CMR is also validated against single photon emission computed tomography (SPECT) for assessment of myocardium at risk up to one week after acute myocardial infarction. However, the clinical standard for quantification of left ventricular mass and volumes is manual delineation which has been shown to have a large bias between observers from different sites and for myocardium at risk and myocardial infarction there is no clinical standard due to varying results shown for the previously suggested threshold methods. The new automatic algorithms were all based on intensity classification by Expectation Maximization (EM) and incorporation of a priori information specific for each application. Validation was performed in large cohorts of patients with regards to bias in clinical parameters and regional agreement as Dice Similarity Coefficient (DSC). Further, images with reference delineation of the left ventricle were made available for future benchmarking of left ventricular segmentation, and the new automatic algorithms for segmentation of myocardium at risk and myocardial infarction were directly compared to the previously suggested intensity threshold methods. Combining intensity classification by EM with a priori information as in the new automatic algorithms was shown superior to previous methods and specifically to the previously suggested threshold methods for myocardium at risk and myocardial infarction. Added value of using a priori information and intensity correction was shown significant measured by DSC even though not significant for bias. For the previously suggested methods of infarct quantification a poorer result was found in the new multi-center, multi-vendor patient data than in the original validation in animal studies or single center patient studies. Thus, the results in this thesis also show the importance ofusing both bias and DSC for validation and performing validation in images of representative quality as in multi-center, multi-vendor patient studies

MRI Evaluation of Injectable Hyaluronic Acid Hydrogel Therapy to Attenuate Myocardial Infarct Remodeling

Left ventricular (LV) remodeling following myocardial infarction (MI) leads to maladaptive processes that often progress to heart failure. Injectable biomaterials can alter the mechanical signaling post-MI to limit this progression. To design optimal therapies, noninvasive techniques are needed to elucidate the reciprocal interaction between the injected material and the surrounding myocardial tissue. Towards this goal, the general hypothesis of this dissertation was that magnetic resonance imaging (MRI) can be used to characterize the properties of injectable materials once delivered to the myocardium and evaluate the temporal effects of injectable materials on myocardial tissue properties post-MI. To test this hypothesis, injectable hyaluronic acid (HA) hydrogels were developed with a range of gelation, degradation and mechanical properties by altering the initiator concentration, macromer modification, and macromer concentration, respectively. Non-contrast MRI was then used to characterize the properties (e.g., distribution, chemical composition) of injectable HA hydrogels in myocardial explants. Altering hydrogel gelation led to differences in distribution in myocardial tissue, as quantified by T2-weighted MRI. As an alternative to conventional (i.e.T2-weighted) MRI where contrast depends on differences in MR properties and thus, is non-specific for the material, chemical exchange saturation transfer (CEST) MRI was used to specifically image hydrogels based on their functional (i.e. exchangeable proton) groups. CEST contrast correlated with changes in material properties, specifically macromer concentration. Furthermore, CEST MRI was shown to simultaneously visualize and discriminate between different injectable materials based on their unique chemistry. Finally, the effect of injectable HA hydrogels on myocardial tissue properties was temporally evaluated in a porcine infarct model up to 12 weeks post-MI. Outcome assessment using MRI (e.g. cine, late-gadolinium enhancement, and spatial modulation of magnetization MRI) and finite element (FE) modeling demonstrated that hydrogel therapy led to improved global LV structure and function, increased wall thickness, preserved borderzone contractility, and increased infarct stiffness, respectively. This work demonstrates that MRI can be used to simultaneously study hydrogel properties after injection into the myocardium and evaluate the ability of injectable hydrogels to alter myocardial tissue properties to ultimately improve cardiac outcomes and enable future optimization of biomaterial therapies to attenuate adverse remodeling post-MI