Deep learning for cardiac image segmentation: A review

Deep learning has become the most widely used approach for cardiac image segmentation in recent years. In this paper, we provide a review of over 100 cardiac image segmentation papers using deep learning, which covers common imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound (US) and major anatomical structures of interest (ventricles, atria and vessels). In addition, a summary of publicly available cardiac image datasets and code repositories are included to provide a base for encouraging reproducible research. Finally, we discuss the challenges and limitations with current deep learning-based approaches (scarcity of labels, model generalizability across different domains, interpretability) and suggest potential directions for future research

Automatic segmentation of multiple cardiovascular structures from cardiac computed tomography angiography images using deep learning.

OBJECTIVES:To develop, demonstrate and evaluate an automated deep learning method for multiple cardiovascular structure segmentation. BACKGROUND:Segmentation of cardiovascular images is resource-intensive. We design an automated deep learning method for the segmentation of multiple structures from Coronary Computed Tomography Angiography (CCTA) images. METHODS:Images from a multicenter registry of patients that underwent clinically-indicated CCTA were used. The proximal ascending and descending aorta (PAA, DA), superior and inferior vena cavae (SVC, IVC), pulmonary artery (PA), coronary sinus (CS), right ventricular wall (RVW) and left atrial wall (LAW) were annotated as ground truth. The U-net-derived deep learning model was trained, validated and tested in a 70:20:10 split. RESULTS:The dataset comprised 206 patients, with 5.130 billion pixels. Mean age was 59.9 ± 9.4 yrs., and was 42.7% female. An overall median Dice score of 0.820 (0.782, 0.843) was achieved. Median Dice scores for PAA, DA, SVC, IVC, PA, CS, RVW and LAW were 0.969 (0.979, 0.988), 0.953 (0.955, 0.983), 0.937 (0.934, 0.965), 0.903 (0.897, 0.948), 0.775 (0.724, 0.925), 0.720 (0.642, 0.809), 0.685 (0.631, 0.761) and 0.625 (0.596, 0.749) respectively. Apart from the CS, there were no significant differences in performance between sexes or age groups. CONCLUSIONS:An automated deep learning model demonstrated segmentation of multiple cardiovascular structures from CCTA images with reasonable overall accuracy when evaluated on a pixel level

Dilated Convolutional Neural Networks for Cardiovascular MR Segmentation in Congenital Heart Disease

We propose an automatic method using dilated convolutional neural networks (CNNs) for segmentation of the myocardium and blood pool in cardiovascular MR (CMR) of patients with congenital heart disease (CHD). Ten training and ten test CMR scans cropped to an ROI around the heart were provided in the MICCAI 2016 HVSMR challenge. A dilated CNN with a receptive field of 131x131 voxels was trained for myocardium and blood pool segmentation in axial, sagittal and coronal image slices. Performance was evaluated within the HVSMR challenge. Automatic segmentation of the test scans resulted in Dice indices of 0.80$\pm$0.06 and 0.93$\pm$0.02, average distances to boundaries of 0.96$\pm$0.31 and 0.89$\pm$0.24 mm, and Hausdorff distances of 6.13$\pm$3.76 and 7.07$\pm$3.01 mm for the myocardium and blood pool, respectively. Segmentation took 41.5$\pm$14.7 s per scan. In conclusion, dilated CNNs trained on a small set of CMR images of CHD patients showing large anatomical variability provide accurate myocardium and blood pool segmentations

Artificial Intelligence in Image-Based Screening, Diagnostics, and Clinical Care of Cardiopulmonary Diseases

Cardiothoracic and pulmonary diseases are a significant cause of mortality and morbidity worldwide. The COVID-19 pandemic has highlighted the lack of access to clinical care, the overburdened medical system, and the potential of artificial intelligence (AI) in improving medicine. There are a variety of diseases affecting the cardiopulmonary system including lung cancers, heart disease, tuberculosis (TB), etc., in addition to COVID-19-related diseases. Screening, diagnosis, and management of cardiopulmonary diseases has become difficult owing to the limited availability of diagnostic tools and experts, particularly in resource-limited regions. Early screening, accurate diagnosis and staging of these diseases could play a crucial role in treatment and care, and potentially aid in reducing mortality. Radiographic imaging methods such as computed tomography (CT), chest X-rays (CXRs), and echo ultrasound (US) are widely used in screening and diagnosis. Research on using image-based AI and machine learning (ML) methods can help in rapid assessment, serve as surrogates for expert assessment, and reduce variability in human performance. In this Special Issue, “Artificial Intelligence in Image-Based Screening, Diagnostics, and Clinical Care of Cardiopulmonary Diseases”, we have highlighted exemplary primary research studies and literature reviews focusing on novel AI/ML methods and their application in image-based screening, diagnosis, and clinical management of cardiopulmonary diseases. We hope that these articles will help establish the advancements in AI

Alignment of contrast enhanced medical images

The re-alignment of series of medical images in which there are multiple contrast variations is difficult. The reason for this is that the popularmeasures of image similarity used to drive the alignment procedure do not separate the influence of intensity variation due to image feature motion and intensity variation due to feature enhancement. In particular, the appearance of new structure poses problems when it has no representation in the original image. The acquisition of many images over time, such as in dynamic contrast enhanced MRI, requires that many images with different contrast be registered to the same coordinate system, compounding the problem. This thesis addresses these issues, beginning by presenting conditions under which conventional registration fails and proposing a solution in the form of a ’progressive principal component registration’. The algorithm uses a statistical analysis of a series of contrast varying images in order to reduce the influence of contrast-enhancement that would otherwise distort the calculation of the image similarity measures used in image registration. The algorithm is shown to be versatile in that it may be applied to series of images in which contrast variation is due to either temporal contrast enhancement changes, as in dynamic contrast-enhanced MRI or intrinsically in the image selection procedure as in diffusion weighted MRI

Coronary Artery Segmentation and Motion Modelling

Conventional coronary artery bypass surgery requires invasive sternotomy and the use of a cardiopulmonary bypass, which leads to long recovery period and has high infectious potential. Totally endoscopic coronary artery bypass (TECAB) surgery based on image guided robotic surgical approaches have been developed to allow the clinicians to conduct the bypass surgery off-pump with only three pin holes incisions in the chest cavity, through which two robotic arms and one stereo endoscopic camera are inserted. However, the restricted field of view of the stereo endoscopic images leads to possible vessel misidentification and coronary artery mis-localization. This results in 20-30% conversion rates from TECAB surgery to the conventional approach. We have constructed patient-specific 3D + time coronary artery and left ventricle motion models from preoperative 4D Computed Tomography Angiography (CTA) scans. Through temporally and spatially aligning this model with the intraoperative endoscopic views of the patient's beating heart, this work assists the surgeon to identify and locate the correct coronaries during the TECAB precedures. Thus this work has the prospect of reducing the conversion rate from TECAB to conventional coronary bypass procedures. This thesis mainly focus on designing segmentation and motion tracking methods of the coronary arteries in order to build pre-operative patient-specific motion models. Various vessel centreline extraction and lumen segmentation algorithms are presented, including intensity based approaches, geometric model matching method and morphology-based method. A probabilistic atlas of the coronary arteries is formed from a group of subjects to facilitate the vascular segmentation and registration procedures. Non-rigid registration framework based on a free-form deformation model and multi-level multi-channel large deformation diffeomorphic metric mapping are proposed to track the coronary motion. The methods are applied to 4D CTA images acquired from various groups of patients and quantitatively evaluated

Faster 3D cardiac CT segmentation with Vision Transformers

Accurate segmentation of the heart is essential for personalized blood flow simulations and surgical intervention planning. A recent advancement in image recognition is the Vision Transformer (ViT), which expands the field of view to encompass a greater portion of the global image context. We adapted ViT for three-dimensional volume inputs. Cardiac computed tomography (CT) volumes from 39 patients, featuring up to 20 timepoints representing the complete cardiac cycle, were utilized. Our network incorporates a modified ResNet50 block as well as a ViT block and employs cascade upsampling with skip connections. Despite its increased model complexity, our hybrid Transformer-Residual U-Net framework, termed TRUNet, converges in significantly less time than residual U-Net while providing comparable or superior segmentations of the left ventricle, left atrium, left atrial appendage, ascending aorta, and pulmonary veins. TRUNet offers more precise vessel boundary segmentation and better captures the heart's overall anatomical structure compared to residual U-Net, as confirmed by the absence of extraneous clusters of missegmented voxels. In terms of both performance and training speed, TRUNet exceeded U-Net, a commonly used segmentation architecture, making it a promising tool for 3D semantic segmentation tasks in medical imaging. The code for TRUNet is available at github.com/ljollans/TRUNet

Computational fluid dynamics indicators to improve cardiovascular pathologies

In recent years, the study of computational hemodynamics within anatomically complex vascular regions has generated great interest among clinicians. The progress in computational fluid dynamics, image processing and high-performance computing haveallowed us to identify the candidate vascular regions for the appearance of cardiovascular diseases and to predict how this disease may evolve. Medicine currently uses a paradigm called diagnosis. In this thesis we attempt to introduce into medicine the predictive paradigm that has been used in engineering for many years. The objective of this thesis is therefore to develop predictive models based on diagnostic indicators for cardiovascular pathologies. We try to predict the evolution of aortic abdominal aneurysm, aortic coarctation and coronary artery disease in a personalized way for each patient. To understand how the cardiovascular pathology will evolve and when it will become a health risk, it is necessary to develop new technologies by merging medical imaging and computational science. We propose diagnostic indicators that can improve the diagnosis and predict the evolution of the disease more efficiently than the methods used until now. In particular, a new methodology for computing diagnostic indicators based on computational hemodynamics and medical imaging is proposed. We have worked with data of anonymous patients to create real predictive technology that will allow us to continue advancing in personalized medicine and generate more sustainable health systems. However, our final aim is to achieve an impact at a clinical level. Several groups have tried to create predictive models for cardiovascular pathologies, but they have not yet begun to use them in clinical practice. Our objective is to go further and obtain predictive variables to be used practically in the clinical field. It is to be hoped that in the future extremely precise databases of all of our anatomy and physiology will be available to doctors. These data can be used for predictive models to improve diagnosis or to improve therapies or personalized treatments.En els últims anys, l'estudi de l'hemodinàmica computacional en regions vasculars anatòmicament complexes ha generat un gran interès entre els clínics. El progrés obtingut en la dinàmica de fluids computacional, en el processament d'imatges i en la computació d'alt rendiment ha permès identificar regions vasculars on poden aparèixer malalties cardiovasculars, així com predir-ne l'evolució. Actualment, la medicina utilitza un paradigma anomenat diagnòstic. En aquesta tesi s'intenta introduir en la medicina el paradigma predictiu utilitzat des de fa molts anys en l'enginyeria. Per tant, aquesta tesi té com a objectiu desenvolupar models predictius basats en indicadors de diagnòstic de patologies cardiovasculars. Tractem de predir l'evolució de l'aneurisma d'aorta abdominal, la coartació aòrtica i la malaltia coronària de forma personalitzada per a cada pacient. Per entendre com la patologia cardiovascular evolucionarà i quan suposarà un risc per a la salut, cal desenvolupar noves tecnologies mitjançant la combinació de les imatges mèdiques i la ciència computacional. Proposem uns indicadors que poden millorar el diagnòstic i predir l'evolució de la malaltia de manera més eficient que els mètodes utilitzats fins ara. En particular, es proposa una nova metodologia per al càlcul dels indicadors de diagnòstic basada en l'hemodinàmica computacional i les imatges mèdiques. Hem treballat amb dades de pacients anònims per crear una tecnologia predictiva real que ens permetrà seguir avançant en la medicina personalitzada i generar sistemes de salut més sostenibles. Però el nostre objectiu final és aconseguir un impacte en l¿àmbit clínic. Diversos grups han tractat de crear models predictius per a les patologies cardiovasculars, però encara no han començat a utilitzar-les en la pràctica clínica. El nostre objectiu és anar més enllà i obtenir variables predictives que es puguin utilitzar de forma pràctica en el camp clínic. Es pot preveure que en el futur tots els metges disposaran de bases de dades molt precises de tota la nostra anatomia i fisiologia. Aquestes dades es poden utilitzar en els models predictius per millorar el diagnòstic o per millorar teràpies o tractaments personalitzats.Postprint (published version

Deep learning analysis of vessel reduction images after EVAR

Máster Universitario en Deep Learning for Audio and Video Signal ProcessingAn aortic aneurysm is an enlargement of the aorta, the largest artery supplying blood to the body. The most common aneurysms are abdominal aortic aneurysms (AAA). AAAs tend to grow and rupture, resulting in a high risk of death from internal bleeding. The most commonly used surgical treatment today is the placement of an aortic stent graft, EVAR (EndoVascular Aneurysm Repair). EVAR is a self-expanding device that is placed inside the diseased artery to exclude the aneurysm from circulation, reducing the pressure on the aneurysm and eliminating the risk of AAA rupture. The best prognostic sign of this treatment is the reduction in size of the AAA over time, once depressurization has occurred. An AAA that is correctly treated with EVAR and excluded from circulation is called an aneurysmal sac. However, EVAR is not free of complications, the most frequent are the so-called leaks, which are small inflows of blood into the treated AAA that condition the pressurization of the aneurysmal sac and therefore the reappearance of the risk of rupture and bleeding, which is always accompanied by a lack of reduction in the size of the sac. There are different types of leaks depending on where blood enters the sac. Early detection is essential to plan appropriate treatment in time. The current diagnostic test to follow up EVARs is CT (Computed Tomography). The images obtained with this technique are studied by physicians looking for contrast spots within the treated AAA that indicate the presence of leaks. These contrast peaks may be evident in some cases, but difficult to see in others, especially considering the large volume of images per CT scan. The model proposed in this study consists of a detection network, based on RetinaNet, to localize the sac in the CT images and remove the surrounding noise. Then using a binary classification model based on convolutional networks, both 2D and 3D, to analyze the images and make a prediction of the evolution of the aneurysm size, which would allow physicians to perform targeted surveillance on patients at higher risk of leaking