Characterizing Single Ventricle Patient-Specific Anatomy Using Segmentation of MRI and 3D Reconstruction to Aid Surgical Planning

Single ventricle congenital heart defects occur 2 per every 1000 live births in the USA. In these cases, cyanosis occurs due to the mixing of venous deoxygenated blood and oxygenated blood from the lungs. These defects are surgically treated by the total cavo-pulmonary connection (TCPC), where the superior and inferior vena cavae are connected to the pulmonary arteries routing the systemic venous return directly to the lungs. However, this Fontan repair results in high energy losses and therefore the optimization of this connection prior to the surgery could significantly improve post-operative performance. In this paper, the in-house segmentation and 3D reconstruction scheme is used in the following studies. First, 3D geometrical analysis of the TCPCs is used to determine the advantages and disadvantages of two commonly performed TCPC palliations intra-atrial and extra-cardiac configurations. Then, a surgical planning outline is proposed with segmentation of pre and post surgical Magnetic Resonance Imaging (MRI) data followed by the 3D reconstruction with emphasis on extracting surrounding vessels and structures. A pediatric surgeon performs a virtual surgery on the reconstruction of the patient s pre-Fontan anatomy prior to the actual surgery. A segmentation of the heart, aorta and surrounding vessels superimposed with the Glenn, when used with the SURGEM® tool, simulates the actual Fontan operation. This outline allows the surgeon to envision numerous scenarios of possible surgical options, and accordingly to predict the post operative procedures. The segmentation tool is improved upon to increase the accuracy and efficiency of the process and enhance the quality of the anatomical reconstructions.Committee Member/Second Reader: Kartik S. Sundareswaran; Faculty Mentor: Dr. Ajit P. Yoganatha

From Fully-Supervised Single-Task to Semi-Supervised Multi-Task Deep Learning Architectures for Segmentation in Medical Imaging Applications

Medical imaging is routinely performed in clinics worldwide for the diagnosis and treatment of numerous medical conditions in children and adults. With the advent of these medical imaging modalities, radiologists can visualize both the structure of the body as well as the tissues within the body. However, analyzing these high-dimensional (2D/3D/4D) images demands a significant amount of time and effort from radiologists. Hence, there is an ever-growing need for medical image computing tools to extract relevant information from the image data to help radiologists perform efficiently. Image analysis based on machine learning has pivotal potential to improve the entire medical imaging pipeline, providing support for clinical decision-making and computer-aided diagnosis. To be effective in addressing challenging image analysis tasks such as classification, detection, registration, and segmentation, specifically for medical imaging applications, deep learning approaches have shown significant improvement in performance. While deep learning has shown its potential in a variety of medical image analysis problems including segmentation, motion estimation, etc., generalizability is still an unsolved problem and many of these successes are achieved at the cost of a large pool of datasets. For most practical applications, getting access to a copious dataset can be very difficult, often impossible. Annotation is tedious and time-consuming. This cost is further amplified when annotation must be done by a clinical expert in medical imaging applications. Additionally, the applications of deep learning in the real-world clinical setting are still limited due to the lack of reliability caused by the limited prediction capabilities of some deep learning models. Moreover, while using a CNN in an automated image analysis pipeline, it’s critical to understand which segmentation results are problematic and require further manual examination. To this extent, the estimation of uncertainty calibration in a semi-supervised setting for medical image segmentation is still rarely reported. This thesis focuses on developing and evaluating optimized machine learning models for a variety of medical imaging applications, ranging from fully-supervised, single-task learning to semi-supervised, multi-task learning that makes efficient use of annotated training data. The contributions of this dissertation are as follows: (1) developing a fully-supervised, single-task transfer learning for the surgical instrument segmentation from laparoscopic images; and (2) utilizing supervised, single-task, transfer learning for segmenting and digitally removing the surgical instruments from endoscopic/laparoscopic videos to allow the visualization of the anatomy being obscured by the tool. The tool removal algorithms use a tool segmentation mask and either instrument-free reference frames or previous instrument-containing frames to fill in (inpaint) the instrument segmentation mask; (3) developing fully-supervised, single-task learning via efficient weight pruning and learned group convolution for accurate left ventricle (LV), right ventricle (RV) blood pool and myocardium localization and segmentation from 4D cine cardiac MR images; (4) demonstrating the use of our fully-supervised memory-efficient model to generate dynamic patient-specific right ventricle (RV) models from cine cardiac MRI dataset via an unsupervised learning-based deformable registration field; and (5) integrating a Monte Carlo dropout into our fully-supervised memory-efficient model with inherent uncertainty estimation, with the overall goal to estimate the uncertainty associated with the obtained segmentation and error, as a means to flag regions that feature less than optimal segmentation results; (6) developing semi-supervised, single-task learning via self-training (through meta pseudo-labeling) in concert with a Teacher network that instructs the Student network by generating pseudo-labels given unlabeled input data; (7) proposing largely-unsupervised, multi-task learning to demonstrate the power of a simple combination of a disentanglement block, variational autoencoder (VAE), generative adversarial network (GAN), and a conditioning layer-based reconstructor for performing two of the foremost critical tasks in medical imaging — segmentation of cardiac structures and reconstruction of the cine cardiac MR images; (8) demonstrating the use of 3D semi-supervised, multi-task learning for jointly learning multiple tasks in a single backbone module – uncertainty estimation, geometric shape generation, and cardiac anatomical structure segmentation of the left atrial cavity from 3D Gadolinium-enhanced magnetic resonance (GE-MR) images. This dissertation summarizes the impact of the contributions of our work in terms of demonstrating the adaptation and use of deep learning architectures featuring different levels of supervision to build a variety of image segmentation tools and techniques that can be used across a wide spectrum of medical image computing applications centered on facilitating and promoting the wide-spread computer-integrated diagnosis and therapy data science

Incorporating Cardiac Substructures Into Radiation Therapy For Improved Cardiac Sparing

Growing evidence suggests that radiation therapy (RT) doses to the heart and cardiac substructures (CS) are strongly linked to cardiac toxicities, though only the heart is considered clinically. This work aimed to utilize the superior soft-tissue contrast of magnetic resonance (MR) to segment CS, quantify uncertainties in their position, assess their effect on treatment planning and an MR-guided environment. Automatic substructure segmentation of 12 CS was completed using a novel hybrid MR/computed tomography (CT) atlas method and was improved upon using a 3-dimensional neural network (U-Net) from deep learning. Intra-fraction motion due to respiration was then quantified. The inter-fraction setup uncertainties utilizing a novel MR-linear accelerator were also quantified. Treatment planning comparisons were performed with and without substructure inclusions and methods to reduce radiation dose to sensitive CS were evaluated. Lastly, these described technologies (deep learning U-Net) were translated to an MR-linear accelerator and a segmentation pipeline was created. Automatic segmentations from the hybrid MR/CT atlas was able to generate accurate segmentations for the chambers and great vessels (Dice similarity coefficient (DSC) \u3e 0.75) but coronary artery segmentations were unsuccessful (DSC\u3c0.3). After implementing deep learning, DSC for the chambers and great vessels was ≥0.85 along with an improvement in the coronary arteries (DSC\u3e0.5). Similar accuracy was achieved when implementing deep learning for MR-guided RT. On average, automatic segmentations required ~10 minutes to generate per patient and deep learning only required 14 seconds. The inclusion of CS in the treatment planning process did not yield statistically significant changes in plan complexity, PTV, or OAR dose. Automatic segmentation results from deep learning pose major efficiency and accuracy gains for CS segmentation offering high potential for rapid implementation into radiation therapy planning for improved cardiac sparing. Introducing CS into RT planning for MR-guided RT presented an opportunity for more effective sparing with limited increase in plan complexity

Incorporating Cardiac Substructures Into Radiation Therapy For Improved Cardiac Sparing

Growing evidence suggests that radiation therapy (RT) doses to the heart and cardiac substructures (CS) are strongly linked to cardiac toxicities, though only the heart is considered clinically. This work aimed to utilize the superior soft-tissue contrast of magnetic resonance (MR) to segment CS, quantify uncertainties in their position, assess their effect on treatment planning and an MR-guided environment. Automatic substructure segmentation of 12 CS was completed using a novel hybrid MR/computed tomography (CT) atlas method and was improved upon using a 3-dimensional neural network (U-Net) from deep learning. Intra-fraction motion due to respiration was then quantified. The inter-fraction setup uncertainties utilizing a novel MR-linear accelerator were also quantified. Treatment planning comparisons were performed with and without substructure inclusions and methods to reduce radiation dose to sensitive CS were evaluated. Lastly, these described technologies (deep learning U-Net) were translated to an MR-linear accelerator and a segmentation pipeline was created. Automatic segmentations from the hybrid MR/CT atlas was able to generate accurate segmentations for the chambers and great vessels (Dice similarity coefficient (DSC) \u3e 0.75) but coronary artery segmentations were unsuccessful (DSC\u3c0.3). After implementing deep learning, DSC for the chambers and great vessels was ≥0.85 along with an improvement in the coronary arteries (DSC\u3e0.5). Similar accuracy was achieved when implementing deep learning for MR-guided RT. On average, automatic segmentations required ~10 minutes to generate per patient and deep learning only required 14 seconds. The inclusion of CS in the treatment planning process did not yield statistically significant changes in plan complexity, PTV, or OAR dose. Automatic segmentation results from deep learning pose major efficiency and accuracy gains for CS segmentation offering high potential for rapid implementation into radiation therapy planning for improved cardiac sparing. Introducing CS into RT planning for MR-guided RT presented an opportunity for more effective sparing with limited increase in plan complexity

Echocardiography

The book "Echocardiography - New Techniques" brings worldwide contributions from highly acclaimed clinical and imaging science investigators, and representatives from academic medical centers. Each chapter is designed and written to be accessible to those with a basic knowledge of echocardiography. Additionally, the chapters are meant to be stimulating and educational to the experts and investigators in the field of echocardiography. This book is aimed primarily at cardiology fellows on their basic echocardiography rotation, fellows in general internal medicine, radiology and emergency medicine, and experts in the arena of echocardiography. Over the last few decades, the rate of technological advancements has developed dramatically, resulting in new techniques and improved echocardiographic imaging. The authors of this book focused on presenting the most advanced techniques useful in today's research and in daily clinical practice. These advanced techniques are utilized in the detection of different cardiac pathologies in patients, in contributing to their clinical decision, as well as follow-up and outcome predictions. In addition to the advanced techniques covered, this book expounds upon several special pathologies with respect to the functions of echocardiography

Automated deep phenotyping of the cardiovascular system using magnetic resonance imaging

Across a lifetime, the cardiovascular system must adapt to a great range of demands from the body. The individual changes in the cardiovascular system that occur in response to loading conditions are influenced by genetic susceptibility, and the pattern and extent of these changes have prognostic value. Brachial blood pressure (BP) and left ventricular ejection fraction (LVEF) are important biomarkers that capture this response, and their measurements are made at high resolution. Relatively, clinical analysis is crude, and may result in lost information and the introduction of noise. Digital information storage enables efficient extraction of information from a dataset, and this strategy may provide more precise and deeper measures to breakdown current phenotypes into their component parts. The aim of this thesis was to develop automated analysis of cardiovascular magnetic resonance (CMR) imaging for more detailed phenotyping, and apply these techniques for new biological insights into the cardiovascular response to different loading conditions. I therefore tested the feasibility and clinical utility of computational approaches for image and waveform analysis, recruiting and acquiring additional patient cohorts where necessary, and then applied these approaches prospectively to participants before and after six-months of exercise training for a first-time marathon. First, a multi-centre, multi-vendor, multi-field strength, multi-disease CMR resource of 110 patients undergoing repeat imaging in a short time-frame was assembled. The resource was used to assess whether automated analysis of LV structure and function is feasible on real-world data, and if it can improve upon human precision. This showed that clinicians can be confident in detecting a 9% change in EF or a 20g change in LV mass. This will be difficult to improve by clinicians because the greatest source of human error was attributable to the observer rather than modifiable factors. Having understood these errors, a convolutional neural network was trained on separate multi-centre data for automated analysis and was successfully generalizable to the real-world CMR data. Precision was similar to human analysis, and performance was 186 times faster. This real-world benchmarking resource has been made freely available (thevolumesresource.com). Precise automated segmentations were then used as a platform to delve further into the LV phenotype. Global LVEFs measured from CMR imaging in 116 patients with severe aortic stenosis were broken down into ~10 million regional measurements of structure and function, represented by computational three-dimensional LV models for each individual. A cardiac atlas approach was used to compile, label, segment and represent these data. Models were compared with healthy matched controls, and co-registered with follow-up one year after aortic valve replacement (AVR). This showed that there is a tendency to asymmetric septal hypertrophy in all patients with severe aortic stenosis (AS), rather than a characteristic specific to predisposed patients. This response to AS was more unfavourable in males than females (associated with higher NT-proBNP, and lower blood pressure), but was more modifiable with AVR. This was not detected using conventional analysis. Because cardiac function is coupled with the vasculature, a novel integrated assessment of the cardiovascular system was developed. Wave intensity theory was used to combine central blood pressure and CMR aortic blood flow-velocity waveforms to represent the interaction of the heart with the vessels in terms of traveling energy waves. This was performed and then validated in 206 individuals (the largest cohort to date), demonstrating inefficient ventriculo-arterial coupling in female sex and healthy ageing. CMR imaging was performed in 236 individuals before training for a first-time marathon and 138 individuals were followed-up after marathon completion. After training, systolic/diastolic blood pressure reduced by 4/3mmHg, descending aortic stiffness decreased by 16%, and ventriculo-arterial coupling improved by 14%. LV mass increased slightly, with a tendency to more symmetrical hypertrophy. The reduction in aortic stiffness was equivalent to a 4-year reduction in estimated biological aortic age, and the benefit was greater in older, male, and slower individuals. In conclusion, this thesis demonstrates that automating analysis of clinical cardiovascular phenotypes is precise with significant time-saving. Complex data that is usually discarded can be used efficiently to identify new biology. Deeper phenotypes developed in this work inform risk reduction behaviour in healthy individuals, and demonstrably deliver a more sensitive marker of LV remodelling, potentially enhancing risk prediction in severe aortic stenosis

Computational fluid dynamics indicators to improve cardiovascular pathologies

In recent years, the study of computational hemodynamics within anatomically complex vascular regions has generated great interest among clinicians. The progress in computational fluid dynamics, image processing and high-performance computing haveallowed us to identify the candidate vascular regions for the appearance of cardiovascular diseases and to predict how this disease may evolve. Medicine currently uses a paradigm called diagnosis. In this thesis we attempt to introduce into medicine the predictive paradigm that has been used in engineering for many years. The objective of this thesis is therefore to develop predictive models based on diagnostic indicators for cardiovascular pathologies. We try to predict the evolution of aortic abdominal aneurysm, aortic coarctation and coronary artery disease in a personalized way for each patient. To understand how the cardiovascular pathology will evolve and when it will become a health risk, it is necessary to develop new technologies by merging medical imaging and computational science. We propose diagnostic indicators that can improve the diagnosis and predict the evolution of the disease more efficiently than the methods used until now. In particular, a new methodology for computing diagnostic indicators based on computational hemodynamics and medical imaging is proposed. We have worked with data of anonymous patients to create real predictive technology that will allow us to continue advancing in personalized medicine and generate more sustainable health systems. However, our final aim is to achieve an impact at a clinical level. Several groups have tried to create predictive models for cardiovascular pathologies, but they have not yet begun to use them in clinical practice. Our objective is to go further and obtain predictive variables to be used practically in the clinical field. It is to be hoped that in the future extremely precise databases of all of our anatomy and physiology will be available to doctors. These data can be used for predictive models to improve diagnosis or to improve therapies or personalized treatments.En els últims anys, l'estudi de l'hemodinàmica computacional en regions vasculars anatòmicament complexes ha generat un gran interès entre els clínics. El progrés obtingut en la dinàmica de fluids computacional, en el processament d'imatges i en la computació d'alt rendiment ha permès identificar regions vasculars on poden aparèixer malalties cardiovasculars, així com predir-ne l'evolució. Actualment, la medicina utilitza un paradigma anomenat diagnòstic. En aquesta tesi s'intenta introduir en la medicina el paradigma predictiu utilitzat des de fa molts anys en l'enginyeria. Per tant, aquesta tesi té com a objectiu desenvolupar models predictius basats en indicadors de diagnòstic de patologies cardiovasculars. Tractem de predir l'evolució de l'aneurisma d'aorta abdominal, la coartació aòrtica i la malaltia coronària de forma personalitzada per a cada pacient. Per entendre com la patologia cardiovascular evolucionarà i quan suposarà un risc per a la salut, cal desenvolupar noves tecnologies mitjançant la combinació de les imatges mèdiques i la ciència computacional. Proposem uns indicadors que poden millorar el diagnòstic i predir l'evolució de la malaltia de manera més eficient que els mètodes utilitzats fins ara. En particular, es proposa una nova metodologia per al càlcul dels indicadors de diagnòstic basada en l'hemodinàmica computacional i les imatges mèdiques. Hem treballat amb dades de pacients anònims per crear una tecnologia predictiva real que ens permetrà seguir avançant en la medicina personalitzada i generar sistemes de salut més sostenibles. Però el nostre objectiu final és aconseguir un impacte en l¿àmbit clínic. Diversos grups han tractat de crear models predictius per a les patologies cardiovasculars, però encara no han començat a utilitzar-les en la pràctica clínica. El nostre objectiu és anar més enllà i obtenir variables predictives que es puguin utilitzar de forma pràctica en el camp clínic. Es pot preveure que en el futur tots els metges disposaran de bases de dades molt precises de tota la nostra anatomia i fisiologia. Aquestes dades es poden utilitzar en els models predictius per millorar el diagnòstic o per millorar teràpies o tractaments personalitzats.Postprint (published version

Evaluation of algorithms for Multi-Modality Whole Heart Segmentation:An open-access grand challenge

Knowledge of whole heart anatomy is a prerequisite for many clinical applications. Whole heart segmentation (WHS), which delineates substructures of the heart, can be very valuable for modeling and analysis of the anatomy and functions of the heart. However, automating this segmentation can be arduous due to the large variation of the heart shape, and different image qualities of the clinical data. To achieve this goal, a set of training data is generally needed for constructing priors or for training. In addition, it is difficult to perform comparisons between different methods, largely due to differences in the datasets and evaluation metrics used. This manuscript presents the methodologies and evaluation results for the WHS algorithms selected from the submissions to the Multi-Modality Whole Heart Segmentation (MM-WHS) challenge, in conjunction with MICCAI 2017. The challenge provides 120 three-dimensional cardiac images covering the whole heart, including 60 CT and 60 MRI volumes, all acquired in clinical environments with manual delineation. Ten algorithms for CT data and eleven algorithms for MRI data, submitted from twelve groups, have been evaluated. The results show that many of the deep learning (DL) based methods achieved high accuracy, even though the number of training datasets was limited. A number of them also reported poor results in the blinded evaluation, probably due to overfitting in their training. The conventional algorithms, mainly based on multi-atlas segmentation, demonstrated robust and stable performance, even though the accuracy is not as good as the best DL method in CT segmentation. The challenge, including the provision of the annotated training data and the blinded evaluation for submitted algorithms on the test data, continues as an ongoing benchmarking resource via its homepage (\url{www.sdspeople.fudan.edu.cn/zhuangxiahai/0/mmwhs/}).Comment: 14 pages, 7 figures, sumitted to Medical Image Analysi

Computational analysis of blood flow and stress patterns in the aorta of patients with Marfan syndrome

Personalised external aortic root support (PEARS) was designed to prevent progressive aortic dilatation, and the associated risk of aortic dissection, in patients with Marfan syndrome by providing an additional support to the aorta. The objective of this thesis was to understand the biomechanical implications of PEARS surgery as well as to investigate the altered haemodynamics associated with the disease and its treatment. Finite element (FE) models were developed using patient-specific aortic geometries reconstructed from pre and post-PEARS magnetic resonance (MR) images of three Marfan patients. The wall and PEARS materials were assumed to be isotropic, incompressible and linearly elastic. A static load on the inner wall corresponding to the patients’ pulse pressure was applied with a zero-displacement constraint at all boundaries. Results showed that peak aortic stresses and displacements before PEARS were located at the sinuses of Valsalva but following PEARS surgery, they were shifted to the aortic arch, at the intersection between the supported and unsupported aorta. The zero-displacement constraint at the aortic root was subsequently removed and replaced with downward motion measured from in vivo images. This revealed significant increases in the longitudinal wall stress, especially in the pre-PEARS models. Computational fluid dynamics (CFD) models were developed to evaluate flow characteristics. The correlation-based transitional Shear Stress Transport (SST-Tran) model was adopted to simulate potential transitional and turbulence flow during part of the cardiac cycle and flow waveforms derived from phase-contrast MR images were imposed at the inlets. Qualitative patterns of the haemodynamics were similar pre- and post-PEARS with variations in mean helicity flow index (HFI) of -10%, 35% and 20% in the post-PEARS aortas of the three patients. A fluid-structure interaction (FSI) model was developed for one patient, pre- and post-PEARS in order to examine the effect of wall compliance on aortic flow as well as the effect of pulsatile flow on wall stress. This model excluded the sinuses and was based on the laminar flow assumption. The results were similar to those obtained using the rigid wall and static structural models, with minor quantitative differences. Considering the higher computational cost of FSI simulations and the relatively small differences observed in peak wall stress, it is reasonable to suggest that static structural models would be sufficient for wall stress prediction. Additionally, aortic root motion had a more profound effect on wall stress than wall compliance. Further studies are required to assess the statistical significance of the findings outlined in this thesis. Recommendations for future work were also highlighted, with emphasis on model assumptions including material properties, residual stress and boundary conditions.Open Acces