Automated Echocardiographic Image Interpretation Using Artificial Intelligence

In addition to remaining as one of the leading causes of global mortality, cardio vascular disease has a significant impact on overall health, well-being, and life expectancy. Therefore, early detection of anomalies in cardiac function has become essential for early treatment, and therefore reduction in mortalities. Echocardiography is the most commonly used modality for evaluating the structure and function of the heart. Analysis of echocardiographic images has an important role in the clinical practice in assessing the cardiac morphology and function and thereby reaching a diagnosis. The process of interpretation of echocardiographic images is considered challenging for several reasons. The manual annotation is still a daily work in the clinical routine due to the lack of reliable automatic interpretation methods. This can lead to time-consuming tasks that are prone to intra- and inter-observer variability. Echocardiographic images inherently suffer from a high level of noise and poor qualities. Therefore, although several studies have attempted automating the process, this re-mains a challenging task, and improving the accuracy of automatic echocardiography interpretation is an ongoing field. Advances in Artificial Intelligence and Deep Learning can help to construct an auto-mated, scalable pipeline for echocardiographic image interpretation steps, includingview classification, phase-detection, image segmentation with a focus on border detection, quantification of structure, and measurement of the clinical markers. This thesis aims to develop optimised automated methods for the three individual steps forming part of an echocardiographic exam, namely view classification, left ventricle segmentation, quantification, and measurement of left ventricle structure. Various Neural Architecture Search methods were employed to design efficient neural network architectures for the above tasks. Finally, an optimisation-based speckle tracking echocardiography algorithm was proposed to estimate the myocardial tissue velocities and cardiac deformation. The algorithm was adopted to measure cardiac strain which is used for detecting myocardial ischaemia. All proposed techniques were compared with the existing state-of-the-art methods. To this end, publicly available patients datasets, as well as two private datasets provided by the clinical partners to this project, were used for developments and comprehensive performance evaluations of the proposed techniques. Results demonstrated the feasibility of using automated tools for reliable echocardiographic image interpretations, which can be used as assistive tools to clinicians in obtaining clinical measurements

Computational Methods for Segmentation of Multi-Modal Multi-Dimensional Cardiac Images

Segmentation of the heart structures helps compute the cardiac contractile function quantified via the systolic and diastolic volumes, ejection fraction, and myocardial mass, representing a reliable diagnostic value. Similarly, quantification of the myocardial mechanics throughout the cardiac cycle, analysis of the activation patterns in the heart via electrocardiography (ECG) signals, serve as good cardiac diagnosis indicators. Furthermore, high quality anatomical models of the heart can be used in planning and guidance of minimally invasive interventions under the assistance of image guidance. The most crucial step for the above mentioned applications is to segment the ventricles and myocardium from the acquired cardiac image data. Although the manual delineation of the heart structures is deemed as the gold-standard approach, it requires significant time and effort, and is highly susceptible to inter- and intra-observer variability. These limitations suggest a need for fast, robust, and accurate semi- or fully-automatic segmentation algorithms. However, the complex motion and anatomy of the heart, indistinct borders due to blood flow, the presence of trabeculations, intensity inhomogeneity, and various other imaging artifacts, makes the segmentation task challenging. In this work, we present and evaluate segmentation algorithms for multi-modal, multi-dimensional cardiac image datasets. Firstly, we segment the left ventricle (LV) blood-pool from a tri-plane 2D+time trans-esophageal (TEE) ultrasound acquisition using local phase based filtering and graph-cut technique, propagate the segmentation throughout the cardiac cycle using non-rigid registration-based motion extraction, and reconstruct the 3D LV geometry. Secondly, we segment the LV blood-pool and myocardium from an open-source 4D cardiac cine Magnetic Resonance Imaging (MRI) dataset by incorporating average atlas based shape constraint into the graph-cut framework and iterative segmentation refinement. The developed fast and robust framework is further extended to perform right ventricle (RV) blood-pool segmentation from a different open-source 4D cardiac cine MRI dataset. Next, we employ convolutional neural network based multi-task learning framework to segment the myocardium and regress its area, simultaneously, and show that segmentation based computation of the myocardial area is significantly better than that regressed directly from the network, while also being more interpretable. Finally, we impose a weak shape constraint via multi-task learning framework in a fully convolutional network and show improved segmentation performance for LV, RV and myocardium across healthy and pathological cases, as well as, in the challenging apical and basal slices in two open-source 4D cardiac cine MRI datasets. We demonstrate the accuracy and robustness of the proposed segmentation methods by comparing the obtained results against the provided gold-standard manual segmentations, as well as with other competing segmentation methods

Computational Anatomy for Multi-Organ Analysis in Medical Imaging: A Review

The medical image analysis field has traditionally been focused on the development of organ-, and disease-specific methods. Recently, the interest in the development of more 20 comprehensive computational anatomical models has grown, leading to the creation of multi-organ models. Multi-organ approaches, unlike traditional organ-specific strategies, incorporate inter-organ relations into the model, thus leading to a more accurate representation of the complex human anatomy. Inter-organ relations are not only spatial, but also functional and physiological. Over the years, the strategies 25 proposed to efficiently model multi-organ structures have evolved from the simple global modeling, to more sophisticated approaches such as sequential, hierarchical, or machine learning-based models. In this paper, we present a review of the state of the art on multi-organ analysis and associated computation anatomy methodology. The manuscript follows a methodology-based classification of the different techniques 30 available for the analysis of multi-organs and multi-anatomical structures, from techniques using point distribution models to the most recent deep learning-based approaches. With more than 300 papers included in this review, we reflect on the trends and challenges of the field of computational anatomy, the particularities of each anatomical region, and the potential of multi-organ analysis to increase the impact of 35 medical imaging applications on the future of healthcare.Comment: Paper under revie

Automatic Cardiac MRI Image Segmentation and Mesh Generation

Segmenting and reconstructing cardiac anatomical structures from magnetic resonance (MR) images is essential for the quantitative measurement and automatic diagnosis of cardiovascular diseases [1]. However, manual evaluation of the time-series cardiac MRI (CMRI) obtained during routine clinical care are laborious, inefficient, and tends to produce biased and non-reproducible results [2]. This thesis proposes an end-to-end pipeline for automatically segmenting short-axis (SAX) CMRI images and generating high-quality 2D and 3D meshes suitable for finite element analysis. The main advantage of our approach is that it can not only work as a stand-alone pipeline for the automatic CMR image segmentation and mesh generation but also functions effectively as a post-processing tool for improving the outcomes of deep learning methods. Our results indicate that the segmentation accuracy outperformed the traditional U-Net-based approach by as much as 82.5% (percent increase in Dice score) for 5 patient types. The mesh models generated from our contoured segmentations had minimized mean distance error of less than 1.3 pixels and optimized mesh quality with an average Kupp index greater than 0.8

A computer vision pipeline for fully automated echocardiogram interpretation

Cardiovascular disease is the leading cause of global mortality and continues to place a significant burden, in economic and resource terms, upon health services. A 2-dimensional transthoracic echocardiogram captures high spatial and temporal images and videos of the heart and is the modality of choice for the rapid assessment of heart function and structure due to it’s non-invasive nature and lack of ionising radiation. The challenging process of analysing echocardiographic images is currently manually performed by trained experts, though this process is vulnerable to intra- and inter-observer variability and is highly time-consuming. Additionally, echocardiographic images suffer from varying degrees of noise and vary drastically in terms of image quality. Exponential advancements in the fields of artificial intelligence, deep learning and computer vision have enabled the rapid development of automated systems capable of high-precision tasks, often out-performing human experts. This thesis aims to investigate the applicability of applying deep learning methods to automate key processes in the modern echocardiographic laboratory. Namely, view classification, quality assessment, cardiac phase detection, segmentation of the left ventricle and keypoint detection on tissue Doppler imaging strips. State-of-the-art deep learning architectures were applied to each task, and evaluated against ground-truth annotations provided by trained experts. The datasets used throughout each Chapter are diverse and, in some cases, have been made public for the benefit of the research community. To encourage transparency and openness, all code and model weights have been published. Should automated deep learning systems, both online (in terms of providing real-time feedback) and offline (behind the scenes), become integrated within clinical practice, there is great potential for improved accuracy and efficiency, thus improving patient outcomes. Furthermore, health services could save valuable resources such as time and money

Exploring variability in medical imaging

Although recent successes of deep learning and novel machine learning techniques improved the perfor- mance of classification and (anomaly) detection in computer vision problems, the application of these methods in medical imaging pipeline remains a very challenging task. One of the main reasons for this is the amount of variability that is encountered and encapsulated in human anatomy and subsequently reflected in medical images. This fundamental factor impacts most stages in modern medical imaging processing pipelines. Variability of human anatomy makes it virtually impossible to build large datasets for each disease with labels and annotation for fully supervised machine learning. An efficient way to cope with this is to try and learn only from normal samples. Such data is much easier to collect. A case study of such an automatic anomaly detection system based on normative learning is presented in this work. We present a framework for detecting fetal cardiac anomalies during ultrasound screening using generative models, which are trained only utilising normal/healthy subjects. However, despite the significant improvement in automatic abnormality detection systems, clinical routine continues to rely exclusively on the contribution of overburdened medical experts to diagnosis and localise abnormalities. Integrating human expert knowledge into the medical imaging processing pipeline entails uncertainty which is mainly correlated with inter-observer variability. From the per- spective of building an automated medical imaging system, it is still an open issue, to what extent this kind of variability and the resulting uncertainty are introduced during the training of a model and how it affects the final performance of the task. Consequently, it is very important to explore the effect of inter-observer variability both, on the reliable estimation of model’s uncertainty, as well as on the model’s performance in a specific machine learning task. A thorough investigation of this issue is presented in this work by leveraging automated estimates for machine learning model uncertainty, inter-observer variability and segmentation task performance in lung CT scan images. Finally, a presentation of an overview of the existing anomaly detection methods in medical imaging was attempted. This state-of-the-art survey includes both conventional pattern recognition methods and deep learning based methods. It is one of the first literature surveys attempted in the specific research area.Open Acces