Preserving known anatomical topology in medical image segmentation using deep learning

Since the rise of deep learning, new medical image segmentation methods have rapidly been proposed with promising results, with each one reporting marginal improvements on the previous state-of-the-art (SOTA) method. However, on visual inspection, errors are often revealed, such as topological mistakes (e.g. holes or folds), that are not detected using traditional evaluation metrics, such as Dice. Moreover, correct topology is often essential in ensuring segmentations are anatomically and pathologically plausible and, ultimately, suitable for downstream image processing tasks. Therefore, there is a need for methods to focus on ensuring that the predicted segmentations are topologically correct, rather than just optimising the pixel-wise accuracy. In this thesis, I propose a method that utilises prior knowledge of anatomy to segment structures, whilst preserving the known topology. The presented model, Topology Encouraging Deformation Segmentation Network (TEDS-Net), performs segmentation by deforming a prior shape with the same topological features as the anatomy of interest using learnt topology-preserving deformation fields. However, here I show that such fields only guarantee topology preservation in the continuous domain and that their properties begin to break down when applied in discrete spaces. To overcome this effect, I introduced additional modifications in TEDS-Net to more strictly enforce topology preservation, a step often overlooked in previous work. Across this thesis, TEDS-Net is applied to a range of natural and medical image segmentation tasks. I show how it can be used for multiple topology types, multiple structures and in both two- and three-dimensions. Further, I show how TEDS-Net can be used to segment whole volumes using minimally annotated training data. Across these experiments, TEDS-Net outperforms all SOTA baselines on topology, whilst maintaining competitive pixel-wise accuracy. Finally, TEDS-Net is integrated into a whole medical imaging pipeline, to illustrate the importance of topologically correct segmentations for downstream tasks. TEDS-Net is used to segment the developing cortical plate from in-utero fetal brain ultrasound scans in 3D, to enable the characterisation of its complex growth and development during gestation. To the best of my knowledge, this task has only been previously attempted from magnetic resonance imaging (MRI), despite ultrasound being the modality of choice in prenatal care. This is likely due to large acoustic shadows obstructing key brain regions in ultrasound. Due to TEDS-Net anatomical constraints, it can anatomically guide the cortical plate segmentation in regions of shadows, producing a complete segmentation that enables accurate downstream analysis

Computer Vision Techniques for Transcatheter Intervention

Minimally invasive transcatheter technologies have demonstrated substantial promise for the diagnosis and treatment of cardiovascular diseases. For example, TAVI is an alternative to AVR for the treatment of severe aortic stenosis and TAFA is widely used for the treatment and cure of atrial fibrillation. In addition, catheter-based IVUS and OCT imaging of coronary arteries provides important information about the coronary lumen, wall and plaque characteristics. Qualitative and quantitative analysis of these cross-sectional image data will be beneficial for the evaluation and treatment of coronary artery diseases such as atherosclerosis. In all the phases (preoperative, intraoperative, and postoperative) during the transcatheter intervention procedure, computer vision techniques (e.g., image segmentation, motion tracking) have been largely applied in the field to accomplish tasks like annulus measurement, valve selection, catheter placement control, and vessel centerline extraction. This provides beneficial guidance for the clinicians in surgical planning, disease diagnosis, and treatment assessment. In this paper, we present a systematical review on these state-of-the-art methods.We aim to give a comprehensive overview for researchers in the area of computer vision on the subject of transcatheter intervention. Research in medical computing is multi-disciplinary due to its nature, and hence it is important to understand the application domain, clinical background, and imaging modality so that methods and quantitative measurements derived from analyzing the imaging data are appropriate and meaningful. We thus provide an overview on background information of transcatheter intervention procedures, as well as a review of the computer vision techniques and methodologies applied in this area

DefCor-Net: Physics-Aware Ultrasound Deformation Correction

The recovery of morphologically accurate anatomical images from deformed ones is challenging in ultrasound (US) image acquisition, but crucial to accurate and consistent diagnosis, particularly in the emerging field of computer-assisted diagnosis. This article presents a novel anatomy-aware deformation correction approach based on a coarse-to-fine, multi-scale deep neural network (DefCor-Net). To achieve pixel-wise performance, DefCor-Net incorporates biomedical knowledge by estimating pixel-wise stiffness online using a U-shaped feature extractor. The deformation field is then computed using polynomial regression by integrating the measured force applied by the US probe. Based on real-time estimation of pixel-by-pixel tissue properties, the learning-based approach enables the potential for anatomy-aware deformation correction. To demonstrate the effectiveness of the proposed DefCor-Net, images recorded at multiple locations on forearms and upper arms of six volunteers are used to train and validate DefCor-Net. The results demonstrate that DefCor-Net can significantly improve the accuracy of deformation correction to recover the original geometry (Dice Coefficient: from $14.3\pm20.9$ to $82.6\pm12.1$ when the force is $6N$).Comment: Accepted by MedIA. code is availabl

Accurate determination and application of local strain for studying tissues with gradients in mechanical properties

Determination of the mechanical behavior of materials requires an understanding of deformation during loading. While this is traditionally accomplished in engineering by examining a force displacement curve for a whole sample, these techniques implicitly ignore local geometric complexities and local material inhomogeneities commonly found in biologic tissues. Techniques such as normalized cross correlation have been classically applied to address this issue and resolve deformation at the local level; however, these techniques have proven unreliable when deformations become large, if the sample undergoes a rotation, and/or if strain fields become incompatible (e.g. at or near failure). Presented here is a toolbox of techniques that addresses the limitations of the prior state-of-the-art for localized strain estimation. The first algorithm, termed 2D direct deformation estimation (2D-DDE), directly incorporates concepts from mechanics into non-rigid registration algorithms from computer vision, eliminating the need to consider displacement fields, as required for all of the prior state-of-the-art techniques. This results in not only an improvement in accuracy and precision of deformation estimation, but also relaxes compatibility of the deformation fields. A second algorithm, 2D Strain Inference with Measures of Probable Local Elevation (2D-SIMPLE), incorporates the results of 2D-DDE with results from algorithms that enforce strain compatibility to develop a robust detector of strain concentrations. While tracking local strain in a vinylidene chloride sheet in tension, 2D-SIMPLE detected strain concentrations which predicted the initiation of a crack in the material and the progression of the crack tip. The third and fourth algorithms generalize the two dimensional algorithms to analyze three dimensional deformations in volumetric images (3D-DDE and 3D-SIMPLE, respectively). Lastly, the 2D-DDE algorithm is modified to estimate two dimensional surface deformation from multi-view imaging systems. The robustness and adaptability of these techniques was then validated and demonstrated on a wide variety of biomedical applications. Using 2D-DDE, a microscale compliant region was discovered at the tendon-to-bone attachment, local heterogeneity of partially mineralized scaffolds was revealed, and gradients in stiffness of partially mineralized nano-fiber scaffolds were demonstrated. Using 2D-SIMPLE, mechanisms of embryonic wound healing and associated strain localizations were elucidated. 3D-DDE confirmed the existence of strain gradients across chordae tendineae in beating murine hearts as well as demonstrated dramatic localized changes in wall deformation before and after myocardial infarction in murine hearts. 2D-DDE was also used to develop a model system to study the effects of applied stress versus the effects of applied strain on cells. The model system was first theorized by considering a system in which gradients of cross sectional area or scaffold shape were composed with gradients in material stiffness. By combining these gradients in novel ways, it was theoretically determined that stress and strain could be locally isolated. A tensile bioreactor was constructed, techniques for fabricating scaffolds with gradients in stiffness and gradients in cross sectional area were developed, and theoretical strain gradients were confirmed experimentally using 2D-DDE. The model system was then validated for in vitro cell studies. Cell adhesion, proliferation, and viability following a seven day loading protocol were explored. Methods for determining single cell responses, which could be correlated back to a specific stress or strain states, were developed using immunocytochemistry and 2D-DDE approaches. Future studies will apply this model system to determine precise mechanotransduction responses of cells. These studies are critical to optimize stem cell tissue engineering strategies as well inform cell mechanobiology mechanisms

Renal Cryoablation: Investigation of Periprocedural Visualization Tools and Treatment Response Quantification

Cryoablation for small renal tumors has demonstrated sufficient clinical efficacy over the past decade as a non-surgical nephron-sparing approach for treating renal masses for patients who are not surgical candidates. Minimally invasive percutaneous cryoablations have been performed with image guidance from CT, ultrasound, and MRI. During the MRI-guided cryoablation procedure, the interventional radiologist visually compares the iceball size on monitoring images with respect to the original tumor on separate planning images. The comparisons made during the monitoring step are time consuming, inefficient and sometimes lack the precision needed for decision making, requiring the radiologist to make further changes later in the procedure. This study sought to mitigate uncertainty in these visual comparisons by quantifying tissue response to cryoablation and providing visualization of the response during the procedure. Based on retrospective analysis of MR-guided cryoablation patient data, registration and segmentation algorithms were investigated and implemented for periprocedural visualization to deliver iceball position/size with respect to planning images registered within 3.3mm with at least 70% overlap and a quantitative logit model was developed to relate perfusion deficit in renal parenchyma visualized in verification images as a result of iceball size visualized in monitoring images. Through retrospective study of 20 patient cases, the relationship between likelihood of perfusion loss in renal parenchyma and distance within iceball was quantified and iteratively fit to a logit curve. Using the parameters from the logit fit, the margin for 95% perfusion loss likelihood was found to be 4.28 mm within the iceball. The observed margin corresponds well with the clinically accepted margin of 3-5mm within the iceball. In order to display the iceball position and perfusion loss likelihood to the radiologist, algorithms were implemented to create a fast segmentation and registration module which executed in under 2 minutes, within the clinically-relevant 3 minute monitoring period. Using 16 patient cases, the average Hausdorff distance was reduced from 10.1mm to 3.21 mm with average DSC increased from 46.6% to 82.6% before and after registration

Multi-modality cardiac image computing: a survey

Multi-modality cardiac imaging plays a key role in the management of patients with cardiovascular diseases. It allows a combination of complementary anatomical, morphological and functional information, increases diagnosis accuracy, and improves the efficacy of cardiovascular interventions and clinical outcomes. Fully-automated processing and quantitative analysis of multi-modality cardiac images could have a direct impact on clinical research and evidence-based patient management. However, these require overcoming significant challenges including inter-modality misalignment and finding optimal methods to integrate information from different modalities. This paper aims to provide a comprehensive review of multi-modality imaging in cardiology, the computing methods, the validation strategies, the related clinical workflows and future perspectives. For the computing methodologies, we have a favored focus on the three tasks, i.e., registration, fusion and segmentation, which generally involve multi-modality imaging data, either combining information from different modalities or transferring information across modalities. The review highlights that multi-modality cardiac imaging data has the potential of wide applicability in the clinic, such as trans-aortic valve implantation guidance, myocardial viability assessment, and catheter ablation therapy and its patient selection. Nevertheless, many challenges remain unsolved, such as missing modality, modality selection, combination of imaging and non-imaging data, and uniform analysis and representation of different modalities. There is also work to do in defining how the well-developed techniques fit in clinical workflows and how much additional and relevant information they introduce. These problems are likely to continue to be an active field of research and the questions to be answered in the future

SUBJECT-SPECIFIC 4D ULTRASOUND LIVER MODEL AND DYNAMIC PHANTOM FOR ROBOTIC-ASSISTED LIVER BIOPSY STUDY

Ph.DDOCTOR OF PHILOSOPH