A Systematic Quality Scoring Analysis to Assess Automated Cardiovascular Magnetic Resonance Segmentation Algorithms

BACKGROUND: The quantitative measures used to assess the performance of automated methods often do not reflect the clinical acceptability of contouring. A quality-based assessment of automated cardiac magnetic resonance (CMR) segmentation more relevant to clinical practice is therefore needed. OBJECTIVE: We propose a new method for assessing the quality of machine learning (ML) outputs. We evaluate the clinical utility of the proposed method as it is employed to systematically analyse the quality of an automated contouring algorithm. METHODS: A dataset of short-axis (SAX) cine CMR images from a clinically heterogeneous population (n = 217) were manually contoured by a team of experienced investigators. On the same images we derived automated contours using a ML algorithm. A contour quality scoring application randomly presented manual and automated contours to four blinded clinicians, who were asked to assign a quality score from a predefined rubric. Firstly, we analyzed the distribution of quality scores between the two contouring methods across all clinicians. Secondly, we analyzed the interobserver reliability between the raters. Finally, we examined whether there was a variation in scores based on the type of contour, SAX slice level, and underlying disease. RESULTS: The overall distribution of scores between the two methods was significantly different, with automated contours scoring better than the manual (OR (95% CI) = 1.17 (1.07–1.28), p = 0.001; n = 9401). There was substantial scoring agreement between raters for each contouring method independently, albeit it was significantly better for automated segmentation (automated: AC2 = 0.940, 95% CI, 0.937–0.943 vs manual: AC2 = 0.934, 95% CI, 0.931–0.937; p = 0.006). Next, the analysis of quality scores based on different factors was performed. Our approach helped identify trends patterns of lower segmentation quality as observed for left ventricle epicardial and basal contours with both methods. Similarly, significant differences in quality between the two methods were also found in dilated cardiomyopathy and hypertension. CONCLUSIONS: Our results confirm the ability of our systematic scoring analysis to determine the clinical acceptability of automated contours. This approach focused on the contours' clinical utility could ultimately improve clinicians' confidence in artificial intelligence and its acceptability in the clinical workflo

Towards automated dynamic scene analysis and augmentation during image-guided radiological and surgical interventions

This thesis proposes non-invasive automated scene analysis and augmentation techniques to improve navigation in image-guided therapy (IGT) applications. IGT refers to procedures in which physicians rely on medical images to plan, perform, and monitor an intervention. In IGT, the tomographic images acquired before the intervention may not directly correspond to what the physician sees via the intraoperative imaging. This is due to many factors such as: time-varying changes in the patient's anatomy (e.g., patient positioning or changes in pathology), risk of overexposure to ionizing radiation (restricted use of X-ray imaging), operational costs, and differences in imaging modalities. This inconsistency often results in a navigational problem that demands substantial additional effort from the physician to piece together a mental representation of complex correspondences between the preoperative images and the intraoperative scene. The first direction explored in this thesis, investigates the application of image-based motion analysis techniques for vessel segmentation. Specifically, we propose novel motion-based segmentation methods to enable safe, fast, and automatic localization of vascular structures from dynamic medical image sequences and demonstrated their efficacy in segmenting vasculature from surgical video and dynamic medical ultrasound sequences. The second direction investigates ways in which navigation uncertainties can be computed, propagated, and visualized in the context of IGT navigation systems that target deformable soft-tissues. Specifically, we present an uncertainty-encoded scene augmentation method for robot-assisted laparoscopic surgery, in which we propose visualization techniques for presenting probabilistic tumor margins. We further present a computationally efficient framework to estimate the uncertainty in deformable image registration and to subsequently propagate the effects of the computed uncertainties through to the visualizations, organ segmentations, and dosimetric evaluations performed in the context of fractionated image-guided brachytherapy. Our contributions constitute a step towards automated and real-time IGT navigation and may, in the near future, help to improve interventional outcomes for patients (improved targeting of pathologies) and increase surgical efficiency (less effort required by the physician).Applied Science, Faculty ofElectrical and Computer Engineering, Department ofGraduat

3D Surface Reconstruction of Organs using Patient-Specific Shape Priors in Robot-Assisted Laparoscopic Surgery

Abstract. With the advent of robot-assisted laparoscopic surgery (RALS), intra-operative stereo endoscopy is becoming a ubiquitous imaging modality in abdominal interventions. This high resolution intra-operative imaging modality enables the reconstruction of 3D soft-tissue surface geometry with the help of computer vision techniques. This reconstructed surface is a prerequisite for many clinical applications such as imageguidance with cross-modality registration, telestration, expansion of the surgical scene by stitching/mosaicing, and collision detection. Reconstructing the surface geometry from camera information alone remains a very challenging problem in RALS mainly due to a small baseline between the optical centres of the cameras, presence of blood and smoke, specular highlights, occlusion, and smooth/textureless regions. In this paper, we propose a method for increasing the overall surface reconstruction accuracy by incorporating patient specific shape priors extracted from pre-operative images. Our method is validated on an in silico phantom and we show that the combination of both pre-operative and intraoperative data significantly improves surface reconstruction as compared to the ground truth. Finally, we verify the clinical potential of the proposed method in the context of abdominal surgery in a phantom study of an ex vivo lamb kidney.

Towards Multi-Modal Image-Guided Tumour Identification in Robot-Assisted Partial Nephrectomy

Abstract — Tumour identification is a critical step in robotassisted partial nephrectomy (RAPN) during which the surgeon determines the tumour localization and resection margins. To help the surgeon in achieving this step, our research work aims at leveraging both pre- and intra-operative imaging modalities (CT, MRI, laparoscopic US, stereo endoscopic video) to provide an augmented reality view of kidney-tumour boundaries with uncertainty-encoded information. We present herein the progress of this research work including segmentation of preoperative scans, biomechanical simulation of deformations, stereo surface reconstruction from stereo endoscopic camera, pre-operative to intra-operative data registration, and augmented reality visualization. I

Predicting post-contrast information from contrast agent free cardiac MRI using machine learning:Challenges and methods

OBJECTIVES: Currently, administering contrast agents is necessary for accurately visualizing and quantifying presence, location, and extent of myocardial infarction (MI) with cardiac magnetic resonance (CMR). In this study, our objective is to investigate and analyze pre- and post-contrast CMR images with the goal of predicting post-contrast information using pre-contrast information only. We propose methods and identify challenges. METHODS: The study population consists of 272 retrospectively selected CMR studies with diagnoses of MI (n = 108) and healthy controls (n = 164). We describe a pipeline for pre-processing this dataset for analysis. After data feature engineering, 722 cine short-axis (SAX) images and segmentation mask pairs were used for experimentation. This constitutes 506, 108, and 108 pairs for the training, validation, and testing sets, respectively. We use deep learning (DL) segmentation (UNet) and classification (ResNet50) models to discover the extent and location of the scar and classify between the ischemic cases and healthy cases (i.e., cases with no regional myocardial scar) from the pre-contrast cine SAX image frames, respectively. We then capture complex data patterns that represent subtle signal and functional changes in the cine SAX images due to MI using optical flow, rate of change of myocardial area, and radiomics data. We apply this dataset to explore two supervised learning methods, namely, the support vector machines (SVM) and the decision tree (DT) methods, to develop predictive models for classifying pre-contrast cine SAX images as being a case of MI or healthy. RESULTS: Overall, for the UNet segmentation model, the performance based on the mean Dice score for the test set (n = 108) is 0.75 (±0.20) for the endocardium, 0.51 (±0.21) for the epicardium and 0.20 (±0.17) for the scar. For the classification task, the accuracy, F1 and precision scores of 0.68, 0.69, and 0.64, respectively, were achieved with the SVM model, and of 0.62, 0.63, and 0.72, respectively, with the DT model. CONCLUSION: We have presented some promising approaches involving DL, SVM, and DT methods in an attempt to accurately predict contrast information from non-contrast images. While our initial results are modest for this challenging task, this area of research still poses several open problems

Retraining Convolutional Neural Networks for Specialized Cardiovascular Imaging Tasks: Lessons from Tetralogy of Fallot

Ventricular contouring of cardiac magnetic resonance imaging is the gold standard for volumetric analysis for repaired tetralogy of Fallot (rTOF), but can be time-consuming and subject to variability. A convolutional neural network (CNN) ventricular contouring algorithm was developed to generate contours for mostly structural normal hearts. We aimed to improve this algorithm for use in rTOF and propose a more comprehensive method of evaluating algorithm performance. We evaluated the performance of a ventricular contouring CNN, that was trained on mostly structurally normal hearts, on rTOF patients. We then created an updated CNN by adding rTOF training cases and evaluated the new algorithm's performance generating contours for both the left and right ventricles (LV and RV) on new testing data. Algorithm performance was evaluated with spatial metrics (Dice Similarity Coefficient (DSC), Hausdorff distance, and average Hausdorff distance) and volumetric comparisons (e.g., differences in RV volumes). The original Mostly Structurally Normal (MSN) algorithm was better at contouring the LV than the RV in patients with rTOF. After retraining the algorithm, the new MSN + rTOF algorithm showed improvements for LV epicardial and RV endocardial contours on testing data to which it was naïve (N = 30; e.g., DSC 0.883 vs. 0.905 for LV epicardium at end diastole, p < 0.0001) and improvements in RV end-diastolic volumetrics (median %error 8.1 vs 11.4, p = 0.0022). Even with a small number of cases, CNN-based contouring for rTOF can be improved. This work should be extended to other forms of congenital heart disease with more extreme structural abnormalities. Aspects of this work have already been implemented in clinical practice, representing rapid clinical translation. The combined use of both spatial and volumetric comparisons yielded insights into algorithm errors