Medical Image Registration Using Deep Neural Networks

Registration is a fundamental problem in medical image analysis wherein images are transformed spatially to align corresponding anatomical structures in each image. Recently, the development of learning-based methods, which exploit deep neural networks and can outperform classical iterative methods, has received considerable interest from the research community. This interest is due in part to the substantially reduced computational requirements that learning-based methods have during inference, which makes them particularly well-suited to real-time registration applications. Despite these successes, learning-based methods can perform poorly when applied to images from different modalities where intensity characteristics can vary greatly, such as in magnetic resonance and ultrasound imaging. Moreover, registration performance is often demonstrated on well-curated datasets, closely matching the distribution of the training data. This makes it difficult to determine whether demonstrated performance accurately represents the generalization and robustness required for clinical use. This thesis presents learning-based methods which address the aforementioned difficulties by utilizing intuitive point-set-based representations, user interaction and meta-learning-based training strategies. Primarily, this is demonstrated with a focus on the non-rigid registration of 3D magnetic resonance imaging to sparse 2D transrectal ultrasound images to assist in the delivery of targeted prostate biopsies. While conventional systematic prostate biopsy methods can require many samples to be taken to confidently produce a diagnosis, tumor-targeted approaches have shown improved patient, diagnostic, and disease management outcomes with fewer samples. However, the available intraoperative transrectal ultrasound imaging alone is insufficient for accurate targeted guidance. As such, this exemplar application is used to illustrate the effectiveness of sparse, interactively-acquired ultrasound imaging for real-time, interventional registration. The presented methods are found to improve registration accuracy, relative to state-of-the-art, with substantially lower computation time and require a fraction of the data at inference. As a result, these methods are particularly attractive given their potential for real-time registration in interventional applications

Real-time multimodal image registration with partial intraoperative point-set data

We present Free Point Transformer (FPT) - a deep neural network architecture for non-rigid point-set registration. Consisting of two modules, a global feature extraction module and a point transformation module, FPT does not assume explicit constraints based on point vicinity, thereby overcoming a common requirement of previous learning-based point-set registration methods. FPT is designed to accept unordered and unstructured point-sets with a variable number of points and uses a "model-free" approach without heuristic constraints. Training FPT is flexible and involves minimizing an intuitive unsupervised loss function, but supervised, semi-supervised, and partially- or weakly-supervised training are also supported. This flexibility makes FPT amenable to multimodal image registration problems where the ground-truth deformations are difficult or impossible to measure. In this paper, we demonstrate the application of FPT to non-rigid registration of prostate magnetic resonance (MR) imaging and sparsely-sampled transrectal ultrasound (TRUS) images. The registration errors were 4.71 mm and 4.81 mm for complete TRUS imaging and sparsely-sampled TRUS imaging, respectively. The results indicate superior accuracy to the alternative rigid and non-rigid registration algorithms tested and substantially lower computation time. The rapid inference possible with FPT makes it particularly suitable for applications where real-time registration is beneficial

A novel NMF-based DWI CAD framework for prostate cancer.

In this thesis, a computer aided diagnostic (CAD) framework for detecting prostate cancer in DWI data is proposed. The proposed CAD method consists of two frameworks that use nonnegative matrix factorization (NMF) to learn meaningful features from sets of high-dimensional data. The first technique, is a three dimensional (3D) level-set DWI prostate segmentation algorithm guided by a novel probabilistic speed function. This speed function is driven by the features learned by NMF from 3D appearance, shape, and spatial data. The second technique, is a probabilistic classifier that seeks to label a prostate segmented from DWI data as either alignat, contain cancer, or benign, containing no cancer. This approach uses a NMF-based feature fusion to create a feature space where data classes are clustered. In addition, using DWI data acquired at a wide range of b-values (i.e. magnetic field strengths) is investigated. Experimental analysis indicates that for both of these frameworks, using NMF producing more accurate segmentation and classification results, respectively, and that combining the information from DWI data at several b-values can assist in detecting prostate cancer

Deep learning for an improved diagnostic pathway of prostate cancer in a small multi-parametric magnetic resonance data regime

Prostate Cancer (PCa) is the second most commonly diagnosed cancer among men, with an estimated incidence of 1.3 million new cases worldwide in 2018. The current diagnostic pathway of PCa relies on prostate-specific antigen (PSA) levels in serum. Nevertheless, PSA testing comes at the cost of under-detection of malignant lesions and a substantial over-diagnosis of indolent ones, leading to unnecessary invasive testing such biopsies and treatment in indolent PCa lesions. Magnetic Resonance Imaging (MRI) is a non-invasive technique that has emerged as a valuable tool for PCa detection, staging, early screening, treatment planning and intervention. However, analysis of MRI relies on expertise, can be time-consuming, requires specialized training and in its absence suffers from inter and intra-reader variability and sub-optimal interpretations. Deep Learning (DL) techniques have the ability to recognize complex patterns in imaging data and are able to automatize certain assessments or tasks while offering a lesser degree of subjectiveness, providing a tool that can help clinicians in their daily tasks. In spite of it, DL success has traditionally relied on the availability of large amounts of labelled data, which are rarely available in the medical field and are costly and hard to obtain due to privacy regulations of patients’ data and required specialized training, among others. This work investigates DL algorithms specially tailored to work in a limited data regime with the final objective of improving the current prostate cancer diagnostic pathway by improving the performance of DL algorithms for PCa MRI applications in a limited data regime scenario. In particular, this thesis starts by exploring Generative Adversarial Networks (GAN) to generate synthetic samples and their effect on tasks such as prostate capsule segmentation and PCa lesion significance classification (triage). Following, we explore the use of Auto-encoders (AEs) to exploit the data imbalance that is usually present in medical imaging datasets. Specifically, we propose a framework based on AEs to detect the presence of prostate lesions (tumours) by uniquely learning from control (healthy) data in an outlier detection-like fashion. This thesis also explores more recent DL paradigms that have shown promising results in natural images: generative and contrastive self-supervised learning (SSL). In both cases, we propose specific prostate MRI image manipulations for a PCa lesion classification downstream task and show the improvements offered by the techniques when compared with other initialization methods such as ImageNet pre-training. Finally, we explore data fusion techniques in order to leverage different data sources in the form of MRI sequences (orthogonal views) acquired by default during patient examinations and that are commonly ignored in DL systems. We show improvements in a PCa lesion significance classification when compared to a single input system (axial view)

Non-Cancerous Abnormalities That Could Mimic Prostate Cancer Like Signal in Multi-Parametric MRI Images

Prostate Cancer (PCa) is the most common non-cutaneous cancer in North American men. Multi-parametric magnatic resonance imaging (mpMRI) has the potential to be used as a non-invasive procedure to predict locations and prognosis of PCa. This study aims to examine non-cancerous pathology lesions and normal histology that could mimic cancer in mpMRI signals. This study includes 19 radical prostatectomy specimens from the London Health Science Centre (LHSC) that were marked with 10 strand-shaped fiducials per specimen which were used as landmarks in histology processing and ex vivo MRI. Initial registration between fiducials on histology and MR images was performed followed by the development of an interactive digital technique for deformable registration of in vivo to ex vivo MRI with digital histopathology images. The relationship between MRI signals and non-cancerous abnormalities that could mimic PCa has not been tested previously in correlation with digital histopathology imaging. The unregistered mp-MRI images are contoured by 4 individual radiology observers according to the Prostate Imaging Reporting and Data System (PI-RADS). Analysis of the radiology data showed prostatic intraepithelial neoplasia (PIN), atrophy and benign prostatic hyperplasia (BPH) as main non-cancerous abnormalities responsible for cancer like signals on mpMRI. This study will help increase the accuracy of detecting PCa and play a role in the diagnosis and classification of confounders that mimic cancer in MR images

Evaluation of an MRI-based screening pathway for prostate cancer

In recent years there has been a wealth of debate regarding prostate cancer screening, with a concurrent increase in new imaging techniques for prostate cancer diagnosis. Imaging has been the technique of choice in lung and breast cancer screening programmes but has not been explored for prostate cancer screening. Herein, this thesis explores the role of magnetic resonance imaging (MRI) as a new approach to screen for prostate cancer. Following an introduction to the current screening landscape, my thesis focuses on the development and validation of a fast MRI, known as a prostagram, that could serve as a viable image-based screening test. Evaluation of this new technique is performed within a prospective, population-based, blinded, cohort study which was conducted at seven primary care practices and two imaging centres. A diverse array of performance characteristics of fast MRI are compared to PSA. These encompass biopsy rates, cancer detection rates, diagnostic accuracy and patient reported experience measures. The second half of this thesis focuses on further optimising the fast MRI protocol for screening and exploring methods of integrating it into an alternative screening pathway. The outcomes point towards a pathway which combines a low threshold PSA and a fast MRI as yielding a more acceptable balance between benefits and harms. This is followed by the development of a risk tool to address the challenges of equivocal MRI lesions. Overall my thesis provides a balanced evaluation of fast MRI as a new screening test and the final chapter highlights outstanding challenges that must be addressed for fast MRI to progress as a legitimate screening modality. There is a requirement for all new screening tests to be evaluated in robust randomised controlled trials and the thesis concludes by setting out a phased research framework for fast MRI to enable a full evaluation over the next decade.Open Acces

The diagnostic accuracy and cost-effectiveness of magnetic resonance spectroscopy and enhanced magnetic resonance imaging techniques in aiding the localisation of prostate abnormalities for biopsy : a systematic review and economic evaluation

Peer reviewedPublisher PD