Software and Hardware-based Tools for Improving Ultrasound Guided Prostate Brachytherapy

Minimally invasive procedures for prostate cancer diagnosis and treatment, including biopsy and brachytherapy, rely on medical imaging such as two-dimensional (2D) and three-dimensional (3D) transrectal ultrasound (TRUS) and magnetic resonance imaging (MRI) for critical tasks such as target definition and diagnosis, treatment guidance, and treatment planning. Use of these imaging modalities introduces challenges including time-consuming manual prostate segmentation, poor needle tip visualization, and variable MR-US cognitive fusion. The objective of this thesis was to develop, validate, and implement software- and hardware-based tools specifically designed for minimally invasive prostate cancer procedures to overcome these challenges. First, a deep learning-based automatic 3D TRUS prostate segmentation algorithm was developed and evaluated using a diverse dataset of clinical images acquired during prostate biopsy and brachytherapy procedures. The algorithm significantly outperformed state-of-the-art fully 3D CNNs trained using the same dataset while a segmentation time of 0.62 s demonstrated a significant reduction compared to manual segmentation. Next, the impact of dataset size, image quality, and image type on segmentation performance using this algorithm was examined. Using smaller training datasets, segmentation accuracy was shown to plateau with as little as 1000 training images, supporting the use of deep learning approaches even when data is scarce. The development of an image quality grading scale specific to 3D TRUS images will allow for easier comparison between algorithms trained using different datasets. Third, a power Doppler (PD) US-based needle tip localization method was developed and validated in both phantom and clinical cases, demonstrating reduced tip error and variation for obstructed needles compared to conventional US. Finally, a surface-based MRI-3D TRUS deformable image registration algorithm was developed and implemented clinically, demonstrating improved registration accuracy compared to manual rigid registration and reduced variation compared to the current clinical standard of physician cognitive fusion. These generalizable and easy-to-implement tools have the potential to improve workflow efficiency and accuracy for minimally invasive prostate procedures

A review of artificial intelligence in prostate cancer detection on imaging

A multitude of studies have explored the role of artificial intelligence (AI) in providing diagnostic support to radiologists, pathologists, and urologists in prostate cancer detection, risk-stratification, and management. This review provides a comprehensive overview of relevant literature regarding the use of AI models in (1) detecting prostate cancer on radiology images (magnetic resonance and ultrasound imaging), (2) detecting prostate cancer on histopathology images of prostate biopsy tissue, and (3) assisting in supporting tasks for prostate cancer detection (prostate gland segmentation, MRI-histopathology registration, MRI-ultrasound registration). We discuss both the potential of these AI models to assist in the clinical workflow of prostate cancer diagnosis, as well as the current limitations including variability in training data sets, algorithms, and evaluation criteria. We also discuss ongoing challenges and what is needed to bridge the gap between academic research on AI for prostate cancer and commercial solutions that improve routine clinical care

Real-time multimodal image registration with partial intraoperative point-set data

We present Free Point Transformer (FPT) - a deep neural network architecture for non-rigid point-set registration. Consisting of two modules, a global feature extraction module and a point transformation module, FPT does not assume explicit constraints based on point vicinity, thereby overcoming a common requirement of previous learning-based point-set registration methods. FPT is designed to accept unordered and unstructured point-sets with a variable number of points and uses a "model-free" approach without heuristic constraints. Training FPT is flexible and involves minimizing an intuitive unsupervised loss function, but supervised, semi-supervised, and partially- or weakly-supervised training are also supported. This flexibility makes FPT amenable to multimodal image registration problems where the ground-truth deformations are difficult or impossible to measure. In this paper, we demonstrate the application of FPT to non-rigid registration of prostate magnetic resonance (MR) imaging and sparsely-sampled transrectal ultrasound (TRUS) images. The registration errors were 4.71 mm and 4.81 mm for complete TRUS imaging and sparsely-sampled TRUS imaging, respectively. The results indicate superior accuracy to the alternative rigid and non-rigid registration algorithms tested and substantially lower computation time. The rapid inference possible with FPT makes it particularly suitable for applications where real-time registration is beneficial

Medical Image Registration Using Deep Neural Networks

Registration is a fundamental problem in medical image analysis wherein images are transformed spatially to align corresponding anatomical structures in each image. Recently, the development of learning-based methods, which exploit deep neural networks and can outperform classical iterative methods, has received considerable interest from the research community. This interest is due in part to the substantially reduced computational requirements that learning-based methods have during inference, which makes them particularly well-suited to real-time registration applications. Despite these successes, learning-based methods can perform poorly when applied to images from different modalities where intensity characteristics can vary greatly, such as in magnetic resonance and ultrasound imaging. Moreover, registration performance is often demonstrated on well-curated datasets, closely matching the distribution of the training data. This makes it difficult to determine whether demonstrated performance accurately represents the generalization and robustness required for clinical use. This thesis presents learning-based methods which address the aforementioned difficulties by utilizing intuitive point-set-based representations, user interaction and meta-learning-based training strategies. Primarily, this is demonstrated with a focus on the non-rigid registration of 3D magnetic resonance imaging to sparse 2D transrectal ultrasound images to assist in the delivery of targeted prostate biopsies. While conventional systematic prostate biopsy methods can require many samples to be taken to confidently produce a diagnosis, tumor-targeted approaches have shown improved patient, diagnostic, and disease management outcomes with fewer samples. However, the available intraoperative transrectal ultrasound imaging alone is insufficient for accurate targeted guidance. As such, this exemplar application is used to illustrate the effectiveness of sparse, interactively-acquired ultrasound imaging for real-time, interventional registration. The presented methods are found to improve registration accuracy, relative to state-of-the-art, with substantially lower computation time and require a fraction of the data at inference. As a result, these methods are particularly attractive given their potential for real-time registration in interventional applications

Registration of magnetic resonance and ultrasound images for guiding prostate cancer interventions

Prostate cancer is a major international health problem with a large and rising incidence in many parts of the world. Transrectal ultrasound (TRUS) imaging is used routinely to guide surgical procedures, such as needle biopsy and a number of minimally-invasive therapies, but its limited ability to visualise prostate cancer is widely recognised. Magnetic resonance (MR) imaging techniques, on the other hand, have recently been developed that can provide clinically useful diagnostic information. Registration (or alignment) of MR and TRUS images during TRUS-guided surgical interventions potentially provides a cost-effective approach to augment TRUS images with clinically useful, MR-derived information (for example, tumour location, shape and size). This thesis describes a deformable image registration framework that enables automatic and/or semi-automatic alignment of MR and 3D TRUS images of the prostate gland. The method combines two technical developments in the field: First, a method for constructing patient-specific statistical shape models of prostate motion/deformation, based on learning from finite element simulations of gland motion using geometric data from a preoperative MR image, is proposed. Second, a novel “model-to-image” registration framework is developed to register this statistical shape model automatically to an intraoperative TRUS image. This registration approach is implemented using a novel model-to-image vector alignment (MIVA) algorithm, which maximises the likelihood of a particular instance of a statistical shape model given a voxel-intensity-based feature vector that represents an estimate of the surface normal vectors at the boundary of the organ in question. Using real patient data, the MR-TRUS registration accuracy of the new algorithm is validated using intra-prostatic anatomical landmarks. A rigorous and extensive validation analysis is also provided for assessing the image registration experiments. The final target registration error after performing 100 MR–TRUS registrations for each patient have a median of 2.40 mm, meaning that over 93% registrations may successfully hit the target representing a clinically significant lesion. The implemented registration algorithms took less than 30 seconds and 2 minutes for manually defined point- and normal vector features, respectively. The thesis concludes with a summary of potential applications and future research directions

Image-guided prostate biopsy robots: A review

At present, the incidence of prostate cancer (PCa) in men is increasing year by year. So, the early diagnosis of PCa is of great significance. Transrectal ultrasonography (TRUS)-guided biopsy is a common method for diagnosing PCa. The biopsy process is performed manually by urologists but the diagnostic rate is only 20%–30% and its reliability and accuracy can no longer meet clinical needs. The image-guided prostate biopsy robot has the advantages of a high degree of automation, does not rely on the skills and experience of operators, reduces the work intensity and operation time of urologists and so on. Capable of delivering biopsy needles to pre-defined biopsy locations with minimal needle placement errors, it makes up for the shortcomings of traditional free-hand biopsy and improves the reliability and accuracy of biopsy. The integration of medical imaging technology and the robotic system is an important means for accurate tumor location, biopsy puncture path planning and visualization. This paper mainly reviews image-guided prostate biopsy robots. According to the existing literature, guidance modalities are divided into magnetic resonance imaging (MRI), ultrasound (US) and fusion image. First, the robot structure research by different guided methods is the main line and the actuators and material research of these guided modalities is the auxiliary line to introduce and compare. Second, the robot image-guided localization technology is discussed. Finally, the image-guided prostate biopsy robot is summarized and suggestions for future development are provided