3D fusion of histology to multi-parametric MRI for prostate cancer imaging evaluation and lesion-targeted treatment planning

Multi-parametric magnetic resonance imaging (mpMRI) of localized prostate cancer has the potential to support detection, staging and localization of tumors, as well as selection, delivery and monitoring of treatments. Delineating prostate cancer tumors on imaging could potentially further support the clinical workflow by enabling precise monitoring of tumor burden in active-surveillance patients, optimized targeting of image-guided biopsies, and targeted delivery of treatments to decrease morbidity and improve outcomes. Evaluating the performance of mpMRI for prostate cancer imaging and delineation ideally includes comparison to an accurately registered reference standard, such as prostatectomy histology, for the locations of tumor boundaries on mpMRI. There are key gaps in knowledge regarding how to accurately register histological reference standards to imaging, and consequently further gaps in knowledge regarding the suitability of mpMRI for tasks, such as tumor delineation, that require such reference standards for evaluation. To obtain an understanding of the magnitude of the mpMRI-histology registration problem, we quantified the position, orientation and deformation of whole-mount histology sections relative to the formalin-fixed tissue slices from which they were cut. We found that (1) modeling isotropic scaling accounted for the majority of the deformation with a further small but statistically significant improvement from modeling affine transformation, and (2) due to the depth (mean±standard deviation (SD) 1.1±0.4 mm) and orientation (mean±SD 1.5±0.9°) of the sectioning, the assumption that histology sections are cut from the front faces of tissue slices, common in previous approaches, introduced a mean error of 0.7 mm. To determine the potential consequences of seemingly small registration errors such as described above, we investigated the impact of registration accuracy on the statistical power of imaging validation studies using a co-registered spatial reference standard (e.g. histology images) by deriving novel statistical power formulae that incorporate registration error. We illustrated, through a case study modeled on a prostate cancer imaging trial at our centre, that submillimeter differences in registration error can have a substantial impact on the required sample sizes (and therefore also the study cost) for studies aiming to detect mpMRI signal differences due to 0.5 – 2.0 cm3 prostate tumors. With the aim of achieving highly accurate mpMRI-histology registrations without disrupting the clinical pathology workflow, we developed a three-stage method for accurately registering 2D whole-mount histology images to pre-prostatectomy mpMRI that allowed flexible placement of cuts during slicing for pathology and avoided the assumption that histology sections are cut from the front faces of tissue slices. The method comprised a 3D reconstruction of histology images, followed by 3D–3D ex vivo–in vivo and in vivo–in vivo image transformations. The 3D reconstruction method minimized fiducial registration error between cross-sections of non-disruptive histology- and ex-vivo-MRI-visible strand-shaped fiducials to reconstruct histology images into the coordinate system of an ex vivo MR image. We quantified the mean±standard deviation target registration error of the reconstruction to be 0.7±0.4 mm, based on the post-reconstruction misalignment of intrinsic landmark pairs. We also compared our fiducial-based reconstruction to an alternative reconstruction based on mutual-information-based registration, an established method for multi-modality registration. We found that the mean target registration error for the fiducial-based method (0.7 mm) was lower than that for the mutual-information-based method (1.2 mm), and that the mutual-information-based method was less robust to initialization error due to multiple sources of error, including the optimizer and the mutual information similarity metric. The second stage of the histology–mpMRI registration used interactively defined 3D–3D deformable thin-plate-spline transformations to align ex vivo to in vivo MR images to compensate for deformation due to endorectal MR coil positioning, surgical resection and formalin fixation. The third stage used interactively defined 3D–3D rigid or thin-plate-spline transformations to co-register in vivo mpMRI images to compensate for patient motion and image distortion. The combined mean registration error of the histology–mpMRI registration was quantified to be 2 mm using manually identified intrinsic landmark pairs. Our data set, comprising mpMRI, target volumes contoured by four observers and co-registered contoured and graded histology images, was used to quantify the positive predictive values and variability of observer scoring of lesions following the Prostate Imaging Reporting and Data System (PI-RADS) guidelines, the variability of target volume contouring, and appropriate expansion margins from target volumes to achieve coverage of histologically defined cancer. The analysis of lesion scoring showed that a PI-RADS overall cancer likelihood of 5, denoting “highly likely cancer”, had a positive predictive value of 85% for Gleason 7 cancer (and 93% for lesions with volumes \u3e0.5 cm3 measured on mpMRI) and that PI-RADS scores were positively correlated with histological grade (ρ=0.6). However, the analysis also showed interobserver differences in PI-RADS score of 0.6 to 1.2 (on a 5-point scale) and an agreement kappa value of only 0.30. The analysis of target volume contouring showed that target volume contours with suitable margins can achieve near-complete histological coverage for detected lesions, despite the presence of high interobserver spatial variability in target volumes. Prostate cancer imaging and delineation have the potential to support multiple stages in the management of localized prostate cancer. Targeted biopsy procedures with optimized targeting based on tumor delineation may help distinguish patients who need treatment from those who need active surveillance. Ongoing monitoring of tumor burden based on delineation in patients undergoing active surveillance may help identify those who need to progress to therapy early while the cancer is still curable. Preferentially targeting therapies at delineated target volumes may lower the morbidity associated with aggressive cancer treatment and improve outcomes in low-intermediate-risk patients. Measurements of the accuracy and variability of lesion scoring and target volume contouring on mpMRI will clarify its value in supporting these roles

Tools for improving high-dose-rate prostate cancer brachytherapy using three-dimensional ultrasound and magnetic resonance imaging

High-dose-rate brachytherapy (HDR-BT) is an interstitial technique for the treatment of intermediate and high-risk localized prostate cancer that involves placement of a radiation source directly inside the prostate using needles. Dose-escalated whole-gland treatments have led to improvements in survival, and tumour-targeted treatments may offer future improvements in therapeutic ratio. The efficacy of tumour-targeted HDR-BT depends on imaging tools to enable accurate dose delivery to prostate sub-volumes. This thesis is focused on implementing ultrasound tools to improve HDR-BT needle localization accuracy and efficiency, and evaluating dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) for tumour localization. First, we implemented a device enabling sagittally-reconstructed 3D (SR3D) ultrasound, which provides sub-millimeter resolution in the needle insertion direction. We acquired SR3D and routine clinical images in a cohort of 12 consecutive eligible HDR-BT patients, with a total of 194 needles. The SR3D technique provided needle insertion depth errors within 5 mm for 93\% of needles versus 76\% for the clinical imaging technique, leading to increased precision in dose delivered to the prostate. Second, we implemented an algorithm to automatically segment multiple HDR-BT needles in a SR3D image. The algorithm was applied to the SR3D images from the first patient cohort, demonstrating mean execution times of 11.0 s per patient and successfully segmenting 82\% of needles within 3 mm. Third, we augmented SR3D imaging with live-2D sagittal ultrasound for needle tip localization. This combined technique was applied to another cohort of 10 HDR-BT patients, reducing insertion depth errors compared to routine imaging from a range of [-8.1 mm, 7.7 mm] to [-6.2 mm, 5.9 mm]. Finally, we acquired DCE-MRI in 16 patients scheduled to undergo prostatectomy, using either high spatial resolution or high temporal resolution imaging, and compared the images to whole-mount histology. The high spatial resolution images demonstrated improved high-grade cancer classification compared to the high temporal resolution images, with areas under the receiver operating characteristic curve of 0.79 and 0.70, respectively. In conclusion, we have translated and evaluated specialized imaging tools for HDR-BT which are ready to be tested in a clinical trial investigating tumour-targeted treatment

Super-Resolution Contrast Enhanced Ultrasound Methodology for the Identification of in-Vivo Vascular Dynamics in 2D

\u3cp\u3eObjectives The aim of this study was to provide an ultrasound-based super-resolution methodology that can be implemented using clinical 2-dimensional ultrasound equipment and standard contrast-enhanced ultrasound modes. In addition, the aim is to achieve this for true-to-life patient imaging conditions, including realistic examination times of a few minutes and adequate image penetration depths that can be used to scan entire organs without sacrificing current super-resolution ultrasound imaging performance. Methods Standard contrast-enhanced ultrasound was used along with bolus or infusion injections of SonoVue (Bracco, Geneva, Switzerland) microbubble (MB) suspensions. An image analysis methodology, translated from light microscopy algorithms, was developed for use with ultrasound contrast imaging video data. New features that are tailored for ultrasound contrast image data were developed for MB detection and segmentation, so that the algorithm can deal with single and overlapping MBs. The method was tested initially on synthetic data, then with a simple microvessel phantom, and then with in vivo ultrasound contrast video loops from sheep ovaries. Tracks detailing the vascular structure and corresponding velocity map of the sheep ovary were reconstructed. Images acquired from light microscopy, optical projection tomography, and optical coherence tomography were compared with the vasculature network that was revealed in the ultrasound contrast data. The final method was applied to clinical prostate data as a proof of principle. Results Features of the ovary identified in optical modalities mentioned previously were also identified in the ultrasound super-resolution density maps. Follicular areas, follicle wall, vessel diameter, and tissue dimensions were very similar. An approximately 8.5-fold resolution gain was demonstrated in vessel width, as vessels of width down to 60 μm were detected and verified (λ = 514 μm). Best agreement was found between ultrasound measurements and optical coherence tomography with 10% difference in the measured vessel widths, whereas ex vivo microscopy measurements were significantly lower by 43% on average. The results were mostly achieved using video loops of under 2-minute duration that included respiratory motion. A feasibility study on a human prostate showed good agreement between density and velocity ultrasound maps with the histological evaluation of the location of a tumor. Conclusions The feasibility of a 2-dimensional contrast-enhanced ultrasound-based super-resolution method was demonstrated using in vitro, synthetic and in vivo animal data. The method reduces the examination times to a few minutes using state-of-the-art ultrasound equipment and can provide super-resolution maps for an entire prostate with similar resolution to that achieved in other studies.\u3c/p\u3

Deformable image registration between pathological images and MR image via an optical macro image

Computed tomography (CT) and magnetic resonance (MR) imaging have been widely used for visualizing the inside of the human body. However, in many cases, pathological diagnosis is conducted through a biopsy or resection of an organ to evaluate the condition of tissues as definitive diagnosis. To provide more advanced information onto CT or MR image, it is necessary to reveal the relationship between tissue information and image signals. We propose a registration scheme for a set of PT images of divided specimens and a 3D-MR image by reference to an optical macro image (OM image) captured by an optical camera. We conducted a fundamental study using a resected human brain after the death of a brain cancer patient. We constructed two kinds of registration processes using the OM image as the base for both registrations to make conversion parameters between the PT and MR images. The aligned PT images had shapes similar to the OM image. On the other hand, the extracted cross-sectional MR image was similar to the OM image. From these resultant conversion parameters, the corresponding region on the PT image could be searched and displayed when an arbitrary pixel on the MR image was selected. The relationship between the PT and MR images of the whole brain can be analyzed using the proposed method. We confirmed that same regions between the PT and MR images could be searched and displayed using resultant information obtained by the proposed method. In terms of the accuracy of proposed method, the TREs were 0.56 ± 0.39 mm and 0.87 ± 0.42 mm. We can analyze the relationship between tissue information and MR signals using the proposed method

Multi-modal matching of 2D images with 3D medical data

Image registration is the process of aligning images of the same object taken at different time points or with different imaging modalities with the aim to compare them in one coordinate system. Image registration is particularly important in biomedical imaging, where a multitude of imaging modalities exist. For example, images can be obtained with X-ray computed tomography (CT) which is based on the object’s X-ray beam attenuation whereas magnetic resonance imaging (MRI) underlines its local proton density. The gold standard in pathology for tissue analysis is histology. Histology, however, provides only 2D information in the selected sections of the 3D tissue. To evaluate the tissue’s 3D structure, volume imaging techniques, such as CT or MRI, are preferable. The combination of functional information from histology with 3D morphological data from CT is essential for tissue analysis. Furthermore, histology can validate anatomical features identified in CT data. Therefore, the registration of these two modalities is indispensable to provide a more complete overview of the tissue. Previously proposed algorithms for the registration of histological slides into 3D volumes usually rely on manual interactions, which is time-consuming and prone to bias. The high complexity of this type of registration originates from the large number of degrees of freedom. The goal of my thesis was to develop an automatic method for histology to 3D volume registration to master these challenges. The first stage of the developed algorithm uses a scale-invariant feature detector to find common matches between the histology slide and each tomography slice in a 3D dataset. A plane of the most likely position is then fitted into the feature point cloud using a robust model fitting algorithm. The second stage builds upon the first one and introduces fine-tuning of the slice position using normalized Mutual Information (NMI). Additionally, using previously developed 2D-2D registration techniques we find the rotation and translation of the histological slide within the plane. Moreover, the framework takes into account any potential nonlinear deformations of the histological slides that might occur during tissue preparation. The application of the algorithm to MRI data is investigated in our third work. The developed extension of the multi-modal feature detector showed promising results, however, the registration of a histological slide to the direct MRI volume remains a challenging task

Registration of histology and magnetic resonance imaging of the brain

Combining histology and non-invasive imaging has been attracting the attention of the medical imaging community for a long time, due to its potential to correlate macroscopic information with the underlying microscopic properties of tissues. Histology is an invasive procedure that disrupts the spatial arrangement of the tissue components but enables visualisation and characterisation at a cellular level. In contrast, macroscopic imaging allows non-invasive acquisition of volumetric information but does not provide any microscopic details. Through the establishment of spatial correspondences obtained via image registration, it is possible to compare micro- and macroscopic information and to recover the original histological arrangement in three dimensions. In this thesis, I present: (i) a survey of the literature relative to methods for histology reconstruction with and without the help of 3D medical imaging; (ii) a graph-theoretic method for histology volume reconstruction from sets of 2D sections, without external information; (iii) a method for multimodal 2D linear registration between histology and MRI based on partial matching of shape-informative boundaries