Multimodal breast imaging: Registration, visualization, and image synthesis

The benefit of registration and fusion of functional images with anatomical images is well appreciated in the advent of combined positron emission tomography and x-ray computed tomography scanners (PET/CT). This is especially true in breast cancer imaging, where modalities such as high-resolution and dynamic contrast-enhanced magnetic resonance imaging (MRI) and F-18-FDG positron emission tomography (PET) have steadily gained acceptance in addition to x-ray mammography, the primary detection tool. The increased interest in combined PET/MRI images has facilitated the demand for appropriate registration and fusion algorithms. A new approach to MRI-to-PET non-rigid breast image registration was developed and evaluated based on the location of a small number of fiducial skin markers (FSMs) visible in both modalities. The observed FSM displacement vectors between MRI and PET, distributed piecewise linearly over the breast volume, produce a deformed Finite-Element mesh that reasonably approximates non-rigid deformation of the breast tissue between the MRI and PET scans. The method does not require a biomechanical breast tissue model, and is robust and fast. The method was evaluated both qualitatively and quantitatively on patients and a deformable breast phantom. The procedure yields quality images with average target registration error (TRE) below 4 mm. The importance of appropriately jointly displaying (i.e. fusing) the registered images has often been neglected and underestimated. A combined MRI/PET image has the benefits of directly showing the spatial relationships between the two modalities, increasing the sensitivity, specificity, and accuracy of diagnosis. Additional information on morphology and on dynamic behavior of the suspicious lesion can be provided, allowing more accurate lesion localization including mapping of hyper- and hypo-metabolic regions as well as better lesion-boundary definition, improving accuracy when grading the breast cancer and assessing the need for biopsy. Eight promising fusion-for-visualization techniques were evaluated by radiologists from University Hospital, in Syracuse, NY. Preliminary results indicate that the radiologists were better able to perform a series of tasks when reading the fused PET/MRI data sets using color tables generated by a newly developed genetic algorithm, as compared to other commonly used schemes. The lack of a known ground truth hinders the development and evaluation of new algorithms for tasks such as registration and classification. A preliminary mesh-based breast phantom containing 12 distinct tissue classes along with tissue properties necessary for the simulation of dynamic positron emission tomography scans was created. The phantom contains multiple components which can be separately manipulated, utilizing geometric transformations, to represent populations or a single individual being imaged in multiple positions. This phantom will support future multimodal breast imaging work

Nonrigid Registration of 3-Dimensional Images of the Carotid Arteries

Atherosclerosis at the carotid bifurcation can result in cerebral emboli, which in turn can block the blood supply to the brain causing ischemic strokes. Non-invasive imaging tools that characterize arterial wall, and atherosclerotic plaque structure and composition may help to determine the factors, which lead to the development of unstable lesions, and identify patients at risk of plaque disruption. Registration of 3D ultrasound (US) images of carotid plaque obtained at different time points, and with Magnetic Resonance (MR) images are required for monitoring of plaque changes in volume and surface morphology, and combining the complementary information of the two modalities for better understanding of factors that define plaque vulnerability. These registration techniques should be nonrigid, to remove deformations caused by bending and torsion in the neck during image acquisition sessions. The high degrees of freedom and large number of parameters associated with nonrigid image registration methods causes several problems including unnatural plaque morphology alteration, high computational complexity, and low reliability. Thus, we used a “twisting and bending” model with only six parameters to model the natural movement of the neck for nonrigid registration. We calculated the Mean Registration Error (MRE) between the segmented vessel surfaces in the target and the registered images using the distance between “matched points” to evaluate registration results. We registered 3D US carotid images acquired at different head positions to simulate images acquired at different times, and obtained an average MRE of 0.8±0.3mm for nonrigid registration. We registered 3D US and MR carotid images at field strengths, 1.5T and 3.0T, of the same subject acquired on the same day, and obtained an average MRE of 1.4±0.3mm for 1.5T and 1.5±0.4mm for 3.0T, using nonrigid registration. Furthermore, we showed that the error metric used here was not significantly different from the widely accepted Target Registration Error (TRE)

Landmark Localization, Feature Matching and Biomarker Discovery from Magnetic Resonance Images

The work presented in this thesis proposes several methods that can be roughly divided into three different categories: I) landmark localization in medical images, II) feature matching for image registration, and III) biomarker discovery in neuroimaging. The first part deals with the identification of anatomical landmarks. The motivation stems from the fact that the manual identification and labeling of these landmarks is very time consuming and prone to observer errors, especially when large datasets must be analyzed. In this thesis we present three methods to tackle this challenge: A landmark descriptor based on local self-similarities (SS), a subspace building framework based on manifold learning and a sparse coding landmark descriptor based on data-specific learned dictionary basis. The second part of this thesis deals with finding matching features between a pair of images. These matches can be used to perform a registration between them. Registration is a powerful tool that allows mapping images in a common space in order to aid in their analysis. Accurate registration can be challenging to achieve using intensity based registration algorithms. Here, a framework is proposed for learning correspondences in pairs of images by matching SS features and random sample and consensus (RANSAC) is employed as a robust model estimator to learn a deformation model based on feature matches. Finally, the third part of the thesis deals with biomarker discovery using machine learning. In this section a framework for feature extraction from learned low-dimensional subspaces that represent inter-subject variability is proposed. The manifold subspace is built using data-driven regions of interest (ROI). These regions are learned via sparse regression, with stability selection. Also, probabilistic distribution models for different stages in the disease trajectory are estimated for different class populations in the low-dimensional manifold and used to construct a probabilistic scoring function.Open Acces

Quantification of tumour heterogenity in MRI

Cancer is the leading cause of death that touches us all, either directly or indirectly. It is estimated that the number of newly diagnosed cases in the Netherlands will increase to 123,000 by the year 2020. General Dutch statistics are similar to those in the UK, i.e. over the last ten years, the age-standardised incidence rate1 has stabilised at around 355 females and 415 males per 100,000. Figure 1 shows the cancer incidence per gender. In the UK, the rise in lifetime risk of cancer is more than one in three and depends on many factors, including age, lifestyle and genetic makeup

Sub-pixel Registration In Computational Imaging And Applications To Enhancement Of Maxillofacial Ct Data

In computational imaging, data acquired by sampling the same scene or object at different times or from different orientations result in images in different coordinate systems. Registration is a crucial step in order to be able to compare, integrate and fuse the data obtained from different measurements. Tomography is the method of imaging a single plane or slice of an object. A Computed Tomography (CT) scan, also known as a CAT scan (Computed Axial Tomography scan), is a Helical Tomography, which traditionally produces a 2D image of the structures in a thin section of the body. It uses X-ray, which is ionizing radiation. Although the actual dose is typically low, repeated scans should be limited. In dentistry, implant dentistry in specific, there is a need for 3D visualization of internal anatomy. The internal visualization is mainly based on CT scanning technologies. The most important technological advancement which dramatically enhanced the clinician\u27s ability to diagnose, treat, and plan dental implants has been the CT scan. Advanced 3D modeling and visualization techniques permit highly refined and accurate assessment of the CT scan data. However, in addition to imperfections of the instrument and the imaging process, it is not uncommon to encounter other unwanted artifacts in the form of bright regions, flares and erroneous pixels due to dental bridges, metal braces, etc. Currently, removing and cleaning up the data from acquisition backscattering imperfections and unwanted artifacts is performed manually, which is as good as the experience level of the technician. On the other hand the process is error prone, since the editing process needs to be performed image by image. We address some of these issues by proposing novel registration methods and using stonecast models of patient\u27s dental imprint as reference ground truth data. Stone-cast models were originally used by dentists to make complete or partial dentures. The CT scan of such stone-cast models can be used to automatically guide the cleaning of patients\u27 CT scans from defects or unwanted artifacts, and also as an automatic segmentation system for the outliers of the CT scan data without use of stone-cast models. Segmented data is subsequently used to clean the data from artifacts using a new proposed 3D inpainting approach

Atlas based image reconstruction for diffuse optical imaging of the human brain

Diffuse Optical Tomography (DOT) has been applied to imaging functional activations in the adult brain. Registered-atlas models are acceptable alternative forward models for the subject-specific models. In this work, different landmark-based rigid registration methods are quantitatively evaluated and compared in geometrical accuracy of the registration result, accuracy of light propagation approximation and recovery accuracy of the brain activations based on the whole head and localized head regions. The most suitable registration methods are selected based on accuracy and efficiency and they vary based on region of interest. For example, the most suitable registration method for recovery of whole cortex activation is the registration method based on 19 landmarks from the EEG 10/20 system and non-iterative Point to Point algorithms (EEG19nP2P registration). Efficiency of the recovery process is another popular research area in DOT. In this work, a modified generation approach of the light propagation approximation is designed based on a reduced sensitivity matrix and parallelisation process. It improves the storage efficiency by >1000% and time efficiency by ~400%. Based on this approach, the brain activation recovery of DOT can be processed on a normal laptop without large memory requirements within 45 minutes which is more suitable for a portable system

Automated analysis and visualization of preclinical whole-body microCT data

In this thesis, several strategies are presented that aim to facilitate the analysis and visualization of whole-body in vivo data of small animals. Based on the particular challenges for image processing, when dealing with whole-body follow-up data, we addressed several aspects in this thesis. The developed methods are tailored to handle data of subjects with significantly varying posture and address the large tissue heterogeneity of entire animals. In addition, we aim to compensate for lacking tissue contrast by relying on approximation of organs based on an animal atlas. Beyond that, we provide a solution to automate the combination of multimodality, multidimensional data.* Advanced School for Computing and Imaging (ASCI), Delft, NL * Bontius Stichting inz Doelfonds Beeldverwerking, Leiden, NL * Caliper Life Sciences, Hopkinton, USA * Foundation Imago, Oegstgeest, NLUBL - phd migration 201