Longitudinal Image Registration with Temporal-Order and Subject-Specificity Discrimination

Morphological analysis of longitudinal MR images plays a key role in monitoring disease progression for prostate cancer patients, who are placed under an active surveillance program. In this paper, we describe a learning-based image registration algorithm to quantify changes on regions of interest between a pair of images from the same patient, acquired at two different time points. Combining intensity-based similarity and gland segmentation as weak supervision, the population-data-trained registration networks significantly lowered the target registration errors (TREs) on holdout patient data, compared with those before registration and those from an iterative registration algorithm. Furthermore, this work provides a quantitative analysis on several longitudinal-data-sampling strategies and, in turn, we propose a novel regularisation method based on maximum mean discrepancy, between differently-sampled training image pairs. Based on 216 3D MR images from 86 patients, we report a mean TRE of 5.6 mm and show statistically significant differences between the different training data sampling strategies

Multidirectional and Topography-based Dynamic-scale Varifold Representations with Application to Matching Developing Cortical Surfaces

The human cerebral cortex is marked by great complexity as well as substantial dynamic changes during early postnatal development. To obtain a fairly comprehensive picture of its age-induced and/or disorder-related cortical changes, one needs to match cortical surfaces to one another, while maximizing their anatomical alignment. Methods that geodesically shoot surfaces into one another as currents (a distribution of oriented normals) and varifolds (a distribution of non-oriented normals) provide an elegant Riemannian framework for generic surface matching and reliable statistical analysis. However, both conventional current and varifold matching methods have two key limitations. First, they only use the normals of the surface to measure its geometry and guide the warping process, which overlooks the importance of the orientations of the inherently convoluted cortical sulcal and gyral folds. Second, the ‘conversion’ of a surface into a current or a varifold operates at a fixed scale under which geometric surface details will be neglected, which ignores the dynamic scales of cortical foldings. To overcome these limitations and improve varifold-based cortical surface registration, we propose two different strategies. The first strategy decomposes each cortical surface into its normal and tangent varifold representations, by integrating principal curvature direction field into the varifold matching framework, thus providing rich information of the orientation of cortical folding and better characterization of the complex cortical geometry. The second strategy explores the informative cortical geometric features to perform a dynamic-scale measurement of the cortical surface that depends on the local surface topography (e.g., principal curvature), thereby we introduce the concept of a topography-based dynamic-scale varifold. We tested the proposed varifold variants for registering 12 pairs of dynamically developing cortical surfaces from 0 to 6 months of age. Both variants improved the matching accuracy in terms of closeness to the target surface and the goodness of alignment with regional anatomical boundaries, when compared with three state-of-the-art methods: (1) diffeomorphic spectral matching, (2) conventional current-based surface matching, and (3) conventional varifold-based surface matching

Longitudinal Image Registration with Temporal-order and Subject-specificity Discrimination

Morphological analysis of longitudinal MR images plays a key role in monitoring disease progression for prostate cancer patients, who are placed under an active surveillance program. In this paper, we describe a learning-based image registration algorithm to quantify changes on regions of interest between a pair of images from the same patient, acquired at two different time points. Combining intensity-based similarity and gland segmentation as weak supervision, the population-data-trained registration networks significantly lowered the target registration errors (TREs) on holdout patient data, compared with those before registration and those from an iterative registration algorithm. Furthermore, this work provides a quantitative analysis on several longitudinal-data-sampling strategies and, in turn, we propose a novel regularisation method based on maximum mean discrepancy, between differently-sampled training image pairs. Based on 216 3D MR images from 86 patients, we report a mean TRE of 5.6 mm and show statistically significant differences between the different training data sampling strategies.Comment: Accepted at MICCAI 202

Heterogeneous volumetric data mapping and its medical applications

With the advance of data acquisition techniques, massive solid geometries are being collected routinely in scientific tasks, these complex and unstructured data need to be effectively correlated for various processing and analysis. Volumetric mapping solves bijective low-distortion correspondence between/among 3D geometric data, and can serve as an important preprocessing step in many tasks in compute-aided design and analysis, industrial manufacturing, medical image analysis, to name a few. This dissertation studied two important volumetric mapping problems: the mapping of heterogeneous volumes (with nonuniform inner structures/layers) and the mapping of sequential dynamic volumes. To effectively handle heterogeneous volumes, first, we studied the feature-aligned harmonic volumetric mapping. Compared to previous harmonic mapping, it supports the point, curve, and iso-surface alignment, which are important low-dimensional structures in heterogeneous volumetric data. Second, we proposed a biharmonic model for volumetric mapping. Unlike the conventional harmonic volumetric mapping that only supports positional continuity on the boundary, this new model allows us to have higher order continuity $C^1$ along the boundary surface. This suggests a potential model to solve the volumetric mapping of complex and big geometries through divide-and-conquer. We also studied the medical applications of our volumetric mapping in lung tumor respiratory motion modeling. We were building an effective digital platform for lung tumor radiotherapy based on effective volumetric CT/MRI image matching and analysis. We developed and integrated in this platform a set of geometric/image processing techniques including advanced image segmentation, finite element meshing, volumetric registration and interpolation. The lung organ/tumor and surrounding tissues are treated as a heterogeneous region and a dynamic 4D registration framework is developed for lung tumor motion modeling and tracking. Compared to the previous 3D pairwise registration, our new 4D parameterization model leads to a significantly improved registration accuracy. The constructed deforming model can hence approximate the deformation of the tissues and tumor

Longitudinal Image Registration With Temporally-Dependent Image Similarity Measure

Longitudinal imaging studies are frequently used to investigate temporal changes in brain morphology and often require spatial correspondence between images achieved through image registration. Beside morphological changes, image intensity may also change over time, for example when studying brain maturation. However, such intensity changes are not accounted for in image similarity measures for standard image registration methods. Hence, (i) local similarity measures, (ii) methods estimating intensity transformations between images, and (iii) metamorphosis approaches have been developed to either achieve robustness with respect to intensity changes or to simultaneously capture spatial and intensity changes. For these methods, longitudinal intensity changes are not explicitly modeled and images are treated as independent static samples. Here, we propose a model-based image similarity measure for longitudinal image registration that estimates a temporal model of intensity change using all available images simultaneously

Single View Reconstruction for Human Face and Motion with Priors

Single view reconstruction is fundamentally an under-constrained problem. We aim to develop new approaches to model human face and motion with model priors that restrict the space of possible solutions. First, we develop a novel approach to recover the 3D shape from a single view image under challenging conditions, such as large variations in illumination and pose. The problem is addressed by employing the techniques of non-linear manifold embedding and alignment. Specifically, the local image models for each patch of facial images and the local surface models for each patch of 3D shape are learned using a non-linear dimensionality reduction technique, and the correspondences between these local models are then learned by a manifold alignment method. Local models successfully remove the dependency of large training databases for human face modeling. By combining the local shapes, the global shape of a face can be reconstructed directly from a single linear system of equations via least square. Unfortunately, this learning-based approach cannot be successfully applied to the problem of human motion modeling due to the internal and external variations in single view video-based marker-less motion capture. Therefore, we introduce a new model-based approach for capturing human motion using a stream of depth images from a single depth sensor. While a depth sensor provides metric 3D information, using a single sensor, instead of a camera array, results in a view-dependent and incomplete measurement of object motion. We develop a novel two-stage template fitting algorithm that is invariant to subject size and view-point variations, and robust to occlusions. Starting from a known pose, our algorithm first estimates a body configuration through temporal registration, which is used to search the template motion database for a best match. The best match body configuration as well as its corresponding surface mesh model are deformed to fit the input depth map, filling in the part that is occluded from the input and compensating for differences in pose and body-size between the input image and the template. Our approach does not require any makers, user-interaction, or appearance-based tracking. Experiments show that our approaches can achieve good modeling results for human face and motion, and are capable of dealing with variety of challenges in single view reconstruction, e.g., occlusion

Automated Extraction of Biomarkers for Alzheimer's Disease from Brain Magnetic Resonance Images

In this work, different techniques for the automated extraction of biomarkers for Alzheimer's disease (AD) from brain magnetic resonance imaging (MRI) are proposed. The described work forms part of PredictAD (www.predictad.eu), a joined European research project aiming at the identification of a unified biomarker for AD combining different clinical and imaging measurements. Two different approaches are followed in this thesis towards the extraction of MRI-based biomarkers: (I) the extraction of traditional morphological biomarkers based on neuronatomical structures and (II) the extraction of data-driven biomarkers applying machine-learning techniques. A novel method for a unified and automated estimation of structural volumes and volume changes is proposed. Furthermore, a new technique that allows the low-dimensional representation of a high-dimensional image population for data analysis and visualization is described. All presented methods are evaluated on images from the Alzheimer's Disease Neuroimaging Initiative (ADNI), providing a large and diverse clinical database. A rigorous evaluation of the power of all identified biomarkers to discriminate between clinical subject groups is presented. In addition, the agreement of automatically derived volumes with reference labels as well as the power of the proposed method to measure changes in a subject's atrophy rate are assessed. The proposed methods compare favorably to state-of-the art techniques in neuroimaging in terms of accuracy, robustness and run-time

3-D lung deformation and function from respiratory-gated 4-D x-ray CT images : application to radiation treatment planning.

Many lung diseases or injuries can cause biomechanical or material property changes that can alter lung function. While the mechanical changes associated with the change of the material properties originate at a regional level, they remain largely asymptomatic and are invisible to global measures of lung function until they have advanced significantly and have aggregated. In the realm of external beam radiation therapy of patients suffering from lung cancer, determination of patterns of pre- and post-treatment motion, and measures of regional and global lung elasticity and function are clinically relevant. In this dissertation, we demonstrate that 4-D CT derived ventilation images, including mechanical strain, provide an accurate and physiologically relevant assessment of regional pulmonary function which may be incorporated into the treatment planning process. Our contributions are as follows: (i) A new volumetric deformable image registration technique based on 3-D optical flow (MOFID) has been designed and implemented which permits the possibility of enforcing physical constraints on the numerical solutions for computing motion field from respiratory-gated 4-D CT thoracic images. The proposed optical flow framework is an accurate motion model for the thoracic CT registration problem. (ii) A large displacement landmark-base elastic registration method has been devised for thoracic CT volumetric image sets containing large deformations or changes, as encountered for example in registration of pre-treatment and post-treatment images or multi-modality registration. (iii) Based on deformation maps from MOFIO, a novel framework for regional quantification of mechanical strain as an index of lung functionality has been formulated for measurement of regional pulmonary function. (iv) In a cohort consisting of seven patients with non-small cell lung cancer, validation of physiologic accuracy of the 4-0 CT derived quantitative images including Jacobian metric of ventilation, Vjac, and principal strains, (V?1, V?2, V?3, has been performed through correlation of the derived measures with SPECT ventilation and perfusion scans. The statistical correlations with SPECT have shown that the maximum principal strain pulmonary function map derived from MOFIO, outperforms all previously established ventilation metrics from 40-CT. It is hypothesized that use of CT -derived ventilation images in the treatment planning process will help predict and prevent pulmonary toxicity due to radiation treatment. It is also hypothesized that measures of regional and global lung elasticity and function obtained during the course of treatment may be used to adapt radiation treatment. Having objective methods with which to assess pre-treatment global and regional lung function and biomechanical properties, the radiation treatment dose can potentially be escalated to improve tumor response and local control

Registration and Analysis of Developmental Image Sequences

Mapping images into the same anatomical coordinate system via image registration is a fundamental step when studying physiological processes, such as brain development. Standard registration methods are applicable when biological structures are mapped to the same anatomy and their appearance remains constant across the images or changes spatially uniformly. However, image sequences of animal or human development often do not follow these assumptions, and thus standard registration methods are unsuited for their analysis. In response, this dissertation tackles the problems of i) registering developmental image sequences with spatially non-uniform appearance change and ii) reconstructing a coherent 3D volume from serially sectioned images with non-matching anatomies between the sections. There are three major contributions presented in this dissertation. First, I develop a similarity metric that incorporates a time-dependent appearance model into the registration framework. The proposed metric allows for longitudinal image registration in the presence of spatially non-uniform appearance change over time—a common medical imaging problem for longitudinal magnetic resonance images of the neonatal brain. Next, a method is introduced for registering longitudinal developmental datasets with missing time points using an appearance atlas built from a population. The proposed method is applied to a longitudinal study of young macaque monkeys with incomplete image sequences. The final contribution is a template-free registration method to reconstruct images of serially sectioned biological samples into a coherent 3D volume. The method is applied to confocal fluorescence microscopy images of serially sectioned embryonic mouse brains.Doctor of Philosoph