Symmetry-guided nonrigid registration: the case for distortion correction in multidimensional photoemission spectroscopy

Image symmetrization is an effective strategy to correct symmetry distortion in experimental data for which symmetry is essential in the subsequent analysis. In the process, a coordinate transform, the symmetrization transform, is required to undo the distortion. The transform may be determined by image registration (i.e. alignment) with symmetry constraints imposed in the registration target and in the iterative parameter tuning, which we call symmetry-guided registration. An example use case of image symmetrization is found in electronic band structure mapping by multidimensional photoemission spectroscopy, which employs a 3D time-of-flight detector to measure electrons sorted into the momentum ($k_x$, $k_y$) and energy ($E$) coordinates. In reality, imperfect instrument design, sample geometry and experimental settings cause distortion of the photoelectron trajectories and, therefore, the symmetry in the measured band structure, which hinders the full understanding and use of the volumetric datasets. We demonstrate that symmetry-guided registration can correct the symmetry distortion in the momentum-resolved photoemission patterns. Using proposed symmetry metrics, we show quantitatively that the iterative approach to symmetrization outperforms its non-iterative counterpart in the restored symmetry of the outcome while preserving the average shape of the photoemission pattern. Our approach is generalizable to distortion corrections in different types of symmetries and should also find applications in other experimental methods that produce images with similar features

3-D lung deformation and function from respiratory-gated 4-D x-ray CT images : application to radiation treatment planning.

Many lung diseases or injuries can cause biomechanical or material property changes that can alter lung function. While the mechanical changes associated with the change of the material properties originate at a regional level, they remain largely asymptomatic and are invisible to global measures of lung function until they have advanced significantly and have aggregated. In the realm of external beam radiation therapy of patients suffering from lung cancer, determination of patterns of pre- and post-treatment motion, and measures of regional and global lung elasticity and function are clinically relevant. In this dissertation, we demonstrate that 4-D CT derived ventilation images, including mechanical strain, provide an accurate and physiologically relevant assessment of regional pulmonary function which may be incorporated into the treatment planning process. Our contributions are as follows: (i) A new volumetric deformable image registration technique based on 3-D optical flow (MOFID) has been designed and implemented which permits the possibility of enforcing physical constraints on the numerical solutions for computing motion field from respiratory-gated 4-D CT thoracic images. The proposed optical flow framework is an accurate motion model for the thoracic CT registration problem. (ii) A large displacement landmark-base elastic registration method has been devised for thoracic CT volumetric image sets containing large deformations or changes, as encountered for example in registration of pre-treatment and post-treatment images or multi-modality registration. (iii) Based on deformation maps from MOFIO, a novel framework for regional quantification of mechanical strain as an index of lung functionality has been formulated for measurement of regional pulmonary function. (iv) In a cohort consisting of seven patients with non-small cell lung cancer, validation of physiologic accuracy of the 4-0 CT derived quantitative images including Jacobian metric of ventilation, Vjac, and principal strains, (V?1, V?2, V?3, has been performed through correlation of the derived measures with SPECT ventilation and perfusion scans. The statistical correlations with SPECT have shown that the maximum principal strain pulmonary function map derived from MOFIO, outperforms all previously established ventilation metrics from 40-CT. It is hypothesized that use of CT -derived ventilation images in the treatment planning process will help predict and prevent pulmonary toxicity due to radiation treatment. It is also hypothesized that measures of regional and global lung elasticity and function obtained during the course of treatment may be used to adapt radiation treatment. Having objective methods with which to assess pre-treatment global and regional lung function and biomechanical properties, the radiation treatment dose can potentially be escalated to improve tumor response and local control

A Heterogeneous and Multi-Range Soft-Tissue Deformation Model for Applications in Adaptive Radiotherapy

During fractionated radiotherapy, anatomical changes result in uncertainties in the applied dose distribution. With increasing steepness of applied dose gradients, the relevance of patient deformations increases. Especially in proton therapy, small anatomical changes in the order of millimeters can result in large range uncertainties and therefore in substantial deviations from the planned dose. To quantify the anatomical changes, deformation models are required. With upcoming MR-guidance, the soft-tissue deformations gain visibility, but so far only few soft-tissue models meeting the requirements of high-precision radiotherapy exist. Most state-of-the-art models either lack anatomical detail or exhibit long computation times. In this work, a fast soft-tissue deformation model is developed which is capable of considering tissue properties of heterogeneous tissue. The model is based on the chainmail (CM)-concept, which is improved by three basic features. For the first time, rotational degrees of freedom are introduced into the CM-concept to improve the characteristic deformation behavior. A novel concept for handling multiple deformation initiators is developed to cope with global deformation input. And finally, a concept for handling various shapes of deformation input is proposed to provide a high flexibility concerning the design of deformation input. To demonstrate the model flexibility, it was coupled to a kinematic skeleton model for the head and neck region, which provides anatomically correct deformation input for the bones. For exemplary patient CTs, the combined model was shown to be capable of generating artificially deformed CT images with realistic appearance. This was achieved for small-range deformations in the order of interfractional deformations, as well as for large-range deformations like an arms-up to arms-down deformation, as can occur between images of different modalities. The deformation results showed a strong improvement in biofidelity, compared to the original chainmail-concept, as well as compared to clinically used image-based deformation methods. The computation times for the model are in the order of 30 min for single-threaded calculations, by simple code parallelization times in the order of 1 min can be achieved. Applications that require realistic forward deformations of CT images will benefit from the improved biofidelity of the developed model. Envisioned applications are the generation of plan libraries and virtual phantoms, as well as data augmentation for deep learning approaches. Due to the low computation times, the model is also well suited for image registration applications. In this context, it will contribute to an improved calculation of accumulated dose, as is required in high-precision adaptive radiotherapy

Deformable Medical Image Registration: A Survey

Deformable image registration is a fundamental task in medical image processing. Among its most important applications, one may cite: i) multi-modality fusion, where information acquired by different imaging devices or protocols is fused to facilitate diagnosis and treatment planning; ii) longitudinal studies, where temporal structural or anatomical changes are investigated; and iii) population modeling and statistical atlases used to study normal anatomical variability. In this technical report, we attempt to give an overview of deformable registration methods, putting emphasis on the most recent advances in the domain. Additional emphasis has been given to techniques applied to medical images. In order to study image registration methods in depth, their main components are identified and studied independently. The most recent techniques are presented in a systematic fashion. The contribution of this technical report is to provide an extensive account of registration techniques in a systematic manner.Le recalage déformable d'images est une des tâches les plus fondamentales dans l'imagerie médicale. Parmi ses applications les plus importantes, on compte: i) la fusion d' information provenant des différents types de modalités a n de faciliter le diagnostic et la planification du traitement; ii) les études longitudinales, oú des changements structurels ou anatomiques sont étudiées en fonction du temps; et iii) la modélisation de la variabilité anatomique normale d'une population et les atlas statistiques. Dans ce rapport de recherche, nous essayons de donner un aperçu des différentes méthodes du recalage déformables, en mettant l'accent sur les avancées les plus récentes du domaine. Nous avons particulièrement insisté sur les techniques appliquées aux images médicales. A n d'étudier les méthodes du recalage d'images, leurs composants principales sont d'abord identifiés puis étudiées de manière indépendante, les techniques les plus récentes étant classifiées en suivant un schéma logique déterminé. La contribution de ce rapport de recherche est de fournir un compte rendu détaillé des techniques de recalage d'une manière systématique

Towards Robust and Accurate Image Registration by Incorporating Anatomical and Appearance Priors

Ph.DDOCTOR OF PHILOSOPH

Non-rigid medical image registration with extended free form deformations: modelling general tissue transitions

Image registration seeks pointwise correspondences between the same or analogous objects in different images. Conventional registration methods generally impose continuity and smoothness throughout the image. However, there are cases in which the deformations may involve discontinuities. In general, the discontinuities can be of different types, depending on the physical properties of the tissue transitions involved and boundary conditions. For instance, in the respiratory motion the lungs slide along the thoracic cage following the tangential direction of their interface. In the normal direction, however, the lungs and the thoracic cage are constrained to be always in contact but they have different material properties producing different compression or expansion rates. In the literature, there is no generic method, which handles different types of discontinuities and considers their directional dependence. The aim of this thesis is to develop a general registration framework that is able to correctly model different types of tissue transitions with a general formalism. This has led to the development of the eXtended Free Form Deformation (XFFD) registration method. XFFD borrows the concept of the interpolation method from the eXtended Finite Element method (XFEM) to incorporate discontinuities by enriching B-spline basis functions, coupled with extra degrees of freedom. XFFD can handle different types of discontinuities and encodes their directional-dependence without any additional constraints. XFFD has been evaluated on digital phantoms, publicly available 3D liver and lung CT images. The experiments show that XFFD improves on previous methods and that it is important to employ the correct model that corresponds to the discontinuity type involved at the tissue transition. The effect of using incorrect models is more evident in the strain, which measures mechanical properties of the tissues

Simulation-Based Joint Estimation of Body Deformation and Elasticity Parameters for Medical Image Analysis

Elasticity parameter estimation is essential for generating accurate and controlled simulation results for computer animation and medical image analysis. However, finding the optimal parameters for a particular simulation often requires iterations of simulation, assessment, and adjustment and can become a tedious process. Elasticity values are especially important in medical image analysis, since cancerous tissues tend to be stiffer. Elastography is a popular type of method for finding stiffness values by reconstructing a dense displacement field from medical images taken during the application of forces or vibrations. These methods, however, are limited by the imaging modality and the force exertion or vibration actuation mechanisms, which can be complicated for deep-seated organs. In this thesis, I present a novel method for reconstructing elasticity parameters without requiring a dense displacement field or a force exertion device. The method makes use of natural deformations within the patient and relies on surface information from segmented images taken on different days. The elasticity value of the target organ and boundary forces acting on surrounding organs are optimized with an iterative optimizer, within which the deformation is always generated by a physically-based simulator. Experimental results on real patient data are presented to show the positive correlation between recovered elasticity values and clinical prostate cancer stages. Furthermore, to resolve the performance issue arising from the high dimensionality of boundary forces, I propose to use a reduced finite element model to improve the convergence of the optimizer. To find the set of bases to represent the dimensions for forces, a statistical training based on real patient data is performed. I demonstrate the trade-off between accuracy and performance by using different numbers of bases in the optimization using synthetic data. A speedup of more than an order of magnitude is observed without sacrificing too much accuracy in recovered elasticity.Doctor of Philosoph

3D-3D Deformable Registration and Deep Learning Segmentation based Neck Diseases Analysis in MRI

Whiplash, cervical dystonia (CD), neck pain and work-related upper limb disorder (WRULD) are the most common diseases in the cervical region. Headaches, stiffness, sensory disturbance to the legs and arms, optical problems, aching in the back and shoulder, and auditory and visual problems are common symptoms seen in patients with these diseases. CD patients may also suffer tormenting spasticity in some neck muscles, with the symptoms possibly being acute and persisting for a long time, sometimes a lifetime. Whiplash-associated disorders (WADs) may occur due to sudden forward and backward movements of the head and neck occurring during a sporting activity or vehicle or domestic accident. These diseases affect private industries, insurance companies and governments, with the socio-economic costs significantly related to work absences, long-term sick leave, early disability and disability support pensions, health care expenses, reduced productivity and insurance claims. Therefore, diagnosing and treating neck-related diseases are important issues in clinical practice. The reason for these afflictions resulting from accident is the impairment of the cervical muscles which undergo atrophy or pseudo-hypertrophy due to fat infiltrating into them. These morphological changes have to be determined by identifying and quantifying their bio-markers before applying any medical intervention. Volumetric studies of neck muscles are reliable indicators of the proper treatments to apply. Radiation therapy, chemotherapy, injection of a toxin or surgery could be possible ways of treating these diseases. However, the dosages required should be precise because the neck region contains some sensitive organs, such as nerves, blood vessels and the trachea and spinal cord. Image registration and deep learning-based segmentation can help to determine appropriate treatments by analyzing the neck muscles. However, this is a challenging task for medical images due to complexities such as many muscles crossing multiple joints and attaching to many bones. Also, their shapes and sizes vary greatly across populations whereas their cross-sectional areas (CSAs) do not change in proportion to the heights and weights of individuals, with their sizes varying more significantly between males and females than ages. Therefore, the neck's anatomical variabilities are much greater than those of other parts of the human body. Some other challenges which make analyzing neck muscles very difficult are their compactness, similar gray-level appearances, intra-muscular fat, sliding due to cardiac and respiratory motions, false boundaries created by intramuscular fat, low resolution and contrast in medical images, noise, inhomogeneity and background clutter with the same composition and intensity. Furthermore, a patient's mode, position and neck movements during the capture of an image create variability. However, very little significant research work has been conducted on analyzing neck muscles. Although previous image registration efforts form a strong basis for many medical applications, none can satisfy the requirements of all of them because of the challenges associated with their implementation and low accuracy which could be due to anatomical complexities and variabilities or the artefacts of imaging devices. In existing methods, multi-resolution- and heuristic-based methods are popular. However, the above issues cause conventional multi-resolution-based registration methods to be trapped in local minima due to their low degrees of freedom in their geometrical transforms. Although heuristic-based methods are good at handling large mismatches, they require pre-segmentation and are computationally expensive. Also, current deformable methods often face statistical instability problems and many local optima when dealing with small mismatches. On the other hand, deep learning-based methods have achieved significant success over the last few years. Although a deeper network can learn more complex features and yields better performances, its depth cannot be increased as this would cause the gradient to vanish during training and result in training difficulties. Recently, researchers have focused on attention mechanisms for deep learning but current attention models face a challenge in the case of an application with compact and similar small multiple classes, large variability, low contrast and noise. The focus of this dissertation is on the design of 3D-3D image registration approaches as well as deep learning-based semantic segmentation methods for analyzing neck muscles. In the first part of this thesis, a novel object-constrained hierarchical registration framework for aligning inter-subject neck muscles is proposed. Firstly, to handle large-scale local minima, it uses a coarse registration technique which optimizes a new edge position difference (EPD) similarity measure to align large mismatches. Also, a new transformation based on the discrete periodic spline wavelet (DPSW), affine and free-form-deformation (FFD) transformations are exploited. Secondly, to avoid the monotonous nature of using transformations in multiple stages, affine registration technique, which uses a double-pushing system by changing the edges in the EPD and switching the transformation's resolutions, is designed to align small mismatches. The EPD helps in both the coarse and fine techniques to implement object-constrained registration via controlling edges which is not possible using traditional similarity measures. Experiments are performed on clinical 3D magnetic resonance imaging (MRI) scans of the neck, with the results showing that the EPD is more effective than the mutual information (MI) and the sum of squared difference (SSD) measures in terms of the volumetric dice similarity coefficient (DSC). Also, the proposed method is compared with two state-of-the-art approaches with ablation studies of inter-subject deformable registration and achieves better accuracy, robustness and consistency. However, as this method is computationally complex and has a problem handling large-scale anatomical variabilities, another 3D-3D registration framework with two novel contributions is proposed in the second part of this thesis. Firstly, a two-stage heuristic search optimization technique for handling large mismatches,which uses a minimal user hypothesis regarding these mismatches and is computationally fast, is introduced. It brings a moving image hierarchically closer to a fixed one using MI and EPD similarity measures in the coarse and fine stages, respectively, while the images do not require pre-segmentation as is necessary in traditional heuristic optimization-based techniques. Secondly, a region of interest (ROI) EPD-based registration framework for handling small mismatches using salient anatomical information (AI), in which a convex objective function is formed through a unique shape created from the desired objects in the ROI, is proposed. It is compared with two state-of-the-art methods on a neck dataset, with the results showing that it is superior in terms of accuracy and is computationally fast. In the last part of this thesis, an evaluation study of recent U-Net-based convolutional neural networks (CNNs) is performed on a neck dataset. It comprises 6 recent models, the U-Net, U-Net with a conditional random field (CRF-Unet), attention U-Net (A-Unet), nested U-Net or U-Net++, multi-feature pyramid (MFP)-Unet and recurrent residual U-Net (R2Unet) and 4 with more comprehensive modifications, the multi-scale U-Net (MS-Unet), parallel multi-scale U-Net (PMSUnet), recurrent residual attention U-Net (R2A-Unet) and R2A-Unet++ in neck muscles segmentation, with analyses of the numerical results indicating that the R2Unet architecture achieves the best accuracy. Also, two deep learning-based semantic segmentation approaches are proposed. In the first, a new two-stage U-Net++ (TS-UNet++) uses two different types of deep CNNs (DCNNs) rather than one similar to the traditional multi-stage method, with the U-Net++ in the first stage and the U-Net in the second. More convolutional blocks are added after the input and before the output layers of the multi-stage approach to better extract the low- and high-level features. A new concatenation-based fusion structure, which is incorporated in the architecture to allow deep supervision, helps to increase the depth of the network without accelerating the gradient-vanishing problem. Then, more convolutional layers are added after each concatenation of the fusion structure to extract more representative features. The proposed network is compared with the U-Net, U-Net++ and two-stage U-Net (TS-UNet) on the neck dataset, with the results indicating that it outperforms the others. In the second approach, an explicit attention method, in which the attention is performed through a ROI evolved from ground truth via dilation, is proposed. It does not require any additional CNN, as does a cascaded approach, to localize the ROI. Attention in a CNN is sensitive with respect to the area of the ROI. This dilated ROI is more capable of capturing relevant regions and suppressing irrelevant ones than a bounding box and region-level coarse annotation, and is used during training of any CNN. Coarse annotation, which does not require any detailed pixel wise delineation that can be performed by any novice person, is used during testing. This proposed ROI-based attention method, which can handle compact and similar small multiple classes with objects with large variabilities, is compared with the automatic A-Unet and U-Net, and performs best

Biomechanical Modeling and Inverse Problem Based Elasticity Imaging for Prostate Cancer Diagnosis

Early detection of prostate cancer plays an important role in successful prostate cancer treatment. This requires screening the prostate periodically after the age of 50. If screening tests lead to prostate cancer suspicion, prostate needle biopsy is administered which is still considered as the clinical gold standard for prostate cancer diagnosis. Given that needle biopsy is invasive and is associated with issues including discomfort and infection, it is desirable to develop a prostate cancer diagnosis system that has high sensitivity and specificity for early detection with a potential to improve needle biopsy outcome. Given the complexity and variability of prostate cancer pathologies, many research groups have been pursuing multi-parametric imaging approach as no single modality imaging technique has proven to be adequate. While imaging additional tissue properties increases the chance of reliable prostate cancer detection and diagnosis, selecting an additional property needs to be done carefully by considering clinical acceptability and cost. Clinical acceptability entails ease with respect to both operating by the radiologist and patient comfort. In this work, effective tissue biomechanics based diagnostic techniques are proposed for prostate cancer assessment with the aim of early detection and minimizing the numbers of prostate biopsies. The techniques take advantage of the low cost, widely available and well established TRUS imaging method. The proposed techniques include novel elastography methods which were formulated based on an inverse finite element frame work. Conventional finite element analysis is known to have high computational complexity, hence computation time demanding. This renders the proposed elastography methods not suitable for real-time applications. To address this issue, an accelerated finite element method was proposed which proved to be suitable for prostate elasticity reconstruction. In this method, accurate finite element analysis of a large number of prostates undergoing TRUS probe loadings was performed. Geometry input and displacement and stress fields output obtained from the analysis were used to train a neural network mapping function to be used for elastopgraphy imaging of prostate cancer patients. The last part of the research presented in this thesis tackles an issue with the current 3D TRUS prostate needle biopsy. Current 3D TRUS prostate needle biopsy systems require registering preoperative 3D TRUS to intra-operative 2D TRUS images. Such image registration is time-consuming while its real-time implementation is yet to be developed. To bypass this registration step, concept of a robotic system was proposed which can reliably determine the preoperative TRUS probe position relative to the prostate to place at the same position relative to the prostate intra-operatively. For this purpose, a contact pressure feedback system is proposed to ensure similar prostate deformation during 3D and 2D image acquisition in order to bypass the registration step