Quicksilver: Fast Predictive Image Registration - a Deep Learning Approach

This paper introduces Quicksilver, a fast deformable image registration method. Quicksilver registration for image-pairs works by patch-wise prediction of a deformation model based directly on image appearance. A deep encoder-decoder network is used as the prediction model. While the prediction strategy is general, we focus on predictions for the Large Deformation Diffeomorphic Metric Mapping (LDDMM) model. Specifically, we predict the momentum-parameterization of LDDMM, which facilitates a patch-wise prediction strategy while maintaining the theoretical properties of LDDMM, such as guaranteed diffeomorphic mappings for sufficiently strong regularization. We also provide a probabilistic version of our prediction network which can be sampled during the testing time to calculate uncertainties in the predicted deformations. Finally, we introduce a new correction network which greatly increases the prediction accuracy of an already existing prediction network. We show experimental results for uni-modal atlas-to-image as well as uni- / multi- modal image-to-image registrations. These experiments demonstrate that our method accurately predicts registrations obtained by numerical optimization, is very fast, achieves state-of-the-art registration results on four standard validation datasets, and can jointly learn an image similarity measure. Quicksilver is freely available as an open-source software.Comment: Add new discussion

Fast Predictive Multimodal Image Registration

We introduce a deep encoder-decoder architecture for image deformation prediction from multimodal images. Specifically, we design an image-patch-based deep network that jointly (i) learns an image similarity measure and (ii) the relationship between image patches and deformation parameters. While our method can be applied to general image registration formulations, we focus on the Large Deformation Diffeomorphic Metric Mapping (LDDMM) registration model. By predicting the initial momentum of the shooting formulation of LDDMM, we preserve its mathematical properties and drastically reduce the computation time, compared to optimization-based approaches. Furthermore, we create a Bayesian probabilistic version of the network that allows evaluation of registration uncertainty via sampling of the network at test time. We evaluate our method on a 3D brain MRI dataset using both T1- and T2-weighted images. Our experiments show that our method generates accurate predictions and that learning the similarity measure leads to more consistent registrations than relying on generic multimodal image similarity measures, such as mutual information. Our approach is an order of magnitude faster than optimization-based LDDMM.Comment: Accepted as a conference paper for ISBI 201

Closed and open source neuroimage analysis tools and libraries at UNC

pre-printThe emergence of open-source libraries and development tools in the last decade has changed the process of academic software development in many ways. In medical image processing and visualization this change is especially evident, also because open source projects are actively furthered by grant funding institutions. This manuscript presents the use of such development tools and libraries at the UNC Neuro-Image Analysis Laboratory for open source applications and tools. We have also experienced in our research that the development of open source in academics raises the issue of access to unpublished methodology. The strategy at our laboratory is to combine all in-house libraries and applications into a single repository that consists of two parts: a fully open source part that is distributed under a Berkley-style license and a private, closed source part with unpublished tools and methods. Access to the open source part is unrestricted, whereas the private parts can only be downloaded via cvs user login. This setup solved our issues regarding unpublished methodology, as migration from the private to the open source part is very simple. Overall our experience with this development environment within the academic setting is very positive

Statistical group differences in anatomical shape analysis using Hotelling T2 Metric

journal articleShape analysis has become of increasing interest to the neuroimaging community due to its potential to precisely locate morphological changes between healthy and pathological structures. This manuscript presents a comprehensive set of tools for the computation of 3D structural statistical shape analysis. It has been applied in several studies on brain morphometry, but can potentially be employed in other 3D shape problems. Its main limitations is the necessity of spherical topology. The input of the proposed shape analysis is a set of binary segmentation of a single brain structure, such as the hippocampus or caudate. These segmentations are converted into a corresponding spherical harmonic description (SPHARM), which is then sampled into a triangulated surfaces (SPHARM-PDM). After alignment, differences between groups of surfaces are computed using the Hotelling T2 two sample metric. Statistical pvalues, both raw and corrected for multiple comparisons, result in significance maps. Additional visualization of the group tests are provided via mean difference magnitude and vector maps, as well as maps of the group covariance information. The correction for multiple comparisons is performed via two separate methods that each have a distinct view of the problem. The first one aims to control the family-wise error rate (FWER) or false-positives via the extrema histogram of non-parametric permutations. The second method controls the false discovery rate and results in a less conservative estimate of the false-negatives. Prior versions of this shape analysis framework have been applied already to clinical studies on hippocampus and lateral ventricle shape in adult schizophrenics. The novelty of this submission is the use of the Hotelling T2 two-sample group difference metric for the computation of a template free statistical shape analysis. Template free group testing allowed this framework to become independent of any template choice, as well as it improved the sensitivity of our method considerably. In addition to our existing correction methodology for the multiple comparison problem using non-parametric permutation tests, we have extended the testing framework to include False Discovery Rate (FDR). FDR provides a significance correction with higher sensitivity while allowing a expected minimal amount of false-positives compared to our prior testing scheme

Role of deep learning in infant brain MRI analysis

Deep learning algorithms and in particular convolutional networks have shown tremendous success in medical image analysis applications, though relatively few methods have been applied to infant MRI data due numerous inherent challenges such as inhomogenous tissue appearance across the image, considerable image intensity variability across the first year of life, and a low signal to noise setting. This paper presents methods addressing these challenges in two selected applications, specifically infant brain tissue segmentation at the isointense stage and presymptomatic disease prediction in neurodevelopmental disorders. Corresponding methods are reviewed and compared, and open issues are identified, namely low data size restrictions, class imbalance problems, and lack of interpretation of the resulting deep learning solutions. We discuss how existing solutions can be adapted to approach these issues as well as how generative models seem to be a particularly strong contender to address them

Minimum description length with local geometry

pre-printEstablishing optimal correspondence across object populations is essential to statistical shape analysis. Minimizing the description length (MDL) is a popular method for finding correspondence. In this work, we extend the MDL method by incorporating various local curvature metrics. Using local curvature can improve performance by ensuring that corresponding points exhibit similar local geometric characteristics that can't always be captured by mere point locations. We illustrate results on a variety of anatomical structures. The MDL method with a combination of point locations and curvature outperforms all the other methods we analyzed, including traditional MDL and spherical harmonics (SPHARM) correspondence, when the analyzed object population exhibits complex structure. When the objects are of simple nature, however, there's no added benefit to using the local curvature. In our experiments, we did not observe a significant difference in the correspondence quality when different curvature metrics (e.g. principal curvatures, mean curvature, Gaussian curvature) were used

Correspondence evaluation in local shape analysis and structural subdivision

journal articleRegional volumetric and local shape analysis has become of increasing interest to the neuroimaging community due to the potential to locate morphological changes. In this paper we compare three common correspondence methods applied to two studies of hippocampal shape in schizophrenia: correspondence via deformable registration, spherical harmonics (SPHARM) and Minimum Description Length (MDL) optimization. These correspondence methods are evaluated in respect to local statistical shape analysis and structural subdivision analysis. Results show a non-negligible influence of the choice of correspondence especially in studies with low numbers of subjects. The differences are especially striking in the structural subdivision analysis and hints at a possible source for the diverging findings in many subdivision studies. Our comparative study is not meant to be exhaustive, but rather raises awareness of the issue and shows that assessing the validity of the correspondence is an important step

Exercise Regulation of Marrow Adipose Tissue

Despite association with low bone density and skeletal fractures, marrow adipose tissue (MAT) remains poorly understood. The marrow adipocyte originates from the mesenchymal stem cell (MSC) pool that also gives rise to osteoblasts, chondrocytes, and myocytes, among other cell types. To date, the presence of MAT has been attributed to preferential biasing of MSC into the adipocyte rather than osteoblast lineage, thus negatively impacting bone formation. Here, we focus on understanding the physiology of MAT in the setting of exercise, dietary interventions, and pharmacologic agents that alter fat metabolism. The beneficial effect of exercise on musculoskeletal strength is known: exercise induces bone formation, encourages growth of skeletally supportive tissues, inhibits bone resorption, and alters skeletal architecture through direct and indirect effects on a multiplicity of cells involved in skeletal adaptation. MAT is less well studied due to the lack of reproducible quantification techniques. In recent work, osmium-based 3D quantification shows a robust response of MAT to both dietary and exercise intervention in that MAT is elevated in response to high-fat diet and can be suppressed following daily exercise. Exercise-induced bone formation correlates with suppression of MAT, such that exercise effects might be due to either calorie expenditure from this depot or from mechanical biasing of MSC lineage away from fat and toward bone, or a combination thereof. Following treatment with the anti-diabetes drug rosiglitazone – a PPARγ-agonist known to increase MAT and fracture risk – mice demonstrate a fivefold higher femur MAT volume compared to the controls. In addition to preventing MAT accumulation in control mice, exercise intervention significantly lowers MAT accumulation in rosiglitazone-treated mice. Importantly, exercise induction of trabecular bone volume is unhindered by rosiglitazone. Thus, despite rosiglitazone augmentation of MAT, exercise significantly suppresses MAT volume and induces bone formation. That exercise can both suppress MAT volume and increase bone quantity, notwithstanding the skeletal harm induced by rosiglitazone, underscores exercise as a powerful regulator of bone remodeling, encouraging marrow stem cells toward the osteogenic lineage to fulfill an adaptive need for bone formation. Thus, exercise represents an effective strategy to mitigate the deleterious effects of overeating and iatrogenic etiologies on bone and fat

Conformal initialization for shape analysis applications in SALT

Shape analysis is an important method used in neuroimaging research community due to its potential to precisely locate morphological changes between healthy and pathological structures. A popular shape analysis framework in the neuroimaging community is based on the encoding surface locations as spherical harmonics for a representation called SPHARM-PDM. The SPHARM-PDM pipeline takes a set of brain segmentation of a single brain structure (for example, hippocampus) as input and converts them into a corresponding spherical harmonic description (SPHARM), which is then sampled into triangulated surface (SPHARM-PDM). At present, the SPHARM-PDM pipeline utilizes an area-preserving optimization of the spherical mapping based on an initial heat-equation based mapping of the surface mesh to the unit sphere. In the case of objects with complex shape, this initial spherical mapping suffers from a high degree of mapping distortion that cannot always be corrected by the following optimization procedure. Here we proposed the use of an alternative initialization based on a conformal flattening. This method adopts a bijective angle preserving conformal flattening scheme to replace the heat equation mapping scheme as initialization for use in the SPHARM-PDM pipeline. After quantitative measures of shape calculated from various complex structures, we concluded that in most cases, the new pipeline produced dramatically better results than the old pipeline. The main contribution of this paper is a command line tool based on the Slicer Execution Model, which merges the conformal flattening into the SPHARM-PDM pipeline for use in the SALT shape analysis toolbox

Three-dimensional regional displacement after surgical-orthodontic correction of Class III malocclusion

To investigate how displacements of maxillo-mandibular structures are associated with each other at splint removal and one year post-surgery following 1-jaw and 2-jaw surgeries for correction of Class III malocclusion