Combining the Band-Limited Parameterization and Semi-Lagrangian Runge–Kutta Integration for Efficient PDE-Constrained LDDMM

The family of PDE-constrained Large Deformation Diffeomorphic Metric Mapping (LDDMM) methods is emerging as a particularly interesting approach for physically meaningful diffeomorphic transformations. The original combination of Gauss–Newton–Krylov optimization and Runge–Kutta integration shows excellent numerical accuracy and fast convergence rate. However, its most significant limitation is the huge computational complexity, hindering its extensive use in Computational Anatomy applied studies. This limitation has been treated independently by the problem formulation in the space of band-limited vector fields and semi-Lagrangian integration. The purpose of this work is to combine both in three variants of band-limited PDE-constrained LDDMM for further increasing their computational efficiency. The accuracy of the resulting methods is evaluated extensively. For all the variants, the proposed combined approach shows a significant increment of the computational efficiency. In addition, the variant based on the deformation state equation is positioned consistently as the best performing method across all the evaluation frameworks in terms of accuracy and efficiency

Partial Differential Equation-Constrained Diffeomorphic Registration from Sum of Squared Differences to Normalized Cross-Correlation, Normalized Gradient Fields, and Mutual Information: A Unifying Framework; 35632143

This work proposes a unifying framework for extending PDE-constrained Large Deformation Diffeomorphic Metric Mapping (PDE-LDDMM) with the sum of squared differences (SSD) to PDE-LDDMM with different image similarity metrics. We focused on the two best-performing variants of PDE-LDDMM with the spatial and band-limited parameterizations of diffeomorphisms. We derived the equations for gradient-descent and Gauss-Newton-Krylov (GNK) optimization with Normalized Cross-Correlation (NCC), its local version (lNCC), Normalized Gradient Fields (NGFs), and Mutual Information (MI). PDE-LDDMM with GNK was successfully implemented for NCC and lNCC, substantially improving the registration results of SSD. For these metrics, GNK optimization outperformed gradient-descent. However, for NGFs, GNK optimization was not able to overpass the performance of gradient-descent. For MI, GNK optimization involved the product of huge dense matrices, requesting an unaffordable memory load. The extensive evaluation reported the band-limited version of PDE-LDDMM based on the deformation state equation with NCC and lNCC image similarities among the best performing PDE-LDDMM methods. In comparison with benchmark deep learning-based methods, our proposal reached or surpassed the accuracy of the best-performing models. In NIREP16, several configurations of PDE-LDDMM outperformed ANTS-lNCC, the best benchmark method. Although NGFs and MI usually underperformed the other metrics in our evaluation, these metrics showed potentially competitive results in a multimodal deformable experiment. We believe that our proposed image similarity extension over PDE-LDDMM will promote the use of physically meaningful diffeomorphisms in a wide variety of clinical applications depending on deformable image registration

LDDMM y GANs: Redes Generativas Antagónicas para Registro Difeomorfico.

El Registro Difeomorfico de imágenes es un problema clave para muchas aplicaciones de la Anatomía Computacional. Tradicionalmente, el registro deformable de imagen ha sido formulado como un problema variacional, resoluble mediante costosos métodos de optimización numérica. En la última década, contribuciones en la forma de nuevos métodos basados en formulaciones tradicionales están decreciendo, mientras que más modelos basados en Aprendizaje profundo están siendo desarrollados para aprender registros deformables de imágenes. En este trabajo contribuimos a esta nueva corriente proponiendo un novedoso método LDDMM para registro difeomorfico de imágenes 3D, basado en redes generativas antagónicas. Combinamos las arquitecturas de generadores y discriminadores con mejores prestaciones en registro deformable con el paradigma LDDMM. Hemos implementado con éxito tres modelos para distintas parametrizaciones de difeomorfismos, los cuales demuestran resultados competitivos en comparación con métodos del estado del arte tanto tradicionales como basados en aprendizaje profundo.<br /

Fourier-Net+: Leveraging Band-Limited Representation for Efficient 3D Medical Image Registration

U-Net style networks are commonly utilized in unsupervised image registration to predict dense displacement fields, which for high-resolution volumetric image data is a resource-intensive and time-consuming task. To tackle this challenge, we first propose Fourier-Net, which replaces the costly U-Net style expansive path with a parameter-free model-driven decoder. Instead of directly predicting a full-resolution displacement field, our Fourier-Net learns a low-dimensional representation of the displacement field in the band-limited Fourier domain which our model-driven decoder converts to a full-resolution displacement field in the spatial domain. Expanding upon Fourier-Net, we then introduce Fourier-Net+, which additionally takes the band-limited spatial representation of the images as input and further reduces the number of convolutional layers in the U-Net style network's contracting path. Finally, to enhance the registration performance, we propose a cascaded version of Fourier-Net+. We evaluate our proposed methods on three datasets, on which our proposed Fourier-Net and its variants achieve comparable results with current state-of-the art methods, while exhibiting faster inference speeds, lower memory footprint, and fewer multiply-add operations. With such small computational cost, our Fourier-Net+ enables the efficient training of large-scale 3D registration on low-VRAM GPUs. Our code is publicly available at \url{https://github.com/xi-jia/Fourier-Net}.Comment: Under review. arXiv admin note: text overlap with arXiv:2211.1634

Medical image analysis via Fréchet means of diffeomorphisms

The construction of average models of anatomy, as well as regression analysis of anatomical structures, are key issues in medical research, e.g., in the study of brain development and disease progression. When the underlying anatomical process can be modeled by parameters in a Euclidean space, classical statistical techniques are applicable. However, recent work suggests that attempts to describe anatomical differences using flat Euclidean spaces undermine our ability to represent natural biological variability. In response, this dissertation contributes to the development of a particular nonlinear shape analysis methodology. This dissertation uses a nonlinear deformable model to measure anatomical change and define geometry-based averaging and regression for anatomical structures represented within medical images. Geometric differences are modeled by coordinate transformations, i.e., deformations, of underlying image coordinates. In order to represent local geometric changes and accommodate large deformations, these transformations are taken to be the group of diffeomorphisms with an associated metric. A mean anatomical image is defined using this deformation-based metric via the Fréchet mean—the minimizer of the sum of squared distances. Similarly, a new method called manifold kernel regression is presented for estimating systematic changes—as a function of a predictor variable, such as age—from data in nonlinear spaces. It is defined by recasting kernel regression in terms of a kernel-weighted Fréchet mean. This method is applied to determine systematic geometric changes in the brain from a random design dataset of medical images. Finally, diffeomorphic image mapping is extended to accommodate extraneous structures—objects that are present in one image and absent in another and thus change image topology—by deflating them prior to the estimation of geometric change. The method is applied to quantify the motion of the prostate in the presence of transient bowel gas

Doctor of Philosophy

dissertationStochastic methods, dense free-form mapping, atlas construction, and total variation are examples of advanced image processing techniques which are robust but computationally demanding. These algorithms often require a large amount of computational power as well as massive memory bandwidth. These requirements used to be ful lled only by supercomputers. The development of heterogeneous parallel subsystems and computation-specialized devices such as Graphic Processing Units (GPUs) has brought the requisite power to commodity hardware, opening up opportunities for scientists to experiment and evaluate the in uence of these techniques on their research and practical applications. However, harnessing the processing power from modern hardware is challenging. The di fferences between multicore parallel processing systems and conventional models are signi ficant, often requiring algorithms and data structures to be redesigned signi ficantly for efficiency. It also demands in-depth knowledge about modern hardware architectures to optimize these implementations, sometimes on a per-architecture basis. The goal of this dissertation is to introduce a solution for this problem based on a 3D image processing framework, using high performance APIs at the core level to utilize parallel processing power of the GPUs. The design of the framework facilitates an efficient application development process, which does not require scientists to have extensive knowledge about GPU systems, and encourages them to harness this power to solve their computationally challenging problems. To present the development of this framework, four main problems are described, and the solutions are discussed and evaluated: (1) essential components of a general 3D image processing library: data structures and algorithms, as well as how to implement these building blocks on the GPU architecture for optimal performance; (2) an implementation of unbiased atlas construction algorithms|an illustration of how to solve a highly complex and computationally expensive algorithm using this framework; (3) an extension of the framework to account for geometry descriptors to solve registration challenges with large scale shape changes and high intensity-contrast di fferences; and (4) an out-of-core streaming model, which enables developers to implement multi-image processing techniques on commodity hardware

Descomposición Ortogonal Propia en registro mediante difeomorfismos.

Los métodos de registro mediante difeomorfismos se han posicionado como los métodos de referencia en el área del registro no-rígido de imágenes médicas. La metodología subyacente proporciona soluciones que mantienen la corrección biológica de los campos de deformación de las anatomías en términos de suavidad y conservación de la topología. El método Large Deformation Diffeomorphic Metric Mapping (LDDMM) fue pionero en el año 2005 sentando las bases en el cálculo de registro difeomorfo en el paradigma de grandes deformaciones. El principal problema de LDDMM es su gran carga computacional. El presente trabajo tiene como objetivo el estudio de la capacidad de los métodos denominados Reduced Order Models (ROM) para reducir la complejidad computacional de una de las variantes de LDDMM basada en la restricción del problema a campos vectoriales iniciales respetando la ecuación diferencial de Euler-Poincaré (EPDiff). El trabajo se centrará en el estudio del artículo Wen et al., Data-driven Model Order Reduction For Diffeomorphic Image Registration. En este, se presenta un ROM denominado Proper Orthogonal Decomposition (POD) para reducir la dimensionalidad de la ecuación de Euler-Poincaré. Así, en este trabajo, se estudiará la reproducibilidad el artículo, tanto a nivel teórico como a nivel experimental. Se estudiará la implementación del algoritmo mediante el registro de distintas imágenes, así como la viabilidad de este, mediante la comparación de distintas métricas como son el error final en el registro o el tiempo de computación.<br /

Geodesic Active Fields:A Geometric Framework for Image Registration

Image registration is the concept of mapping homologous points in a pair of images. In other words, one is looking for an underlying deformation field that matches one image to a target image. The spectrum of applications of image registration is extremely large: It ranges from bio-medical imaging and computer vision, to remote sensing or geographic information systems, and even involves consumer electronics. Mathematically, image registration is an inverse problem that is ill-posed, which means that the exact solution might not exist or not be unique. In order to render the problem tractable, it is usual to write the problem as an energy minimization, and to introduce additional regularity constraints on the unknown data. In the case of image registration, one often minimizes an image mismatch energy, and adds an additive penalty on the deformation field regularity as smoothness prior. Here, we focus on the registration of the human cerebral cortex. Precise cortical registration is required, for example, in statistical group studies in functional MR imaging, or in the analysis of brain connectivity. In particular, we work with spherical inflations of the extracted hemispherical surface and associated features, such as cortical mean curvature. Spatial mapping between cortical surfaces can then be achieved by registering the respective spherical feature maps. Despite the simplified spherical geometry, inter-subject registration remains a challenging task, mainly due to the complexity and inter-subject variability of the involved brain structures. In this thesis, we therefore present a registration scheme, which takes the peculiarities of the spherical feature maps into particular consideration. First, we realize that we need an appropriate hierarchical representation, so as to coarsely align based on the important structures with greater inter-subject stability, before taking smaller and more variable details into account. Based on arguments from brain morphogenesis, we propose an anisotropic scale-space of mean-curvature maps, built around the Beltrami framework. Second, inspired by concepts from vision-related elements of psycho-physical Gestalt theory, we hypothesize that anisotropic Beltrami regularization better suits the requirements of image registration regularization, compared to traditional Gaussian filtering. Different objects in an image should be allowed to move separately, and regularization should be limited to within the individual Gestalts. We render the regularization feature-preserving by limiting diffusion across edges in the deformation field, which is in clear contrast to the indifferent linear smoothing. We do so by embedding the deformation field as a manifold in higher-dimensional space, and minimize the associated Beltrami energy which represents the hyperarea of this embedded manifold as measure of deformation field regularity. Further, instead of simply adding this regularity penalty to the image mismatch in lieu of the standard penalty, we propose to incorporate the local image mismatch as weighting function into the Beltrami energy. The image registration problem is thus reformulated as a weighted minimal surface problem. This approach has several appealing aspects, including (1) invariance to re-parametrization and ability to work with images defined on non-flat, Riemannian domains (e.g., curved surfaces, scalespaces), and (2) intrinsic modulation of the local regularization strength as a function of the local image mismatch and/or noise level. On a side note, we show that the proposed scheme can easily keep up with recent trends in image registration towards using diffeomorphic and inverse consistent deformation models. The proposed registration scheme, called Geodesic Active Fields (GAF), is non-linear and non-convex. Therefore we propose an efficient optimization scheme, based on splitting. Data-mismatch and deformation field regularity are optimized over two different deformation fields, which are constrained to be equal. The constraint is addressed using an augmented Lagrangian scheme, and the resulting optimization problem is solved efficiently using alternate minimization of simpler sub-problems. In particular, we show that the proposed method can easily compete with state-of-the-art registration methods, such as Demons. Finally, we provide an implementation of the fast GAF method on the sphere, so as to register the triangulated cortical feature maps. We build an automatic parcellation algorithm for the human cerebral cortex, which combines the delineations available on a set of atlas brains in a Bayesian approach, so as to automatically delineate the corresponding regions on a subject brain given its feature map. In a leave-one-out cross-validation study on 39 brain surfaces with 35 manually delineated gyral regions, we show that the pairwise subject-atlas registration with the proposed spherical registration scheme significantly improves the individual alignment of cortical labels between subject and atlas brains, and, consequently, that the estimated automatic parcellations after label fusion are of better quality