HNSF Log-Demons: Diffeomorphic demons registration using hierarchical neighbourhood spectral features

© 2021 The Authors. Many biomedical applications require accurate non-rigid image registration that can cope with complex deformations. However, popular diffeomorphic Demons registration algorithms suffer from difficulties for complex and serious distortions since they only use image greyscale and gradient information. To address these difficulties, a new diffeomorphic Demons registration algorithm is proposed using hierarchical neighbourhood spectral features namely HNSF Log-Demons in this paper. In view of three important properties of hierarchical neighbourhood spectral features based on line graph such as rotation invariance, invariance of linear changes of brightness, and robustness to noise, the hierarchical neighbourhood spectral features of a reference image and a moving image is first extracted and these novel spectral features are incorporated into the energy function of the diffeomorphic registration framework to improve the capability of capturing complex distortions. Secondly, the Nystr ö o ̈ m approximation based on random singular value decomposition is employed to effectively enhance the computational efficiency of HNSF Log-Demons. Finally, the hybrid multi-resolution strategy based on wavelet decomposition in the registration process is utilised to further improve the registration accuracy and efficiency. Experimental results show that the proposed HNSF Log-Demons not only effectively ensures the generation of smooth and reversible deformation field, but also achieves better performance than state-of-the-art algorithms.National Natural Science Foundation of China. Grant Numbers: 61762058, 61861024, 61871259; Natural Science Foundation of Gansu Province of China. Grant Number: 20JR5RA404; Natural Science Basic Research Program of Shaanxi. Grant Number: 2021JC-47

Deformable Medical Image Registration: A Survey

Deformable image registration is a fundamental task in medical image processing. Among its most important applications, one may cite: i) multi-modality fusion, where information acquired by different imaging devices or protocols is fused to facilitate diagnosis and treatment planning; ii) longitudinal studies, where temporal structural or anatomical changes are investigated; and iii) population modeling and statistical atlases used to study normal anatomical variability. In this technical report, we attempt to give an overview of deformable registration methods, putting emphasis on the most recent advances in the domain. Additional emphasis has been given to techniques applied to medical images. In order to study image registration methods in depth, their main components are identified and studied independently. The most recent techniques are presented in a systematic fashion. The contribution of this technical report is to provide an extensive account of registration techniques in a systematic manner.Le recalage déformable d'images est une des tâches les plus fondamentales dans l'imagerie médicale. Parmi ses applications les plus importantes, on compte: i) la fusion d' information provenant des différents types de modalités a n de faciliter le diagnostic et la planification du traitement; ii) les études longitudinales, oú des changements structurels ou anatomiques sont étudiées en fonction du temps; et iii) la modélisation de la variabilité anatomique normale d'une population et les atlas statistiques. Dans ce rapport de recherche, nous essayons de donner un aperçu des différentes méthodes du recalage déformables, en mettant l'accent sur les avancées les plus récentes du domaine. Nous avons particulièrement insisté sur les techniques appliquées aux images médicales. A n d'étudier les méthodes du recalage d'images, leurs composants principales sont d'abord identifiés puis étudiées de manière indépendante, les techniques les plus récentes étant classifiées en suivant un schéma logique déterminé. La contribution de ce rapport de recherche est de fournir un compte rendu détaillé des techniques de recalage d'une manière systématique

Development of High Angular Resolution Diffusion Imaging Analysis Paradigms for the Investigation of Neuropathology

Diffusion weighted magnetic resonance imaging (DW-MRI), provides unique insight into the microstructure of neural white matter tissue, allowing researchers to more fully investigate white matter disorders. The abundance of clinical research projects incorporating DW-MRI into their acquisition protocols speaks to the value this information lends to the study of neurological disease. However, the most widespread DW-MRI technique, diffusion tensor imaging (DTI), possesses serious limitations which restrict its utility in regions of complex white matter. Fueled by advances in DW-MRI acquisition protocols and technologies, a group of exciting new DW-MRI models, developed to address these concerns, are now becoming available to clinical researchers. The emergence of these new imaging techniques, categorized as high angular resolution diffusion imaging (HARDI), has generated the need for sophisticated computational neuroanatomic techniques able to account for the high dimensionality and structure of HARDI data. The goal of this thesis is the development of such techniques utilizing prominent HARDI data models. Specifically, methodologies for spatial normalization, population atlas building and structural connectivity have been developed and validated. These methods form the core of a comprehensive analysis paradigm allowing the investigation of local white matter microarcitecture, as well as, systemic properties of neuronal connectivity. The application of this framework to the study of schizophrenia and the autism spectrum disorders demonstrate its sensitivity sublte differences in white matter organization, as well as, its applicability to large population DW-MRI studies

Atlas Construction for Measuring the Variability of Complex Anatomical Structures

RÉSUMÉ La recherche sur l'anatomie humaine, en particulier sur le cœur et le cerveau, est d'un intérêt particulier car leurs anomalies entraînent des pathologies qui sont parmi les principales causes de décès dans le monde et engendrent des coûts substantiels. Heureusement, les progrès en imagerie médicale permettent des diagnostics et des traitements autrefois impossibles. En contrepartie, la quantité phénoménale de données produites par ces technologies nécessite le développement d'outils efficaces pour leur traitement. L'objectif de cette thèse est de proposer un ensemble d'outils permettant de normaliser des mesures prélevées sur différents individus, essentiels à l'étude des caractéristiques de structures anatomiques complexes. La normalisation de mesures consiste à rassembler une collection d'images dans une référence commune, aussi appelée construction d'atlas numériques, afin de combiner des mesures provenant de différents patients. Le processus de construction inclut deux étapes principales; la segmentation d'images pour trouver des régions d'intérêts et le recalage d'images afin de déterminer les correspondances entres régions d'intérêts. Les méthodes actuelles de constructions d'atlas peuvent nécessiter des interventions manuelles, souvent fastidieuses, variables, et sont en outre limitées par leurs mécanismes internes. Principalement, le recalage d'images dépend d'une déformation incrémentales d'images sujettes a des minimums locaux. Le recalage n'est ainsi pas optimal lors de grandes déformations et ces limitations requièrent la nécessite de proposer de nouvelles approches pour la construction d'atlas. Les questions de recherche de cette thèse se concentrent donc sur l'automatisation des méthodes actuelles ainsi que sur la capture de déformations complexes de structures anatomiques, en particulier sur le cœur et le cerveau. La méthodologie adoptée a conduit à trois objectifs de recherche spécifiques. Le premier prévoit un nouveau cadre de construction automatise d'atlas afin de créer le premier atlas humain de l'architecture de fibres cardiaques. Le deuxième vise à explorer une nouvelle approche basée sur la correspondance spectrale, nommée FOCUSR, afin de capturer une grande variabilité de formes sur des maillages. Le troisième aboutit finalement à développer une approche fondamentalement différente pour le recalage d'images à fortes déformations, nommée les démons spectraux. Le premier objectif vise plus particulièrement à construire un atlas statistique de l'architecture des fibres cardiaques a partir de 10 cœurs ex vivo humains. Le système développé a mené à deux contributions techniques et une médicale, soit l'amélioration de la segmentation de structures cardiaques et l'automatisation du calcul de forme moyenne, ainsi que notamment la première étude chez l'homme de la variabilité de l'architecture des fibres cardiaques. Pour résumer les principales conclusions, les fibres du cœur humain moyen varient de +- 12 degrés, l'angle d'helix s'étend entre -41 degrés (+- 26 degrés) sur l'épicarde à +66 degrés (+- 15 degrés) sur l'endocarde, tandis que l'angle transverse varie entre +9 degrés (+- 12 degrés) et +34 degrés (+- 29 degrés) à travers le myocarde. Ces résultats sont importants car ces fibres jouent un rôle clef dans diverses fonctions mécaniques et électrophysiologiques du cœur. Le deuxième objectif cherche à capturer une grande variabilité de formes entre structures anatomiques complexes, plus particulièrement entre cortex cérébraux à cause de l'extrême variabilité de ces surfaces et de leur intérêt pour l'étude de fonctions cognitives. La nouvelle méthode de correspondance surfacique, nommée FOCUSR, exploite des représentations spectrales car l'appariement devient plus facile et rapide dans le domaine spectral plutôt que dans l'espace Euclidien classique. Dans sa forme la plus simple, FOCUSR améliore les méthodes spectrales actuelles par un recalage non rigide des représentations spectrales, toutefois, son plein potentiel est atteint en exploitant des données supplémentaires lors de la mise en correspondance. Par exemple, les résultats ont montré que la profondeur des sillons et de la courbure du cortex cérébral améliore significativement la correspondance de surfaces de cerveaux. Enfin, le troisième objectif vise à améliorer le recalage d'images d'organes ayant des fortes variabilités entre individus ou subis de fortes déformations, telles que celles créées par le mouvement cardiaque. La méthodologie amenée par la correspondance spectrale permet d'améliorer les approches conventionnelles de recalage d'images. En effet, les représentations spectrales, capturant des similitudes géométriques globales entre différentes formes, permettent de surmonter les limitations actuelles des méthodes de recalage qui restent guidées par des forces locales. Le nouvel algorithme, nommé démons spectraux, peut ainsi supporter de très grandes déformations locales et complexes entre images, et peut être tout autant adapté a d'autres approches, telle que dans un cadre de recalage conjoint d'images. Il en résulte un cadre complet de construction d'atlas, nommé démons spectraux multijoints, où la forme moyenne est calculée directement lors du processus de recalage plutôt qu'avec une approche séquentielle de recalage et de moyennage. La réalisation de ces trois objectifs spécifiques a permis des avancées dans l'état de l'art au niveau des méthodes de correspondance spectrales et de construction d'atlas, en permettant l'utilisation d'organes présentant une forte variabilité de formes. Dans l'ensemble, les différentes stratégies fournissent de nouvelles contributions sur la façon de trouver et d'exploiter des descripteurs globaux d'images et de surfaces. D'un point de vue global, le développement des objectifs spécifiques établit un lien entre : a) la première série d'outils, mettant en évidence les défis à recaler des images à fortes déformations, b) la deuxième série d'outils, servant à capturer de fortes déformations entre surfaces mais qui ne reste pas directement applicable a des images, et c) la troisième série d'outils, faisant un retour sur le traitement d'images en permettant la construction d'atlas a partir d'images ayant subies de fortes déformations. Il y a cependant plusieurs limitations générales qui méritent d'être investiguées, par exemple, les données partielles (tronquées ou occluses) ne sont pas actuellement prises en charge les nouveaux outils, ou encore, les stratégies algorithmiques utilisées laissent toujours place à l'amélioration. Cette thèse donne de nouvelles perspectives dans les domaines de l'imagerie cardiaque et de la neuroimagerie, toutefois, les nouveaux outils développés sont assez génériques pour être appliqués a tout recalage d'images ou de surfaces. Les recommandations portent sur des recherches supplémentaires qui établissent des liens avec la segmentation à base de graphes, pouvant conduire à un cadre complet de construction d'atlas où la segmentation, le recalage, et le moyennage de formes seraient tous interdépendants. Il est également recommandé de poursuivre la recherche sur la construction de meilleurs modèles électromécaniques cardiaques à partir des résultats de cette thèse. En somme, les nouveaux outils offrent de nouvelles bases de recherche et développement pour la normalisation de formes, ce qui peut potentiellement avoir un impact sur le diagnostic, ainsi que la planification et la pratique d'interventions médicales.----------ABSTRACT Research on human anatomy, in particular on the heart and the brain, is a primary concern for society since their related diseases are among top killers across the globe and have exploding associated costs. Fortunately, recent advances in medical imaging offer new possibilities for diagnostics and treatments. On the other hand, the growth in data produced by these relatively new technologies necessitates the development of efficient tools for processing data. The focus of this thesis is to provide a set of tools for normalizing measurements across individuals in order to study complex anatomical characteristics. The normalization of measurements consists of bringing a collection of images into a common reference, also known as atlas construction, in order to combine measurements made on different individuals. The process of constructing an atlas involves the topics of segmentation, which finds regions of interest in the data (e.g., an organ, a structure), and registration, which finds correspondences between regions of interest. Current frameworks may require tedious and hardly reproducible user interactions, and are additionally limited by their computational schemes, which rely on slow iterative deformations of images, prone to local minima. Image registration is, therefore, not optimal with large deformations. Such limitations indicate the need to research new approaches for atlas construction. The research questions are consequently addressing the problems of automating current frameworks and capturing global and complex deformations between anatomical structures, in particular between human hearts and brains. More precisely, the methodology adopted in the thesis led to three specific research objectives. Briefly, the first step aims at developing a new automated framework for atlas construction in order to build the first human atlas of the cardiac fiber architecture. The second step intends to explore a new approach based on spectral correspondence, named FOCUSR, in order to precisely capture large shape variability. The third step leads, finally, to a fundamentally new approach for image registration with large deformations, named the Spectral Demons algorithm. The first objective aims more specifically at constructing a statistical atlas of the cardiac fiber architecture from a unique human dataset of 10 ex vivo hearts. The developed framework made two technical, and one medical, contributions, that are the improvement of the segmentation of cardiac structures, the automation of the shape averaging process, and more importantly, the first human study on the variability of the cardiac fiber architecture. To summarize the main finding, the fiber orientations in human hearts has been found to vary with about +- 12 degrees, the range of the helix angle spans from -41 degrees (+- 26 degrees) on the epicardium to +66 degrees (+- 15 degrees) on the endocardium, while, the range of the transverse angle spans from +9 degrees (+- 12 degrees) to +34 degrees (+- 29 degrees) across the myocardial wall. These findings are significant in cardiology since the fiber architecture plays a key role in cardiac mechanical functions and in electrophysiology. The second objective intends to capture large shape variability between complex anatomical structures, in particular between cerebral cortices due to their highly convoluted surfaces and their high anatomical and functional variability across individuals. The new method for surface correspondence, named FOCUSR, exploits spectral representations since matching is easier in the spectral domain rather than in the conventional Euclidean space. In its simplest form, FOCUSR improves current spectral approaches by refining spectral representations with a nonrigid alignment; however, its full power is demonstrated when using additional features during matching. For instance, the results showed that sulcal depth and cortical curvature improve significantly the accuracy of cortical surface matching. Finally, the third objective is to improve image registration for organs with a high inter-subject variability or undergoing very large deformations, such as the heart. The new approach brought by the spectral matching technique allows the improvement of conventional image registration methods. Indeed, spectral representations, which capture global geometric similarities and large deformations between different shapes, may be used to overcome a major limitation of current registration methods, which are in fact guided by local forces and restrained to small deformations. The new algorithm, named Spectral Demons, can capture very large and complex deformations between images, and can additionally be adapted to other approaches, such as in a groupwise configuration. This results in a complete framework for atlas construction, named Groupwise Spectral Demons, where the average shape is computed during the registration process rather than in sequential steps. The achievements of these three specific objectives permitted advances in the state-of-the-art of spectral matching methods and of atlas construction, enabling the registration of organs with significant shape variability. Overall, the investigation of these different strategies provides new contributions on how to find and exploit global descriptions of images and surfaces. From a global perspective, these objectives establish a link between: a) the first set of tools, that highlights the challenges in registering images with very large deformations, b) the second set of tools, that captures very large deformations between surfaces but are not applicable to images, and c) the third set of tools, that comes back on processing images and allows a natural construction of atlases from images with very large deformations. There are, however, several general remaining limitations, for instance, partial data (truncated or occluded) is currently not supported by the new tools, or also, the strategy for computing and using spectral representations still leaves room for improvement. This thesis gives new perspectives in cardiac and neuroimaging, yet at the same time, the new tools remain general enough for virtually any application that uses surface or image registration. It is recommended to research additional links with graph-based segmentation methods, which may lead to a complete framework for atlas construction where segmentation, registration and shape averaging are all interlinked. It is also recommended to pursue research on building better cardiac electromechanical models from the findings of this thesis. Nevertheless, the new tools provide new grounds for research and application of shape normalization, which may potentially impact diagnostic, as well as planning and performance of medical interventions

Efficient dense non-rigid registration using the free-form deformation framework

Medical image registration consists of finding spatial correspondences between two images or more. It is a powerful tool which is commonly used in various medical image processing tasks. Even though medical image registration has been an active topic of research for the last two decades, significant challenges in the field remain to be solved. This thesis addresses some of these challenges through extensions to the Free-Form Deformation (FFD) registration framework, which is one of the most widely used and well-established non-rigid registration algorithm. Medical image registration is a computationally expensive task because of the high degrees of freedom of the non-rigid transformations. In this work, the FFD algorithm has been re-factored to enable fast processing, while maintaining the accuracy of the results. In addition, parallel computing paradigms have been employed to provide near real-time image registration capabilities. Further modifications have been performed to improve the registration robustness to artifacts such as tissues non-uniformity. The plausibility of the generated deformation field has been improved through the use of bio-mechanical models based regularization. Additionally, diffeomorphic extensions to the algorithm were also developed. The work presented in this thesis has been extensively validated using brain magnetic resonance imaging of patients diagnosed with dementia or patients undergoing brain resection. It has also been applied to lung X-ray computed tomography and imaging of small animals. Alongside with this thesis an open-source package, NiftyReg, has been developed to release the presented work to the medical imaging community

ADVANCED MOTION MODELS FOR RIGID AND DEFORMABLE REGISTRATION IN IMAGE-GUIDED INTERVENTIONS

Image-guided surgery (IGS) has been a major area of interest in recent decades that continues to transform surgical interventions and enable safer, less invasive procedures. In the preoperative contexts, diagnostic imaging, including computed tomography (CT) and magnetic resonance (MR) imaging, offers a basis for surgical planning (e.g., definition of target, adjacent anatomy, and the surgical path or trajectory to the target). At the intraoperative stage, such preoperative images and the associated planning information are registered to intraoperative coordinates via a navigation system to enable visualization of (tracked) instrumentation relative to preoperative images. A major limitation to such an approach is that motions during surgery, either rigid motions of bones manipulated during orthopaedic surgery or brain soft-tissue deformation in neurosurgery, are not captured, diminishing the accuracy of navigation systems. This dissertation seeks to use intraoperative images (e.g., x-ray fluoroscopy and cone-beam CT) to provide more up-to-date anatomical context that properly reflects the state of the patient during interventions to improve the performance of IGS. Advanced motion models for inter-modality image registration are developed to improve the accuracy of both preoperative planning and intraoperative guidance for applications in orthopaedic pelvic trauma surgery and minimally invasive intracranial neurosurgery. Image registration algorithms are developed with increasing complexity of motion that can be accommodated (single-body rigid, multi-body rigid, and deformable) and increasing complexity of registration models (statistical models, physics-based models, and deep learning-based models). For orthopaedic pelvic trauma surgery, the dissertation includes work encompassing: (i) a series of statistical models to model shape and pose variations of one or more pelvic bones and an atlas of trajectory annotations; (ii) frameworks for automatic segmentation via registration of the statistical models to preoperative CT and planning of fixation trajectories and dislocation / fracture reduction; and (iii) 3D-2D guidance using intraoperative fluoroscopy. For intracranial neurosurgery, the dissertation includes three inter-modality deformable registrations using physic-based Demons and deep learning models for CT-guided and CBCT-guided procedures

Global optimization algorithms for image registration and clustering

Global optimization is a classical problem of finding the minimum or maximum value of an objective function. It has applications in many areas, such as biological image analysis, chemistry, mechanical engineering, financial analysis, deep learning and image processing. For practical applications, it is important to understand the efficiency of global optimization algorithms. This dissertation develops and analyzes some new global optimization algorithms and applies them to practical problems, mainly for image registration and data clustering. First, the dissertation presents a new global optimization algorithm which approximates the optimum using only function values. The basic idea is to use the points at which the function has been evaluated to decompose the domain into a collection of hyper-rectangles. At each step of the algorithm, it chooses a hyper-rectangle according to a certain criterion and the next function evaluation is at the center of the hyper-rectangle. The dissertation includes a proof that the algorithm converges to the global optimum as the number of function evaluations goes to infinity, and also establishes the convergence rate. Standard test functions are used to experimentally evaluate the algorithm. The second part focuses on applying algorithms from the first part to solve some practical problems. Image processing tasks often require optimizing over some set of parameters. In the image registration problem, one attempts to determine the best transformation for aligning similar images. Such problems typically require minimizing a dissimilarity measure with multiple local minima. The dissertation describes a global optimization algorithm and applies it to the problem of identifying the best transformation for aligning two images. Global optimization algorithms can also be applied to the data clustering problem. The basic purpose of clustering is to categorize data into different groups by their similarity. The objective cost functions for clustering usually are non-convex. $k$-means is a popular algorithm which can find local optima quickly but may not obtain global optima. The different starting points for $k$-means can output different local optima. This dissertation describes a global optimization algorithm for approximating the global minimum of the clustering problem. The third part of the dissertation presents variations of the proposed algorithm that work with different assumptions on the available information, including a version that uses derivatives

Geodesic Active Fields:A Geometric Framework for Image Registration

Image registration is the concept of mapping homologous points in a pair of images. In other words, one is looking for an underlying deformation field that matches one image to a target image. The spectrum of applications of image registration is extremely large: It ranges from bio-medical imaging and computer vision, to remote sensing or geographic information systems, and even involves consumer electronics. Mathematically, image registration is an inverse problem that is ill-posed, which means that the exact solution might not exist or not be unique. In order to render the problem tractable, it is usual to write the problem as an energy minimization, and to introduce additional regularity constraints on the unknown data. In the case of image registration, one often minimizes an image mismatch energy, and adds an additive penalty on the deformation field regularity as smoothness prior. Here, we focus on the registration of the human cerebral cortex. Precise cortical registration is required, for example, in statistical group studies in functional MR imaging, or in the analysis of brain connectivity. In particular, we work with spherical inflations of the extracted hemispherical surface and associated features, such as cortical mean curvature. Spatial mapping between cortical surfaces can then be achieved by registering the respective spherical feature maps. Despite the simplified spherical geometry, inter-subject registration remains a challenging task, mainly due to the complexity and inter-subject variability of the involved brain structures. In this thesis, we therefore present a registration scheme, which takes the peculiarities of the spherical feature maps into particular consideration. First, we realize that we need an appropriate hierarchical representation, so as to coarsely align based on the important structures with greater inter-subject stability, before taking smaller and more variable details into account. Based on arguments from brain morphogenesis, we propose an anisotropic scale-space of mean-curvature maps, built around the Beltrami framework. Second, inspired by concepts from vision-related elements of psycho-physical Gestalt theory, we hypothesize that anisotropic Beltrami regularization better suits the requirements of image registration regularization, compared to traditional Gaussian filtering. Different objects in an image should be allowed to move separately, and regularization should be limited to within the individual Gestalts. We render the regularization feature-preserving by limiting diffusion across edges in the deformation field, which is in clear contrast to the indifferent linear smoothing. We do so by embedding the deformation field as a manifold in higher-dimensional space, and minimize the associated Beltrami energy which represents the hyperarea of this embedded manifold as measure of deformation field regularity. Further, instead of simply adding this regularity penalty to the image mismatch in lieu of the standard penalty, we propose to incorporate the local image mismatch as weighting function into the Beltrami energy. The image registration problem is thus reformulated as a weighted minimal surface problem. This approach has several appealing aspects, including (1) invariance to re-parametrization and ability to work with images defined on non-flat, Riemannian domains (e.g., curved surfaces, scalespaces), and (2) intrinsic modulation of the local regularization strength as a function of the local image mismatch and/or noise level. On a side note, we show that the proposed scheme can easily keep up with recent trends in image registration towards using diffeomorphic and inverse consistent deformation models. The proposed registration scheme, called Geodesic Active Fields (GAF), is non-linear and non-convex. Therefore we propose an efficient optimization scheme, based on splitting. Data-mismatch and deformation field regularity are optimized over two different deformation fields, which are constrained to be equal. The constraint is addressed using an augmented Lagrangian scheme, and the resulting optimization problem is solved efficiently using alternate minimization of simpler sub-problems. In particular, we show that the proposed method can easily compete with state-of-the-art registration methods, such as Demons. Finally, we provide an implementation of the fast GAF method on the sphere, so as to register the triangulated cortical feature maps. We build an automatic parcellation algorithm for the human cerebral cortex, which combines the delineations available on a set of atlas brains in a Bayesian approach, so as to automatically delineate the corresponding regions on a subject brain given its feature map. In a leave-one-out cross-validation study on 39 brain surfaces with 35 manually delineated gyral regions, we show that the pairwise subject-atlas registration with the proposed spherical registration scheme significantly improves the individual alignment of cortical labels between subject and atlas brains, and, consequently, that the estimated automatic parcellations after label fusion are of better quality