Graph matching using position coordinates and local features for image analysis

Encontrar las correspondencias entre dos imágenes es un problema crucial en el campo de la visión por ordenador i el reconocimiento de patrones. Es relevante para un amplio rango de propósitos des de aplicaciones de reconocimiento de objetos en las áreas de biometría, análisis de documentos i análisis de formas hasta aplicaciones relacionadas con la geometría desde múltiples puntos de vista tales cómo la recuperación de la pose, estructura desde el movimiento y localización y mapeo. La mayoría de las técnicas existentes enfocan este problema o bien usando características locales en la imagen o bien usando métodos de registro de conjuntos de puntos (o bien una mezcla de ambos). En las primeras, un conjunto disperso de características es primeramente extraído de las imágenes y luego caracterizado en la forma de vectores descriptores usando evidencias locales de la imagen. Las características son asociadas según la similitud entre sus descriptores. En las segundas, los conjuntos de características son considerados cómo conjuntos de puntos los cuales son asociados usando técnicas de optimización no lineal. Estos son procedimientos iterativos que estiman los parámetros de correspondencia y de alineamiento en pasos alternados. Los grafos son representaciones que contemplan relaciones binarias entre las características. Tener en cuenta relaciones binarias al problema de la correspondencia a menudo lleva al llamado problema del emparejamiento de grafos. Existe cierta cantidad de métodos en la literatura destinados a encontrar soluciones aproximadas a diferentes instancias del problema de emparejamiento de grafos, que en la mayoría de casos es del tipo "NP-hard". El cuerpo de trabajo principal de esta tesis está dedicado a formular ambos problemas de asociación de características de imagen y registro de conjunto de puntos como instancias del problema de emparejamiento de grafos. En todos los casos proponemos algoritmos aproximados para solucionar estos problemas y nos comparamos con un número de métodos existentes pertenecientes a diferentes áreas como eliminadores de "outliers", métodos de registro de conjuntos de puntos y otros métodos de emparejamiento de grafos. Los experimentos muestran que en la mayoría de casos los métodos propuestos superan al resto. En ocasiones los métodos propuestos o bien comparten el mejor rendimiento con algún método competidor o bien obtienen resultados ligeramente peores. En estos casos, los métodos propuestos normalmente presentan tiempos computacionales inferiores.Trobar les correspondències entre dues imatges és un problema crucial en el camp de la visió per ordinador i el reconeixement de patrons. És rellevant per un ampli ventall de propòsits des d’aplicacions de reconeixement d’objectes en les àrees de biometria, anàlisi de documents i anàlisi de formes fins aplicacions relacionades amb geometria des de múltiples punts de vista tals com recuperació de pose, estructura des del moviment i localització i mapeig. La majoria de les tècniques existents enfoquen aquest problema o bé usant característiques locals a la imatge o bé usant mètodes de registre de conjunts de punts (o bé una mescla d’ambdós). En les primeres, un conjunt dispers de característiques és primerament extret de les imatges i després caracteritzat en la forma de vectors descriptors usant evidències locals de la imatge. Les característiques son associades segons la similitud entre els seus descriptors. En les segones, els conjunts de característiques son considerats com conjunts de punts els quals son associats usant tècniques d’optimització no lineal. Aquests son procediments iteratius que estimen els paràmetres de correspondència i d’alineament en passos alternats. Els grafs son representacions que contemplen relacions binaries entre les característiques. Tenir en compte relacions binàries al problema de la correspondència sovint porta a l’anomenat problema de l’emparellament de grafs. Existeix certa quantitat de mètodes a la literatura destinats a trobar solucions aproximades a diferents instàncies del problema d’emparellament de grafs, el qual en la majoria de casos és del tipus “NP-hard”. Una part del nostre treball està dedicat a investigar els beneficis de les mesures de ``bins'' creuats per a la comparació de característiques locals de les imatges. La resta està dedicat a formular ambdós problemes d’associació de característiques d’imatge i registre de conjunt de punts com a instàncies del problema d’emparellament de grafs. En tots els casos proposem algoritmes aproximats per solucionar aquests problemes i ens comparem amb un nombre de mètodes existents pertanyents a diferents àrees com eliminadors d’“outliers”, mètodes de registre de conjunts de punts i altres mètodes d’emparellament de grafs. Els experiments mostren que en la majoria de casos els mètodes proposats superen a la resta. En ocasions els mètodes proposats o bé comparteixen el millor rendiment amb algun mètode competidor o bé obtenen resultats lleugerament pitjors. En aquests casos, els mètodes proposats normalment presenten temps computacionals inferiors

Computerized Analysis of Magnetic Resonance Images to Study Cerebral Anatomy in Developing Neonates

The study of cerebral anatomy in developing neonates is of great importance for the understanding of brain development during the early period of life. This dissertation therefore focuses on three challenges in the modelling of cerebral anatomy in neonates during brain development. The methods that have been developed all use Magnetic Resonance Images (MRI) as source data. To facilitate study of vascular development in the neonatal period, a set of image analysis algorithms are developed to automatically extract and model cerebral vessel trees. The whole process consists of cerebral vessel tracking from automatically placed seed points, vessel tree generation, and vasculature registration and matching. These algorithms have been tested on clinical Time-of- Flight (TOF) MR angiographic datasets. To facilitate study of the neonatal cortex a complete cerebral cortex segmentation and reconstruction pipeline has been developed. Segmentation of the neonatal cortex is not effectively done by existing algorithms designed for the adult brain because the contrast between grey and white matter is reversed. This causes pixels containing tissue mixtures to be incorrectly labelled by conventional methods. The neonatal cortical segmentation method that has been developed is based on a novel expectation-maximization (EM) method with explicit correction for mislabelled partial volume voxels. Based on the resulting cortical segmentation, an implicit surface evolution technique is adopted for the reconstruction of the cortex in neonates. The performance of the method is investigated by performing a detailed landmark study. To facilitate study of cortical development, a cortical surface registration algorithm for aligning the cortical surface is developed. The method first inflates extracted cortical surfaces and then performs a non-rigid surface registration using free-form deformations (FFDs) to remove residual alignment. Validation experiments using data labelled by an expert observer demonstrate that the method can capture local changes and follow the growth of specific sulcus

A Closest Point Proposal for MCMC-based Probabilistic Surface Registration

We propose to view non-rigid surface registration as a probabilistic inference problem. Given a target surface, we estimate the posterior distribution of surface registrations. We demonstrate how the posterior distribution can be used to build shape models that generalize better and show how to visualize the uncertainty in the established correspondence. Furthermore, in a reconstruction task, we show how to estimate the posterior distribution of missing data without assuming a fixed point-to-point correspondence. We introduce the closest-point proposal for the Metropolis-Hastings algorithm. Our proposal overcomes the limitation of slow convergence compared to a random-walk strategy. As the algorithm decouples inference from modeling the posterior using a propose-and-verify scheme, we show how to choose different distance measures for the likelihood model. All presented results are fully reproducible using publicly available data and our open-source implementation of the registration framework

From Noisy Point Clouds to Complete Ear Shapes: Unsupervised Pipeline

Funding Information: This work was supported in part by the European Union’s Horizon 2020 Research And Innovation Programme through the Marie Skłodowska-Curie Project BIGMATH, under Agreement 812912, and in part by the Eureka Eurostars under Project E!11439 FacePrint. The work of Cláudia Soares was supported in part by the Strategic Project NOVA LINCS under Grant UIDB/04516/2020. Funding Information: This work was supported in part by the European Union's Horizon 2020 Research And Innovation Programme through the Marie Skiodowska-Curie Project BIGMATH, under Agreement 812912, and in part by the Eureka Eurostars under Project E11439 FacePrint. The work of Claudia Soares was supported in part by the Strategic Project NOVA LINCS under Grant UIDB/04516/2020. Publisher Copyright: © 2013 IEEE.Ears are a particularly difficult region of the human face to model, not only due to the non-rigid deformations existing between shapes but also to the challenges in processing the retrieved data. The first step towards obtaining a good model is to have complete scans in correspondence, but these usually present a higher amount of occlusions, noise and outliers when compared to most face regions, thus requiring a specific procedure. Therefore, we propose a complete pipeline taking as input unordered 3D point clouds with the aforementioned problems, and producing as output a dataset in correspondence, with completion of the missing data. We provide a comparison of several state-of-the-art registration and shape completion methods, concluding on the best choice for each of the steps.publishersversionpublishe

Registration of 3D Point Clouds and Meshes: A Survey From Rigid to Non-Rigid

Three-dimensional surface registration transforms multiple three-dimensional data sets into the same coordinate system so as to align overlapping components of these sets. Recent surveys have covered different aspects of either rigid or nonrigid registration, but seldom discuss them as a whole. Our study serves two purposes: 1) To give a comprehensive survey of both types of registration, focusing on three-dimensional point clouds and meshes and 2) to provide a better understanding of registration from the perspective of data fitting. Registration is closely related to data fitting in which it comprises three core interwoven components: model selection, correspondences and constraints, and optimization. Study of these components 1) provides a basis for comparison of the novelties of different techniques, 2) reveals the similarity of rigid and nonrigid registration in terms of problem representations, and 3) shows how overfitting arises in nonrigid registration and the reasons for increasing interest in intrinsic techniques. We further summarize some practical issues of registration which include initializations and evaluations, and discuss some of our own observations, insights and foreseeable research trends

Calculating Sparse and Dense Correspondences for Near-Isometric Shapes

Comparing and analysing digital models are basic techniques of geometric shape processing. These techniques have a variety of applications, such as extracting the domain knowledge contained in the growing number of digital models to simplify shape modelling. Another example application is the analysis of real-world objects, which itself has a variety of applications, such as medical examinations, medical and agricultural research, and infrastructure maintenance. As methods to digitalize physical objects mature, any advances in the analysis of digital shapes lead to progress in the analysis of real-world objects. Global shape properties, like volume and surface area, are simple to compare but contain only very limited information. Much more information is contained in local shape differences, such as where and how a plant grew. Sadly the computation of local shape differences is hard as it requires knowledge of corresponding point pairs, i.e. points on both shapes that correspond to each other. The following article thesis (cumulative dissertation) discusses several recent publications for the computation of corresponding points: - Geodesic distances between points, i.e. distances along the surface, are fundamental for several shape processing tasks as well as several shape matching techniques. Chapter 3 introduces and analyses fast and accurate bounds on geodesic distances. - When building a shape space on a set of shapes, misaligned correspondences lead to points moving along the surfaces and finally to a larger shape space. Chapter 4 shows that this also works the other way around, that is good correspondences are obtain by optimizing them to generate a compact shape space. - Representing correspondences with a “functional map” has a variety of advantages. Chapter 5 shows that representing the correspondence map as an alignment of Green’s functions of the Laplace operator has similar advantages, but is much less dependent on the number of eigenvectors used for the computations. - Quadratic assignment problems were recently shown to reliably yield sparse correspondences. Chapter 6 compares state-of-the-art convex relaxations of graphics and vision with methods from discrete optimization on typical quadratic assignment problems emerging in shape matching

A Probabilistic Framework for Statistical Shape Models and Atlas Construction: Application to Neuroimaging

Accurate and reliable registration of shapes and multi-dimensional point sets describing the morphology/physiology of anatomical structures is a pre-requisite for constructing statistical shape models (SSMs) and atlases. Such statistical descriptions of variability across populations (regarding shape or other morphological/physiological quantities) are based on homologous correspondences across the multiple samples that comprise the training data. The notion of exact correspondence can be ambiguous when these data contain noise and outliers, missing data, or significant and abnormal variations due to pathology. But, these phenomena are common in medical image-derived data, due, for example, to inconsistencies in image quality and acquisition protocols, presence of motion artefacts, differences in pre-processing steps, and inherent variability across patient populations and demographics. This thesis therefore focuses on formulating a unified probabilistic framework for the registration of shapes and so-called \textit{generalised point sets}, which is robust to the anomalies and variations described. Statistical analysis of shapes across large cohorts demands automatic generation of training sets (image segmentations delineating the structure of interest), as manual and semi-supervised approaches can be prohibitively time consuming. However, automated segmentation and landmarking of images often result in shapes with high levels of outliers and missing data. Consequently, a robust method for registration and correspondence estimation is required. A probabilistic group-wise registration framework for point-based representations of shapes, based on Student’s t-mixture model (TMM) and a multi-resolution extension to the same (mrTMM), are formulated to this end. The frameworks exploit the inherent robustness of Student’s t-distributions to outliers, which is lacking in existing Gaussian mixture model (GMM)-based approaches. The registration accuracy of the proposed approaches was quantitatively evaluated and shown to outperform the state-of-the-art, using synthetic and clinical data. A corresponding improvement in the quality of SSMs generated subsequently was also shown, particularly for data sets containing high levels of noise. In general, the proposed approach requires fewer user specified parameters than existing methods, whilst affording much improved robustness to outliers. Registration of generalised point sets, which combine disparate features such as spatial positions, directional/axial data, and scalar-valued quantities, was studied next. A hybrid mixture model (HMM), combining different types of probability distributions, was formulated to facilitate the joint registration and clustering of multi-dimensional point sets of this nature. Two variants of the HMM were developed for modelling: (1) axial data; and (2) directional data. The former, based on a combination of Student’s t, Watson and Gaussian distributions, was used to register hybrid point sets comprising magnetic resonance diffusion tensor image (DTI)-derived quantities, such as voxel spatial positions (defining a region/structure of interest), associated fibre orientations, and scalar measures reflecting tissue anisotropy. The latter meanwhile, formulated using a combination of Student’s t and Von-Mises-Fisher distributions, is used for the registration of shapes represented as hybrid point sets comprising spatial positions and associated surface normal vectors. The Watson-variant of the HMM facilitates statistical analysis and group-wise comparisons of DTI data across patient populations, presented as an exemplar application of the proposed approach. The Fisher-variant of the HMM on the other hand, was used to register hybrid representations of shapes, providing substantial improvements over point-based registration approaches in terms of anatomical validity in the estimated correspondences