Cross-source point cloud matching by exploring structure property

University of Technology Sydney. Faculty of Engineering and Information Technology.Cross-source point cloud are 3D data coming from heterogeneous sensors. The matching of cross-source point cloud is extremely difficult because they contain mixture of different variations, such as missing data, noise and outliers, different viewpoint, density and spatial transformation. In this thesis, cross-source point cloud matching is solved from three aspects, utilizing of structure information, statistical model and learning representation. Chapter 1 introduces the value of cross-source point cloud registration and summarizes the key challenges of cross-source point cloud registration problem. Chapter 2 reviews the existing registration methods and analyse their limitation in solving the cross-source point cloud registration problem. Chapter 3 proposes two algorithms to discuss how to utilize structure information to solve the cross-source point cloud registration problem. In the first part of this chapter, macro and micro structures are extracted based on 3D point cloud segmentation. Then, these macro and micro structure components are integrated into a graph. With novel descriptors generated, the registration problem is successfully converted into graph matching problem. In the second part, weak region affinity and pixel-wise refinement are proposed to solve the cross-source point cloud. These two components are unified represented into a tensor space and the registration problem is converted into tensor optimization problem. In this method, the tensor space is updated when the transformation matrix is updated to get feedback from the recent transformation estimation step. Chapter 4 discusses how to utilize the statistical distribution of cross-source point cloud to solve matching problem. The goal is to find the potential matching region and estimate the accurate registration relationship. In this chapter, ensemble of shape functions (ESF) is utilized to select potential regions and a novel registration is proposed to solve the matching problem. For the registration, Gaussian mixture models (GMM) is selected as our mathematical tool. However, different to previous GMM-based registration methods, which assume a GMM for each point cloud, the proposed algorithm assumes a virtual GMM and the cross-source point clouds are samples from the virtual GMM. Then, the transformation is optimized to project the samples into a same virtual GMM. When the optimization is convergence, both the parameters of GMM and the transformation matrices are estimated. In Chapter 5, a deep learning method is proposed to represent the local structure information. Because of arbitrary rotation in cross-source point clouds, a rotation-invariant 3D representation method is proposed to robust represent the 3D point cloud although there are arbitrary rotation and translation. Also, there is no robust keypoints in these cross-source point cloud because of they come from heterogenous sensors, train the network is very difficult. A region-based method is proposed to generate regions for each point cloud and synthetic labelled dataset is constructed for training the network. All these algorithms are aimed to solve the cross-source point cloud registration problem. The performance of these algorithms is tested on many datasets, which shows the effective and correctness. These algorithms also provide insightful knowledge for 3D computer vision workers to process 3D point cloud

Three-dimensional measurements with a novel technique combination of confocal and focus variation with a simultaneous scan

The most common optical measurement technologies used today for the three dimensional measurement of technical surfaces are Coherence Scanning Interferometry (CSI), Imaging Confocal Microscopy (IC), and Focus Variation (FV). Each one has its benefits and its drawbacks. FV will be the ideal technology for the measurement of those regions where the slopes are high and where the surface is very rough, while CSI and IC will provide better results for smoother and flatter surface regions. In this work we investigated the benefits and drawbacks of combining Interferometry, Confocal and focus variation to get better measurement of technical surfaces. We investigated a way of using Microdisplay Scanning type of Confocal Microscope to acquire on a simultaneous scan confocal and focus Variation information to reconstruct a three dimensional measurement. Several methods are presented to fuse the optical sectioning properties of both techniques as well as the topographical information. This work shows the benefit of this combination technique on several industrial samples where neither confocal nor focus variation is able to provide optimal results.Postprint (author's final draft

Surveying and Three-Dimensional Modeling for Preservation and Structural Analysis of Cultural Heritage

Dense point clouds can be used for three important steps in structural analysis, in the field of cultural heritage, regardless of which instrument it was used for acquisition data. Firstly, they allow deriving the geometric part of a finite element (FE) model automatically or semi-automatically. User input is mainly required to complement invisible parts and boundaries of the structure, and to assign meaningful approximate physical parameters. Secondly, FE model obtained from point clouds can be used to estimate better and more precise parameters of the structural analysis, i.e., to train the FE model. Finally, the definition of a correct Level of Detail about the three-dimensional model, deriving from the initial point cloud, can be used to define the limit beyond which the structural analysis is compromised, or anyway less precise. In this work of research, this will be demonstrated using three different case studies of buildings, consisting mainly of masonry, measured through terrestrial laser scanning and photogrammetric acquisitions. This approach is not a typical study for geomatics analysis, but its challenges allow studying benefits and limitations. The results and the proposed approaches could represent a step towards a multidisciplinary approach where Geomatics can play a critical role in the monitoring and civil engineering field. Furthermore, through a geometrical reconstruction, different analyses and comparisons are possible, in order to evaluate how the numerical model is accurate. In fact, the discrepancies between the different results allow to evaluate how, from a geometric and simplified modeling, important details can be lost. This causes, for example, modifications in terms of mass and volume of the structure

Multi-scale data fusion for surface metrology

The major trends in manufacturing are miniaturization, convergence of the traditional research fields and creation of interdisciplinary research areas. These trends have resulted in the development of multi-scale models and multi-scale surfaces to optimize the performance. Multi-scale surfaces that exhibit specific properties at different scales for a specific purpose require multi-scale measurement and characterization. Researchers and instrument developers have developed instruments that are able to perform measurements at multiple scales but lack the much required multi- scale characterization capability. The primary focus of this research was to explore possible multi-scale data fusion strategies and options for surface metrology domain and to develop enabling software tools in order to obtain effective multi-scale surface characterization, maximizing fidelity while minimizing measurement cost and time. This research effort explored the fusion strategies for surface metrology domain and narrowed the focus on Discrete Wavelet Frame (DWF) based multi-scale decomposition. An optimized multi-scale data fusion strategy ‘FWR method’ was developed and was successfully demonstrated on both high aspect ratio surfaces and non-planar surfaces. It was demonstrated that the datum features can be effectively characterized at a lower resolution using one system (Vision CMM) and the actual features of interest could be characterized at a higher resolution using another system (Coherence Scanning Interferometer) with higher capability while minimizing the measurement time

A Learning Health System for Radiation Oncology

The proposed research aims to address the challenges faced by clinical data science researchers in radiation oncology accessing, integrating, and analyzing heterogeneous data from various sources. The research presents a scalable intelligent infrastructure, called the Health Information Gateway and Exchange (HINGE), which captures and structures data from multiple sources into a knowledge base with semantically interlinked entities. This infrastructure enables researchers to mine novel associations and gather relevant knowledge for personalized clinical outcomes. The dissertation discusses the design framework and implementation of HINGE, which abstracts structured data from treatment planning systems, treatment management systems, and electronic health records. It utilizes disease-specific smart templates for capturing clinical information in a discrete manner. HINGE performs data extraction, aggregation, and quality and outcome assessment functions automatically, connecting seamlessly with local IT/medical infrastructure. Furthermore, the research presents a knowledge graph-based approach to map radiotherapy data to an ontology-based data repository using FAIR (Findable, Accessible, Interoperable, Reusable) concepts. This approach ensures that the data is easily discoverable and accessible for clinical decision support systems. The dissertation explores the ETL (Extract, Transform, Load) process, data model frameworks, ontologies, and provides a real-world clinical use case for this data mapping. To improve the efficiency of retrieving information from large clinical datasets, a search engine based on ontology-based keyword searching and synonym-based term matching tool was developed. The hierarchical nature of ontologies is leveraged to retrieve patient records based on parent and children classes. Additionally, patient similarity analysis is conducted using vector embedding models (Word2Vec, Doc2Vec, GloVe, and FastText) to identify similar patients based on text corpus creation methods. Results from the analysis using these models are presented. The implementation of a learning health system for predicting radiation pneumonitis following stereotactic body radiotherapy is also discussed. 3D convolutional neural networks (CNNs) are utilized with radiographic and dosimetric datasets to predict the likelihood of radiation pneumonitis. DenseNet-121 and ResNet-50 models are employed for this study, along with integrated gradient techniques to identify salient regions within the input 3D image dataset. The predictive performance of the 3D CNN models is evaluated based on clinical outcomes. Overall, the proposed Learning Health System provides a comprehensive solution for capturing, integrating, and analyzing heterogeneous data in a knowledge base. It offers researchers the ability to extract valuable insights and associations from diverse sources, ultimately leading to improved clinical outcomes. This work can serve as a model for implementing LHS in other medical specialties, advancing personalized and data-driven medicine

Computerized Analysis of Magnetic Resonance Images to Study Cerebral Anatomy in Developing Neonates

The study of cerebral anatomy in developing neonates is of great importance for the understanding of brain development during the early period of life. This dissertation therefore focuses on three challenges in the modelling of cerebral anatomy in neonates during brain development. The methods that have been developed all use Magnetic Resonance Images (MRI) as source data. To facilitate study of vascular development in the neonatal period, a set of image analysis algorithms are developed to automatically extract and model cerebral vessel trees. The whole process consists of cerebral vessel tracking from automatically placed seed points, vessel tree generation, and vasculature registration and matching. These algorithms have been tested on clinical Time-of- Flight (TOF) MR angiographic datasets. To facilitate study of the neonatal cortex a complete cerebral cortex segmentation and reconstruction pipeline has been developed. Segmentation of the neonatal cortex is not effectively done by existing algorithms designed for the adult brain because the contrast between grey and white matter is reversed. This causes pixels containing tissue mixtures to be incorrectly labelled by conventional methods. The neonatal cortical segmentation method that has been developed is based on a novel expectation-maximization (EM) method with explicit correction for mislabelled partial volume voxels. Based on the resulting cortical segmentation, an implicit surface evolution technique is adopted for the reconstruction of the cortex in neonates. The performance of the method is investigated by performing a detailed landmark study. To facilitate study of cortical development, a cortical surface registration algorithm for aligning the cortical surface is developed. The method first inflates extracted cortical surfaces and then performs a non-rigid surface registration using free-form deformations (FFDs) to remove residual alignment. Validation experiments using data labelled by an expert observer demonstrate that the method can capture local changes and follow the growth of specific sulcus

Assisting digital volume correlation with mechanical image-based modeling: application to the measurement of kinematic fields at the architecture scale in cellular materials

La mesure de champs de déplacement et de déformation aux petites échelles dans des microstructures complexes représente encore un défi majeur dans le monde de la mécanique expérimentale. Ceci est en partie dû aux acquisitions d'images et à la pauvreté de la texture à ces échelles. C'est notamment le cas pour les matériaux cellulaires lorsqu'ils sont imagés avec des micro-tomographes conventionnels et qu'ils peuvent être sujets à des mécanismes de déformation complexes. Comme la validation de modèles numériques et l'identification des propriétés mécaniques de matériaux se base sur des mesures précises de déplacements et de déformations, la conception et l'implémentation d'algorithmes robustes et fiables de corrélation d'images semble nécessaire. Lorsque l'on s'intéresse à l'utilisation de la corrélation d'images volumiques (DVC) pour les matériaux cellulaires, on est confronté à un paradoxe: l'absence de texture à l'échelle du constituant conduit à considérer l'architecture comme marqueur pour la corrélation. Ceci conduit à l'échec des techniques ordinaires de DVC à mesurer des cinématiques aux échelles subcellulaires en lien avec des comportements mécaniques locaux complexes tels que la flexion ou le flambement de travées. L'objectif de cette thèse est la conception d'une technique de DVC pour la mesure de champs de déplacement dans des matériaux cellulaires à l'échelle de leurs architectures. Cette technique assiste la corrélation d'images par une régularisation élastique faible en utilisant un modèle mécanique généré automatiquement et basé sur les images. La méthode suggérée introduit une séparation d'échelles au dessus desquelles la DVC est dominante et en dessous desquelles elle est assistée par le modèle mécanique basé sur l'image. Une première étude numérique consistant à comparer différentes techniques de construction de modèles mécaniques basés sur les images est conduite. L'accent est mis sur deux méthodes de calcul particulières: la méthode des éléments finis (FEM) et la méthode des cellules finies (FCM) qui consiste à immerger la géométrie complexe dans une grille régulière de haut ordre sans utiliser de mailleurs. Si la FCM évite une première phase délicate de discrétisation, plusieurs paramètres restent néanmoins délicats à fixer. Dans ce travail, ces paramètres sont ajustés afin d'obtenir (a) la meilleure précision (bornée par les erreurs de pixellisation) tout en (b) assurant une complexité minimale. Pour l'aspect mesure par corrélation d'images régularisée, plusieurs expérimentations virtuelles à partir de différentes simulations numériques (en élasticité, en plasticité et en non-linéarité géométrique) sont d'abord réalisées afin d'analyser l'influence des paramètres de régularisation introduits. Les erreurs de mesures peuvent dans ce cas être quantifiées à l'aide des solutions de référence éléments finis. La capacité de la méthode à mesurer des cinématiques complexes en absence de texture est démontrée pour des régimes non-linéaires tels que le flambement. Finalement, le travail proposé est généralisé à la corrélation volumique des différents états de déformation du matériau et à la construction automatique de la micro-architecture cellulaire en utilisant soit une grille B-spline d'ordre arbitraire (FCM) soit un maillage éléments finis (FEM). Une mise en évidence expérimentale de l'efficacité et de la justesse de l'approche proposée est effectuée à travers de la mesure de cinématiques complexes dans une mousse polyuréthane sollicitée en compression lors d'un essai in situ.Measuring displacement and strain fields at low observable scales in complex microstructures still remains a challenge in experimental mechanics often because of the combination of low definition images with poor texture at this scale. The problem is particularly acute in the case of cellular materials, when imaged by conventional micro-tomographs, for which complex highly non-linear local phenomena can occur. As the validation of numerical models and the identification of mechanical properties of materials must rely on accurate measurements of displacement and strain fields, the design and implementation of robust and faithful image correlation algorithms must be conducted. With cellular materials, the use of digital volume correlation (DVC) faces a paradox: in the absence of markings of exploitable texture on/or in the struts or cell walls, the available speckle will be formed by the material architecture itself. This leads to the inability of classical DVC codes to measure kinematics at the cellular and a fortiori sub-cellular scales, precisely because the interpolation basis of the displacement field cannot account for the complexity of the underlying kinematics, especially when bending or buckling of beams or walls occurs. The objective of the thesis is to develop a DVC technique for the measurement of displacement fields in cellular materials at the scale of their architecture. The proposed solution consists in assisting DVC by a weak elastic regularization using an automatic image-based mechanical model. The proposed method introduces a separation of scales above which DVC is dominant and below which it is assisted by image-based modeling. First, a numerical investigation and comparison of different techniques for building automatically a geometric and mechanical model from tomographic images is conducted. Two particular methods are considered: the finite element method (FEM) and the finite-cell method (FCM). The FCM is a fictitious domain method that consists in immersing the complex geometry in a high order structured grid and does not require meshing. In this context, various discretization parameters appear delicate to choose. In this work, these parameters are adjusted to obtain (a) the best possible accuracy (bounded by pixelation errors) while (b) ensuring minimal complexity. Concerning the ability of the mechanical image-based models to regularize DIC, several virtual experimentations are performed in two-dimensions in order to finely analyze the influence of the introduced regularization lengths for different input mechanical behaviors (elastic, elasto-plastic and geometrically non-linear) and in comparison with ground truth. We show that the method can estimate complex local displacement and strain fields with speckle-free low definition images, even in non-linear regimes such as local buckling. Finally a three-dimensional generalization is performed through the development of a DVC framework. It takes as an input the reconstructed volumes at the different deformation states of the material and constructs automatically the cellular micro-architeture geometry. It considers either an immersed structured B-spline grid of arbitrary order or a finite-element mesh. An experimental evidence is performed by measuring the complex kinematics of a polyurethane foam under compression during an in situ test

ULTRA CLOSE-RANGE DIGITAL PHOTOGRAMMETRY AS A TOOL TO PRESERVE, STUDY, AND SHARE SKELETAL REMAINS

Skeletal collections around the world hold valuable and intriguing knowledge about humanity. Their potential value could be fully exploited by overcoming current limitations in documenting and sharing them. Virtual anthropology provides effective ways to study and value skeletal collections using three-dimensional (3D) data, e.g. allowing powerful comparative and evolutionary studies, along with specimen preservation and dissemination. CT- and laser scanning are the most used techniques for three-dimensional reconstruction. However, they are resource-intensive and, therefore, difficult to be applied to large samples or skeletal collections. Ultra close-range digital photogrammetry (UCR-DP) enables photorealistic 3D reconstructions from simple photographs of the specimen. However, it is the least used method in skeletal anthropology and the lack of appropriate protocols often limit the quality of its outcomes. This Ph.D. thesis explored UCR-DP application in skeletal anthropology. The state-of-the-art of this technique was studied, and a new approach based on cloud computing was proposed and validated against current gold standards. This approach relies on the processing capabilities of remote servers and a free-for-academic use software environment; it proved to produce measurements equivalent to those of osteometry and, in many cases, they were more precise than those of CT-scanning. Cloud-based UCR-DP allowed the processing of multiple 3D models at once, leading to a low-cost, quick, and effective 3D production. The technique was successfully used to digitally preserve an initial sample of 534 crania from the skeletal collections of the Museo Sardo di Antropologia ed Etnografia (MuSAE, Università degli Studi di Cagliari). Best practices in using the technique for skeletal collection dissemination were studied and several applications were developed including MuSAE online virtual tours, virtual physical anthropology labs and distance learning, durable online dissemination, and values-led participatorily designed interactive and immersive exhibitions at the MuSAE. The sample will be used in a future population study of Sardinian skeletal characteristics from the Neolithic to modern times. In conclusion, cloud-based UCR-DP offers many significant advantages over other 3D scanning techniques: greater versatility in terms of application range and technical implementation, scalability, photorealistic restitution, reduced requirements relating to hardware, labour, time, and cost, and is, therefore, the best choice to document and value effectively large skeletal samples and collections