Developing a Semantic-Driven Hybrid Segmentation Method for Point Clouds of 3D Shapes

With the rapid development of point cloud processing technologies and the availability of a wide range of 3D capturing devices, a geometric object from the real world can be directly represented digitally as a dense and fine point cloud. Decomposing a 3D shape represented in point cloud into meaningful parts has very important practical implications in the fields of computer graphics, virtual reality and mixed reality. In this paper, a semantic-driven automated hybrid segmentation method is proposed for 3D point cloud shapes. Our method consists of three stages: semantic clustering, variational merging, and region remerging. In the first stage, a new feature of point cloud, called Local Concave-Convex Histogram, is introduced to first extract saddle regions complying with the semantic boundary feature. All other types of regions are then aggregated according to this extracted feature. This stage often leads to multiple over-segmentation convex regions, which are then remerged by a variational method established based on the narrow-band theory. Finally, in order to recombine the regions with the approximate shapes, order relation is introduced to improve the weighting forms in calculating the conventional Shape Diameter Function. We have conducted extensive experiments with the Princeton Dataset. The results show that the proposed algorithm outperforms the state-of-the-art algorithms in this area. We have also applied the proposed algorithm to process the point cloud data acquired directly from the real 3D objects. It achieves excellent results too. These results demonstrate that the method proposed in this paper is effective and universal

Improvement of Geometric Quality Inspection and Process Efficiency in Additive Manufacturing

Additive manufacturing (AM) has been known for its ability of producing complex geometries in flexible production environments. In recent decades, it has attracted increasing attention and interest of different industrial sectors. However, there are still some technical challenges hindering the wide application of AM. One major barrier is the limited dimensional accuracy of AM produced parts, especially for industrial sectors such as aerospace and biomedical engineering, where high geometric accuracy is required. Nevertheless, traditional quality inspection techniques might not perform well due to the complexity and flexibility of AM fabricated parts. Another issue, which is brought up from the growing demand for large-scale 3D printing in these industry sectors, is the limited fabrication speed of AM processes. However, how to improve the fabrication efficiency without sacrificing the geometric quality is still a challenging problem that has not been well addressed. In this work, new geometric inspection methods are proposed for both offline and online inspection paradigms, and a layer-by-layer toolpath optimization model is proposed to further improve the fabrication efficiency of AM processes without degrading the resolution. First, a novel Location-Orientation-Shape (LOS) distribution derived from 3D scanning output is proposed to improve the offline inspection in detecting and distinguishing positional and dimensional non-conformities of features. Second, the online geometric inspection is improved by a multi-resolution alignment and inspection framework based on wavelet decomposition and design of experiments (DOE). The new framework is able to improve the alignment accuracy and to distinguish different sources of error based on the shape deviation of each layer. In addition, a quickest change point detection method is used to identify the layer where the earliest change of systematic deviation distribution occurs during the printing process. Third, to further improve the printing efficiency without sacrificing the quality of each layer, a toolpath allocation and scheduling optimization model is proposed based on a concurrent AM process that allows multiple extruders to work collaboratively on the same layer. For each perspective of improvements, numerical studies are provided to emphasize the theoretical and practical meanings of proposed methodologies

Evaluation System for Craniosynostosis Surgeries with Computer Simulation and Statistical Modelling

Craniosynostosis is a pathology in infants when one or more sutures prematurely closed, leading to abnormal skull shape. It has been classified according to the specific suture that has been closed, each of which has a typical skull shape. Surgery is the common treatment to correct the deformed skull shape and to reduce the excessive intracranial pressure. Since every case is unique, the cranial facial teams have difficulties to select an optimum solution for a specific patient from multiple options. In addition, there is not an appropriate quantified measurement existed currently to help cranial facial team to quantitatively evaluate their surgeries. We aimed to develop a head model of a craniosynostosis patient, which allows neurosurgeons to perform any potential surgeries on it so as to simulate the postoperative head development. Therefore, neurosurgeons could foresee the surgical results and is able to select the optimal one. In this thesis, we have developed a normal head model, and built mathematical models for possible dynamic behaviors. We also modified this model by closing one or two sutures to simulate common types of craniosynostosis. The abnormal simulation results showed a qualitative match with real cases and the normal simulation indicated a higher growth rate of cranial index than clinical data. We believed that this discrepancy caused by the rigidity of our skull plates, which will be adapted to deformable object in the future. In order to help neurosurgeons to better evaluate a surgery, we hope to develop an algorithm to quantify the level of deformity of a skull. We have designed a set of work flow and targeted curvatures as the key role. A training data was carefully selected to search for an optimal system to characterize different shapes. A set of test data was used to validate our algorithm to assess the performance of the optimal system. With a stable evaluating system, we can evaluate a surgery by comparing the preoperative and postoperative skulls from the patient. An effective surgery can be considered if the postoperative skull shifted toward normal shape from preoperative shape

Collaborative visualization environment using P2P technology and ellipsoidal mesh partitioning

Ph.DDOCTOR OF PHILOSOPH

Toward knowledge-based automatic 3D spatial topological modeling from LiDAR point clouds for urban areas

Le traitement d'un très grand nombre de données LiDAR demeure très coûteux et nécessite des approches de modélisation 3D automatisée. De plus, les nuages de points incomplets causés par l'occlusion et la densité ainsi que les incertitudes liées au traitement des données LiDAR compliquent la création automatique de modèles 3D enrichis sémantiquement. Ce travail de recherche vise à développer de nouvelles solutions pour la création automatique de modèles géométriques 3D complets avec des étiquettes sémantiques à partir de nuages de points incomplets. Un cadre intégrant la connaissance des objets à la modélisation 3D est proposé pour améliorer la complétude des modèles géométriques 3D en utilisant un raisonnement qualitatif basé sur les informations sémantiques des objets et de leurs composants, leurs relations géométriques et spatiales. De plus, nous visons à tirer parti de la connaissance qualitative des objets en reconnaissance automatique des objets et à la création de modèles géométriques 3D complets à partir de nuages de points incomplets. Pour atteindre cet objectif, plusieurs solutions sont proposées pour la segmentation automatique, l'identification des relations topologiques entre les composants de l'objet, la reconnaissance des caractéristiques et la création de modèles géométriques 3D complets. (1) Des solutions d'apprentissage automatique ont été proposées pour la segmentation sémantique automatique et la segmentation de type CAO afin de segmenter des objets aux structures complexes. (2) Nous avons proposé un algorithme pour identifier efficacement les relations topologiques entre les composants d'objet extraits des nuages de points afin d'assembler un modèle de Représentation Frontière. (3) L'intégration des connaissances sur les objets et la reconnaissance des caractéristiques a été développée pour inférer automatiquement les étiquettes sémantiques des objets et de leurs composants. Afin de traiter les informations incertitudes, une solution de raisonnement automatique incertain, basée sur des règles représentant la connaissance, a été développée pour reconnaître les composants du bâtiment à partir d'informations incertaines extraites des nuages de points. (4) Une méthode heuristique pour la création de modèles géométriques 3D complets a été conçue en utilisant les connaissances relatives aux bâtiments, les informations géométriques et topologiques des composants du bâtiment et les informations sémantiques obtenues à partir de la reconnaissance des caractéristiques. Enfin, le cadre proposé pour améliorer la modélisation 3D automatique à partir de nuages de points de zones urbaines a été validé par une étude de cas visant à créer un modèle de bâtiment 3D complet. L'expérimentation démontre que l'intégration des connaissances dans les étapes de la modélisation 3D est efficace pour créer un modèle de construction complet à partir de nuages de points incomplets.The processing of a very large set of LiDAR data is very costly and necessitates automatic 3D modeling approaches. In addition, incomplete point clouds caused by occlusion and uneven density and the uncertainties in the processing of LiDAR data make it difficult to automatic creation of semantically enriched 3D models. This research work aims at developing new solutions for the automatic creation of complete 3D geometric models with semantic labels from incomplete point clouds. A framework integrating knowledge about objects in urban scenes into 3D modeling is proposed for improving the completeness of 3D geometric models using qualitative reasoning based on semantic information of objects and their components, their geometric and spatial relations. Moreover, we aim at taking advantage of the qualitative knowledge of objects in automatic feature recognition and further in the creation of complete 3D geometric models from incomplete point clouds. To achieve this goal, several algorithms are proposed for automatic segmentation, the identification of the topological relations between object components, feature recognition and the creation of complete 3D geometric models. (1) Machine learning solutions have been proposed for automatic semantic segmentation and CAD-like segmentation to segment objects with complex structures. (2) We proposed an algorithm to efficiently identify topological relationships between object components extracted from point clouds to assemble a Boundary Representation model. (3) The integration of object knowledge and feature recognition has been developed to automatically obtain semantic labels of objects and their components. In order to deal with uncertain information, a rule-based automatic uncertain reasoning solution was developed to recognize building components from uncertain information extracted from point clouds. (4) A heuristic method for creating complete 3D geometric models was designed using building knowledge, geometric and topological relations of building components, and semantic information obtained from feature recognition. Finally, the proposed framework for improving automatic 3D modeling from point clouds of urban areas has been validated by a case study aimed at creating a complete 3D building model. Experiments demonstrate that the integration of knowledge into the steps of 3D modeling is effective in creating a complete building model from incomplete point clouds

Towards parameter-less 3D mesh segmentation

This thesis focuses on the 3D mesh segmentation process. The research demonstrated how the process can be done in a parameterless approach which allows full automation with accurate results. Applications of this research include, but not limited to, 3D search engines, 3D character animation, robotics environment recognition, and augmented reality