Extraction robuste de primitives géométriques 3D dans un nuage de points et alignement basé sur les primitives

Dans ce projet, nous étudions les problèmes de rétro-ingénierie et de contrôle de la qualité qui jouent un rôle important dans la fabrication industrielle. La rétro-ingénierie tente de reconstruire un modèle 3D à partir de nuages de points, qui s’apparente au problème de la reconstruction de la surface 3D. Le contrôle de la qualité est un processus dans lequel la qualité de tous les facteurs impliqués dans la production est abordée. En fait, les systèmes ci-dessus nécessitent beaucoup d’intervention de la part d’un utilisateur expérimenté, résultat souhaité est encore loin soit une automatisation complète du processus. Par conséquent, de nombreux défis doivent encore être abordés pour atteindre ce résultat hautement souhaitable en production automatisée. La première question abordée dans la thèse consiste à extraire les primitives géométriques 3D à partir de nuages de points. Un cadre complet pour extraire plusieurs types de primitives à partir de données 3D est proposé. En particulier, une nouvelle méthode de validation est proposée pour évaluer la qualité des primitives extraites. À la fin, toutes les primitives présentes dans le nuage de points sont extraites avec les points de données associés et leurs paramètres descriptifs. Ces résultats pourraient être utilisés dans diverses applications telles que la reconstruction de scènes on d’édifices, la géométrie constructive et etc. La seconde question traiée dans ce travail porte sur l’alignement de deux ensembles de données 3D à l’aide de primitives géométriques, qui sont considérées comme un nouveau descripteur robuste. L’idée d’utiliser les primitives pour l’alignement arrive à surmonter plusieurs défis rencontrés par les méthodes d’alignement existantes. Ce problème d’alignement est une étape essentielle dans la modélisation 3D, la mise en registre, la récupération de modèles. Enfin, nous proposons également une méthode automatique pour extraire les discontinutés à partir de données 3D d’objets manufacturés. En intégrant ces discontinutés au problème d’alignement, il est possible d’établir automatiquement les correspondances entre primitives en utilisant l’appariement de graphes relationnels avec attributs. Nous avons expérimenté tous les algorithmes proposés sur différents jeux de données synthétiques et réelles. Ces algorithmes ont non seulement réussi à accomplir leur tâches avec succès mais se sont aussi avérés supérieus aux méthodes proposées dans la literature. Les résultats présentés dans le thèse pourraient s’avérér utilises à plusieurs applications.In this research project, we address reverse engineering and quality control problems that play significant roles in industrial manufacturing. Reverse engineering attempts to rebuild a 3D model from the scanned data captured from a object, which is the problem similar to 3D surface reconstruction. Quality control is a process in which the quality of all factors involved in production is monitored and revised. In fact, the above systems currently require significant intervention from experienced users, and are thus still far from being fully automated. Therefore, many challenges still need to be addressed to achieve the desired performance for automated production. The first proposition of this thesis is to extract 3D geometric primitives from point clouds for reverse engineering and surface reconstruction. A complete framework to extract multiple types of primitives from 3D data is proposed. In particular, a novel validation method is also proposed to assess the quality of the extracted primitives. At the end, all primitives present in the point cloud are extracted with their associated data points and descriptive parameters. These results could be used in various applications such as scene and building reconstruction, constructive solid geometry, etc. The second proposition of the thesis is to align two 3D datasets using the extracted geometric primitives, which is introduced as a novel and robust descriptor. The idea of using primitives for alignment is addressed several challenges faced by existing registration methods. This alignment problem is an essential step in 3D modeling, registration and model retrieval. Finally, an automatic method to extract sharp features from 3D data of man-made objects is also proposed. By integrating the extracted sharp features into the alignment framework, it is possible implement automatic assignment of primitive correspondences using attribute relational graph matching. Each primitive is considered as a node of the graph and an attribute relational graph is created to provide a structural and relational description between primitives. We have experimented all the proposed algorithms on different synthetic and real scanned datasets. Our algorithms not only are successful in completing their tasks with good results but also outperform other methods. We believe that the contribution of them could be useful in many applications

Feature-based hybrid inspection planning for complex mechanical parts

Globalization and emerging new powers in the manufacturing world are among many challenges, major manufacturing enterprises are facing. This resulted in increased alternatives to satisfy customers\u27 growing needs regarding products\u27 aesthetic and functional requirements. Complexity of part design and engineering specifications to satisfy such needs often require a better use of advanced and more accurate tools to achieve good quality. Inspection is a crucial manufacturing function that should be further improved to cope with such challenges. Intelligent planning for inspection of parts with complex geometric shapes and free form surfaces using contact or non-contact devices is still a major challenge. Research in segmentation and localization techniques should also enable inspection systems to utilize modern measurement technologies capable of collecting huge number of measured points. Advanced digitization tools can be classified as contact or non-contact sensors. The purpose of this thesis is to develop a hybrid inspection planning system that benefits from the advantages of both techniques. Moreover, the minimization of deviation of measured part from the original CAD model is not the only characteristic that should be considered when implementing the localization process in order to accept or reject the part; geometric tolerances must also be considered. A segmentation technique that deals directly with the individual points is a necessary step in the developed inspection system, where the output is the actual measured points, not a tessellated model as commonly implemented by current segmentation tools. The contribution of this work is three folds. First, a knowledge-based system was developed for selecting the most suitable sensor using an inspection-specific features taxonomy in form of a 3D Matrix where each cell includes the corresponding knowledge rules and generate inspection tasks. A Travel Salesperson Problem (TSP) has been applied for sequencing these hybrid inspection tasks. A novel region-based segmentation algorithm was developed which deals directly with the measured point cloud and generates sub-point clouds, each of which represents a feature to be inspected and includes the original measured points. Finally, a new tolerance-based localization algorithm was developed to verify the functional requirements and was applied and tested using form tolerance specifications. This research enhances the existing inspection planning systems for complex mechanical parts with a hybrid inspection planning model. The main benefits of the developed segmentation and tolerance-based localization algorithms are the improvement of inspection decisions in order not to reject good parts that would have otherwise been rejected due to misleading results from currently available localization techniques. The better and more accurate inspection decisions achieved will lead to less scrap, which, in turn, will reduce the product cost and improve the company potential in the market

A framework for hull form reverse engineering and geometry integration into numerical simulations

The thesis presents a ship hull form specific reverse engineering and CAD integration framework. The reverse engineering part proposes three alternative suitable reconstruction approaches namely curves network, direct surface fitting, and triangulated surface reconstruction. The CAD integration part includes surface healing, region identification, and domain preparation strategies which used to adapt the CAD model to downstream application requirements. In general, the developed framework bridges a point cloud and a CAD model obtained from IGES and STL file into downstream applications

Segmentation And Spatial Depth Ridge Detection Of Unorganized Point Cloud Data

Visual 3D data are of interest to a number of fields: medical professionals, game designers, graphic designers, and (in the interest of this paper) ichthyologists interested in the taxonomy of fish. Since the release of the Kinect for the Microsoft XBox, game designers have been interested in using the 3D data returned by the device to understand human movement and translate that movement into an interface with which to interact with game systems. In the medical field, researchers must use computer vision tools to navigate through the data found in CT scans and MRI scans. These tools must segment images into the parts that are relevant to researchers and account for noise related to the scanning process all while ignoring other types of noise such as foreign elements in the body that might indicate signs of illness. 3D point cloud data represents some unique challenges. Consider an object scanned with a laser scanner. The scanner returns the surface points of the object, but nothing more. Using the tool Qhull, a researcher can quickly compute the convex hull of an object (which is an interesting challenge in itself), but the convex hull (obviously) leaves out any description of an object\u27s concave features. Several algorithms have been proposed to illustrate an object\u27s complete features based on unorganized 3D point cloud data as accurately as possible, most notably Boissonnat\u27s tetrahedral culling algorithm and The Power Crust algorithm. We introduce a new approach to the area partitioning problem that takes into consideration these algorithms\u27 strengths and weaknesses. In this paper we propose a methodology for approximating a shape\u27s solid geometry using the unorganized 3D point cloud data of that shape primarily by utilizing localized principal component analysis information. Our model accounts for three comissues that arise in the scanning of 3D objects: noise in surface points, poorly sampled surface area, and narrow corners. We explore each of these areas of concern and outline our approach to each. Our technique uses a growing algorithm that labels points as it progresses and uses those labels with a simple priority queue. We found that our approach works especially well for approximating surfaces under the condition where a local surface is poorly sampled (i.e a significant hole is present in the point cloud). We then turn to study the medial axis of a shape for the purposes of `unfolding\u27 that structure. Our approach uses a ridge formulation based on the spatial depth statistic to create the medial axis. We conclude the paper with visual results of our technique

Cylinders extraction in non-oriented point clouds as a clustering problem

Finding geometric primitives in 3D point clouds is a fundamental task in many engineering applications such as robotics, autonomous-vehicles and automated industrial inspection. Among all solid shapes, cylinders are frequently found in a variety of scenes, comprising natural or man-made objects. Despite their ubiquitous presence, automated extraction and fitting can become challenging if performed ”in-the-wild”, when the number of primitives is unknown or the point cloud is noisy and not oriented. In this paper we pose the problem of extracting multiple cylinders in a scene by means of a Game-Theoretic inlier selection process exploiting the geometrical relations between pairs of axis candidates. First, we formulate the similarity between two possible cylinders considering the rigid motion aligning the two axes to the same line. This motion is represented with a unitary dual-quaternion so that the distance between two cylinders is induced by the length of the shortest geodesic path in SE(3). Then, a Game-Theoretical process exploits such similarity function to extract sets of primitives maximizing their inner mutual consensus. The outcome of the evolutionary process consists in a probability distribution over the sets of candidates (ie axes), which in turn is used to directly estimate the final cylinder parameters. An extensive experimental section shows that the proposed algorithm offers a high resilience to noise, since the process inherently discards inconsistent data. Compared to other methods, it does not need point normals and does not require a fine tuning of multiple parameters

State-of-The-Art and Applications of 3D Imaging Sensors in Industry, Cultural Heritage, Medicine, and Criminal Investigation

3D imaging sensors for the acquisition of three dimensional (3D) shapes have created, in recent years, a considerable degree of interest for a number of applications. The miniaturization and integration of the optical and electronic components used to build them have played a crucial role in the achievement of compactness, robustness and flexibility of the sensors. Today, several 3D sensors are available on the market, even in combination with other sensors in a “sensor fusion” approach. An importance equal to that of physical miniaturization has the portability of the measurements, via suitable interfaces, into software environments designed for their elaboration, e.g., CAD-CAM systems, virtual renders, and rapid prototyping tools. In this paper, following an overview of the state-of-art of 3D imaging sensors, a number of significant examples of their use are presented, with particular reference to industry, heritage, medicine, and criminal investigation applications

Automatic tolerance inspection through Reverse Engineering: a segmentation technique for plastic injection moulded parts

This work studies segmentations procedures to recognise features in a Reverse Engineering (RE) application that is oriented to computer-aided tolerance inspection of injection moulding die set-up, necessary to manufacture electromechanical components. It will discuss all steps of the procedures, from the initial acquisition to the final measure data management, but specific original developments will be focused on the RE post-processing method, that should solve the problem related to the automation of the surface recognition and then of the inspection process. As it will be explained in the first two Chapters, automation of the inspection process pertains, eminently, to feature recognition after the segmentation process. This work presents a voxel-based approach with the aim of reducing the computation efforts related to tessellation and curvature analysis, with or without filtering. In fact, a voxel structure approximates the shape through parallelepipeds that include small sub-set of points. In this sense, it represents a filter, since the number of voxels is less than the total number of points, but also a local approximation of the surface, if proper fitting models are applied. Through sensitivity analysis and industrial applications, limits and perspectives of the proposed algorithms are discussed and validated in terms of accuracy and save of time. Validation case-studies are taken from real applications made in ABB Sace S.p.A., that promoted this research. Plastic injection moulding of electromechanical components has a time-consuming die set-up. It is due to the necessity of providing dies with many cavities, which during the cooling phase may present different stamping conditions, thus defects that include lengths outside their dimensional tolerance, and geometrical errors. To increase the industrial efficiency, the automation of the inspection is not only due to the automatic recognition of features but also to a computer-aided inspection protocol (path planning and inspection data management). For this reason, also these steps will be faced, as the natural framework of the thesis research activity. The work structure concerns with six chapters. In Chapter 1, an introduction to the whole procedure is presented, focusing on reasons and utilities of the application of RE techniques in industrial engineering. Chapter 2 analyses acquisition issues and methods that are related to our application, describing: (a) selected hardware; (b) adopted strategy related to the cloud of point acquisition. In Chapter 3, the proposed RE post-processing is described together with a state of art about data segmentation and surface reconstruction. Chapter 4 discusses the proposed algorithms through sensitivity studies concerning thresholds and parameters utilised in segmentation phase and surface reconstruction. Chapter 5 explains briefly the inspection workflow, PDM requirements and solution, together with a preliminary assessing of measures and their reliability. These three chapters (3, 4 and 5) report final sections, called “Discussion”, in which specific considerations are given. Finally, Chapter 6 gives examples of the proposed segmentation technique in the framework of the industrial applications, through specific case studies

Repair of metallic components using hybrid manufacturing

Many high-performance metal parts users extend the service of these damaged parts by employing repair technology. Hybrid manufacturing, which includes additive manufacturing (AM) and subtractive manufacturing, provides greater build capability, better accuracy, and surface finish for component repair. However, most repair processes still rely on manual operations, which are not satisfactory in terms of time, cost, reliability, and accuracy. This dissertation aims to improve the application of hybrid manufacturing for repairing metallic components by addressing the following three research topics. The first research topic is to investigate and develop an efficient best-fit and shape adaption algorithm for automating 3D models\u27 the alignment and defect reconstruction. A multi-feature fitting algorithm and cross-section comparison method are developed. The second research topic is to develop a smooth toolpath generation method for laser metal deposition to improve the deposition quality for metallic component fabrication and repair. Smooth connections or transitions in toolpath planning are achieved to provide a constant feedrate and controllable deposition idle time for each single deposition pass. The third research topic is to develop an automated repair process could efficiently obtain the spatial information of a worn component for defect detection, alignment, and 3D scanning with the integration of stereo vision and laser displacement sensor. This dissertation investigated and developed key technologies to improve the efficiency, repair quality, precision, and automation for the repair of metallic components using hybrid manufacturing. Moreover, the research results of this dissertation can benefit a wide range of industries, such as additive manufacturing, manufacturing and measurement automation, and part inspection --Abstract, page iv

Point clouds and thermal data fusion for automated gbXML-based building geometry model generation

Existing residential and small commercial buildings now represent the greatest opportunity to improve building energy efficiency. Building energy simulation analysis is becoming increasingly important because the analysis results can assist the decision makers to make decisions on improving building energy efficiency and reducing environmental impacts. However, manually measuring as-is conditions of building envelops including geometry and thermal value is still a labor-intensive, costly, and slow process. Thus, the primary objective of this research was to automatically collect and extract the as-is geometry and thermal data of the building envelope components and create a gbXML-based building geometry model. In the proposed methodology, a rapid and low-cost data collection hardware system was designed by integrating 3D laser scanners and an infrared (IR) camera. Secondly, several algorithms were created to automatically recognize various components of building envelope as objects from collected raw data. The extracted 3D semantic geometric model was then automatically saved as an industry standard file format for data interoperability. The feasibility of the proposed method was validated through three case studies. The contributions of this research include 1) a customized low-cost hybrid data collection system development to fuse various data into a thermal point cloud; 2) an automatic method of extracting building envelope components and its geometry data to generate gbXML-based building geometry model. The broader impacts of this research are that it could offer a new way to collect as is building data without impeding occupants’ daily life, and provide an easier way for laypeople to understand the energy performance of their buildings via 3D thermal point cloud visualization.Ph.D