Validation of Portable Mobile Mapping System for inspection tasks in thermal and fluid-mechanical facilities

20 p.The three-dimensional registration of industrial facilities has a great importance for maintenance, inspection, and safety tasks and it is a starting point for new improvements and expansions in the industrial facilities context. In this paper, a comparison between the results obtained using a novel portable mobile mapping system (PMMS) and a static terrestrial laser scanner (TLS), widely used for 3D reconstruction in civil and industrial scenarios, is carried out. This comparison is performed in the context of industrial inspection tasks, specifically in the thermal and fluid-mechanics facilities in a hospital. The comparison addresses the general reconstruction of a machine room, focusing on the quantitative and qualitative analysis of different elements (e.g., valves, regulation systems, burner systems and tanks, etc.). The validation of the PMMS is provided considering the TLS as ground truth and applying a robust statistical analysis. Results come to confirm the suitability of the PMMS to perform inspection tasks in industrial facilities.S

Extraction robuste de primitives géométriques 3D dans un nuage de points et alignement basé sur les primitives

Dans ce projet, nous étudions les problèmes de rétro-ingénierie et de contrôle de la qualité qui jouent un rôle important dans la fabrication industrielle. La rétro-ingénierie tente de reconstruire un modèle 3D à partir de nuages de points, qui s’apparente au problème de la reconstruction de la surface 3D. Le contrôle de la qualité est un processus dans lequel la qualité de tous les facteurs impliqués dans la production est abordée. En fait, les systèmes ci-dessus nécessitent beaucoup d’intervention de la part d’un utilisateur expérimenté, résultat souhaité est encore loin soit une automatisation complète du processus. Par conséquent, de nombreux défis doivent encore être abordés pour atteindre ce résultat hautement souhaitable en production automatisée. La première question abordée dans la thèse consiste à extraire les primitives géométriques 3D à partir de nuages de points. Un cadre complet pour extraire plusieurs types de primitives à partir de données 3D est proposé. En particulier, une nouvelle méthode de validation est proposée pour évaluer la qualité des primitives extraites. À la fin, toutes les primitives présentes dans le nuage de points sont extraites avec les points de données associés et leurs paramètres descriptifs. Ces résultats pourraient être utilisés dans diverses applications telles que la reconstruction de scènes on d’édifices, la géométrie constructive et etc. La seconde question traiée dans ce travail porte sur l’alignement de deux ensembles de données 3D à l’aide de primitives géométriques, qui sont considérées comme un nouveau descripteur robuste. L’idée d’utiliser les primitives pour l’alignement arrive à surmonter plusieurs défis rencontrés par les méthodes d’alignement existantes. Ce problème d’alignement est une étape essentielle dans la modélisation 3D, la mise en registre, la récupération de modèles. Enfin, nous proposons également une méthode automatique pour extraire les discontinutés à partir de données 3D d’objets manufacturés. En intégrant ces discontinutés au problème d’alignement, il est possible d’établir automatiquement les correspondances entre primitives en utilisant l’appariement de graphes relationnels avec attributs. Nous avons expérimenté tous les algorithmes proposés sur différents jeux de données synthétiques et réelles. Ces algorithmes ont non seulement réussi à accomplir leur tâches avec succès mais se sont aussi avérés supérieus aux méthodes proposées dans la literature. Les résultats présentés dans le thèse pourraient s’avérér utilises à plusieurs applications.In this research project, we address reverse engineering and quality control problems that play significant roles in industrial manufacturing. Reverse engineering attempts to rebuild a 3D model from the scanned data captured from a object, which is the problem similar to 3D surface reconstruction. Quality control is a process in which the quality of all factors involved in production is monitored and revised. In fact, the above systems currently require significant intervention from experienced users, and are thus still far from being fully automated. Therefore, many challenges still need to be addressed to achieve the desired performance for automated production. The first proposition of this thesis is to extract 3D geometric primitives from point clouds for reverse engineering and surface reconstruction. A complete framework to extract multiple types of primitives from 3D data is proposed. In particular, a novel validation method is also proposed to assess the quality of the extracted primitives. At the end, all primitives present in the point cloud are extracted with their associated data points and descriptive parameters. These results could be used in various applications such as scene and building reconstruction, constructive solid geometry, etc. The second proposition of the thesis is to align two 3D datasets using the extracted geometric primitives, which is introduced as a novel and robust descriptor. The idea of using primitives for alignment is addressed several challenges faced by existing registration methods. This alignment problem is an essential step in 3D modeling, registration and model retrieval. Finally, an automatic method to extract sharp features from 3D data of man-made objects is also proposed. By integrating the extracted sharp features into the alignment framework, it is possible implement automatic assignment of primitive correspondences using attribute relational graph matching. Each primitive is considered as a node of the graph and an attribute relational graph is created to provide a structural and relational description between primitives. We have experimented all the proposed algorithms on different synthetic and real scanned datasets. Our algorithms not only are successful in completing their tasks with good results but also outperform other methods. We believe that the contribution of them could be useful in many applications

Automatic tolerance inspection through Reverse Engineering: a segmentation technique for plastic injection moulded parts

This work studies segmentations procedures to recognise features in a Reverse Engineering (RE) application that is oriented to computer-aided tolerance inspection of injection moulding die set-up, necessary to manufacture electromechanical components. It will discuss all steps of the procedures, from the initial acquisition to the final measure data management, but specific original developments will be focused on the RE post-processing method, that should solve the problem related to the automation of the surface recognition and then of the inspection process. As it will be explained in the first two Chapters, automation of the inspection process pertains, eminently, to feature recognition after the segmentation process. This work presents a voxel-based approach with the aim of reducing the computation efforts related to tessellation and curvature analysis, with or without filtering. In fact, a voxel structure approximates the shape through parallelepipeds that include small sub-set of points. In this sense, it represents a filter, since the number of voxels is less than the total number of points, but also a local approximation of the surface, if proper fitting models are applied. Through sensitivity analysis and industrial applications, limits and perspectives of the proposed algorithms are discussed and validated in terms of accuracy and save of time. Validation case-studies are taken from real applications made in ABB Sace S.p.A., that promoted this research. Plastic injection moulding of electromechanical components has a time-consuming die set-up. It is due to the necessity of providing dies with many cavities, which during the cooling phase may present different stamping conditions, thus defects that include lengths outside their dimensional tolerance, and geometrical errors. To increase the industrial efficiency, the automation of the inspection is not only due to the automatic recognition of features but also to a computer-aided inspection protocol (path planning and inspection data management). For this reason, also these steps will be faced, as the natural framework of the thesis research activity. The work structure concerns with six chapters. In Chapter 1, an introduction to the whole procedure is presented, focusing on reasons and utilities of the application of RE techniques in industrial engineering. Chapter 2 analyses acquisition issues and methods that are related to our application, describing: (a) selected hardware; (b) adopted strategy related to the cloud of point acquisition. In Chapter 3, the proposed RE post-processing is described together with a state of art about data segmentation and surface reconstruction. Chapter 4 discusses the proposed algorithms through sensitivity studies concerning thresholds and parameters utilised in segmentation phase and surface reconstruction. Chapter 5 explains briefly the inspection workflow, PDM requirements and solution, together with a preliminary assessing of measures and their reliability. These three chapters (3, 4 and 5) report final sections, called “Discussion”, in which specific considerations are given. Finally, Chapter 6 gives examples of the proposed segmentation technique in the framework of the industrial applications, through specific case studies

Recognition of one class of quadrics from 3D point clouds

Within cyber physical production systems 3D vision as a source of information from real-world provides enormous possibilities. While the hardware of contemporary 3D scanners is characterized by high speed along with high resolution and accuracy, there is a lack of real-time online data processing algorithms that would give certain elements of intelligence to the sensory system. Critical elements of data processing software are efficient, real-time applicable methods for fully automatic recognition of high level geometric primitives from point cloud (surface segmentation and fitting). This paper presents a method for recognition of one class of quadrics from 3D point clouds, in particular for recognition of cylinders, elliptical cylinders and ellipsoids. The method is based on the properties of scatter matrix during direct least squares fitting of ellipsoids. Presented recognition procedure can be employed for segmentation of regions with G1 or higher continuity, and this is its comparative advantage to similar methods. The applicability of the method is illustrated and experimentally verified using two case studies. First case study refers to a synthesized, and the second to a real-world scanned point cloud

Recognition of one class of quadrics from 3D point clouds

Within cyber physical production systems 3D vision as a source of information from real-world provides enormous possibilities. While the hardware of contemporary 3D scanners is characterized by high speed along with high resolution and accuracy, there is a lack of real-time online data processing algorithms that would give certain elements of intelligence to the sensory system. Critical elements of data processing software are efficient, real-time applicable methods for fully automatic recognition of high level geometric primitives from point cloud (surface segmentation and fitting). This paper presents a method for recognition of one class of quadrics from 3D point clouds, in particular for recognition of cylinders, elliptical cylinders and ellipsoids. The method is based on the properties of scatter matrix during direct least squares fitting of ellipsoids. Presented recognition procedure can be employed for segmentation of regions with G1 or higher continuity, and this is its comparative advantage to similar methods. The applicability of the method is illustrated and experimentally verified using two case studies. First case study refers to a synthesized, and the second to a real-world scanned point cloud

Progress in industrial photogrammetry by means of markerless solutions

174 p.La siguiente tesis está enfocada al desarrollo y uso avanzado de metodologías fotogramétrica sin dianas en aplicaciones industriales. La fotogrametría es una técnica de medición óptica 3D que engloba múltiples configuraciones y aproximaciones. En este estudio se han desarrollado procedimientos de medición, modelos y estrategias de procesamiento de imagen que van más allá que la fotogrametría convencional y buscan el emplear soluciones de otros campos de la visión artificial en aplicaciones industriales. Mientras que la fotogrametría industrial requiere emplear dianas artificiales para definir los puntos o elementos de interés, esta tesis contempla la reducción e incluso la eliminación de las dianas tanto pasivas como activas como alternativas prácticas. La mayoría de los sistemas de medida utilizan las dianas tanto para definir los puntos de control, relacionar las distintas perspectivas, obtener precisión, así como para automatizar las medidas. Aunque en muchas situaciones el empleo de dianas no sea restrictivo existen aplicaciones industriales donde su empleo condiciona y restringe considerablemente los procedimientos de medida empleados en la inspección. Un claro ejemplo es la verificación y control de calidad de piezas seriadas, o la medición y seguimiento de elementos prismáticos relacionados con un sistema de referencia determinado. Es en este punto donde la fotogrametría sin dianas puede combinarse o complementarse con soluciones tradicionales para tratar de mejorar las prestaciones actuales

Feature-based hybrid inspection planning for complex mechanical parts

Globalization and emerging new powers in the manufacturing world are among many challenges, major manufacturing enterprises are facing. This resulted in increased alternatives to satisfy customers\u27 growing needs regarding products\u27 aesthetic and functional requirements. Complexity of part design and engineering specifications to satisfy such needs often require a better use of advanced and more accurate tools to achieve good quality. Inspection is a crucial manufacturing function that should be further improved to cope with such challenges. Intelligent planning for inspection of parts with complex geometric shapes and free form surfaces using contact or non-contact devices is still a major challenge. Research in segmentation and localization techniques should also enable inspection systems to utilize modern measurement technologies capable of collecting huge number of measured points. Advanced digitization tools can be classified as contact or non-contact sensors. The purpose of this thesis is to develop a hybrid inspection planning system that benefits from the advantages of both techniques. Moreover, the minimization of deviation of measured part from the original CAD model is not the only characteristic that should be considered when implementing the localization process in order to accept or reject the part; geometric tolerances must also be considered. A segmentation technique that deals directly with the individual points is a necessary step in the developed inspection system, where the output is the actual measured points, not a tessellated model as commonly implemented by current segmentation tools. The contribution of this work is three folds. First, a knowledge-based system was developed for selecting the most suitable sensor using an inspection-specific features taxonomy in form of a 3D Matrix where each cell includes the corresponding knowledge rules and generate inspection tasks. A Travel Salesperson Problem (TSP) has been applied for sequencing these hybrid inspection tasks. A novel region-based segmentation algorithm was developed which deals directly with the measured point cloud and generates sub-point clouds, each of which represents a feature to be inspected and includes the original measured points. Finally, a new tolerance-based localization algorithm was developed to verify the functional requirements and was applied and tested using form tolerance specifications. This research enhances the existing inspection planning systems for complex mechanical parts with a hybrid inspection planning model. The main benefits of the developed segmentation and tolerance-based localization algorithms are the improvement of inspection decisions in order not to reject good parts that would have otherwise been rejected due to misleading results from currently available localization techniques. The better and more accurate inspection decisions achieved will lead to less scrap, which, in turn, will reduce the product cost and improve the company potential in the market

CLOI-NET: Class segmentation of industrial facilities' point cloud datasets

Shape segmentation from point cloud data is a core step of the digital twinning process for industrial facilities. However, it is also a very labor intensive step, which counteracts the perceived value of the resulting model. The state-of-the-art method for automating cylinder detection can detect cylinders with 62% precision and 70% recall, while other shapes must then be segmented manually and shape segmentation is not achieved. This performance is promising, but it is far from drastically eliminating the manual labor cost. We argue that the use of class segmentation deep learning algorithms has the theoretical potential to perform better in terms of per point accuracy and less manual segmentation time needed. However, such algorithms could not be used so far due to the lack of a pre-trained dataset of laser scanned industrial shapes as well as the lack of appropriate geometric features in order to learn these shapes. In this paper, we tackle both problems in three steps. First, we parse the industrial point cloud through a novel class segmentation solution (CLOI-NET) that consists of an optimized PointNET++ based deep learning network and post-processing algorithms that enforce stronger contextual relationships per point. We then allow the user to choose the optimal manual annotation of a test facility by means of active learning to further improve the results. We achieve the first step by clustering points in meaningful spatial 3D windows based on their location. Then, we apply a class segmentation deep network, and output a probability distribution of all label categories per point and improve the predicted labels by enforcing post-processing rules. We finally optimize the results by finding the optimal amount of data to be used for training experiments. We validate our method on the largest richly annotated dataset of the most important to model industrial shapes (CLOI) and yield 82% average accuracy per point, 95.6% average AUC among all classes and estimated 70% labor hour savings in class segmentation. This proves that it is the first to automatically segment industrial point cloud shapes with no prior knowledge at commercially viable performance and is the foundation for efficient industrial shape modeling in cluttered point clouds