WELDMAP: A Photogrammetric Suite Applied to the Inspection of Welds

[EN] This paper presents a new tool for external quality control in welds using close-range photogrammetry. The main contribution of the developed approach is the automatic assessment of welds based on 3D photogrammetric models, enabling objective and accurate analyses through an in-house tool that was developed, WELDMAP. As a result, inspectors can perform external quality control of welds in a simple and efficient way without requiring visual inspections or external tools, and thus avoiding the subjectivity and imprecisions of the classical protocol. The tool was validated with a large dataset in laboratory tests as well as in real scenarios.SIMinistry of Science and Innovation, Government of Spai

Weld bead detection based on 3D geometric features and machine learning approaches

14 p.Weld bead detection is essential for automated welding inspection processes. The non-invasive passive techniques, such as photogrammetry, are quickly evolving to provide a 3D point cloud with submillimeter precision and spatial resolution. However, its application in weld visual inspection has not been extensively studied. The derived 3D point clouds, despite the lack of topological information, store significant information for the weld-plaque segmentation. Although the weld bead detection is being carried out over images or based on laser profiles, its characterization by means of 3D geometrical features has not been assessed. Moreover, it is possible to combine machine learning approaches and the 3D features in order to realize the full potential of the weld bead segmentation of 3D submillimeter point clouds. In this paper, the novelty is focused on the study of 3D features on real cases to identify the most relevant ones for weld bead detection on the basis of the information gain. For this novel contribution, the influence of neighborhood size for covariance matrix computation, decision tree algorithms, and split criteria are analyzed to assess the optimal results. The classification accuracy is evaluated by the degree of agreement of the classified data by the kappa index and the area under the receiver operating characteristic (ROC) curve. The experimental results show that the proposed novel methodology performs better than 0.85 for the kappa index and better than 0.95 for ROC area.S

Development of a real-time ultrasonic sensing system for automated and robotic welding

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.The implementation of robotic technology into welding processes is made difficult by the inherent process variables of part location, fit up, orientation and repeatability. Considering these aspects, to ensure weld reproducibility consistency and quality, advanced adaptive control techniques are essential. These involve not only the development of adequate sensors for seam tracking and joint recognition but also developments of overall machines with a level of artificial intelligence sufficient for automated welding. The development of such a prototype system which utilizes a manipulator arm, ultrasonic sensors and a transistorised welding power source is outlined. This system incorporates three essential aspects. It locates and tracks the welding seam ensuring correct positioning of the welding head relatively to the joint preparation. Additionally, it monitors the joint profile of the molten weld pool and modifies the relevant heat input parameters ensuring consistent penetration, joint filling and acceptable weld bead shape. Finally, it makes use of both the above information to reconstruct three-dimensional images of the weld pool silhouettes providing in-process inspection capabilities of the welded joints. Welding process control strategies have been incorporated into the system based on quantitative relationships between input parameters and weld bead shape configuration allowing real-time decisions to be made during the process of welding, without the need for operation intervention.British Technology Group (BTG

Machine-human Cooperative Control of Welding Process

An innovative auxiliary control system is developed to cooperate with an unskilled welder in a manual GTAW in order to obtain a consistent welding performance. In the proposed system, a novel mobile sensing system is developed to non-intrusively monitor a manual GTAW by measuring three-dimensional (3D) weld pool surface. Specifically, a miniature structured-light laser amounted on torch projects a dot matrix pattern on weld pool surface during the process; Reflected by the weld pool surface, the laser pattern is intercepted by and imaged on the helmet glass, and recorded by a compact camera on it. Deformed reflection pattern contains the geometry information of weld pool, thus is utilized to reconstruct its $3$D surface. An innovative image processing algorithm and a reconstruction scheme have been developed for (3D) reconstruction. The real-time spatial relations of the torch and the helmet is formulated during welding. Two miniature wireless inertial measurement units (WIMU) are mounted on the torch and the helmet, respectively, to detect their rotation rates and accelerations. A quaternion based unscented Kalman filter (UKF) has been designed to estimate the helmet/torch orientations based on the data from the WIMUs. The distance between the torch and the helmet is measured using an extra structure-light low power laser pattern. Furthermore, human welder\u27s behavior in welding performance has been studied, e.g., a welder`s adjustments on welding current were modeled as response to characteristic parameters of the three-dimensional weld pool surface. This response model as a controller is implemented both automatic and manual gas tungsten arc welding process to maintain a consistent full penetration

A Tutorial on Learning Human Welder\u27s Behavior: Sensing, Modeling, and Control

Human welder\u27s experiences and skills are critical for producing quality welds in manual GTAW process. Learning human welder\u27s behavior can help develop next generation intelligent welding machines and train welders faster. In this tutorial paper, various aspects of mechanizing the welder\u27s intelligence are surveyed, including sensing of the weld pool, modeling of the welder\u27s adjustments and this model-based control approach. Specifically, different sensing methods of the weld pool are reviewed and a novel 3D vision-based sensing system developed at University of Kentucky is introduced. Characterization of the weld pool is performed and human intelligent model is constructed, including an extensive survey on modeling human dynamics and neuro-fuzzy techniques. Closed-loop control experiment results are presented to illustrate the robustness of the model-based intelligent controller despite welding speed disturbance. A foundation is thus established to explore the mechanism and transformation of human welder\u27s intelligence into robotic welding system. Finally future research directions in this field are presented

A Laser-Based Vision System for Weld Quality Inspection

Welding is a very complex process in which the final weld quality can be affected by many process parameters. In order to inspect the weld quality and detect the presence of various weld defects, different methods and systems are studied and developed. In this paper, a laser-based vision system is developed for non-destructive weld quality inspection. The vision sensor is designed based on the principle of laser triangulation. By processing the images acquired from the vision sensor, the geometrical features of the weld can be obtained. Through the visual analysis of the acquired 3D profiles of the weld, the presences as well as the positions and sizes of the weld defects can be accurately identified and therefore, the non-destructive weld quality inspection can be achieved

Process control for WAAM using computer vision

This study is mainly about the vision system and control algorithm programming for wire arc additive manufacturing (WAAM). Arc additive manufacturing technology is formed by the principle of heat source cladding produced by welders using molten inert gas shielded welding (MIG), tungsten inert gas shielded welding (TIG) and layered plasma welding power supply (PA). It has high deposition efficiency, short manufacturing cycle, low cost, and easy maintenance. Although WAAM has very good uses in various fields, the inability to control the adding process in real time has led to defects in the weld and reduced quality. Therefore, it is necessary to develop the real-time feedback through computer vision and algorithms for WAAM to ensure that the thickness and the width of each layer during the addition process are the same

Desarrollo de geotecnologías aplicadas a la inspección y monitorización de entornos industriales

Tesis por compendio de publicaciones[ES]El desarrollo tecnológico de las últimas dos décadas ha supuesto un cambio radical que está llevando a un nuevo paradigma en el que se entremezclan el mundo físico y el digital. Estos cambios han influido enormemente en la sociedad, modificando las formas de comunicación, acceso a información, ocio, trabajo, etc. Asimismo, la industria ha adoptado estas tecnologías disruptivas, las cuales están contribuyendo a lograr un mayor control y automatización del proceso productivo. En el ámbito industrial, las tareas de mantenimiento son críticas para garantizar el correcto funcionamiento de una planta o instalación, ya que influyen directamente en la productividad y pueden suponer un elevado costo adicional. Las nuevas tecnologías están posibilitando la monitorización continua y a la inspección automatizada, proporcionando herramientas auxiliares a los inspectores que mejoran la detección de fallos y permiten anticipar y optimizar la planificación de las tareas de mantenimiento. Con el objetivo de desarrollar herramientas que aporten mejoras en las tareas de mantenimiento en industria, la presente tesis doctoral se basa en el estudio de como las geotecnologías pueden aportar soluciones óptimas en la monitorización e inspección. Debido a la gran variedad de entornos industriales, las herramientas de apoyo al mantenimiento deben adaptarse a cada caso en concreto. En este aspecto, y con el fin de demostrar la adaptabilidad de la geomática y las geotecnologías, se han estudiado instalaciones industriales de ámbitos muy diversos, como una sala de máquinas (escenario interior), plantas fotovoltaicas (escenario exterior) y soldaduras (interior y exterior). La escala de los escenarios objeto de estudio ha sido muy variada, desde las escalas más pequeñas, para el estudio de las soldaduras y la sala de máquinas, a las escalas más grandes, en los estudios de evolución de la vegetación y presencia de masas de agua en plantas fotovoltaicas. Las geotecnologías demuestran su versatilidad para trabajar a distintas escalas, con soluciones que permiten un gran detalle y precisión, como la fotogrametría de rango cercano y el sistema de escaneado portátil (Portable Mobile Mapping System - PMMS), y otras que pueden abarcar zonas más amplias del territorio, como es el caso de la teledetección o la fotogrametría con drones. Según lo expuesto anteriormente, el enfoque de la tesis ha sido el estudio de elementos o instalaciones industriales a diferentes escalas. En el primer caso se desarrolló una herramienta para el control de calidad externo de soldaduras utilizando fotogrametría de rango cercano y algoritmos para la detección automática de defectos. En el segundo caso se propuso el uso de un PMMS para optimizar la toma de datos en las tareas de inspección en instalaciones fluidomecánicas. En el tercer caso se utilizó la fotogrametría con drones y la combinación de imágenes RGB y térmicas con algoritmos de visión computacional para la detección de patologías en paneles fotovoltaicos. Finalmente, para la monitorización de la vegetación y la detección de masas de agua en el entorno de plantas fotovoltaicas, se empleó la teledetección mediante el cálculo de índices espectrales. [EN]The technological development of the last two decades has brought about a radical change that is leading to a new paradigm in which the physical and digital worlds are intertwined. These changes have had a great impact on society, modifying communication methods, access to information, leisure, work, etc. In addition, the industry has adopted these disruptive technologies, which are contributing to achieving greater control and automation of the production process. In the industrial sector, maintenance tasks are critical to ensuring the proper operation of a plant or facility, as they directly influence productivity and can involve high additional costs. New technologies are making continuous monitoring and automated inspection possible, providing auxiliary tools to inspectors that improve fault detection and allow for the anticipation and optimization of maintenance task planning. With the aim of developing tools that provide improvements in maintenance tasks in industry, this doctoral thesis is based on the study of how geotechnologies can provide optimal solutions in monitoring and inspection. Due to the great variety of industrial environments, maintenance support tools must adapt to each specific case. In this regard, and in order to demonstrate the adaptability of geomatics and geotechnologies, industrial installations from very diverse areas have been studied, such as a machine room (indoor scenario), photovoltaic plants (outdoor scenario), and welding (indoor and outdoor scenarios). The scale of the studied scenarios has been very varied, ranging from smaller scales for the study of welds and machine rooms, to larger scales in the studies of vegetation evolution and the presence of bodies of water in photovoltaic plants. Geotechnologies demonstrate their versatility to work at different scales, with solutions that allow for great detail and precision, such as close-range photogrammetry and the Portable Mobile Mapping System (PMMS), as well as others that can cover larger areas of the territory, such as remote sensing or photogrammetry with drones. The focus of the thesis has been the study of industrial elements or installations at different scales. In the first case, a tool was developed for external quality control of welding, using close-range photogrammetry and algorithms for automatic defect detection. In the second case, the use of a PMMS is proposed to optimize data collection in fluid-mechanical installation inspection tasks. In the third case, drone photogrammetry and the combination of RGB and thermal images with computer vision algorithms were used for the detection of pathologies in photovoltaic panels. Finally, for the monitoring of vegetation and the detection of water masses in the environment of photovoltaic plants, remote sensing was employed through the calculation of spectral indices