Data Analysis and Modeling Techniques of Welding Processes: The State-of-the-Art

Information contributes to the improvement of decision-making, process improvement, error detection, and prevention. The new requirements of the coming Industry 4.0 will make these new information technologies help in the improvement and decision-making of industrial processes. In case of the welding processes, several techniques have been used. Welding processes can be analyzed as a stochastic system with several inputs and outputs. This allows a study with a data analysis perspective. Data mining processes, machine learning, deep learning, and reinforcement learning techniques have had good results in the analysis and control of systems as complex as the welding process. The increase of information acquisition and information quality by sensors developed at present, allows a large volume of data that benefits the analysis of these techniques. This research aims to make a bibliographic analysis of the techniques used in the welding area, the advantages that these new techniques can provide, and how some researchers are already using them. The chapter is organized according to some stages of the data mining process. This was defined with the objective of highlighting evolution and potential for each stage for welding processes

Visual Sensing and Defect Detection of Gas Tungsten Arc Welding

Weld imperfections or defects such as incomplete penetration and lack of fusion are critical issues that affect the integration of welding components. The molten weld pool geometry is the major source of information related to the formation of these defects. In this dissertation, a new visual sensing system has been designed and set up to obtain weld pool images during GTAW. The weld pool dynamical behavior can be monitored using both active and passive vision method with the interference of arc light in the image significantly reduced through the narrow band pass filter and laser based auxiliary light source.Computer vision algorithms based on passive vision images were developed to measure the 3D weld pool surface geometry in real time. Specifically, a new method based on the reversed electrode image (REI) was developed to calculate weld pool surface height in real time. Meanwhile, the 2D weld pool boundary was extracted with landmarks detection algorithms. The method was verified with bead-on-plate and butt-joint welding experiments.Supervised machine learning was used to develop the capability to predict, in real-time, the incomplete penetration on thin SS304 plate with the key features extracted from weld pool images. An integrated self-adaptive close loop control system consisting the non-contact visual sensor, machine learning based defect predictor, and welding power source was developed for real-time welding penetration control for bead on plate welding. Moreover, the data driven methods were first applied to detect incomplete penetration and LOF in multi-pass U groove welding. New features extracted from reversed electrode image played the most important role to predict these defects. Finally, real time welding experiments were conducted to verify the feasibility of the developed models

WELD PENETRATION IDENTIFICATION BASED ON CONVOLUTIONAL NEURAL NETWORK

Weld joint penetration determination is the key factor in welding process control area. Not only has it directly affected the weld joint mechanical properties, like fatigue for example. It also requires much of human intelligence, which either complex modeling or rich of welding experience. Therefore, weld penetration status identification has become the obstacle for intelligent welding system. In this dissertation, an innovative method has been proposed to detect the weld joint penetration status using machine-learning algorithms. A GTAW welding system is firstly built. Project a dot-structured laser pattern onto the weld pool surface during welding process, the reflected laser pattern is captured which contains all the information about the penetration status. An experienced welder is able to determine weld penetration status just based on the reflected laser pattern. However, it is difficult to characterize the images to extract key information that used to determine penetration status. To overcome the challenges in finding right features and accurately processing images to extract key features using conventional machine vision algorithms, we propose using convolutional neural network (CNN) to automatically extract key features and determine penetration status. Data-label pairs are needed to train a CNN. Therefore, an image acquiring system is designed to collect reflected laser pattern and the image of work-piece backside. Data augmentation is performed to enlarge the training data size, which resulting in 270,000 training data, 45,000 validation data and 45,000 test data. A six-layer convolutional neural network (CNN) has been designed and trained using a revised mini-batch gradient descent optimizer. Final test accuracy is 90.7% and using a voting mechanism based on three consequent images further improve the prediction accuracy

TOWARD INTELLIGENT WELDING BY BUILDING ITS DIGITAL TWIN

To meet the increasing requirements for production on individualization, efficiency and quality, traditional manufacturing processes are evolving to smart manufacturing with the support from the information technology advancements including cyber-physical systems (CPS), Internet of Things (IoT), big industrial data, and artificial intelligence (AI). The pre-requirement for integrating with these advanced information technologies is to digitalize manufacturing processes such that they can be analyzed, controlled, and interacted with other digitalized components. Digital twin is developed as a general framework to do that by building the digital replicas for the physical entities. This work takes welding manufacturing as the case study to accelerate its transition to intelligent welding by building its digital twin and contributes to digital twin in the following two aspects (1) increasing the information analysis and reasoning ability by integrating deep learning; (2) enhancing the human user operative ability to physical welding manufacturing via digital twins by integrating human-robot interaction (HRI). Firstly, a digital twin of pulsed gas tungsten arc welding (GTAW-P) is developed by integrating deep learning to offer the strong feature extraction and analysis ability. In such a system, the direct information including weld pool images, arc images, welding current and arc voltage is collected by cameras and arc sensors. The undirect information determining the welding quality, i.e., weld joint top-side bead width (TSBW) and back-side bead width (BSBW), is computed by a traditional image processing method and a deep convolutional neural network (CNN) respectively. Based on that, the weld joint geometrical size is controlled to meet the quality requirement in various welding conditions. In the meantime, this developed digital twin is visualized to offer a graphical user interface (GUI) to human users for their effective and intuitive perception to physical welding processes. Secondly, in order to enhance the human operative ability to the physical welding processes via digital twins, HRI is integrated taking virtual reality (VR) as the interface which could transmit the information bidirectionally i.e., transmitting the human commends to welding robots and visualizing the digital twin to human users. Six welders, skilled and unskilled, tested this system by completing the same welding job but demonstrate different patterns and resulted welding qualities. To differentiate their skill levels (skilled or unskilled) from their demonstrated operations, a data-driven approach, FFT-PCA-SVM as a combination of fast Fourier transform (FFT), principal component analysis (PCA), and support vector machine (SVM) is developed and demonstrates the 94.44% classification accuracy. The robots can also work as an assistant to help the human welders to complete the welding tasks by recognizing and executing the intended welding operations. This is done by a developed human intention recognition algorithm based on hidden Markov model (HMM) and the welding experiments show that developed robot-assisted welding can help to improve welding quality. To further take the advantages of the robots i.e., movement accuracy and stability, the role of the robot upgrades to be a collaborator from an assistant to complete a subtask independently i.e., torch weaving and automatic seam tracking in weaving GTAW. The other subtask i.e., welding torch moving along the weld seam is completed by the human users who can adjust the travel speed to control the heat input and ensure the good welding quality. By doing that, the advantages of humans (intelligence) and robots (accuracy and stability) are combined together under this human-robot collaboration framework. The developed digital twin for welding manufacturing helps to promote the next-generation intelligent welding and can be applied in other similar manufacturing processes easily after small modifications including painting, spraying and additive manufacturing

A Numerical and Experimental Study on Effect of Composition of Ar-N2 Shielding Gas on the Weld Bead Profile and its Prediction for Hot Wire Arc Additive Manufacturing

Wire arc additive manufacturing is a process of making three-dimensional metal parts in a layer-by-layer approach using a feed wire and electric arc as a heat source. Wire arc additive manufacturing (WAAM) is becoming more popular due to its ability to create complex 3D parts, less build time, high deposition rate, and significant cost savings. Out of the many parameters used in WAAM, one of the important parameters is shielding gas which plays a significant role in material quality, properties, and defects. In this study, a controlled amount of Argon (Ar) and Nitrogen (N2) shielding gases are used to see the effect on the weld bead depth and width. In addition, a computational fluid dynamics (CFD) model is used to perform numerical analysis. The data collected from the experiment is used to perform a regression analysis to predict future values. The amount of shielding gas mixture is controlled through a flowmeter to three different total flowrates. The result shows there is an increase in the depth and width of the weld bead with the increase in N2 percentage in the Ar-N2 shielding gas mixture. With the increase in Nitrogen percentage, the tungsten arc is observed unstable and spattering is noticed. The temperature of the surface of the base metal is increased while using the Ar-N2 mixture. The experiment result is further verified by developing and analyzing a three-dimensional computational fluid dynamics model using a volume of fluid (VOF) method. Support vector machine (SVM) regression model with Gaussian kernel function is used to perform the predictive regression analysis. Out of all the regression models, SVM has the lowest model loss for the collected data

Detecting Process Anomalies in the GMAW Process by Acoustic Sensing with a Convolutional Neural Network (CNN) for Classification

Today, the quality of welded seams is often examined off-line with either destructive or non-destructive testing. These test procedures are time-consuming and therefore costly. This is especially true if the welds are not welded accurately due to process anomalies. In manual welding, experienced welders are able to detect process anomalies by listening to the sound of the welding process. In this paper, an approach to transfer the “hearing” of an experienced welder into an automated testing process is presented. An acoustic measuring device for recording audible sound is installed for this purpose on a fully automated welding fixture. The processing of the sound information by means of machine learning methods enables in-line process control. Existing research results until now show that the arc is the main sound source. However, both the outflow of the shielding gas and the wire feed emit sound information. Other investigations describe welding irregularities by evaluating and assessing existing sound recordings. Descriptive analysis was performed to find a connection between certain sound patterns and welding irregularities. Recent contributions have used machine learning to identify the degree of welding penetration. The basic assumption of the presented investigations is that process anomalies are the cause of welding irregularities. The focus was on detecting deviating shielding gas flow rates based on audio recordings, processed by a convolutional neural network (CNN). After adjusting the hyperparameters of the CNN it was capable of distinguishing between different flow rates of shielding gas

MACHINE VISION RECOGNITION OF THREE-DIMENSIONAL SPECULAR SURFACE FOR GAS TUNGSTEN ARC WELD POOL

Observing the weld pool surface and measuring its geometrical parameters is a key to developing the next-generation intelligent welding machines that can mimic a skilled human welder who observes the weld pool to adjust welding parameters. It also provides us an effective way to improve and validate welding process modeling. Although different techniques have been applied in the past few years, the dynamic specular weld pool surface and the strong weld arc complicate these approaches and make the observation /measurement difficult. In this dissertation, a novel machine vision system to measure three-dimensional gas tungsten arc weld pool surface is proposed, which takes advantage of the specular reflection. In the designed system, a structured laser pattern is projected onto the weld pool surface and its reflection from the specular weld pool surface is imaged on an imaging plane and recorded by a high-speed camera with a narrow band-pass filter. The deformation of the molten weld pool surface distorts the reflected pattern. To derive the deformed surface of the weld pool, an image processing algorithm is firstly developed to detect the reflection points in the reflected laser pattern. The reflection points are then matched with their respective incident rays according to the findings of correspondence simulations. As a result, a set of matched incident ray and reflection point is obtained and an iterative surface reconstruction scheme is proposed to derive the three-dimensional pool surface from this set of data based on the reflection law. The reconstructed results proved the effectiveness of the system. Using the proposed surface measurement (machine vision) system, the fluctuation of weld pool surface parameters has been studied. In addition, analysis has been done to study the measurement error and identify error sources in order to improve the measurement system for better accuracy. The achievements in this dissertation provide a useful guidance for the further studies in on-line pool measurement and welding quality control

VISION BASED REAL-TIME MONITORING AND CONTROL OF METAL TRANSFER IN LASER ENHANCED GAS METAL

Laser enhanced gas metal arc welding (GMAW) is a novel welding process where a laser is applied to provide an auxiliary detaching force to help detach the droplet such that welds may be made in gas tungsten arc welding high quality at GMAW high speeds. The current needed to generate the electromagnetic (detaching) force is thus reduced. The reduction in the current helps reduce the impact on the weld pool and over-heat fumes/smokes. However, in the previous studies, a continuous laser is applied. Since the auxiliary is only needed each time the droplet needs to be detached and the detachment time is relatively short in the transfer cycle, the laser energy is greatly wasted in the rest of the transfer cycle. In addition, the unnecessary application of the laser on the droplet causes additional over-heat fumes. Hence, this study proposes to use a pulsed laser such that the peak pulse is applied only when the droplet is ready to detach. To this end, the state of the droplet development needs to be closely monitored in real-time. Since the metal transfer is an ultra-high speed process and the most reliable method to monitor should be based on visual feedback, a high imaging system has been proposed to monitor the real-time development of the droplet. A high-speed image processing system has been developed to real-time extract the developing droplet. A closed-loop control system has been established to use the real-time imaging processing result on the monitoring of the developing droplet to determine if the laser peak pulse needs to be applied. Experiments verified the effectiveness of the proposed methods and established system. A controlled novel process – pulsed laser-enhanced GMAW - is thus established for possible applications in producing high-quality welds at GMAW speeds

REAL-TIME SENSING AND CONTROL OF DEVELOPING WELD PENETRATION THROUGH REFLECTION VIBRATION IN GTAW

GTAW (Gas Tungsten Arc Welding) weld pool surface is believed to contain sufficient information to determine the weld penetration, from which skilled welders are able to control the welding process to desired penetration states. However, it is unclear how human welders extract the weld penetration from the observed weld pool surface. In this research, a novel method is studied to determine the weld penetration based on the dynamic change of the weld pool surface. This study observes/measures/analyzes the development of a weld pool from partial to full penetration and correlates such change to the weld penetration. Similar trends in the weld pool surface are observed when the weld penetration changes from partial to full penetration despite the amperage used and material welded. Correlating the weld pool surface reflecting grayness and the development of the weld penetration from experiments shows: (1) the weld pool reflection intensity will increase while the weld penetration is increasing; (2) the increasing trends of weld pool reflection intensity will decrease when the full penetration is achieved; (3) the weld pool reflection intensity will increase after the full penetration is achieved. Such trend in the weld pool surface reflection intensity when the weld penetration develops is used as feedback signal to detect the weld pool penetration. To control the weld pool penetration, a first-order dynamic model is identified. Model Predictive Control (MPC) is used to control the weld penetration. Experiments verified the feasibility of this proposed method and established system

END-TO-END PREDICTION OF WELD PENETRATION IN REAL TIME BASED ON DEEP LEARNING

Welding is an important joining technique that has been automated/robotized. In automated/robotic welding applications, however, the parameters are preset and are not adaptively adjusted to overcome unpredicted disturbances, which cause these applications to not be able to meet the standards from welding/manufacturing industry in terms of quality, efficiency, and individuality. Combining information sensing and processing with traditional welding techniques is a significant step toward revolutionizing the welding industry. In practical welding, the weld penetration as measured by the back-side bead width is a critical factor when determining the integrity of the weld produced. However, the back-side bead width is difficult to be directly monitored during manufacturing because it occurs underneath the surface of the welded workpiece. Therefore, predicting back-side bead width based on conveniently sensible information from the welding process is a fundamental issue in intelligent welding. Traditional research methods involve an indirect process that includes defining and extracting key characteristic information from the sensed data and building a model to predict the target information from the characteristic information. Due to a lack of feature information, the cumulative error of the extracted information and the complex sensing process directly affect prediction accuracy and real-time performance. An end-to-end, data-driven prediction system is proposed to predict the weld penetration status from top-side images during welding. In this method, a passive-vision sensing system with two cameras to simultaneously monitor the top-side and back-bead information is developed. Then the weld joints are classified into three classes (i.e., under penetration, desirable penetration, and excessive penetration) according to the back-bead width. Taking the weld pool-arc images as inputs and corresponding penetration statuses as labels, an end-to-end convolutional neural network (CNN) is designed and trained so the features are automatically defined and extracted. In order to increase accuracy and training speed, a transfer learning approach based on a residual neural network (ResNet) is developed. This ResNet-based model is pre-trained on an ImageNet dataset to process a better feature-extracting ability, and its fully connected layers are modified based on our own dataset. Our experiments show that this transfer learning approach can decrease training time and improve performance. Furthermore, this study proposes that the present weld pool-arc image is fused with two previous images that were acquired 1/6s and 2/6s earlier. The fused single image thus reflects the dynamic welding phenomena, and prediction accuracy is significantly improved with the image-sequence data by fusing temporal information to the input layer of the CNN (early fusion). Due to the critical role of weld penetration and the negligible impact on system implementation, this method represents major progress in the field of weld-penetration monitoring and is expected to provide more significant improvements during welding using pulsed current where the process becomes highly dynamic