Machine Learning Based Defect Detection in Robotic Wire Arc Additive Manufacturing

In the last ten years, research interests in various aspects of the Wire Arc Additive Manufacturing (WAAM) processes have grown exponentially. More recently, efforts to integrate an automatic quality assurance system for the WAAM process are increasing. No reliable online monitoring system for the WAAM process is a key gap to be filled for the commercial application of the technology, as it will enable the components produced by the process to be qualified for the relevant standards and hence be fit for use in critical applications in the aerospace or naval sectors. However, most of the existing monitoring methods only detect or solve issues from a specific sensor, no monitoring system integrated with different sensors or data sources is developed in WAAM in the last three years. In addition, complex principles and calculations of conventional algorithms make it hard to be applied in the manufacturing of WAAM as the character of a long manufacturing cycle. Intelligent algorithms provide in-built advantages in processing and analysing data, especially for large datasets generated during the long manufacturing cycles. In this research, in order to establish an intelligent WAAM defect detection system, two intelligent WAAM defect detection modules are developed successfully. The first module takes welding arc current / voltage signals during the deposition process as inputs and uses algorithms such as support vector machine (SVM) and incremental SVM to identify disturbances and continuously learn new defects. The incremental learning module achieved more than a 90% f1-score on new defects. The second module takes CCD images as inputs and uses object detection algorithms to predict the unfused defect during the WAAM manufacturing process with above 72% mAP. This research paves the path for developing an intelligent WAAM online monitoring system in the future. Together with process modelling, simulation and feedback control, it reveals the future opportunity for a digital twin system

Visual Sensing and Defect Detection of Gas Tungsten Arc Welding

Weld imperfections or defects such as incomplete penetration and lack of fusion are critical issues that affect the integration of welding components. The molten weld pool geometry is the major source of information related to the formation of these defects. In this dissertation, a new visual sensing system has been designed and set up to obtain weld pool images during GTAW. The weld pool dynamical behavior can be monitored using both active and passive vision method with the interference of arc light in the image significantly reduced through the narrow band pass filter and laser based auxiliary light source.Computer vision algorithms based on passive vision images were developed to measure the 3D weld pool surface geometry in real time. Specifically, a new method based on the reversed electrode image (REI) was developed to calculate weld pool surface height in real time. Meanwhile, the 2D weld pool boundary was extracted with landmarks detection algorithms. The method was verified with bead-on-plate and butt-joint welding experiments.Supervised machine learning was used to develop the capability to predict, in real-time, the incomplete penetration on thin SS304 plate with the key features extracted from weld pool images. An integrated self-adaptive close loop control system consisting the non-contact visual sensor, machine learning based defect predictor, and welding power source was developed for real-time welding penetration control for bead on plate welding. Moreover, the data driven methods were first applied to detect incomplete penetration and LOF in multi-pass U groove welding. New features extracted from reversed electrode image played the most important role to predict these defects. Finally, real time welding experiments were conducted to verify the feasibility of the developed models

TOWARD INTELLIGENT WELDING BY BUILDING ITS DIGITAL TWIN

To meet the increasing requirements for production on individualization, efficiency and quality, traditional manufacturing processes are evolving to smart manufacturing with the support from the information technology advancements including cyber-physical systems (CPS), Internet of Things (IoT), big industrial data, and artificial intelligence (AI). The pre-requirement for integrating with these advanced information technologies is to digitalize manufacturing processes such that they can be analyzed, controlled, and interacted with other digitalized components. Digital twin is developed as a general framework to do that by building the digital replicas for the physical entities. This work takes welding manufacturing as the case study to accelerate its transition to intelligent welding by building its digital twin and contributes to digital twin in the following two aspects (1) increasing the information analysis and reasoning ability by integrating deep learning; (2) enhancing the human user operative ability to physical welding manufacturing via digital twins by integrating human-robot interaction (HRI). Firstly, a digital twin of pulsed gas tungsten arc welding (GTAW-P) is developed by integrating deep learning to offer the strong feature extraction and analysis ability. In such a system, the direct information including weld pool images, arc images, welding current and arc voltage is collected by cameras and arc sensors. The undirect information determining the welding quality, i.e., weld joint top-side bead width (TSBW) and back-side bead width (BSBW), is computed by a traditional image processing method and a deep convolutional neural network (CNN) respectively. Based on that, the weld joint geometrical size is controlled to meet the quality requirement in various welding conditions. In the meantime, this developed digital twin is visualized to offer a graphical user interface (GUI) to human users for their effective and intuitive perception to physical welding processes. Secondly, in order to enhance the human operative ability to the physical welding processes via digital twins, HRI is integrated taking virtual reality (VR) as the interface which could transmit the information bidirectionally i.e., transmitting the human commends to welding robots and visualizing the digital twin to human users. Six welders, skilled and unskilled, tested this system by completing the same welding job but demonstrate different patterns and resulted welding qualities. To differentiate their skill levels (skilled or unskilled) from their demonstrated operations, a data-driven approach, FFT-PCA-SVM as a combination of fast Fourier transform (FFT), principal component analysis (PCA), and support vector machine (SVM) is developed and demonstrates the 94.44% classification accuracy. The robots can also work as an assistant to help the human welders to complete the welding tasks by recognizing and executing the intended welding operations. This is done by a developed human intention recognition algorithm based on hidden Markov model (HMM) and the welding experiments show that developed robot-assisted welding can help to improve welding quality. To further take the advantages of the robots i.e., movement accuracy and stability, the role of the robot upgrades to be a collaborator from an assistant to complete a subtask independently i.e., torch weaving and automatic seam tracking in weaving GTAW. The other subtask i.e., welding torch moving along the weld seam is completed by the human users who can adjust the travel speed to control the heat input and ensure the good welding quality. By doing that, the advantages of humans (intelligence) and robots (accuracy and stability) are combined together under this human-robot collaboration framework. The developed digital twin for welding manufacturing helps to promote the next-generation intelligent welding and can be applied in other similar manufacturing processes easily after small modifications including painting, spraying and additive manufacturing

Data Analysis and Modeling Techniques of Welding Processes: The State-of-the-Art

Information contributes to the improvement of decision-making, process improvement, error detection, and prevention. The new requirements of the coming Industry 4.0 will make these new information technologies help in the improvement and decision-making of industrial processes. In case of the welding processes, several techniques have been used. Welding processes can be analyzed as a stochastic system with several inputs and outputs. This allows a study with a data analysis perspective. Data mining processes, machine learning, deep learning, and reinforcement learning techniques have had good results in the analysis and control of systems as complex as the welding process. The increase of information acquisition and information quality by sensors developed at present, allows a large volume of data that benefits the analysis of these techniques. This research aims to make a bibliographic analysis of the techniques used in the welding area, the advantages that these new techniques can provide, and how some researchers are already using them. The chapter is organized according to some stages of the data mining process. This was defined with the objective of highlighting evolution and potential for each stage for welding processes

Effects of part-to-part gap and the variation of weld seam on the laser welding quality

Human and Systems EngineeringLaser welding is an emerging joining technique for automotive industry. Due to the complex process, the control of laser welding quality problem such as spatter, porosity, and undercut is not easy. Traditional mechanical testing of weld is time consuming, costly, and inefficient for on-line monitoring of laser welding quality. In this regards, this thesis aims to estimate the laser welding quality in a non-destructive way by using the geometry of top weld seam of weldment. The most challenging task which is covered in this thesis is the estimation of the tensile shear strength in a non-destructive way. Experimental analysis reveals that the existence of two types of correlation in the variation of weld seam and tensile shear strength in the laser welding of galvanized steel: (1) positive correlation exists between the log-transformed average width of the top weld seam and maximum tensile shear strength, and (2) negative correlation exists between the variation of the width of top weld seam and maximum tensile shear strength. These correlations exist only when top weld seam having non uniform seam boundary. However, weld seam having uniform seam boundary do not have sufficient evidence of the correlation between the variation of weld seam and tensile shear strength. The reason for this is due to the high value of standard deviation of the average width of the top weld seam. Width of weld seam is more responsible for tensile strength rather than variation of weld seam.ope

WELD PENETRATION IDENTIFICATION BASED ON CONVOLUTIONAL NEURAL NETWORK

Weld joint penetration determination is the key factor in welding process control area. Not only has it directly affected the weld joint mechanical properties, like fatigue for example. It also requires much of human intelligence, which either complex modeling or rich of welding experience. Therefore, weld penetration status identification has become the obstacle for intelligent welding system. In this dissertation, an innovative method has been proposed to detect the weld joint penetration status using machine-learning algorithms. A GTAW welding system is firstly built. Project a dot-structured laser pattern onto the weld pool surface during welding process, the reflected laser pattern is captured which contains all the information about the penetration status. An experienced welder is able to determine weld penetration status just based on the reflected laser pattern. However, it is difficult to characterize the images to extract key information that used to determine penetration status. To overcome the challenges in finding right features and accurately processing images to extract key features using conventional machine vision algorithms, we propose using convolutional neural network (CNN) to automatically extract key features and determine penetration status. Data-label pairs are needed to train a CNN. Therefore, an image acquiring system is designed to collect reflected laser pattern and the image of work-piece backside. Data augmentation is performed to enlarge the training data size, which resulting in 270,000 training data, 45,000 validation data and 45,000 test data. A six-layer convolutional neural network (CNN) has been designed and trained using a revised mini-batch gradient descent optimizer. Final test accuracy is 90.7% and using a voting mechanism based on three consequent images further improve the prediction accuracy

Methods for Quality Monitoring in Ultrasonic Welding of Carbon Fiber Reinforced Polymer Composites

Carbon fiber reinforced composites have been increasingly used in various industrial sectors, especially in the automotive industry. Ultrasonic welding is considered as an effective approach to joining such composites. Reliable weld quality classification and prediction methods are needed to ensure quality and reduce manufacturing costs. However, existing methods have two weaknesses. The first one is that the majority of the existing methods are based on signal feature data extracted from the original experimental time-series data. Feature-based models may not take full advantage of the information contained in the large amounts of time-series data available, even though the models are simple and easy to program. On the other hand, when using experimental time-series data to conduct weld quality monitoring, the data size may be insufficient for training neural network-based methods for quality monitoring or classification. Therefore, a method is needed to augment experimental data while preserving the statistical characteristics of the experimental data. To find reliable quality monitoring models in various situations, this dissertation proposes two neural network models that are respectively applied to feature-based data and full time-series-based data and compares their performances. The dissertation first investigates the relationship between weld energy and joint performance in ultrasonic welding of carbon fiber reinforced polymer (CFRP) sheets through weld experiments. The weld quality classes for training quality monitoring algorithms are determined from welded joint lap-shear strength and the microstructure of the weld zone. These pre-defined weld quality classes are the output criteria for weld quality monitoring on feature-based models and time-series-based models. For feature- based weld quality monitoring, a simple and efficient feature selection method is first developed to screen the most significant features for classification from multiple weld quality classes. A Bayesian regularized neural network (BRNN) is then demonstrated to be more accurate and robust when classifying weld quality classes in ultrasonic composite welding when using feature-based data as the input than the previously proposed methods of support vector machine (SVM), k-nearest neighbors (kNN), and linear discriminant analysis (LDA). To address the limited size of experimental data, a Multivariate Monte Carlo (MMC) simulation with copulas approach is proposed to reasonably generate large amounts of time-series process signals for ultrasonic composite welding. With both experimental data and a large quantity of simulated data, a deep convolutional neural network (CNN) is applied to weld quality classification. The CNN model is found to be more accurate and robust, not only under small training data set sizes, but also under large training data set sizes when compared with previously researched classification methods applied in ultrasonic welding. In conclusion, neural network-based models could achieve high accuracy using feature signals and the full time-series process signals.Ph.D.Manufacturing EngineeringUniversity of Michiganhttp://deepblue.lib.umich.edu/bitstream/2027.42/168232/1/Dissertation_Lei Sun.pd