Bildbasierte Charakterisierung und Regelung von Lasertiefschweißprozessen

Das Laserstrahltiefschweißen ist ein weit verbreitetes Verfahren in der industriellen Fertigung und obwohl bereits seit Jahrzehnten erfolgreich im Einsatz, mangelt es bis heute an Möglichkeiten der Prozessregelung. Verfügbare Regelsysteme beschränken sich meist auf die Positions- oder Abstandsregelung, lassen den eigentlichen Schweißprozess jedoch in aller Regel außen vor. Bisherige Ansätze zur Regelung des Schweißprozesses scheiterten regelmäßig an zu geringer Messgeschwindigkeit oder nicht robust messenden Integraldetektoren. Obgleich die Prozessüberwachung von Laserschweißprozessen bereits in vielen Bereichen Anwendung findet, handelt es sich auch hierbei meist um Verfahren mit integral messenden Detektoren, deren Messkurven lediglich über Korrelationsverfahren mit der erreichten Nahtqualität in Verbindung stehen. Kamerabasierte Verfahren zur Prozessüberwachung wurden zwar in den vergangenen Jahren massiv weiterentwickelt, abgesehen von Systemen zur Positionsüberwachung und -Regelung, kommen jedoch auch bei diesen meist Algorithmen zum Einsatz, die den Prozess auf Helligkeitsschwankungen hin untersuchen. Die Verwendung von Bildverarbeitungsalgorithmen, welche auf der Auswertung von geometrischen Formparametern beruhen, ermöglichen eine weit robustere und aussagekräftigere Beurteilung des Prozesszustandes, als es die eingangs genannten helligkeitsbasierten Algorithmen vermögen. Der notwendige hohe Rechenaufwand verhindert jedoch bis dato die Nutzung solcher Algorithmen für ein echtzeitfähiges System zur Prozessreglung. In dieser Arbeit wird basierend auf spektroskopischen Untersuchungen der elektromagnetischen Prozessemissionen und der Störeinflüsse durch Metalldampf und Schweißrauchpartikel, ein spektrales Fenster identifiziert, welches optimale Bedingungen für die Beobachtung der thermischen Prozessemission mit siliziumbasierten Kameras ermöglicht. Grundlagenuntersuchungen mittels kombiniertem Einsatz von Röntgenvideotechnik und Hochgeschwindigkeitskameras im nahen und mittleren Infrarot, erlauben einen dreidimensionalen Einblick in den Schweißprozess, auch unterhalb der Schmelzebadoberfläche. Die gewonnenen Erkenntnisse bilden die Basis für die Entwicklung einer kamerabasierten Prozessüberwachung, welche über eine koaxial zum Bearbeitungslaserstrahl angeordnete Kamera, die thermische Strahlungsemission des Prozesses erfasst und die entstehenden Bilder anhand geometrischer Bildmerkmale auswertet. Die identifizierten Bildmerkmale beschreiben die jeweiligen transienten Fehler eindeutig und liefern eine Charakterisierung des Prozesszustandes. Aus den evaluierten geometrischen Bildmerkmalen wird das Merkmal Durchschweißloch ausgewählt, um mittels eines geschlossenen Regelkreises den Durchschweißgrad von Lasertiefschweißprozessen zu regeln. Die Regelung wird dabei mittels einer neuartigen Rechnerarchitektur realisiert, der sogenannten Cellularen Neuronalen Netze (CNN). Die CNN-Architektur integriert dabei ein Netzwerk analoger Prozessoren direkt auf dem Kamerachip. Jeder einzelne Pixel verfügt bei diesem System über einen eigen simplen Prozessor. Diese Architektur ermöglicht es durch die Vernetzung der einzelnen Pixel eine Bildverarbeitung direkt auf dem Kamerachip durchzuführen, deren Berechnungen innerhalb eines Belichtungszyklus abgeschlossen sind. Auf diese Weise wurde ein Regelsystem implementiert, welches mit Regelfrequenzen bis zu 14 kHz bei minimaler Latenz, eine robuste Regelung der Durchschweißung und Einschweißung an I-Naht-Überlappverbindungen ermöglicht.Deep-penetration laser welding is still an emerging application in the world of metal joining. It is increasingly replacing traditional resistance spot welding, particularly in automotive body construction, because of its higher productivity, lower costs and better quality. However, while process monitoring and control have found widespread usein classical joining processes, monitoring or closed-loop control of deep-penetration laser welding has only been established in a few applications so far. Previously developed methods for direct in-process monitoring of laser welding processes were usually based on photo diodes. While allowing very high sampling rates, the spatial resolution of such systems is very limited. Camera-based systems on the other hand, offer a high spatial resolution but usually a very limited sampling rate, due to the limited bandwidth of the underlying data processing system. Both approaches are usually based on application-specific correlations between certain measuring signals and typical weld defects. Hence, the systems involved have to be calibrated to the specific application. The goal of this work was the development of a fast, robust, camera-based, closedloop control system for the penetration depth during laser welding. The emphasis of the work is on the overlap joint geometry commonly used in car body construction. This work includes process diagnostics and camera-based process monitoring to build up an accurate picture of the three-dimensional geometry of the welding process. This knowledge is crucial in order to achieve the goal of a non-correlation based measurement of the process characteristics, necessary for the development of a robust closedloop control system. Examination of spectral process emissions in the near infrared range below 1 μm enabled the identification of suitable wavelength ranges for process observation with silicon-based cameras detecting the thermal emission of the process itself. It is demonstrated within this work that conclusions drawn in many older publications have been based on measurements from uncalibrated spectrometers, leading to misinterpretation of various optical effects. By calibrated spectral measurement of the process emissions, two spectral windows free of characteristic line-emissions were identified for steel as well as for aluminium. Furthermore, it is shown that the spectral window be low 500 nm is not usable because of strong scattering effects in the welding plume. These scattering effects increase strongly with shorter wavelengths and cannot be avoided by using external illumination. Consequently the only practically usable spectral window for silicon-based cameras is between 650 nm and 1000 nm. It should also be noted here that the dynamic range of silicon-based cameras is usually not sufficient for a simultaneous detection of the thermal emissions originating from both the keyhole and the weld bead. This is only possible in the infrared spectral range on the longwavelength side of the thermal emission maximum of the welding process (above 3 μm). Investigation of capillary and melt pool dynamics in deep penetration laser welding processes was the next step. High-speed cameras in the visual and infrared spectrum range offer excellent image quality and high frame rates but access to the process details is limited due to the small surface area of the weld zone. With these conventional diagnostic techniques it is thus not possible to observe the key mechanisms inside the volume of the material which essentially determine the behaviour of the welding process. To gain insight into process phenomena such as the shape and movement of the capillary or the melt flow behaviour in the weld bead, X-ray videography is the ideal instrument. In this work the development and implementation of a high-speed X-ray video system is described, which enables the observation of internal process phenomena with high frame rates combined with outstanding spatial resolution. A combined time-synchronous measurement system involving X-ray video and near infrared high-speed video was used to gather three-dimensional information about the geometry of the weld zone and its dynamic behaviour. It was possible to precisely measure the capillary depth and its dynamic movement as well as the direction and velocity of fluid flows inside the molten pool and their influence on pore formation. These findings are crucial for the understanding of possible limitations on the monitoring and controllability of deep-penetration laser welding in an industrial setting. Investigations with high-speed cameras revealed important geometric information on the formation of different welding failures and possibilities for their distinct detection with a passive camera mounted on the welding head coaxially to the laser beam. The geometric image properties observed are directly linked to specific failure mechanisms and do not relay on simple correlations between standard measures and observed welding failures. The automatic recognition of characteristic geometric image-properties was performed by software algorithms developed in Matlab as a proof-of-concept. The robustness of the algorithms developed was tested in an extensive experimental study to identify possible candidates for the development of the closed-loop control system. The so called full penetration hole (FPH) was identified as the most important image feature characterising the state of the process in terms of welding depth for a full penetration weld. Using the aforementioned results it was possible to build up a camera-based closedloop control system to ensure stable welding results even under changing welding conditions. Optical integration into the laser welding head was realised by means of a dichroitic beam-splitter to ensure a coaxial configuration of the camera’s line of sight with the laser beam. This enables the system to be combined with two- or threedimensional scanning heads in remote welding processes. The key-technology to overcome the above mentioned performance issue of camera based control systems is a novel architecture called a “Cellular Neural Network” (CNN). With “Cellular Neural Networks” it is possible to integrate basic processor elements in the electronic circuitry of a CMOS camera resulting in a Single-Instruction-Multiple-Data (SIMD)-architecture on the camera chip itself. Such pixelparallel systems provide extremely fast real-time image processing, since there is no need to transfer image data from the camera to a processor. The closed-loop control system developed in this work uses a CNN based camera surveying the contour of the full-penetration hole with a control frequency of up to 14 kHz for linear welding processes and up to 9 kHz for processes with variable welding trajectory, whereas the latency of the system is in the range of only one single frame. An extensive experimental program was performed to validate the capabilities of the closed-loop control system. It was shown that the system is able to control the degree of full penetration in overlap joints with different steel grades. The closed-loop control successfully compensated for various external disturbances such as variations in material thickness and welding speed, as well as defocusing and the contamination of protective windows. These phenomena could be controlled with either linear or varying direction welding trajectories. Controlled full penetration welds are also possible in aluminum, but not with all alloys. In particular, the alloy AA5182 produces thermal camera images that are strongly disturbed by extensive fluctuations in brightness, which prevents reliable control. On the other hand, a closed-loop control of full penetration with the alloy group AA6000 is possible without difficulty. In closed-loop controlled full penetration welding conditions there is no need for the otherwise obligatory 10 % excess of laser power. This directly leads to a better quality of the root surface due to reduced spattering and smoke residue, as well as an energy saving or a corresponding gain in welding speed. As well as optimizing full penetration, the control system is also able to maintain a stable penetration depth in a partial penetration condition during welding of overlap joints in different steel grades. This is due to the fact that the image feature FPH is also visible when the laser beam reaches the gap between the two welding parts, enabling the system to use this working point as a datum for closed-loop control. Stable closedloop control of partial penetration welds in aluminum alloys could not be achieved in this work. Although the image feature itself was visible, the rate of false detection by the software algorithm was too high to ensure a stable operation. In the partial penetration mode the weld seam does not penetrate through the lower sheet bottom surface. This reduces the need for subsequent machining and also provides higher resistance against corrosion in car body welding. The further decreased laser power in comparison to full penetration mode offers the possibility of higher welding speeds, leading to higher productivity and cost efficiency. The limits of the camera-based closed-loop control system described in this work are basically due to the fact that the image feature of the full penetration hole (FPH) must be visible in the desired parameter field. This means also that a closed-loop control of an arbitrary welding depth, independent of the presence of any boundary surface, is generally not possible. The first welding results using this closed-loop control system were presented at the ICALEO-conference 2008 in Temecula, CA, USA, and the fully developed control system was honored with the third place in the “Berthold Leibinger Innovation Award 2012” and the third place in the “Steel Innovation Award 2012”

END-TO-END PREDICTION OF WELD PENETRATION IN REAL TIME BASED ON DEEP LEARNING

Welding is an important joining technique that has been automated/robotized. In automated/robotic welding applications, however, the parameters are preset and are not adaptively adjusted to overcome unpredicted disturbances, which cause these applications to not be able to meet the standards from welding/manufacturing industry in terms of quality, efficiency, and individuality. Combining information sensing and processing with traditional welding techniques is a significant step toward revolutionizing the welding industry. In practical welding, the weld penetration as measured by the back-side bead width is a critical factor when determining the integrity of the weld produced. However, the back-side bead width is difficult to be directly monitored during manufacturing because it occurs underneath the surface of the welded workpiece. Therefore, predicting back-side bead width based on conveniently sensible information from the welding process is a fundamental issue in intelligent welding. Traditional research methods involve an indirect process that includes defining and extracting key characteristic information from the sensed data and building a model to predict the target information from the characteristic information. Due to a lack of feature information, the cumulative error of the extracted information and the complex sensing process directly affect prediction accuracy and real-time performance. An end-to-end, data-driven prediction system is proposed to predict the weld penetration status from top-side images during welding. In this method, a passive-vision sensing system with two cameras to simultaneously monitor the top-side and back-bead information is developed. Then the weld joints are classified into three classes (i.e., under penetration, desirable penetration, and excessive penetration) according to the back-bead width. Taking the weld pool-arc images as inputs and corresponding penetration statuses as labels, an end-to-end convolutional neural network (CNN) is designed and trained so the features are automatically defined and extracted. In order to increase accuracy and training speed, a transfer learning approach based on a residual neural network (ResNet) is developed. This ResNet-based model is pre-trained on an ImageNet dataset to process a better feature-extracting ability, and its fully connected layers are modified based on our own dataset. Our experiments show that this transfer learning approach can decrease training time and improve performance. Furthermore, this study proposes that the present weld pool-arc image is fused with two previous images that were acquired 1/6s and 2/6s earlier. The fused single image thus reflects the dynamic welding phenomena, and prediction accuracy is significantly improved with the image-sequence data by fusing temporal information to the input layer of the CNN (early fusion). Due to the critical role of weld penetration and the negligible impact on system implementation, this method represents major progress in the field of weld-penetration monitoring and is expected to provide more significant improvements during welding using pulsed current where the process becomes highly dynamic

ESTABLISHING THE FOUNDATION TO ROBOTIZE COMPLEX WELDING PROCESSES THROUGH LEARNING FROM HUMAN WELDERS BASED ON DEEP LEARNING TECHNIQUES

As the demand for customized, efficient, and high-quality production increases, traditional manufacturing processes are transforming into smart manufacturing with the aid of advancements in information technology, such as cyber-physical systems (CPS), the Internet of Things (IoT), big data, and artificial intelligence (AI). The key requirement for integration with these advanced information technologies is to digitize manufacturing processes to enable analysis, control, and interaction with other digitized components. The integration of deep learning algorithm and massive industrial data will be critical components in realizing this process, leading to enhanced manufacturing in the Future of Work at the Human-Technology Frontier (FW-HTF). This work takes welding manufacturing as the case study to accelerate its transition to intelligent welding by robotize a complex welding process. By integrate process sensing, data visualization, deep learning-based modeling and optimization, a complex welding system is established, with the systematic solution to generalize domain-specific knowledge from experienced human welder. Such system can automatically perform complex welding processes that can only be handled by human in the past. To enhance the system\u27s tracking capabilities, we trained an image segmentation network to offer precise position information. We incorporated a recurrent neural network structure to analyze dynamic variations during welding. Addressing the challenge of human heterogeneity in data collection, we conducted experiments illustrating that even inaccurate datasets can effectively train deep learning models with zero mean error. Fine-tuning the model with a small portion of accurate data further elevates its performance

Two-stage quality monitoring of a laser welding process using machine learning – An approach for fast yet precise quality monitoring

In production, quality monitoring is essential to detect defective elements. State-of-the-art approaches are single-sensor systems (SSS) and multi-sensor systems (MSS). Yet, these approaches might not be suitable: Nowadays, one component may comprise several hundred meters of the weld seam, necessitating high-speed welding to produce enough components. To detect as many defects as possible in time, fast yet precise monitoring is required. However, information captured by SSS might not be sufficient and MSS suffer from long inference times. Therefore, we present a confidence-based cascaded system (CS). The key idea of the CS is that not all data are analyzed to obtain the quality weld, but only selected ones. As evidenced by our results, all CS outperform SSS in terms of accuracy and inference time. Further, compared to MSS, the CS has hardware advantages

Novel Approaches for Nondestructive Testing and Evaluation

Nondestructive testing and evaluation (NDT&E) is one of the most important techniques for determining the quality and safety of materials, components, devices, and structures. NDT&E technologies include ultrasonic testing (UT), magnetic particle testing (MT), magnetic flux leakage testing (MFLT), eddy current testing (ECT), radiation testing (RT), penetrant testing (PT), and visual testing (VT), and these are widely used throughout the modern industry. However, some NDT processes, such as those for cleaning specimens and removing paint, cause environmental pollution and must only be considered in limited environments (time, space, and sensor selection). Thus, NDT&E is classified as a typical 3D (dirty, dangerous, and difficult) job. In addition, NDT operators judge the presence of damage based on experience and subjective judgment, so in some cases, a flaw may not be detected during the test. Therefore, to obtain clearer test results, a means for the operator to determine flaws more easily should be provided. In addition, the test results should be organized systemically in order to identify the cause of the abnormality in the test specimen and to identify the progress of the damage quantitatively

Local Binary Patterns in Focal-Plane Processing. Analysis and Applications

Feature extraction is the part of pattern recognition, where the sensor data is transformed into a more suitable form for the machine to interpret. The purpose of this step is also to reduce the amount of information passed to the next stages of the system, and to preserve the essential information in the view of discriminating the data into different classes. For instance, in the case of image analysis the actual image intensities are vulnerable to various environmental effects, such as lighting changes and the feature extraction can be used as means for detecting features, which are invariant to certain types of illumination changes. Finally, classification tries to make decisions based on the previously transformed data. The main focus of this thesis is on developing new methods for the embedded feature extraction based on local non-parametric image descriptors. Also, feature analysis is carried out for the selected image features. Low-level Local Binary Pattern (LBP) based features are in a main role in the analysis. In the embedded domain, the pattern recognition system must usually meet strict performance constraints, such as high speed, compact size and low power consumption. The characteristics of the final system can be seen as a trade-off between these metrics, which is largely affected by the decisions made during the implementation phase. The implementation alternatives of the LBP based feature extraction are explored in the embedded domain in the context of focal-plane vision processors. In particular, the thesis demonstrates the LBP extraction with MIPA4k massively parallel focal-plane processor IC. Also higher level processing is incorporated to this framework, by means of a framework for implementing a single chip face recognition system. Furthermore, a new method for determining optical flow based on LBPs, designed in particular to the embedded domain is presented. Inspired by some of the principles observed through the feature analysis of the Local Binary Patterns, an extension to the well known non-parametric rank transform is proposed, and its performance is evaluated in face recognition experiments with a standard dataset. Finally, an a priori model where the LBPs are seen as combinations of n-tuples is also presentedSiirretty Doriast

Engineering Principles

Over the last decade, there has been substantial development of welding technologies for joining advanced alloys and composites demanded by the evolving global manufacturing sector. The evolution of these welding technologies has been substantial and finds numerous applications in engineering industries. It is driven by our desire to reverse the impact of climate change and fuel consumption in several vital sectors. This book reviews the most recent developments in welding. It is organized into three sections: “Principles of Welding and Joining Technology,” “Microstructural Evolution and Residual Stress,” and “Applications of Welding and Joining.” Chapters address such topics as stresses in welding, tribology, thin-film metallurgical manufacturing processes, and mechanical manufacturing processes, as well as recent advances in welding and novel applications of these technologies for joining different materials such as titanium, aluminum, and magnesium alloys, ceramics, and plastics

Evaluation of Generative Models for Predicting Microstructure Geometries in Laser Powder Bed Fusion Additive Manufacturing

In-situ process monitoring for metals additive manufacturing is paramount to the successful build of an object for application in extreme or high stress environments. In selective laser melting additive manufacturing, the process by which a laser melts metal powder during the build will dictate the internal microstructure of that object once the metal cools and solidifies. The difficulty lies in that obtaining enough variety of data to quantify the internal microstructures for the evaluation of its physical properties is problematic, as the laser passes at high speeds over powder grains at a micrometer scale. Imaging the process in-situ is complex and cost-prohibitive. However, generative modes can provide new artificially generated data. Generative adversarial networks synthesize new computationally derived data through a process that learns the underlying features corresponding to the different laser process parameters in a generator network, then improves upon those artificial renderings by evaluating through the discriminator network. While this technique was effective at delivering high-quality images, modifications to the network through conditions showed improved capabilities at creating these new images. Using multiple evaluation metrics, it has been shown that generative models can be used to create new data for various laser process parameter combinations, thereby allowing a more comprehensive evaluation of ideal laser conditions for any particular build

Electric Vehicle Battery Module Dismantling "Analysis and Evaluation of Robotic Dismantling Techniques for Irre- versible Fasteners, including Object Detection of Components."

This thesis examines a study of The Litium-Ion Battery (LIB) from a electric vehicle, and it’s recycling processes. A Battery Module (BM) from the LIB is shredded when considered an End-Of-Life product, and motivates for automated dismantling concepts to separate the components to save raw materials. From State-of-the-art (SoA) research projects and background theory, automatic module dis- mantling concepts have been evaluated for a Volkswagen E-Golf 2019 battery module. The presence of irreversible fasteners make the use off destructive dismantling techniques neces- sary. This study evaluates two different concepts to disconnect laser welds holding together the compressive plates made of steel. A hydraulic actuated concept is first investigated to separate the welded compressive plates within the casing. A FEM analysis with different configurations is performed to evaluate the most effective hydraulic solution when analysing the Von Mises stress. This solution is further compared with another automatic dismantling concept, namely milling. For the purpose of an automated milling concept, manipulators from ABB are assessed and the feasibility is verified based on results from manual milling operation. The proposed dismantling operation is made possible by developing a system architecture combining robotic control and computer vision. Open source software based on Robot Op- erating System (ROS) and MoveIt connect and control an ABB IRB4400 industrial robot whereas the computer vision setup involves a cutting edge 3D camera, Zivid, and object detection algorithm YOLOv5 best suited for this task. Adjustable acquisition settings in services from Zivid’s ROS driver are tested to accomplish the optimal capture configuration. Two datasets generated with RoboFlow were exported in the YOLOv5 PyTorch format. Custom object detection models with annotated components from the BM was trained and tested with image captures. All in all, this study demonstrates that the automatic dismantling of battery modules can be achieved even though they include irreversible fasteners. The proposed methods are verified on a specific battery module (Egolf 2019) but are flexible enough to be easily extended to a large variety of EV battery modules