Image-Based Feature Tracking Algorithms for Real-Time Clad Height Detection in Laser Cladding

In laser cladding, a material, usually in the form of powder, is deposited on a substrate. Powder particles are intermingled with inert gas and fed by a powder feeder system on the substrate. Laser is employed to melt the additive material and a small layer of surface of the substrate simultaneously. While the powder is being deposited, the laser melts the powder particles and the melted powder particles join the melt pool on the substrate beneath the laser beam. Generating relative motion between the laser focal point and the substrate will result in moving melt pool on the substrate. This will lead to addition of a desired material to the substrate with desired thickness and good bonding as well as minimum dilution. In addition, by producing clads beside and on the top of each other a functional component can be built in a layer by layer fashion. Despite many advantages of laser cladding, it is highly sensitive to internal and external disturbances. This makes a closed-loop control system for laser cladding inevitable. Utilizing a closed-loop control system in laser cladding makes the system insensitive to external and internal disturbances. Having a closed-loop control system for laser cladding would contribute to substantial improvement in clad quality and cost reduction. Feedback sensor is an essential part in a closed-loop control system. Among different parameters that can be used as feedback signals in a closed-loop control of laser cladding, melt pool geometry and in particular clad height is of great importance specifically for the purpose of rapid prototyping. This thesis presents novel algorithms for real-time detection of clad height in laser cladding. This is accomplished by the following: Tackling the issues pertinent to image acquisition in the presence of harsh and intensive light is scrutinized. Important parameters of digital cameras related to selection of proper type of CCD cameras in order to overcome the existent harsh condition are presented. Also, the existent light in laser cladding arisen from different sources is analyzed and based upon that proper bandpass filters and neutral filters are selected. All these lead to capture relatively sharp and clear images of the melt pool. Capturing good quality pictures potentially would provide valuable information about the process. This information could include, but is not limited to, melt pool geometry (i.e., melt pool height, width, melt pool profile, and wet angle), angle of solidification, melt pool temperature, and melt pool temperature distribution. Furthermore, the issues regarding path dependency of the melt pool image are addressed by using a trinocular cameras configuration. By utilizing this, always two cameras monitor the front end of the melt pool regardless of the direction of the clad. Image analysis of the grabbed images is also discussed. Image thresholding is one of the most formidable tasks in image processing and this difficulty is intensified due to characteristics of the grabbed images of the melt pool (e.g., surrounding hazy area around the melt pool). Applying hard partitioning thresholding method did not lead to detec- tion of the melt pool accurately. As a result, fuzzy thresholding by minimizing of the measure of fuzziness is developed and its performance is investigated. The effect of three important membership functions, triangular, Gaussian, and generalized Bell on the performance of the thresholding method is investigated. Also, Image thresholding by utilizing fuzzy c-means clustering is developed. Applying the developed thresholding methods show promising results. Among the developed thresholding methods, fuzzy thresholding with minimizing the measure of fuzziness with Gaussian membership function is selected for the implementation in the algorithm. Finally, Image feature tracking module is presented. The detected borders of the melt pool images are transformed from image plane to the world plane by using a perspective transformation. Four features of the elliptical features of the projected melt pool borders are selected. These four features along with the angle of tangential path vector with respect to the corresponding right hand side camera's axis are fed into an Elman recurrent neural network. The proposed algorithms and the trained neural network are utilized in the process resulting in acceptable detection of the clad height in deposition of straight clads for a specific direction. It is concluded that the system can detect the clad height with about ±0.15 mm maximum error

Constructing a reference standard for sports science and clinical movement sets using IMU-based motion capture technology

Motion analysis has improved greatly over the years through the development of low-cost inertia sensors. Such sensors have shown promising accuracy for both sport and medical applications, facilitating the possibility of a new reference standard to be constructed. Current gold standards within motion capture, such as high-speed camera-based systems and image processing, are not suitable for many movement-sets within both sports science and clinical movement analysis due to restrictions introduced by the movement sets. These restrictions include cost, portability, local environment constraints (such as light level) and poor line of sight accessibility. This thesis focusses on developing a magnetometer-less IMU-based motion capturing system to detect and classify two challenging movement sets: Basic stances during a Shaolin Kung Fu dynamic form, and severity levels from the modified UPDRS (Unified Parkinson’s Disease Rating Scale) analysis tapping exercise. This project has contributed three datasets. The Shaolin Kung Fu dataset is comprised of 5 dynamic movements repeated over 350 times by 8 experienced practitioners. The dataset was labelled by a professional Shaolin Kung Fu master. Two modified UPDRS datasets were constructed, one for each of the two locations measured. The modified UPDRS datasets comprised of 5 severity levels each with 100 self-emulated movement samples. The modified UPDRS dataset was labelled by a researcher in neuropsychological assessment. The errors associated with IMU systems has been reduced significantly through a combination of a Complementary filter and applying the constraints imposed by the range of movements available in human joints. Novel features have been extracted from each dataset. A piecewise feature set based on a moving window approach has been applied to the Shaolin Kung Fu dataset. While a combination of standard statistical features and a Durbin Watson analysis has been extracted from the modified UPDRS measurements. The project has also contributed a comparison of 24 models has been done on all 3 datasets and the optimal model for each dataset has been determined. The resulting models were commensurate with current gold standards. The Shaolin Kung Fu dataset was classified with the computational costly fine decision tree algorithm using 400 splits, resulting in: an accuracy of 98.9%, a precision of 96.9%, a recall value of 99.1%, and a F1-score of 98.0%. A novel approach of using sequential forward feature analysis was used to determine the minimum number of IMU devices required as well as the optimal number of IMU devices. The modified UPDRS datasets were then classified using a support vector machine algorithm requiring various kernels to achieve their highest accuracies. The measurements were repeated with a sensor located on the wrist and finger, with the wrist requiring a linear kernel and the finger a quadratic kernel. Both locations achieved an accuracy, precision, recall, and F1-score of 99.2%. Additionally, the project contributed an evaluation to the effect sensor location has on the proposed models. It was concluded that the IMU-based system has the potential to construct a reference standard both in sports science and clinical movement analysis. Data protection security and communication speeds were limitations in the system constructed due to the measured data being transferred from the devices via Bluetooth Low Energy communication. These limitations were considered and evaluated in the future works of this project

Non-Contact Height Estimation for Material Extrusion Additive Systems via Monocular Imagery

Additive manufacturing is a dynamic technology with a compelling potential to advance the manufacturing industry. Despite its capacity to produce intricate designs in an efficient manner, industry still has not widely adopted additive manufacturing since its commercialization as a result of its many challenges related to quality control. The Air Force Research Laboratory (AFRL), Materials and Manufacturing Directorate, Functional Materials Division, Soft Matter Materials Branch (RXAS) requires a practical and reliable method for maintaining quality control for the production of printed flexible electronics. Height estimation is a crucial component for maintaining quality control in Material Extrusion Additive Manufacturing (MEAM), as the fundamental process for constructing any structure relies on the consecutive layering of precise extrusions. This work presents a computer vision solution to the problem of height estimation using monocular imagery as applicable to MEAM

Pornographic Image Recognition via Weighted Multiple Instance Learning

In the era of Internet, recognizing pornographic images is of great significance for protecting children's physical and mental health. However, this task is very challenging as the key pornographic contents (e.g., breast and private part) in an image often lie in local regions of small size. In this paper, we model each image as a bag of regions, and follow a multiple instance learning (MIL) approach to train a generic region-based recognition model. Specifically, we take into account the region's degree of pornography, and make three main contributions. First, we show that based on very few annotations of the key pornographic contents in a training image, we can generate a bag of properly sized regions, among which the potential positive regions usually contain useful contexts that can aid recognition. Second, we present a simple quantitative measure of a region's degree of pornography, which can be used to weigh the importance of different regions in a positive image. Third, we formulate the recognition task as a weighted MIL problem under the convolutional neural network framework, with a bag probability function introduced to combine the importance of different regions. Experiments on our newly collected large scale dataset demonstrate the effectiveness of the proposed method, achieving an accuracy with 97.52% true positive rate at 1% false positive rate, tested on 100K pornographic images and 100K normal images.Comment: 9 pages, 3 figure

USE OF UAS IN A HIGH MOUNTAIN LANDSCAPE: THE CASE OF GRAN SOMMETTA ROCK GLACIER (AO)

Photogrammetry has been used since long time to periodically control the evolution of landslides, either from aerial images as well as from ground. Landslides control and monitoring systems face a large variety of cases and situations: in hardly accessible environments, like glacial areas and high mountain locations, it is not simple finding a survey method and a measurement control system, which are capable to reliably assess, with low costs, the expected displacement and its accuracy. For this reason, the behaviour of these events presents the geologists and the surveyor each time with different challenges. The use of UAS (Unmanned Aerial System) represents, in this context, a recent and valid option to perform the data acquisition both in safety and quickly, avoiding hazards and risks for the operators while at the same time containing the costs. The paper presents an innovative monitoring system based on UAS-photogrammetry, GNSS survey and DSM change detection techniques to evaluate the Gran Sommetta rock glacier surface movements over the period 2012-2014. Since 2012, the surface movements of the glacier are monitored by ARPAVdA (a regional environmental protection agency) as a case study for the impact of climate change on high-mountain infrastructures. In such scenarios, in fact, a low-cost monitoring activity can provide important data to improve our knowledge about glacier dynamics connected to climate changes and to prevent risks in anthropic Alps areas. To evaluate the displacements of the rock glacier different techniques were proposed: the most reliable uses the orthophoto of the area and rely on a manual identification of corresponding features performed by a trained operator. To further limit the costs and improve the density of displacement information two automatic procedures were developed as well

Real-Time Monitoring of Thermal Processes

In this research, a monitoring system for thermal processes was developed which measures the most critical process phenomena, such as thermal dynamics (peak temperature, heating rate, and cooling rate) and geometric features, in real-time, which can be used for quality assurance and real-time feedback control. Thermal processes are a subset of manufacturing processes that are characterized by heating materials with a concentrated heat source to alter the properties of the materials or join them. Metal additive manufacturing and arc welding processes are considered thermal processes, where the concentrated energy source may be in the form of a laser, electron beam, electric arc, etc. While thermal processes can be used to create complex components without the limitations of traditional manufacturing, process disturbances may cause deviations from expected results. During thermal processing, geometry and thermal dynamics of the heat affected zone (HAZ) directly influence the quality of the produced products. Therefore, it is critical to have an accurate tool to monitor the geometry and thermal dynamics in real time to better assure the quality of the part. Various sensors are available to measure these properties, though imaging is a common theme among thermal process monitoring. Imaging is an effective technique since it allows for non-contact in-situ measurements. Imaging in different wavelengths can provide different information regarding the HAZ, such as the temperature distribution from infrared (IR) light. While high resolution, and high frame rate geometry measurements from visible light can be monitored directly. Moreover, processing images with machine learning algorithms has also been shown to be capable of predicting porosity and detecting defects in the part being manufactured. Therefore, the monitoring system designed in this research features high dynamic range (HDR) visible light and IR dual camera sensors with a common optical path to monitor the geometry and thermal dynamics, with the potential to implement machine learning to monitor other features in the future. An enclosure was designed to house both sensors with a common optical setup for the sensors to have a similar field of view (FOV). In this work, the IR sensor was used to create a dataset to predict the temperature distribution of the HAZ with the HDR sensor. From the temperature distribution, thermal dynamics such as peak temperature, cooling rate, heating rate, solidification time, and melting time were calculated in real-time to estimate the material properties of the final part. The HDR sensor was also used to predict the geometry of the deposited material (clad). Using the same sensors, the height and width of the deposition are estimated from the captured images in real-time which are used for deposition geometry control. The geometry prediction algorithm evolved during this work with different algorithms and features used in the measurements to improve the robustness and accuracy of geometry measurements. To test the effectiveness of the monitoring system, laser heat treatment (LHT) experiments were conducted to initially validate the thermal dynamics measurements. Thermal dynamics were then further validated during laser directed energy deposition (LDED), which was additionally used to validate the geometry measurements of the clad. Moreover, gas metal arc welding (GMAW) experiments were conducted as well to demonstrate the potential for using this system for different energy sources and materials. The developed dual sensor camera was shown to be capable of capturing images in real-time during thermal processes. Processing the visible-light images allows the geometry of the HAZ to be monitored, while the IR sensor provides its temperature distribution. The system was shown to be robust enough to capture data with multiple materials (stainless steel and nickel-based alloys) and with different energy sources (laser and electric arc). The thermal dynamics measured with this tool have been shown to correlate to the material properties of the produced parts, thus demonstrating the potential to infer the material properties from these measurements. It has also been shown that a cost-effective alternative design using the visible light sensor to predict the temperature distribution with calibrated measurements from a pyrometer may be used for temperature measurements in thermal processes. Therefore, the developed monitoring system is shown to be an effective monitoring and control tool for various thermal processes

Microstresses and microstructure in thick cobalt-based laser deposited coatings

Microstresses in a thick laser clad Co-based coating on steel substrate were investigated with 3D X-ray microscopy using an intense synchrotron microfocused beam. The microstructure was examined with tight microscopy and field emission scanning electron microscopy equipped with X-ray energy dispersive spectroscopy and Electron Back Scattering Diffraction (orientation imaging microscopy). Microhardness and scratch resistance variations inside the coating are related to the local microstructure influenced by additional heating and by melt convection during the laser track overlapping. The residual microstrains were accessed with a high spatial resolution defined by the size of the synchrotron microbeam. Type 11 residual strains and stresses on the level of individual grains and dendrites were analyzed in terms of tensor invariants, hydrostatic and von Mises shear stress, along the depth of a slightly diluted clad track. The upper part of the coating shows a constant spread of hydrostatic stresses between -500 and 500 MPa; towards the bottom of the track the spread of these stresses increases almost linearly with depth. A correlation between the microstructural features and the spread of the hydrostatic microstresses was found. It is concluded that microstresses in individual neighboring grains are inhomogeneously dispersed. (c) 2007 Elsevier B.V. All rights reserved

AUTOMATIC ERROR DETECTION AND CORRECTION IN LASER METAL WIRE DEPOSITION - AN ADDITIVE MANUFACTURING TECHNOLOGY

Additive manufacturing (AM) technology involves building three-dimensional objects by adding material layer-upon-layer under computer control. Metal additive manufacturing offers new possibilities, not only in design, but also in the choice of materials. However, the additive process remains at a lower maturity level compared to the conventional subtractive processes such as milling, drilling and machining among others. Scientifically, there is a safety concern relating to the accuracy of the AM process, how printed products will perform over time and the consistency of their quality. Process accuracy and eventual part quality is compromised due to errors introduced by each of the building steps in the process.Laser metal deposition with wire (LMD-w) is an additive manufacturing technology that involves feeding metal wire through a nozzle and melting the wire with a high-power laser. The technology is being largely researched for use in the aerospace industry to fabricate large aircraft components. With efficient process control, i.e. sensing, processing, and feedback correction of errors, the LMD-w technology has the potential to change the course of manufacturing. However, a prominent limitation in LMD-w is the difficulty in controlling the process.This work proposes a method for detecting surface geometry errors in a deposited layer in the LMD-w process via laser height scanning and high-speed image processing. The controlled process is simplified into a linear system. The aim is to develop an effective sensing and correction module that automatically detects irregularities in each layer before proceeding to subsequent layers, which will reduce part porosity and improve inter-layer bond integrity

The use of custom beam profiles in laser deposition

The work presented in this thesis discovers that through the use of shaped laser beam profiles the microstructure of the deposition can be modified. It has been seen that though modifying the beam profile melt pool flow observed during the preposition process has altered. A number of problems have been identified with current laser deposition processes, typically porosity, cracking and undesired deposition profile. The work identifies that thermal profiles are a major factor influencing both the microstructure and deposition. Methods for observing and measuring thermal profiles are explored. A number of beam profiles are used in this study showing a number of effects on the thermal profiles present during the deposition on Inconel 625 onto mild steel substrate. EBSD and ESD analysis is used to examine the properties of the depositions. Further imaging and analysis of melt pool flow during the process is undertaken using high speed camera imaging, motion tracking and novel pyrometry techniques. As was expected the use of modified beam profiles had an influence on the microstructure of the depositions formed, large variations in grain size an orientation were observed along with alloy element segregation. Through the melt pool imaging techniques developed it was observed that the material transport mechanisms were modified by the shaped laser beam dramatically reducing the material transport velocity, indicating a reduced thermal gradient. This work shows that through modifying the laser beam profile factors influencing the quality of a resulting deposition can be changed. Through further work this principle can be expanded to use the laser beam profile as an input factor to allow the used design of deposition profiles