NASA Tech Briefs Index, 1977, volume 2, numbers 1-4

Announcements of new technology derived from the research and development activities of NASA are presented. Abstracts, and indexes for subject, personal author, originating center, and Tech Brief number are presented for 1977

END-TO-END PREDICTION OF WELD PENETRATION IN REAL TIME BASED ON DEEP LEARNING

Welding is an important joining technique that has been automated/robotized. In automated/robotic welding applications, however, the parameters are preset and are not adaptively adjusted to overcome unpredicted disturbances, which cause these applications to not be able to meet the standards from welding/manufacturing industry in terms of quality, efficiency, and individuality. Combining information sensing and processing with traditional welding techniques is a significant step toward revolutionizing the welding industry. In practical welding, the weld penetration as measured by the back-side bead width is a critical factor when determining the integrity of the weld produced. However, the back-side bead width is difficult to be directly monitored during manufacturing because it occurs underneath the surface of the welded workpiece. Therefore, predicting back-side bead width based on conveniently sensible information from the welding process is a fundamental issue in intelligent welding. Traditional research methods involve an indirect process that includes defining and extracting key characteristic information from the sensed data and building a model to predict the target information from the characteristic information. Due to a lack of feature information, the cumulative error of the extracted information and the complex sensing process directly affect prediction accuracy and real-time performance. An end-to-end, data-driven prediction system is proposed to predict the weld penetration status from top-side images during welding. In this method, a passive-vision sensing system with two cameras to simultaneously monitor the top-side and back-bead information is developed. Then the weld joints are classified into three classes (i.e., under penetration, desirable penetration, and excessive penetration) according to the back-bead width. Taking the weld pool-arc images as inputs and corresponding penetration statuses as labels, an end-to-end convolutional neural network (CNN) is designed and trained so the features are automatically defined and extracted. In order to increase accuracy and training speed, a transfer learning approach based on a residual neural network (ResNet) is developed. This ResNet-based model is pre-trained on an ImageNet dataset to process a better feature-extracting ability, and its fully connected layers are modified based on our own dataset. Our experiments show that this transfer learning approach can decrease training time and improve performance. Furthermore, this study proposes that the present weld pool-arc image is fused with two previous images that were acquired 1/6s and 2/6s earlier. The fused single image thus reflects the dynamic welding phenomena, and prediction accuracy is significantly improved with the image-sequence data by fusing temporal information to the input layer of the CNN (early fusion). Due to the critical role of weld penetration and the negligible impact on system implementation, this method represents major progress in the field of weld-penetration monitoring and is expected to provide more significant improvements during welding using pulsed current where the process becomes highly dynamic

Local Binary Patterns in Focal-Plane Processing. Analysis and Applications

Feature extraction is the part of pattern recognition, where the sensor data is transformed into a more suitable form for the machine to interpret. The purpose of this step is also to reduce the amount of information passed to the next stages of the system, and to preserve the essential information in the view of discriminating the data into different classes. For instance, in the case of image analysis the actual image intensities are vulnerable to various environmental effects, such as lighting changes and the feature extraction can be used as means for detecting features, which are invariant to certain types of illumination changes. Finally, classification tries to make decisions based on the previously transformed data. The main focus of this thesis is on developing new methods for the embedded feature extraction based on local non-parametric image descriptors. Also, feature analysis is carried out for the selected image features. Low-level Local Binary Pattern (LBP) based features are in a main role in the analysis. In the embedded domain, the pattern recognition system must usually meet strict performance constraints, such as high speed, compact size and low power consumption. The characteristics of the final system can be seen as a trade-off between these metrics, which is largely affected by the decisions made during the implementation phase. The implementation alternatives of the LBP based feature extraction are explored in the embedded domain in the context of focal-plane vision processors. In particular, the thesis demonstrates the LBP extraction with MIPA4k massively parallel focal-plane processor IC. Also higher level processing is incorporated to this framework, by means of a framework for implementing a single chip face recognition system. Furthermore, a new method for determining optical flow based on LBPs, designed in particular to the embedded domain is presented. Inspired by some of the principles observed through the feature analysis of the Local Binary Patterns, an extension to the well known non-parametric rank transform is proposed, and its performance is evaluated in face recognition experiments with a standard dataset. Finally, an a priori model where the LBPs are seen as combinations of n-tuples is also presentedSiirretty Doriast

Nonterrestrial utilization of materials: Automated space manufacturing facility

Four areas related to the nonterrestrial use of materials are included: (1) material resources needed for feedstock in an orbital manufacturing facility, (2) required initial components of a nonterrestrial manufacturing facility, (3) growth and productive capability of such a facility, and (4) automation and robotics requirements of the facility

Vision-based Monitoring System for High Quality TIG Welding

The current study evaluates an automatic system for real-time arc welding quality assessment and defect detection. The system research focuses on the identification of defects that may arise during the welding process by analysing the occurrence of any changes in the visible spectrum of the weld pool and the surrounding area. Currently, the state-of-the-art is very simplistic, involving an operator observing the process continuously. The operator assessment is subjective, and the criteria of acceptance based solely on operator observations can change over time due to the fatigue leading to incorrect classification. Variations in the weld pool are the initial result of the chosen welding parameters and torch position and at the same time the very first indication of the resulting weld quality. The system investigated in this research study consists of a camera used to record the welding process and a processing unit which analyse the frames giving an indication of the quality expected. The categorisation is achieved by employing artificial neural networks and correlating the weld pool appearance with the resulting quality. Six categories denote the resulting quality of a weld for stainless steel and aluminium. The models use images to learn the correlation between the aspect of the weld pool and the surrounding area and the state of the weld as denoted by the six categories, similar to a welder categorisation. Therefore the models learn the probability distribution of images’ aspect over the categories considered

ESTABLISHING THE FOUNDATION TO ROBOTIZE COMPLEX WELDING PROCESSES THROUGH LEARNING FROM HUMAN WELDERS BASED ON DEEP LEARNING TECHNIQUES

As the demand for customized, efficient, and high-quality production increases, traditional manufacturing processes are transforming into smart manufacturing with the aid of advancements in information technology, such as cyber-physical systems (CPS), the Internet of Things (IoT), big data, and artificial intelligence (AI). The key requirement for integration with these advanced information technologies is to digitize manufacturing processes to enable analysis, control, and interaction with other digitized components. The integration of deep learning algorithm and massive industrial data will be critical components in realizing this process, leading to enhanced manufacturing in the Future of Work at the Human-Technology Frontier (FW-HTF). This work takes welding manufacturing as the case study to accelerate its transition to intelligent welding by robotize a complex welding process. By integrate process sensing, data visualization, deep learning-based modeling and optimization, a complex welding system is established, with the systematic solution to generalize domain-specific knowledge from experienced human welder. Such system can automatically perform complex welding processes that can only be handled by human in the past. To enhance the system\u27s tracking capabilities, we trained an image segmentation network to offer precise position information. We incorporated a recurrent neural network structure to analyze dynamic variations during welding. Addressing the challenge of human heterogeneity in data collection, we conducted experiments illustrating that even inaccurate datasets can effectively train deep learning models with zero mean error. Fine-tuning the model with a small portion of accurate data further elevates its performance

Spatial Programming for Industrial Robots through Task Demonstration

We present an intuitive system for the programming of industrial robots using markerless gesture recognition and mobile augmented reality in terms of programming by demonstration. The approach covers gesture-based task definition and adaption by human demonstration, as well as task evaluation through augmented reality. A 3D motion tracking system and a handheld device establish the basis for the presented spatial programming system. In this publication, we present a prototype toward the programming of an assembly sequence consisting of several pick-and-place tasks. A scene reconstruction provides pose estimation of known objects with the help of the 2D camera of the handheld. Therefore, the programmer is able to define the program through natural bare-hand manipulation of these objects with the help of direct visual feedback in the augmented reality application. The program can be adapted by gestures and transmitted subsequently to an arbitrary industrial robot controller using a unified interface. Finally, we discuss an application of the presented spatial programming approach toward robot-based welding tasks

TOWARD INTELLIGENT WELDING BY BUILDING ITS DIGITAL TWIN

To meet the increasing requirements for production on individualization, efficiency and quality, traditional manufacturing processes are evolving to smart manufacturing with the support from the information technology advancements including cyber-physical systems (CPS), Internet of Things (IoT), big industrial data, and artificial intelligence (AI). The pre-requirement for integrating with these advanced information technologies is to digitalize manufacturing processes such that they can be analyzed, controlled, and interacted with other digitalized components. Digital twin is developed as a general framework to do that by building the digital replicas for the physical entities. This work takes welding manufacturing as the case study to accelerate its transition to intelligent welding by building its digital twin and contributes to digital twin in the following two aspects (1) increasing the information analysis and reasoning ability by integrating deep learning; (2) enhancing the human user operative ability to physical welding manufacturing via digital twins by integrating human-robot interaction (HRI). Firstly, a digital twin of pulsed gas tungsten arc welding (GTAW-P) is developed by integrating deep learning to offer the strong feature extraction and analysis ability. In such a system, the direct information including weld pool images, arc images, welding current and arc voltage is collected by cameras and arc sensors. The undirect information determining the welding quality, i.e., weld joint top-side bead width (TSBW) and back-side bead width (BSBW), is computed by a traditional image processing method and a deep convolutional neural network (CNN) respectively. Based on that, the weld joint geometrical size is controlled to meet the quality requirement in various welding conditions. In the meantime, this developed digital twin is visualized to offer a graphical user interface (GUI) to human users for their effective and intuitive perception to physical welding processes. Secondly, in order to enhance the human operative ability to the physical welding processes via digital twins, HRI is integrated taking virtual reality (VR) as the interface which could transmit the information bidirectionally i.e., transmitting the human commends to welding robots and visualizing the digital twin to human users. Six welders, skilled and unskilled, tested this system by completing the same welding job but demonstrate different patterns and resulted welding qualities. To differentiate their skill levels (skilled or unskilled) from their demonstrated operations, a data-driven approach, FFT-PCA-SVM as a combination of fast Fourier transform (FFT), principal component analysis (PCA), and support vector machine (SVM) is developed and demonstrates the 94.44% classification accuracy. The robots can also work as an assistant to help the human welders to complete the welding tasks by recognizing and executing the intended welding operations. This is done by a developed human intention recognition algorithm based on hidden Markov model (HMM) and the welding experiments show that developed robot-assisted welding can help to improve welding quality. To further take the advantages of the robots i.e., movement accuracy and stability, the role of the robot upgrades to be a collaborator from an assistant to complete a subtask independently i.e., torch weaving and automatic seam tracking in weaving GTAW. The other subtask i.e., welding torch moving along the weld seam is completed by the human users who can adjust the travel speed to control the heat input and ensure the good welding quality. By doing that, the advantages of humans (intelligence) and robots (accuracy and stability) are combined together under this human-robot collaboration framework. The developed digital twin for welding manufacturing helps to promote the next-generation intelligent welding and can be applied in other similar manufacturing processes easily after small modifications including painting, spraying and additive manufacturing