Vision is one of the most powerful forms of non-contact sensing for monitoring and control of manufacturing processes. However, processes involving an arc plasma or flame such as welding or thermal spraying pose particularly challenging problems to conventional vision sensing and processing techniques. The arc or plasma is not typically limited to a single spectral region and thus cannot be easily filtered out optically. This paper presents an innovative vision sensing system that uses intense stroboscopic illumination to overpower the arc light and produce a video image that is free of arc light or glare and dedicated image processing and analysis schemes that can enhance the video images or extract features of interest and produce quantitative process measures which can be used for process monitoring and control. Results of two SBIR programs sponsored by NASA and DOE and focusing on the application of this innovative vision sensing and processing technology to thermal spraying and welding process monitoring and control are discussed

Agapakis, John E.

Bolstad, Jon

English

NASA Technical Reports Server

VISION-AIDED MONITORING & CONTROL OF THERMAL SPRAY, SPRAY FORMING, 
AND WELDING PROCESSES 
John E. Agapakis Jon Bolstad 
Auto111atix Inc. & Control Vision Inc. 
Billerica, MA 01821 
ABSTRACT 
Vision is one of the most powerful forms of non-contact sensing for monitoring and control of manufactusang 
processes. However, processes involving an arc plasma or flame such as welding or thermal spraying pose 
particularly challenging problems to conventional vision sensing and processing techniques. The arc or plasma is not 
typically limited to a single spectral region and thus cannot be easily filtered out optically. This paper presents an 
innovative vision sensing system that uses intense stroboscopic illumination to overpower the arc light mad produce 
a video image that is free of arc light or glare and dedicated image processing and analysis schemes that can enhance 
the video images or extract features of interest and produce quantitative process measures which can be used for 
process monitoring and control. Results of two SBIR programs sponsored by NASA and DOE and fmushg an the 
application of this innovative vision sensing and processing technology to thermal spraying and welding process 
monitoring and control are discussed. 
INTRODUCTION 
Thermal spray processes, electric arc welding, laser welding, and other energy-intensive high luminosi~ i i n d a r s ~  
processes are normally quite difficult to monitor with the human eye because the luminous volume of the plasma or 
flame obscures the details of the behavior of the solid or molten material in the heat affected area F d e m o r e ,  when 
one attempts to use a photographic or video camera, the viewing is further degraded by the extreme bH-ighmess 
variation across the image area, making it impossible to achieve proper exposure throughout the image - except 
possibly for small areas of comparable brightness. Optical filtering with neutral density fdters -like the ones used 
in a welder's helmet - do not particularly help either. In the case of arc welding, one can expect to see a bright 
fireball at the center of the welding pool, but most of the detail at the edge of the welding pool and in the area of the 
welding seam and groove will be lost in relative darkness. With thermal spray processes, the injection m d  flow of 
particles within the plasrna flame is almost totally concealed by the extreme brightness of the flame. bn addition, ejne 
particles quickly accelerate to very high speeds, making their detection even more difficult. 
Over the last six years, Control Vision has developed a unique viewing system capable of overcoming the extreme 
variation in scene brightness created by high luminosity phenomena such as flames, arcs, or plasma and 
electronically producing a video image virtually free of arc glare. This patented system incorporates external 
illumination in the form of intense pulsed laser light. The laser light reflected from the site is for an instant much 
brighter than either the direct or reflected light of the process. The system exploits this situation by viewhg b e  
process with a special-purpose video camera equipped with a CCD video sensor and a very high sped elec~onic 
shutter synchronized with the laser flash and the framing of the video sensor. ' 
This innovative viewing system has already been used in a variety of applications in order to provide visual fedback 
to an operator for in-process monitoring during production or to allow observation of phenomena not oherwfse 
visible during process research and development. In order to incorporate such a sensor in automated process 
monitoring or control applications, it is necessary to develop approaches for processing and analyzing the images 
produced by the sensor and extracting features that can be used for process control. Such customized image 
processing and analysis capabilities have been developed by Automatix on standard computer plagom 'baed nnachine 
vision systems. These developments are being pursued under two SBIR programs sponsored by NASA and DOE. 
The DOE sponsored SBIR program focuses on welding process monitoring and control. Under this effort, Control 
Vision is developing a next-generation vision sensing system that is more compact and can be readily interfaced to a 
https://ntrs.nasa.gov/search.jsp?R=19930016415 2020-03-24T07:27:21+00:00Z
robot or other automated welding equipment, whereas Automatix is developing vision processing techniques for 
image enkaracement and image analysis for the detection of important weld features, such as the weld seam, molten 
mela4 pudde, or keyhole and the calculation of relevant dimensional measurements, such as seam-to-puddle offsets or 
puddle gesmeq.  
The NASA sponsored SBIR program focuses on thermal spraying process monitoring and analysis. Special 
twhniques are been developed to suppress the intense light of the flame or plasma and to allow the visualization of 
h e  powder parlicle flow carried by the flameor plasma with particular emphasis placed on low-pressure or vacuum 
spraying processes inside chambers. In conjunction to these sensing developments pursued by Control Vision, 
inulonah is developing image processing and analysis schemes for the automated extraction of quantitative process 
measures such as particle distribution, velocity, and flow rates from the video images produced by the sensor. Figure 
9 schemaGcally depicts the viewing system used in conjunction with the image processing and analysis system for 
computes-assisted visual monitoring of thermal spraying. 
Plasma Feed 
or Flame Nozzle 
Torch 
Pulsed 
Nitrogen 
Fiberoptic Cable I/, 
Laser L
Fiberoptic 
Feed Unit 
Ima$e Processing \ LaserICamera -I-\ I 
Analysis System Control Unit 
& Filters 
Figure 1 Typical experimental setup for thermal spraying process monitoring 
In both welding and thermal spraying, the developed vision sensing and processing schemes can be used both during 
process development for parameter selection and process understanding or modeling as well as during production for 
real time process monitoring, process alarming, and ultimately process conwl (Figure 2). 
Level Automatic Process 
Control 
Computer Assisted 
Process Monitoring 
j/ /i R a e s s  Research I 
Figure 2 Different levels of 
application for the proposed 
monitoring systems 
Specifically, such systems can be used during process research and development to evaluate the effect of different 
process parameters, explore alternative hardware designs, allow for better record keeping during process 
experimentation, and validate analytical process models. During actual production, these systems can be used for 
computer-assisted monitoring by computing different process measures and providing this information to an operator 
through graphical displays, for automatic alarming by comparing these computed measures to expected values and 
tolerances, and ultimately for real-time process control where the process parameters are automatically adjusted on h e  
basis of the process feedback. 
VISION SENSING 
As mentioned above, the patented Control Vision viewing system incorporates external illumination in the form of 
intense pulsed laser light to overcome the extreme variation in scene brightness created by the flame or arc. The laser 
energy is transported to the welding site through a fiberoptic cable. A xenon flash lamp has also been used as a 
source of intense pulsed light. The system is also equipped with a narrow-band optical filter to match ehe laser 
wavelength and further suppress the arc lighting. The net combination of both temporal and specual filtering resuits 
in a video image that is free of all of the adverse arc Lighting effects. 
Welding Applications 
Figure 3. Gas Metal Arc welding viewed using the Figure 4 Gas Tungsten Arc welding viewed using the 
stroboscopic vision sensing approach stroboscopic vision sensing approach 
The viewing system has been applied to a variety of conventional welding processes including Gas Tungsten Arc 
Welding (GTAW), Plasma Arc Welding (PAW), Variable Polarity Arc Welding (VPPAW), Gas Metal Arc Welding 
(GMAW) as well as high energy welding processes such as Electron Beam Welding (EBW) and Laser Weldiing &W). 
The VPPAW and GTAW processes are of particular interest to NASA since they are extensively used for the 
production of flight hardware such as the Space Shuttle External Tank and Main Engine. 
Phenomena not easily seen through a conventional viewing system are readily visualized using the s~oboscopic 
viewing approach making it a unique tool for welding process R&D. Figure 3 clearly shows metal boplet I;amsfer, 
weld puddle depression, spatter formation, cathodic cleaning, and related phenomena observed during GlaIPPI wel&ng 
of an Aluminum-Bronze alloy. The three frames of Figure 5 provide a clear back side view of the keyhole drilled by 
the plasma during VPPA welding. No evidence of the intense plasma remains in the image. The viewing system can 
be used for process monitoring and help a supervisory operator detect setup problems as in Figure 5@) where the 
keyhole has moved away from the weld seam or process stability problems such as the unstable keyhole fornation 
of Figure 5(c), 
Figwe 5 Backside view of the keyhole drilled by the plasma during VPPA welding 
Figure 6. DC plasma spraying showing the effect of varying powder feed parameters in the particle distribution 
within the plasma plume. As the process parameters are changed the particles ride on the plasma plume (top left and 
right), overshoot (bottom left), and are optimally camed by the plasma Wttom right). 
The visson sensing rechnology presented in this paper has been used in other manufacturing processes involving a 
phsrna. arc or ibn-ie. A family of such processes of particular interest to NASA and the aerospace industq is 
thermal spraying which is widely used today for spray coatings and spray forming applications. Themd coatkg 
pmesses (flame powder, flame wire, wire arc, plasma, etc) utilizing a heat source (chemical combus~on or elecisrc 
arc) to generate an intense flame, arc, or plasma which in turn is used to heat to a molten or elevated empenme 
solid state, accelerate them to a high , and carry and deposit them on a workpiece. Such coahgs can minimize 
or impede environmenlal effects, improve wear resistance, develop abradable seals or thermal barriers, &ow warn- 
part build-up, or control tolerances. Thermal spraying is also used in forming applications whereby molten material 
is sprayed layered and rapidly solidified onto a semi-hished product. This approach reduces the nmber of steps 
required to arrive at a finishied product, compared to more mditional processes. The resultant malerid Ass exhibits 
superior micro-structures by eliminating macro-segregation and promoting finer micro-structures. 
Figure 7. Active viewing of twin-wire arc spray in the Figure 8 Passive viewing of wire arc spray 
immediate area of the gun nozzle downstream from the gun nozzle 
eters can be adjusted so that the plasma can 'be completely removed from the image. This is 
demonstrated in Figure 7 showing twin wire arc spray process in the immediate region of the gun. At the opposite 
end, the laser strobing can also be removed for completely passive viewing. This is only meaningful the region 
away from the gun where no evidence of the flame exists. Passive viewing of the plasma is shown in! Figure 8, Note 
that the particles now are shown as light streaks corresponding to the short path they traverse during the sensor 
integration time. The length of the streaks is thus proportional to the velocity of the particles. 
Figure 9 Twin images 
fired sequentially. 
obtained using 
h o h e r  appmach for visualizing the velocity field that has been explored is the use of two lasers or strobes fired in 
rapid succession. This results in two particle images shifted with respect to one another by an amount that is 
p r s p d o n d  to the local direction and size of the velocity vector. This is depicted in Figure 9. 
Other hemd spraying processes that can benefit from the unique capabilities of the sensor include single wire arc 
sprayhirng, ITA hardfacing, and gas atomization. Figure 10 shows two digitized images from a videotaped single wire 
m w u e n e .  
Figure I0 Single wire spraying images 
VISION PROCESSING 
Under the previously mentioned research programs, Automatix is developing image processing and analysis 
appoaches for pmessing and analyzing the video images produced by the sensor. These capabilities are developed on 
commerc% machine vision systems implemented on industrially hardened standard computer platforms using the 
vision appficacion development environment highlighted in [Schurr 19911. 
Figure 11 Unprocessed image showing remaining Figure 12 Sample image processing operation to 
evidence sf the plasma plume remove the plasma plume 
Image prwessing refers to operations that produce a new image from an original image so as to enhance the 
snb j~ f ive  innage quality, reduce noise, accentuate features of interest, or eliminate image formation problems. Such 
prmssing may be useful to make it easier for a human operator to interpret the image or as a preprocessing step 
prior to m y  subsequent image analysis. Image enhancement or restoration may also be necessary when analyzing a 
recorded sequence of weld images. In such a case, it is no longer possible to change the viewing or picme W n g  
parameters. 
For example in thermal spraying applications, the main objective of image enhancement is to process caphwd 
images and clean up any remaining evidence of the plasma and/or isolate the particles or the plasma plume. This 
separation of the particles from the plasma can be based on the fact that the plume has a lower spatial fpequency 
content than that of the particles. This difference is clearly demonstrated by looking at the intensity &s$-ibudon 
through the particles and the plasma (Figure 11). Figures 12 and 13 show examples of successful innage prmesshg 
to isolate the powder particles. 
Figure 13 Another example of image processing to remove the plasma plume from the image. 
Imane Analvsis 
The main objective of image analysis is to extract features of interest and other quantitative measuremenls from an 
image. In the context of welding applications, image analysis is used to: 
- to detect the weld joint ahead of the welding torch (its location and size can be used for real-time torch & h c e ,  
pre-weld joint inspection, and real-time welding process parameter adjustment) [Agapakis 19901; 
- to detect the molten metal puddle or keyhole under the arc (its shape and size can be used for process anoim~ng 
and real-time process control); 
- to detect the solidified weld bead behind the arc and puddle (its shape and dimensions may be used for p s t w l d  
inspection or process control); 
- to detect other features of potential interest such as the welding torch, electrode, filler metal wire, molten metal 
droplets, or the welding arc and plasma. 
In thermal spraying applications, image analysis is used: 
- to detect the particle detection through image segmentation 
- to compute particle area and other particle characteristics 
- to compute measures of particle spatial distribution 
- to determine the mean particle path 
- to compute the particle velocity distribution 
- to extract the geometry of the plasma plume or flame 
Examples of results of such image analysis for welding applications are shown in Figure 14, where both h e  centraid 
of the detected keyhole or puddle and the centerline of the detected joint seam are shown graphically. The dismce 
between the puddlekeyhole centroid and of the seam centerline can be computed during the process and used for seam 
tracking. 
Sample results of image analysis for thermal spraying applications are shown in Figures 15 and 16. 
Figure 14 Results of image 
analysis during VPPA and GTA 
welding demonstrating the 
detection of the seam centerline 
and of the keyhole or puddle 
centroid. 
Figm 15 kernination of particle spatial distribution 
ACKNOWLEDGMENTS 
The work prwnred in this paper has been pamally sponsored by a Department of Energy Phase I1 SBIR grant and a 
NASA Phase I S B R  contract. In addition to the authors, Richard Lu, Phil Lamoreaux, and Thad Hoffman, and John 
hgerqulan have contributed to these research efforts. 
REFERENCES 
Bagaplkis, JE. ,  Katz, J.M., Friedman, J.M., and Epstein GN. ,  "Vision-Aided Robotic Welding: An Approach and a 
Flexible Implemenration" International Journal of Robotics Research, Vol9, No 5, October 1990, pp 17-34. 
S c h m ,  6.- "A Multi-level Application Development Environment for Machine Vision System Integration, prcx. 
, Detroit, MI, October, 1991. 


Vision-aided Monitoring and Control of Thermal Spray, Spray Forming, and Welding Processes

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server