Design of automatic vision-based inspection system for solder joint segmentation

Purpose: Computer vision has been widely used in the inspection of electronic components. This paper proposes a computer vision system for the automatic detection, localisation, and segmentation of solder joints on Printed Circuit Boards (PCBs) under different illumination conditions. Design/methodology/approach: An illumination normalization approach is applied to an image, which can effectively and efficiently eliminate the effect of uneven illumination while keeping the properties of the processed image the same as in the corresponding image under normal lighting conditions. Consequently special lighting and instrumental setup can be reduced in order to detect solder joints. These normalised images are insensitive to illumination variations and are used for the subsequent solder joint detection stages. In the segmentation approach, the PCB image is transformed from an RGB color space to a YIQ color space for the effective detection of solder joints from the background. Findings: The segmentation results show that the proposed approach improves the performance significantly for images under varying illumination conditions. Research limitations/implications: This paper proposes a front-end system for the automatic detection, localisation, and segmentation of solder joint defects. Further research is required to complete the full system including the classification of solder joint defects. Practical implications: The methodology presented in this paper can be an effective method to reduce cost and improve quality in production of PCBs in the manufacturing industry. Originality/value: This research proposes the automatic location, identification and segmentation of solder joints under different illumination conditions

Inspection System And Method For Bond Detection And Validation Of Surface Mount Devices Using Sensor Fusion And Active Perception

A hybrid surface mount component inspection system which includes both vision and infrared inspection techniques to determine the presence of surface mount components on a printed wiring board, and the quality of solder joints of surface mount components on printed wiring boards by using data level sensor fusion to combine data from two infrared sensors to obtain emissivity independent thermal signatures of solder joints, and using feature level sensor fusion with active perception to assemble and process inspection information from any number of sensors to determine characteristic feature sets of different defect classes to classify solder defects.Georgia Tech Research Corporatio

Application Of Machine Vision On Solder Joint Inspection Using Orthogonal And Oblique Views

Machine vision has been widely deployed in many industrial applications. However, the use of machine vision to perform solder joint inspection of electronic assembly has yet to reach the desired maturity level. Solder joint surfaces are minute, curved and the shapes tend to vary greatly with the soldering conditions. These characteristics have posted a challenging task to develop an effective machine vision with acceptable level of classification accuracy. This research aims to investigate a new methodology of inspecting solder joint through analysis of the combined image from two viewing directions; one from orthogonal view while the other from oblique view. The concept based on the physics of surface tension, contact angle and slant angle of solder joint fillet

A Review and Analysis of Automatic Optical Inspection and Quality Monitoring Methods in Electronics Industry

Electronics industry is one of the fastest evolving, innovative, and most competitive industries. In order to meet the high consumption demands on electronics components, quality standards of the products must be well-maintained. Automatic optical inspection (AOI) is one of the non-destructive techniques used in quality inspection of various products. This technique is considered robust and can replace human inspectors who are subjected to dull and fatigue in performing inspection tasks. A fully automated optical inspection system consists of hardware and software setups. Hardware setup include image sensor and illumination settings and is responsible to acquire the digital image, while the software part implements an inspection algorithm to extract the features of the acquired images and classify them into defected and non-defected based on the user requirements. A sorting mechanism can be used to separate the defective products from the good ones. This article provides a comprehensive review of the various AOI systems used in electronics, micro-electronics, and opto-electronics industries. In this review the defects of the commonly inspected electronic components, such as semiconductor wafers, flat panel displays, printed circuit boards and light emitting diodes, are first explained. Hardware setups used in acquiring images are then discussed in terms of the camera and lighting source selection and configuration. The inspection algorithms used for detecting the defects in the electronic components are discussed in terms of the preprocessing, feature extraction and classification tools used for this purpose. Recent articles that used deep learning algorithms are also reviewed. The article concludes by highlighting the current trends and possible future research directions.Framework of the IQONIC Project; European Union’s Horizon 2020 Research and Innovation Program

Multi-stage Inspection of Laser Welding Defects using Machine Learning

As welding processes become faster and components consist of many more welds compared to previous applications, there is a need for fast but still precise quality inspection. The aim of this paper is to compare already existing approaches, namely single-sensor systems (SSS) and multi-sensor systems (MSS) with a proposed cascaded system (CS). The introduced CS is characterized by the fact that not all available data are analyzed, but only cleverly selected ones. The different approaches consisting of neural networks are compared in terms of their accuracy and computational effort. The data are recorded from scratch and include two common sensor systems for quality control, namely a photodiode (PD) and a high-speed camera (HSC). As a result, when the CS makes half of the final decisions based on a SSS with PD signals and the other half based on a SSS with HSC images, the estimated computational effort is reduced by almost 50% compared to the SSS with HSC images as input. At the same time, the accuracy decreases only by 0.25% to 95.96%. Additionally, based on the CS, a general cascaded system (GCS) for quality inspection is proposed

Understanding, Modeling and Predicting Hidden Solder Joint Shape Using Active Thermography

Characterizing hidden solder joint shapes is essential for electronics reliability. Active thermography is a methodology to identify hidden defects inside an object by means of surface abnormal thermal response after applying a heat flux. This research focused on understanding, modeling, and predicting hidden solder joint shapes. An experimental model based on active thermography was used to understand how the solder joint shapes affect the surface thermal response (grand average cooling rate or GACR) of electronic multi cover PCB assemblies. Next, a numerical model simulated the active thermography technique, investigated technique limitations and extended technique applicability to characterize hidden solder joint shapes. Finally, a prediction model determined the optimum active thermography conditions to achieve an adequate hidden solder joint shape characterization. The experimental model determined that solder joint shape plays a higher role for visible than for hidden solder joints in the GACR; however, a MANOVA analysis proved that hidden solder joint shapes are significantly different when describe by the GACR. An artificial neural networks classifier proved that the distances between experimental solder joint shapes GACR must be larger than 0.12 to achieve 85% of accuracy classifying. The numerical model achieved minimum agreements of 95.27% and 86.64%, with the experimental temperatures and GACRs at the center of the PCB assembly top cover, respectively. The parametric analysis proved that solder joint shape discriminability is directly proportional to heat flux, but inversely proportional to covers number and heating time. In addition, the parametric analysis determined that active thermography is limited to five covers to discriminate among hidden solder joint shapes. A prediction model was developed based on the parametric numerical data to determine the appropriate amount of energy to discriminate among solder joint shapes for up to five covers. The degree of agreement between the prediction model and the experimental model was determined to be within a 90.6% for one and two covers. The prediction model is limited to only three solder joints, but these research principles can be applied to generate more realistic prediction models for large scale electronic assemblies like ball grid array assemblies having as much as 600 solder joints

Automatic surface defect quantification in 3D

Three-dimensional (3D) non-contact optical methods for surface inspection are of significant interest to many industrial sectors. Many aspects of manufacturing processes have become fully automated resulting in high production volumes. However, this is not necessarily the case for surface defect inspection. Existing human visual analysis of surface defects is qualitative and subject to varying interpretation. Automated 3D non-contact analysis should provide a robust and systematic quantitative approach. However, different 3D optical measurement technologies use different physical principles, interact with surfaces and defects in diverse ways, leading to variation in measurement data. Instrument s native software processing of the data may be non-traceable in nature, leading to significant uncertainty about data quantisation. Sub-millimetric level surface defect artefacts have been created using Rockwell and Vickers hardness testing equipment on various substrates. Four different non-contact surface measurement instruments (Alicona InfiniteFocus G4, Zygo NewView 5000, GFM MikroCAD Lite and Heliotis H3) have been utilized to measure different defect artefacts. The four different 3D optical instruments are evaluated by calibrated step-height created using slipgauges and reference defect artefacts. The experimental results are compared to select the most suitable instrument capable of measuring surface defects in robust manner. This research has identified a need for an automatic tool to quantify surface defect and thus a mathematical solution has been implemented for automatic defect detection and quantification (depth, area and volume) in 3D. A simulated defect softgauge with a known geometry has been developed in order to verify the implemented algorithm and provide mathematical traceability. The implemented algorithm has been identified as a traceable, highly repeatable, and high speed solution to quantify surface defect in 3D. Various industrial components with suspicious features and solder joints on PCB are measured and quantified in order to demonstrate applicability