The Use of a Convolutional Neural Network in Detecting Soldering Faults from a Printed Circuit Board Assembly

Automatic Optical Inspection (AOI) is any method of detecting defects during a Printed Circuit Board (PCB) manufacturing process. Early AOI methods were based on classic image processing algorithms using a reference PCB. The traditional methods require very complex and inflexible preprocessing stages. With recent advances in the field of deep learning, especially Convolutional Neural Networks (CNN), automating various computer vision tasks has been established. Limited research has been carried out in the past on using CNN for AOI. The present systems are inflexible and require a lot of preprocessing steps or a complex illumination system to improve the accuracy. This paper studies the effectiveness of using CNN to detect soldering bridge faults in a PCB assembly. The paper presents a method for designing an optimized CNN architecture to detect soldering faults in a PCBA. The proposed CNN architecture is compared with the state-of-the-art object detection architecture, namely YOLO, with respect to detection accuracy, processing time, and memory requirement. The results of our experiments show that the proposed CNN architecture has a 3.0% better average precision, has 50% less number of parameters and infers in half the time as YOLO. The experimental results prove the effectiveness of using CNN in AOI by using images of a PCB assembly without any reference image, any complex preprocessing stage, or a complex illumination system. Doi: 10.28991/HIJ-2022-03-01-01 Full Text: PD

A Review and Analysis of Automatic Optical Inspection and Quality Monitoring Methods in Electronics Industry

Electronics industry is one of the fastest evolving, innovative, and most competitive industries. In order to meet the high consumption demands on electronics components, quality standards of the products must be well-maintained. Automatic optical inspection (AOI) is one of the non-destructive techniques used in quality inspection of various products. This technique is considered robust and can replace human inspectors who are subjected to dull and fatigue in performing inspection tasks. A fully automated optical inspection system consists of hardware and software setups. Hardware setup include image sensor and illumination settings and is responsible to acquire the digital image, while the software part implements an inspection algorithm to extract the features of the acquired images and classify them into defected and non-defected based on the user requirements. A sorting mechanism can be used to separate the defective products from the good ones. This article provides a comprehensive review of the various AOI systems used in electronics, micro-electronics, and opto-electronics industries. In this review the defects of the commonly inspected electronic components, such as semiconductor wafers, flat panel displays, printed circuit boards and light emitting diodes, are first explained. Hardware setups used in acquiring images are then discussed in terms of the camera and lighting source selection and configuration. The inspection algorithms used for detecting the defects in the electronic components are discussed in terms of the preprocessing, feature extraction and classification tools used for this purpose. Recent articles that used deep learning algorithms are also reviewed. The article concludes by highlighting the current trends and possible future research directions.Framework of the IQONIC Project; European Union’s Horizon 2020 Research and Innovation Program

A Hierarchical, Fuzzy Inference Approach to Data Filtration and Feature Prioritization in the Connected Manufacturing Enterprise

The current big data landscape is one such that the technology and capability to capture and storage of data has preceded and outpaced the corresponding capability to analyze and interpret it. This has led naturally to the development of elegant and powerful algorithms for data mining, machine learning, and artificial intelligence to harness the potential of the big data environment. A competing reality, however, is that limitations exist in how and to what extent human beings can process complex information. The convergence of these realities is a tension between the technical sophistication or elegance of a solution and its transparency or interpretability by the human data scientist or decision maker. This dissertation, contextualized in the connected manufacturing enterprise, presents an original Fuzzy Approach to Feature Reduction and Prioritization (FAFRAP) approach that is designed to assist the data scientist in filtering and prioritizing data for inclusion in supervised machine learning models. A set of sequential filters reduces the initial set of independent variables, and a fuzzy inference system outputs a crisp numeric value associated with each feature to rank order and prioritize for inclusion in model training. Additionally, the fuzzy inference system outputs a descriptive label to assist in the interpretation of the feature’s usefulness with respect to the problem of interest. Model testing is performed using three publicly available datasets from an online machine learning data repository and later applied to a case study in electronic assembly manufacture. Consistency of model results is experimentally verified using Fisher’s Exact Test, and results of filtered models are compared to results obtained by the unfiltered sets of features using a proposed novel metric of performance-size ratio (PSR)

Automating Fault Detection and Quality Control in PCBs: A Machine Learning Approach to Handle Imbalanced Data

Printed Circuit Boards (PCBs) are fundamental to the operation of a wide array of electronic devices, from consumer electronics to sophisticated industrial machinery. Given this pivotal role, quality control and fault detection are especially significant, as they are essential for ensuring the devices' long-term reliability and efficiency. To address this, the thesis explores advancements in fault detection and quality control methods for PCBs, with a focus on Machine Learning (ML) and Deep Learning (DL) techniques. The study begins with an in-depth review of traditional approaches like visual and X-ray inspections, then delves into modern, data-driven methods, such as automated anomaly detection in PCB manufacturing using tabular datasets. The core of the thesis is divided into three specific tasks: firstly, applying ML and DL models for anomaly detection in PCBs, particularly focusing on solder-pasting issues and the challenges posed by imbalanced datasets; secondly, predicting human inspection labels through specially designed tabular models like TabNet; and thirdly, implementing multi-classification methods to automate repair labeling on PCBs. The study is structured to offer a comprehensive view, beginning with background information, followed by the methodology and results of each task, and concluding with a summary and directions for future research. Through this systematic approach, the research not only provides new insights into the capabilities and limitations of existing fault detection techniques but also sets the stage for more intelligent and efficient systems in PCB manufacturing and quality control

FICS PCB X-ray: A dataset for automated printed circuit board inter-layers inspection

Advancements in computer vision and machine learning breakthroughs over the years have paved the way for automated X-ray inspection (AXI) of printed circuit boards (PCBs). However, there is no standard dataset to verify the capabilities and limitations of such advancements in practice due to the lack of publicly available datasets for PCB X-ray inspection. Furthermore, there is a lack of diverse PCB X-ray datasets that encompass images from X-ray Computed Tomography (CT). To address the lack of data, we developed the first comprehensive publicly available dataset, FICS PCB X-ray, to aid in the development of robust PCB-AXI methodologies. The dataset consists of diverse images from the tomographic image domain, along with challenging cases of unaligned, raw X-ray data of PCBs. Further, the dataset contains projection data and the reconstructed volume which is converted into a Tiff stack. Annotated X-ray layer images are also available for image processing and machine learning tasks. This paper summarizes the existing databases and their limitations, and proposes a new dataset, FICS PCB X-ray\u27\u27

Multi-stage Inspection of Laser Welding Defects using Machine Learning

As welding processes become faster and components consist of many more welds compared to previous applications, there is a need for fast but still precise quality inspection. The aim of this paper is to compare already existing approaches, namely single-sensor systems (SSS) and multi-sensor systems (MSS) with a proposed cascaded system (CS). The introduced CS is characterized by the fact that not all available data are analyzed, but only cleverly selected ones. The different approaches consisting of neural networks are compared in terms of their accuracy and computational effort. The data are recorded from scratch and include two common sensor systems for quality control, namely a photodiode (PD) and a high-speed camera (HSC). As a result, when the CS makes half of the final decisions based on a SSS with PD signals and the other half based on a SSS with HSC images, the estimated computational effort is reduced by almost 50% compared to the SSS with HSC images as input. At the same time, the accuracy decreases only by 0.25% to 95.96%. Additionally, based on the CS, a general cascaded system (GCS) for quality inspection is proposed

Predictive model-based quality inspection using Machine Learning and Edge Cloud Computing

© 2020 The Authors The supply of defect-free, high-quality products is an important success factor for the long-term competitiveness of manufacturing companies. Despite the increasing challenges of rising product variety and complexity and the necessity of economic manufacturing, a comprehensive and reliable quality inspection is often indispensable. In consequence, high inspection volumes turn inspection processes into manufacturing bottlenecks. In this contribution, we investigate a new integrated solution of predictive model-based quality inspection in industrial manufacturing by utilizing Machine Learning techniques and Edge Cloud Computing technology. In contrast to state-of-the-art contributions, we propose a holistic approach comprising the target-oriented data acquisition and processing, modelling and model deployment as well as the technological implementation in the existing IT plant infrastructure. A real industrial use case in SMT manufacturing is presented to underline the procedure and benefits of the proposed method. The results show that by employing the proposed method, inspection volumes can be reduced significantly and thus economic advantages can be generated

Inspection of the integrity of surface mounted integrated circuits on a printed circuit board using vision

Machine vision technology has permeated many areas of industry, and automated inspection systems are playing increasingly important roles in many production processes. Electronic manufacturing is a good example of the integration of vision based feedback in manufacturing and the assembly of surface mount PCBs is typical of the technology involved. There are opportunities to use machine vision during different stages of the surface mount process. The problem in the inspection of solder joints on surface mount printed circuit board is much more difficult than many other inspection problems. In this thesis, an approach for inspecting surface mounted integrated circuits (SMICs) is presented. It is based on the variance of intensity values of pixels in an image. This method is able to cope with 4 kinds of soldering defects in SMICs. A set of modules for the system is proposed. The computer program which performs the image processing and analyzing has been written in C. It has been linked with a number of image processing routines from MAVIS1 to perform some image processing tasks, and the result is a compact executable module which works under MS-DOS2 3.30

Automatic surface mount solder joints inspection

This thesis reports the research results on automatic inspection of solder joints on printed circuit boards. The previous work on this subject has been advanced significantly in the following three aspects. With the support of AT&T Bell Lab, the most updated surface mount solder joints are inspected in this work instead of larger simulation solder joints or traditional through hole solder joints in the previous work. A small set of features is extracted for surface mount solder joints in both infrared and visual light inspection. A new image processing software named Khoros has been applied to improve the quality of images. It has been demonstrated that infrared imaging technique can identify solder joints of surface mount printed circuit boards according to their solder volumn. The correct classification rate was found to be in the range of 89% to 100%. For the sample joints provided by AT&T Bell Laboratory, reasonably good inspection results have been obtained. The experimental results demonstrate that infrared imaging technique can be utilized to discriminate solder joints on surface mount printed circuit boards with different solder volumes quite reliably