The Use of a Convolutional Neural Network in Detecting Soldering Faults from a Printed Circuit Board Assembly

Automatic Optical Inspection (AOI) is any method of detecting defects during a Printed Circuit Board (PCB) manufacturing process. Early AOI methods were based on classic image processing algorithms using a reference PCB. The traditional methods require very complex and inflexible preprocessing stages. With recent advances in the field of deep learning, especially Convolutional Neural Networks (CNN), automating various computer vision tasks has been established. Limited research has been carried out in the past on using CNN for AOI. The present systems are inflexible and require a lot of preprocessing steps or a complex illumination system to improve the accuracy. This paper studies the effectiveness of using CNN to detect soldering bridge faults in a PCB assembly. The paper presents a method for designing an optimized CNN architecture to detect soldering faults in a PCBA. The proposed CNN architecture is compared with the state-of-the-art object detection architecture, namely YOLO, with respect to detection accuracy, processing time, and memory requirement. The results of our experiments show that the proposed CNN architecture has a 3.0% better average precision, has 50% less number of parameters and infers in half the time as YOLO. The experimental results prove the effectiveness of using CNN in AOI by using images of a PCB assembly without any reference image, any complex preprocessing stage, or a complex illumination system. Doi: 10.28991/HIJ-2022-03-01-01 Full Text: PD

A convolutional neural network based deep learning methodology for recognition of partial discharge patterns from high voltage cables

It is a great challenge to differentiate partial discharge (PD) induced by different types of insulation defects in high-voltage cables. Some types of PD signals have very similar characteristics and are specifically difficult to differentiate, even for the most experienced specialists. To overcome the challenge, a convolutional neural network (CNN)-based deep learning methodology for PD pattern recognition is presented in this paper. First, PD testing for five types of artificial defects in ethylene-propylene-rubber cables is carried out in high voltage laboratory to generate signals containing PD data. Second, 3500 sets of PD transient pulses are extracted, and then 33 kinds of PD features are established. The third stage applies a CNN to the data; typical CNN architecture and the key factors which affect the CNN-based pattern recognition accuracy are described. Factors discussed include the number of the network layers, convolutional kernel size, activation function, and pooling method. This paper presents a flowchart of the CNN-based PD pattern recognition method and an evaluation with 3500 sets of PD samples. Finally, the CNN-based pattern recognition results are shown and the proposed method is compared with two more traditional analysis methods, i.e., support vector machine (SVM) and back propagation neural network (BPNN). The results show that the proposed CNN method has higher pattern recognition accuracy than SVM and BPNN, and that the novel method is especially effective for PD type recognition in cases of signals of high similarity, which is applicable for industrial applications

Fine-grained Classification of Solder Joints with {\alpha}-skew Jensen-Shannon Divergence

Solder joint inspection (SJI) is a critical process in the production of printed circuit boards (PCB). Detection of solder errors during SJI is quite challenging as the solder joints have very small sizes and can take various shapes. In this study, we first show that solders have low feature diversity, and that the SJI can be carried out as a fine-grained image classification task which focuses on hard-to-distinguish object classes. To improve the fine-grained classification accuracy, penalizing confident model predictions by maximizing entropy was found useful in the literature. Inline with this information, we propose using the {\alpha}-skew Jensen-Shannon divergence ({\alpha}-JS) for penalizing the confidence in model predictions. We compare the {\alpha}-JS regularization with both existing entropyregularization based methods and the methods based on attention mechanism, segmentation techniques, transformer models, and specific loss functions for fine-grained image classification tasks. We show that the proposed approach achieves the highest F1-score and competitive accuracy for different models in the finegrained solder joint classification task. Finally, we visualize the activation maps and show that with entropy-regularization, more precise class-discriminative regions are localized, which are also more resilient to noise. Code will be made available here upon acceptance.Comment: Submitted to IEEE Transactions on Components, Packaging and Manufacturing Technolog

FICS-PCB: A Multi-Modal Image Dataset for Automated Printed Circuit Board Visual Inspection

Over the years, computer vision and machine learn- ing disciplines have considerably advanced the field of automated visual inspection for Printed Circuit Board (PCB-AVI) assurance. However, in practice, the capabilities and limitations of these advancements remain unknown because there are few publicly accessible datasets for PCB visual inspection and even fewer that contain images that simulate realistic application scenarios. To address this need, we propose a publicly available dataset, “FICS-PCB”, to facilitate the development of robust methods for PCB-AVI. The proposed dataset includes challenging cases from three variable aspects: illumination, image scale, and image sensor. This dataset consists of 9,912 images of 31 PCB samples and contains 77,347 annotated components. This paper reviews the existing datasets and methodologies used for PCB- AVI, discusses challenges, describes the proposed dataset, and presents baseline performances using feature engineering and deep learning methods for PCB component classification

JujubeNet: A high-precision lightweight jujube surface defect classification network with an attention mechanism

Surface Defect Detection (SDD) is a significant research content in Industry 4.0 field. In the real complex industrial environment, SDD is often faced with many challenges, such as small difference between defect imaging and background, low contrast, large variation of defect scale and diverse types, and large amount of noise in defect images. Jujubes are naturally growing plants, and the appearance of the same type of surface defect can vary greatly, so it is more difficult than industrial products produced according to the prescribed process. In this paper, a ConvNeXt-based high-precision lightweight classification network JujubeNet is presented to address the practical needs of Jujube Surface Defect (JSD) classification. In the proposed method, a Multi-branching module using Depthwise separable Convolution (MDC) is designed to extract more feature information through multi-branching and substantially reduces the number of parameters in the model by using depthwise separable convolutions. What’s more, in our proposed method, the Convolutional Block Attention Module (CBAM) is introduced to make the model concentrate on different classes of JSD features. The proposed JujubeNet is compared with other mainstream networks in the actual production environment. The experimental results show that the proposed JujubeNet can achieve 99.1% classification accuracy, which is significantly better than the current mainstream classification models. The FLOPS and parameters are only 30.7% and 30.6% of ConvNeXt-Tiny respectively, indicating that the model can quickly and effectively classify JSD and is of great practical value

Photoreversible interconversion of a phytochrome photosensory module in the crystalline state.

A major barrier to defining the structural intermediates that arise during the reversible photointerconversion of phytochromes between their biologically inactive and active states has been the lack of crystals that faithfully undergo this transition within the crystal lattice. Here, we describe a crystalline form of the cyclic GMP phosphodiesterases/adenylyl cyclase/FhlA (GAF) domain from the cyanobacteriochrome PixJ in Thermosynechococcus elongatus assembled with phycocyanobilin that permits reversible photoconversion between the blue light-absorbing Pb and green light-absorbing Pg states, as well as thermal reversion of Pg back to Pb. The X-ray crystallographic structure of Pb matches previous models, including autocatalytic conversion of phycocyanobilin to phycoviolobilin upon binding and its tandem thioether linkage to the GAF domain. Cryocrystallography at 150 K, which compared diffraction data from a single crystal as Pb or after irradiation with blue light, detected photoconversion product(s) based on Fobs - Fobs difference maps that were consistent with rotation of the bonds connecting pyrrole rings C and D. Further spectroscopic analyses showed that phycoviolobilin is susceptible to X-ray radiation damage, especially as Pg, during single-crystal X-ray diffraction analyses, which could complicate fine mapping of the various intermediate states. Fortunately, we found that PixJ crystals are amenable to serial femtosecond crystallography (SFX) analyses using X-ray free-electron lasers (XFELs). As proof of principle, we solved by room temperature SFX the GAF domain structure of Pb to 1.55-Å resolution, which was strongly congruent with synchrotron-based models. Analysis of these crystals by SFX should now enable structural characterization of the early events that drive phytochrome photoconversion