Semantic segmentation in flaw detection

The paper presents a review of study on detection and classification of defects using semantic image segmentation based on convolutional neural networks. Taking into account the revealed general features of flaw detection tasks of various industries related to the lack of a large marked data set and the need to detect defects of small sizes. The convolutional neural network of the u-net architecture was chosen as the basis for the decision support system. Testing of this architecture on several datasets yielded positive results regardless of the area of use. © 2020 IOP Publishing Ltd. All rights reserved

Impurities Detection in Intensity Inhomogeneous Edible Bird’s Nest (EBN) Using a U-Net Deep Learning Model

As an important export, cleanliness control on edible bird’s nest (EBN) is paramount. Automatic impurities detection is in urgent need to replace manual practices. However, effective impurities detection algorithm is yet to be developed due to the unresolved inhomogeneous optical properties of EBN. The objective of this work is to develop a novel U-net based algorithm for accurate impurities detection. The algorithm leveraged the convolution mechanisms of U-net for precise and localized features extraction. Output probability tensors were then generated from the deconvolution layers for impurities detection and positioning. The U-net based algorithm outperformed previous image processing-based methods with a higher impurities detection rate of 96.69% and a lower misclassification rate of 10.08%. The applicability of the algorithm was further confirmed with a reasonably high dice coefficient of more than 0.8. In conclusion, the developed U-net based algorithm successfully mitigated intensity inhomogeneity in EBN and improved the impurities detection rate

이상 탐지를 이용한 불분명한 표면을 가지는 웨이퍼 표면에서의 크랙 검출

학위논문(석사) -- 서울대학교대학원 : 공과대학 기계공학부, 2023. 2. 김도년.Defect detection is a crucial process to improve the productivity and quality of products in the industry. However, defects in the nanoscale-manufacture become difficult to detect, since the shapes of the defects are complex and noises and unclean backgrounds cover the defects frequently. It is laborious and inefficient to utilize human resources for defect detection because the rate of defects in the industry is extremely low and it requires professional knowledge to detect the defects in some cases. Applying an anomaly detection model as a defect detector in the industry is the best solution which will save time and human resources. However, there are many difficulties to apply the data-driven based anomaly detection model to real industry inspection. In our research, we found that our target product wafers contain resin bleed, which hinders detecting cracks on the wafer surfaces. The resin bleed impedes the anomaly detection on wafers because it is similar to the cracks in the wafer and at the same time it belongs to the normal components. In this paper, we propose a method to improve the crack detection performance of the anomaly detection model by enhancing the edge information of cracks. Our model achieved 96.7% at the image level AUROC and 98.6% at pixel level AUROC by improving 4.5% and 2.0% respectively without additional annotation.결함 탐지는 산업에서 제품의 생산성이나 질을 향상시키는데 매우 중요한 과정입니다. 그러나 나노 스케일 공정에서 결함의 형상이나 노이즈, 불분명한 배경 같은 요소들은 결함 탐지를 어렵게 만듭니다. 산업에서 결함의 비율은 매우 작고 결함 탐지를 위해서 전문적인 지식을 필요로 하는 경우도 많기 때문에 사람이 직접 결함 탐지를 수행하는 것은 매우 소모적이고 비효율적입니다. 그러므로 산업에서 컴퓨터 비전 기반 결함 탐지 모델을 활용하는 것은 시간이나 물적, 인적 자원을 절약하고 부족한 결함 데이터 문제도 해결할 수 있는 훌륭한 방법입니다. 그러나 데이터 기반 이상 탐지 모델을 실제 산업 검사에 활용하는 것은 많은 어려움을 가지고 있습니다. 해당 연구에서 우리는 결함 탐지의 목표로 하는 웨이퍼 제품에서 resin bleed 라는 크랙 검출을 방해하는 요소를 확인했습니다. resin bleed는 정상 요소에 속하지만 머신 비전의 관점에서는 크랙과 비슷합니다. 이러한 특징들은 데이터 셋 전체에 분포되어 있는 Resin bleed가 결함 탐지 모델이 크랙들을 정상 요소들과 분명하게 구별할 수 있는 능력을 저해합니다. 이 논문에서 우리는 크랙의 엣지 성분을 강화하여 이상 탐지 모델이 크랙을 더 잘 검출할 수 있도록 하는 방법을 제시합니다. 저희가 제안하는 방법들은 결함 탐지 성능을 이미지 레벨에서 96.7%, 픽셀 레벨에서 98.6% 성능을 달성했습니다. 저희가 달성한 성과들은 기존 이상 탐지 모델을 사용했을 때와 비교하여 추가 데이터 주석 없이 이미지 레벨에서 4.5%, 픽셀 레벨에서 2.0% 성능 향상한 결과입니다.Abstract 1 Table of contents 2 List of tables, figures 3 Chapter 1. Introduction 5 1.1 Anomaly Detection 7 1.2 Wafer Defect Detection 9 1.3 Crack Detection 10 Chapter 2. Edge-Enhanced Anomaly Detection 12 2.1 Edge Information Extraction 13 2.2 Edge-Enhanced Features into a Memory Bank 17 2.3 Effective Memory Bank Subset Search 19 2.4 Algorithm for Anomaly Detection and Localization 20 Chapter 3. Model Validation on Wafer Dataset 24 3.1 Experiments Detail 3.1.1 Datasets and Training Details 24 3.1.2 Evaluation Metrics for Anomaly Detection 25 3.2 Anomaly Detection on Wafer Surface 25 3.3 Result Analysis 40 3.3 Comparison Study for Selecting Edge Features 44 Chapter 4. Conclusions 50 Bibliography 51 Abstract in Korean 55석

Deep CNN-Based Automated Optical Inspection for Aerospace Components

ABSTRACT The defect detection problem is of outmost importance in high-tech industries such as aerospace manufacturing and is widely employed using automated industrial quality control systems. In the aerospace manufacturing industry, composite materials are extensively applied as structural components in civilian and military aircraft. To ensure the quality of the product and high reliability, manual inspection and traditional automatic optical inspection have been employed to identify the defects throughout production and maintenance. These inspection techniques have several limitations such as tedious, time- consuming, inconsistent, subjective, labor intensive, expensive, etc. To make the operation effective and efficient, modern automated optical inspection needs to be preferred. In this dissertation work, automatic defect detection techniques are tested on three levels using a novel aerospace composite materials image dataset (ACMID). First, classical machine learning models, namely, Support Vector Machine and Random Forest, are employed for both datasets. Second, deep CNN-based models, such as improved ResNet50 and MobileNetV2 architectures are trained on ACMID datasets. Third, an efficient defect detection technique that combines the features of deep learning and classical machine learning model is proposed for ACMID dataset. To assess the aerospace composite components, all the models are trained and tested on ACMID datasets with distinct sizes. In addition, this work investigates the scenario when defective and non-defective samples are scarce and imbalanced. To overcome the problems of imbalanced and scarce datasets, oversampling techniques and data augmentation using improved deep convolutional generative adversarial networks (DCGAN) are considered. Furthermore, the proposed models are also validated using one of the benchmark steel surface defects (SSD) dataset

Machine learning for advanced characterisation of silicon solar cells

Improving the efficiency, reliability, and durability of photovoltaic cells and modules is key to accelerating the transition towards a carbon-free society. With tens of millions of solar cells manufactured every day, this thesis aims to leverage the available characterisation data to identify defects in solar cells using powerful machine learning techniques. Firstly, it explores temperature and injection dependent lifetime data to characterise bulk defects in silicon solar cells. Machine learning algorithms were trained to model the recombination statistics’ inverse function and predict the defect parameters. The proposed image representation of lifetime data and access to powerful deep learning techniques surpasses traditional defect parameter extraction techniques and enables the extraction of temperature dependent defect parameters. Secondly, it makes use of end-of-line current-voltage measurements and luminescence images to demonstrate how luminescence imaging can satisfy the needs of end-of-line binning. By introducing a deep learning framework, the cell efficiency is correlated to the luminescence image and shows that a luminescence-based binning does not impact the mismatch losses of the fabricated modules while having a greater capability of detecting defects in solar cells. The framework is shown in multiple transfer learning and fine-tuning applications such as half-cut and shingled cells. The method is then extended for automated efficiency-loss analysis, where a new deep learning framework identifies the defective regions in the luminescence image and their impact on the overall cell efficiency. Finally, it presents a machine learning algorithm to model the relationship between input process parameters and output efficiency to identify the recipe for achieving the highest solar cell efficiency with the help of a genetic algorithm optimiser. The development of machine learning-powered characterisation truly unlocks new insight and brings the photovoltaic industry to the next level, making the most of the available data to accelerate the rate of improvement of solar cell and module efficiency while identifying the potential defects impacting their reliability and durability

Semi-automatic liquid filling system using NodeMCU as an integrated Iot Learning tool

Computer programming and IoT are the key skills required in Industrial Revolution 4.0 (IR4.0). The industry demand is very high and therefore related students in this field should grasp adequate knowledge and skill in college or university prior to employment. However, learning technology related subject without applying it to an actual hardware can pose difficulty to relate the theoretical knowledge to problems in real application. It is proven that learning through hands-on activities is more effective and promotes deeper understanding of the subject matter (He et al. in Integrating Internet of Things (IoT) into STEM undergraduate education: Case study of a modern technology infused courseware for embedded system course. Erie, PA, USA, pp 1–9 (2016)). Thus, to fulfill the learning requirement, an integrated learning tool that combines learning of computer programming and IoT control for an industrial liquid filling system model is developed and tested. The integrated learning tool uses NodeMCU, Blynk app and smartphone to enable the IoT application. The system set-up is pre-designed for semi-automation liquid filling process to enhance hands-on learning experience but can be easily programmed for full automation. Overall, it is a user and cost friendly learning tool that can be developed by academic staff to aid learning of IoT and computer programming in related education levels and field