Coleman, Sonya

Kerr, Dermot

Sweeney, Terence

English

Automated inspection has become a vital part of quality control in many industries, including duringsemiconductor wafer production. Current processes often focus on finding defects by comparing imageswith a ‘golden’ image pixel to pixel or, more recently, using shallow or deep learning based approaches.We present an alternative approach which uses the Bag of Visual Words technique to determine localfeatures that correspond to specific defects within a wafer image, known as a custom vocabulary. Usingthis custom vocabulary combined with machine learning, we can characterise and accurately classifydefects found on wafer images

Ulster University's Research Portal

     Defect Exclusive Custom Vocabulary for Classification Anonymous Submission Anonymous Affiliation   Abstract Automated inspection has become a vital part of quality control in many industries, including during semiconductor wafer production. Current processes often focus on finding defects by comparing images with a ‘golden’ image pixel to pixel or, more recently, using shallow or deep learning based approaches. We present an alternative approach which uses the Bag of Visual Words technique to determine local features that correspond to specific defects within a wafer image, known as a custom vocabulary. Using this custom vocabulary combined with machine learning, we can characterise and accurately classify defects found on wafer images.  Keywords: Defect Detection, Local Features, Classification, Bag of Visual Words, Machine Vision  1 Introduction  Semiconductor wafers are a component used in most electronic devices, including phones and hard drive media. During the manufacture of these wafers’ inspection is vital to detect defects and ensure high quality. There is a multitude of methods that have been proposed for detecting these defects, with many techniques focusing on defects present across the whole wafer. In this case, when defects are detected they are marked on a wafer bin map (Figure 1(a)) to identify the total amount of defects present. This approach has been tested extensively [Ooi 2013, Mital, 1991] and is very good when looking for widespread defects across a production line and removing an irreparable product early in the process. However, it is sometimes desirable to not just determine the location of a defect but also to classify the defect type as some defects can be repaired with a cleaning phase, increasing overall wafer yield on the production line and reducing waste, which is critical in today’s competitive world. Classifying types of defects is very difficult using the wafer bin map, however high-resolution images of individual defects on single dies are often taken across the production line. An example of a high-resolution image is illustrated in Figure 1 (b) and a high-resolution defect image in Figure 1(c).    (a) (b) (c) Figure 1: (a) Wafer bin map with detected defects coloured blue, (b) Single high-resolution die (Golden) image and (c) a scratch defect image   When considering the use of die images, most previous work is focussed on the use of global image features.     Tobin’s [Tobin, 2001] content based image retrieval golden image comparison is an example of a method commonly employed by most of the prolific Automated Defect Classification (ADC) machines [Tarasemi, 2019, Chou 1997] with it commonly being known as the ‘Golden Image’ approach. The images used in the experiments presented in this paper are captured by an ADC machine known as the Rudolph NSX105. The Rudolph NSX105 [Tarasemi, 2019] is a commonly used industry standard inspection device which uses the golden image approach. An initial set of golden images are manually selected and added to the inspection system which then uses its initial stage camera to strobe over the wafer, comparing captured images with the corresponding set of golden images. Hence, every single time a wafer product is updated, or a new wafer product developed, a new set of golden images must be created, and the system updated. Another critical problem with the NSX105’s inspection process is that while it can determine a defect at a specific location, it cannot determine the type of defect that has been found on the die; hence the severity of the defect is unknown. This may result in more serious defect types, such as corrosion damage on critical parts, going unnoticed until later in the production process or products being removed from the production line with defects that are not critical which can be costly. Therefore, in this paper, we propose defect exclusive visual words for defect classification by using local image features. The combination of these custom visual word vocabularies along with machine learning, enables accurate defect classification which is a promising step towards an automated inspection algorithm. 2  Methodology  2. 1 Bag of Visual Words  The proposed ADC system is based on the use of the Bag of Visual Words method. This is an extension of the Bag of Words (BoW) text retrieval method making it suitable for use with image data. When using the BoW technique on a text document, a normalised histogram of word counts is computed as well as a sparse-term vector, where each bin corresponds to a term in the vocabulary. In the context of image data, this technique [Csurka, 2004] enables the generalisation of local image feature descriptors which are similar. 2.2 Support Vector Machines  Support vector machines are a machine learning classifier that find the most efficient hyperplane to separate data into a number of classes. It can do this by utilising three different types of mathematical kernel functions that take the data and transform it into a useful classification metric. The kernel types are: linear, polynomial and radial basis function (RBF). Linear is often useful for binary classification tasks. The polynomial kernel is a popular approach often used in image processing. The RBF kernel is often seen as a general-purpose kernel.  In addition, a penalty parameter, known as C, is used to adjust how the SVM avoids misclassification of each training example. This parameter is useful when working with datasets where features are homogenous, such as the images used in the experiments here, as we can optimise the classification response. Support Vector Machines have been used in conjunction with Bag of Visual words in previous research [Henschel, 2014] with promising results but have not been used in combination with Custom Vocabularies or been used with variation of the C parameter. 2.3 Custom Vocabulary A custom vocabulary is an augmentation of Bag of Visual Words, where the visual codebook that is created is augmented or pruned to focus on the features of most interest in the image. Examples of this approach include using two codebooks [Devi, 2017], where two vocabularies are created using different training set classes before being tested in order to observe which vocabulary returns the highest accuracy for each testing class. Therefore, selecting the features which are important results in a stronger final codebook, which can be seen as comparable to a boosting classifier.  The proposed custom vocabulary is based on the use of the SURF feature detector and descriptor as previous work found that it outperforms SIFT for this purpose [Sweeney 2019].  Although the results from this experiment were promising we observed that when using full images, several visual words focussed on background features rather than defect features. Hence, we decided to produce a defect-only image dataset by removing the background and focussing on only the defect. The original 648*494 image is segmented into 35 images of 100*100 pixels in size. We then create a defect only dataset by using a subset of these images. An example of a scratch defect image with cropping locations is illustrated in Figure 2(a) and a selected defect-only image is illustrated in Figure 2(b).              (a) (b) Figure 2–Example of the grid cropping system (a) and the resultant image (b)       In the proposed methodology the SURF interest point detector is used to obtain key-points 𝑘𝑛 and corresponding SURF descriptors 𝑑𝑛 where 𝑖 = 1 … 𝑛 such that a keypoint is represented as: 𝑘𝑖 = (𝑥𝑖 , 𝑦𝑖 , 𝑑𝑖), where 𝑥 and 𝑦 are the coordinates of a point in an image. The SURF keypoint descriptors are of 64 dimensions. An image feature set 𝑆 can be represented by the set of local key-point descriptors such that  𝑆𝐼 = {𝑘1, 𝑘2, … , 𝑘𝑛}. where 𝐼 = 1 … 𝑚 and 𝑚 is the number of images in the image set. The BoVW algorithm 𝐵 is considered to quantize the descriptor 𝑑 ∈ 𝑅𝐼  𝐵: 𝑅𝐼 → [1, 𝐾]𝑑 → 𝐵(𝑑). The 𝐵 assigns descriptor 𝑑 ∈ 𝑅𝐼 to the appropriate cluster 𝐾, where each cluster represents a visual word and the set of visual words is the initial defect vocabulary. Using the cropped images, we use all the words as they are all focussed on the defect only.  With the use of this automated cropping approach the total number of detected key-points kn is restricted to be between 10-80 due to the size of the defect-only images (100x100px), thus the number of clusters k becomes more specialised and the number of images in the image set m grows from 70 images to 316 in this case.  3  Classification results      The methodology is applied to two classes, 70 scratch defect images and 100 control non-defect images. For testing the system, we reserve 20 images from each class and the remainder is used to train the system.  The scratch defect images are subsequently cropped, resulting in 316 defect-only images. The BoVW methodology is applied to obtain the defect exclusive custom vocabulary. Next, we apply the custom vocabulary to the complete (uncropped) images for both the control non-defect and scratch defect classes and use the resulting BoVW histograms as input to a Support Vector Machine to train a binary classifier as scratch or no defect. System performance is then tested using the 20 scratch defect and 20 control non-defect training images. The overall classification results are recorded in Table 1.   SVM   Original approach [Sweeney, 2019] Defect Exclusive Custom Vocabulary Linear C=1 50% 50% Linear C=10 50% 50% Linear C=100 75% 95% Poly C=1 50% 50% Poly C=10 62% 87% Poly C=100 50% 95% RBF C=1 75% 80% RBF C=10 80% 95% RBF C=100 85% 97%  Table 1 – Accuracy Results for SVM  Table 1 shows the overall classification results of the SVM where the 3 kernel types, Linear, Polynomial and Radial Basis Function were used along with 3 different variations of the penalty parameter C to observe which of these combinations returned the most promising results. The results demonstrate that an overall improvement in performance is obtained when a defect exclusive custom vocabulary is utilised, compared with a general vocabulary [Sweeney, 2019] for a full image method which contains background information. As seen from Table 1, the overall accuracy from the proposed approach is significantly higher than the previous method, resulting in and overall classification accuracy of 97% using a Radial Basis Function kernel compared with an accuracy of 85% in the previous work. 4  Conclusion and Further work This paper presents an efficient approach for the development of a custom vocabulary for defects in semiconductor wafers by utilizing the Bag of Visual Words approach.  The custom vocabulary is combined with a number of support vector machine algorithms to determine defects in the semiconductor wafers. The results for this approach demonstrate high accuracy for one defect type and therefore further work with focus on making this a multiclass problem with further defect types being included.   Acknowledgement  This work was funded by a DfE CAST scholarship in collaboration with Seagate Technology. We would also like to thank Seagate Technology for providing the image dataset used in the research. References [Tarasemi, 2019] Tarasemi, "Rudolph August NSX 105 Automated Defect Inspection," 2019. [Online]. Available: https://www.tarasemi.com/product/rudolph-august-nsx-105-automated-defect-inspection/. [Accessed 2 June 2019]. [Csurka, 2004]  G. Csurka, "Visual categorization with bags of keypoints," in Workshop on Statistical learning in computer vision, 2004.  [Hentschel and Sack, 2014] C. Hentschel and H. Sack, "Does one size really fit all? Evaluating classifiers in Bag-of-Visual-Words classification," in 14th International Conference on Knowledge Technologies and Data-driven Business., Graz, Austria, 2014.  [Mital and Tobin, 1991]  D. Mital and E. Teoh, "Computer based wafer inspection system," in Proceeding of international conference on industrial electronics, control and instrumentation, Kobe, 1991.  [Ooi, 2013] M. Ooi, "Defect cluster recognition system for fabricated semiconductor wafers," Engineering Applications of Artificial Intelligence, pp. 1029-1043, 2013.  [Sweeney, 2019] T. Sweeney, S. Coleman and D. Kerr, "A Machine Learning Approach to Wafer Defect Classification using Bag of Visual Words," in Irish Machine Vision and Image Processing Conference. 2019, Dublin, 2019.  [Devi et al., 2017] S. Devi et al., "Better object recognition using bag of visual word model with compact vocabulary," in 13th International Conference on Emerging Technologies (ICET), Islamabad, 2017.  [Chou, 1997] P. Chou et al., "Automatic defect classification for semiconductor manufacturing," Machine Vision and Applications, vol. 9, no. 4, pp. 201-214, 1997.  [Tobin et al., 2001] K. Tobin, et al., "Integrated applications of inspection data in the semiconductor manufacturing environment," Metrology-based Control for Micro-Manufacturing, pp. 31-41, 2001.   

Defect Exclusive Custom Vocabulary for Classification

https://pure.ulster.ac.uk/ws/files/87539197/IMVIP2020_1_.pdf

Defect Exclusive Custom Vocabulary for Classification

Abstract

Similar works

Full text

Available Versions

Ulster University's Research Portal