MathematicS in action

Xavier Jonsson

Jérôme Lelong

Rare event simulation for electronic circuit design

Thomas Deschatre

Joseph Mikael

Deep combinatorial optimisation for optimal stopping time problems: application to swing options pricing.

Annabelle Collin

Mélanie Prague

Philippe Moireau

Estimation for dynamical systems using a population-based Kalman filter – Applications in computational biology

G. Balarac

F. Basile

P. Bénard

F. Bordeu

J.-B. Chapelier

L. Cirrottola

G. Caumon

C. Dapogny

P. Frey

A. Froehly

G. Ghigliotti

R. Laraufie

G. Lartigue

C. Legentil

R. Mercier

V. Moureau

C. Nardoni

S. Pertant

M. Zakari

Tetrahedral remeshing in the context of large-scale numerical simulation and high performance computing

Mike Ludkovski

Regression Monte Carlo for Impulse Control

Meriam El Mansour

Emmanuel Lépinette

Robust discrete-time super-hedging strategies under AIP condition and under price uncertainty

Jean-François Boulanger

Franck Corset

Franck Iutzeler

MathematicSIn ActionJean-François Boulanger, Franck Corset,Franck Iutzeler & Jérôme LelongClassifying and explaining defects with small data for the semiconductor industryVolume 11 (2022), p. 109-114.https://doi.org/10.5802/msia.20© Les auteurs, 2022.Cet article est mis à disposition selon les termesde la licence Creative Commons attribution 4.0.http://creativecommons.org/licenses/by/4.0/C EN T R EMER S ENN EMathematicS In Action est membre duCentre Mersenne pour l’édition scientifique ouvertehttp://www.centre-mersenne.org/e-ISSN : 2102-5754MathematicS In ActionVol. 11, 109-114 (2022)Classifying and explaining defects with small data for thesemiconductor industryJean-François Boulanger ∗Franck Corset ∗∗Franck Iutzeler ∗ ∗ ∗Jérôme Lelong †∗ UnitySC , 611 rue Aristide Bergès, Z.A. de Pré Millet, 38330, Montbonnot-Saint-Martin,FranceE-mail address: jf.boulanger@unity-sc.com∗∗ Univ. Grenoble Alpes, CNRS, Grenoble INP, LJK, 38000 Grenoble, FranceE-mail address: franck.corset@univ-grenoble-alpes.fr∗ ∗ ∗ Univ. Grenoble Alpes, CNRS, Grenoble INP, LJK, 38000 Grenoble, FranceE-mail address: franck.iutzeler@univ-grenoble-alpes.fr† Univ. Grenoble Alpes, CNRS, Grenoble INP, LJK, 38000 Grenoble, FranceE-mail address: jerome.lelong@univ-grenoble-alpes.fr.AbstractIn this work, we present an automatic classifier of wafer defects for the semiconductor industry. Hopefullydefects are rare, but this puts the classifying problem in a small data context. We propose a fast and fullyreproducible approach based on decision trees. The main interest of using decision trees lies in obtaining ahighly explicable classifier, which makes the origin of the defect easy to identify.1. IntroductionIn this note, we provide an overview of our work on the classification of wafer defects as part ofan industrial collaboration between Univ. Grenoble Alpes and UnitySC .Context. Maimosine (https://www.maimosine.fr/) is a federative research structure hostedby Laboratoire Jean Kuntzmann, Université Grenoble Alpes and CNRS, that aims at promotingthe development of cross-disciplinary projects based on mathematical modelling and simulation.One of Maimosine’s main goals is to boost collaborations between academic research and inno-vative companies in the Grenoble area.UnitySC develops and manufactures high precision tools for the semiconductor industry world-wide. Research focused, the company’s strategy is to design and integrate cutting-edge opticalsensors and advanced algorithms to achieve the best measurement performance. The applicationsof the company automatically detect and classify small-dimension defects on customer productsand measure feature sizes for process control.The goal of this collaboration is to classify manufacturing defects on wafers from real dataprovided by the company. The data comes from physical measurements and various quantitiesautomatically derived from proprietary image processing techniques. So far, this classificationtask was performed manually by highly qualified engineers which was very costly and time-consuming; in addition, the produced classification suffered from consistency issues dependingon who was in charge of the classification. This work aims at replacing the manual work by asupervised learning approach of the defect classes based on data already labeled by experts. Asan additional requirement, the approach should return a set of constraint boxes on the inputfeatures, each constraint box corresponds to the prediction of a given defect. This additional109Jean-François Boulanger, Franck Corset, et al.requirement has emerged during preliminary discussions with the company, stemming frominterpretability and real-time needs on the prediction process.Learning a classifier. A classification problem consists in learning a mapping h from someinput space X to one of K possibles classes. To do so, from a data set consisting of n pairs ofexample/class {xi, yi} ∈ X × {1, . . . , K}, i = 1, . . . , n, the objective is to find a mapping thatfits well the given data and also generalizes well to other pairs, provided that they are not toodifferent from the original data.1Lots of models and associated algorithms exist to find such predictors (Nearest Neighbors,Support Vector Machines, Logistic Regression, Random Forests, Neural Networks to name afew; see [2, 3] for a review and [4] for a Python implementation and comparisons). Beyondtheir performance, these models differ a lot in terms of prediction functions. Indeed, if a NeuralNetwork is trained, the prediction consists in feeding it the new example to classify, which canbe costly in terms of storage and prediction time on small processing units. Furthermore, thesekind of methods are sometimes decried in industrial applications for their lack of interpretation.Linear methods such as SVM or logistic regression predict a class from a linear combination of theinput coordinates/features, which is easier to interpret in particular when K = 2, but may sufferfrom performance limitations. Decision trees offer a good compromise between performance andinterpretability. These methods consist in sequentially partitioning the input space by findingthe feature and value that separate best the different classes.Specificity of the task. Interpretability was indeed a key concern in our work. The classifierproduced by the supervised learning approach had to be understandable by the senior engineersat Unity who used to carry out this task manually. Thus, our goal was to output simple rulesin order to enable them to quickly identify the production step responsible for the defect in themanufacturing process. Moreover, the classifier had to be encoded in XML using only boundaryconstraints on the input features. Then, it was obvious that the classifier had to expressedin terms of “boxes” written as a product of one dimension intervals. Decision tree classifiersare precisely designed to produce such classification rules and were thus our workhorse in thiscollaboration.2. Modelling and Methodology2.1. Data setThe data provided by the company is a set of features obtained from a multi-stage imageprocessing on 1190 defective wafers, representing 13 common types of defects. Each wafer isphotographed using three acquisition modes leading to as many data layers. Then, these layersare analyzed to detect potential defects. They are presented as small thumbnails from which apredefined set of features are measured. Our features correspond to these 195 extracted valuesand the defect class is annotated manually by a senior engineer.Obviously, some defects may not be visible on all layers, which implies that the data setobtained by aggregating the data from all the layers has a lot of missing data. Note that in thiscontext a missing data is actually very meaningful since a value is missing when the associatedfeature could not be measured because it was not present in the thumbnail. As almost all thefeatures were actually representing lengths, it was crystal clear that missing data had to be setto 0. Another possibility would be to add for every layer a Boolean variable stating the presenceof absence of the defect in the given layer in order to obtain a more easily readable classifier,but this solution was found to be less robust while offering virtually the same performance.1From a mathematical point of view, we can model the data set as a sampling from some unknown distribution.Then, we expect our predictor to perform well on other pairs sampled from the same distribution.110Classifying defects with small dataClassification results highly depend on the training data. In particular, anyone who intendsto use a classification algorithm must be fully aware that any bias in the training data willautomatically be passed to the classifier. This phenomenon is well-known by researchers workingon automatic classification and naturally tends to expand when using reinforcement learning asthe algorithm learns from its own decisions. As a guarantee against these bias issues or outliers,we advise to remove from the input dataset any defect class with too few samples, which wouldproduce classification rules without any real foundation, especially since some of these defectswere flagged as borderline by the experts. In practice, we remove any defect class with less than10 samples in our tests.2.2. Supervised learning and decision treesConsider a training set built of n labeled samples Dn = {(xi, yi) ∈ X × K, i = 1, . . . , n}, whereX is the feature space and K the types or classes of defects. A classification can be modeled asa function h : X −→ K, which associates a defect class to an element of X (typically a subsetof Rp where p is the number of features). To measure the fitness of the classifier, we usuallydefine a merit function. For instance, it can be its accuracy defined as the fraction of correctlyclassified dataM(Dn; h) =1nn∑i=1`(xi, yi; h) where `(x, y; h) ={1 if h(x) = y0 otherwiseIn this work, a classifier can be identified to a partition of X in which every element of thepartition (referred to as a box) can be written as a product of intervals, ie. X is split into disjointsets B1, . . . ,Bb taking the formBj = {x ∈ X , j1 ≤ x[1] ≤ j1, . . . , jp ≤ x[p] ≤ jp}where x[k] represents the k-th element of x (hence the k-th feature among the p features).Then, for every j, Bj is associated to a given defect class. Let us denote this mapping bya : {B1, . . . ,Bb} → K. In this context, the classifier h can be writtenh(x) =b∑j=1a(Bj)1x∈Bj .Decision Trees [1] are known to produce a partition of the feature space of the form {B1, . . . ,Bb}along with the mapping a. Indeed, they consist in iteratively partitioning the input space alongone feature at a time, resulting in a tree-like structure whose leaves represent the boxes of ourclassifier. Their main parameters are how to find the best possible splitting thresholds, whichis done by looking at the split that increases most the purity of the leaves measured in termsof Gini coefficient or entropy; and how many leaves to output, which can be set by either con-straining the depth of the tree or its width. Although, decision tree classifiers are sub-optimal,their accuracy is already very good and they are obtained using cross-validation.The number of boxes b (or equivalently of leaves in our case) plays a major role in the balancebetween goodness of fit and generalization. Indeed, with b sufficiently large, one can form a boxaround every input point to achieve a perfect accuracy; however, the generalization to unseendata can be arbitrarily bad. Thus, we fix a maximal number of boxes B by cross-validationbefore finding the optimal classifier with at most B boxes on the full dataset. As an illustration,we display in Figure 2.1 a decision tree for our problem with 6 boxes (the number of boxes isactually much greater in practice).111Jean-François Boulanger, Franck Corset, et al.Figure 2.1. Example of a Decision Trees with 6 leaves/boxes2.3. Performance & ValidationDecision trees are prone to over fitting, that is why we cross validate the parameters of the tree(maximal width, depth, minimal number of samples in a leaf, impurity measure) by performing10-fold cross validation.We compared the obtained accuracy to other classifiers in Table 2.1. We observe that theDecision Tree offers a much better accuracy than the Logistic regression, this is mostly due tothe highly non-linear aspect of the problem.Table 2.1. 10-fold cross validation performance of several methods. We displaythe average accuracy on the 10 folds as well as the standard deviation.Method Accuracy Commentsave. sdev.Decision Tree 0.605 (0.083) method usedLogistic regression 0.420 (0.076) linear classifier minimizing the cross-entropy lossRandom Forest 0.635 (0.072) ensemble of 100 random decision treesGradient Boosting 0.720 (0.086) gradient boosted ensemble of 100 decision treesWe also compare decision trees to ensemble methods. These methods are based on the ag-gregation of several trees and are known to overcome the tendency to over fitting of decisiontrees; they also usually offer good performances. Thus, they can be seen as a performance targetin our case. Compared to decision trees, theses approaches perform respectively 3% and 11%better. However, these methods aggregate several trees (usually 100) and thus produce decisionfrontiers that are much less interpretable and implementable, making them unfit for our targetproblem.3. Impact for the companyThe method described above has been integrated in the processing tools of UnitySC . A learnermodule is first presented to the operator to label imagelets of defects detected on a wafer likerepresented on Figure 3.1. This process is done offline during the recipe creation. Then, from thelearned classification tree, an automatic classification module for production has been integrated.It allows process engineers to see in a glance the results of defect classification as represented on112Classifying defects with small dataFigure 3.1. Interface of the learner module interface. Example of learned de-fects: contaminant(a) Contaminant (b) Particle (c) StainFigure 3.2. Image examples of 3 different defect classes.Figure 3.3 where the location and labels of defects are displayed on the wafer map. The grayleveland binary mask imagelets of the defects are also accessible through that interface.The example presented across the different figures corresponds to an automatic defect clas-sification applied to data acquired by the edge inspection module developed at UnitySC . Thismodule composed of several cameras acquires graylevel images of the edge of wafers. Then, thedefects are detected through an automatic image processing framework.For this example more than 80000 defects are detected on the edge of the test wafer (which isa highly contaminated one compared to what is usually inspected). Depending on the nature ofdefects, several classes are defined as represented on Figure 3.2. Defects of different nature arepresent and correspond to process residue or contamination by the environment. Automaticallymaking the difference between the cases is often a key for the final customers.113Jean-François Boulanger, Franck Corset, et al.Figure 3.3. Interface of the classification presented to the final user. Locationsand labels are displayed on the wafer maps. Examples of defect grey image andbinary mask are visible.When tackling application cases where many singularities (i.e. area on wafers declared as de-fects on the detection phase) are present on the surface of wafers, the software module developedduring this project presents several advantages in terms of reliability and flexibility.4. ConclusionIn this work, we have designed a reliable classification tree which reached the specified perfor-mance in term of precision. The method allows us to save time and increases the reliability of theclassification process. The produced classification tree eases the work of application engineersand avoids them to waste time on complex data visualization and manual classification tree setup. The effort is actually focused on acquisition and labelling, which are straightforward tasks.The result is also reproducible and maintainable since it does not depend on the subjectivechoice of an operator on what feature to select and what threshold to set.References[1] Leo Breiman, Jerome Friedman, Richard A. Olshen, and Charles J. Stone. Classification and regressiontrees. Routledge, 2017.[2] László Györfi, Michael Kohler, Adam Krzyżak, and Harro Walk. A distribution-free theory of non-parametric regression, volume 1. Springer, 2002.[3] Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The elements of statistical learning: datamining, inference, and prediction. Springer, 2009.[4] Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, OlivierGrisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machinelearning in Python. J. Mach. Learn. Res., 12:2825–2830, 2011.114

Classifying and explaining defects with small data for the semiconductor industry

Diala Wehbe

Nicolas Wicker

Baydaa Al-Ayoubi

Luc Moulinier

Fixed-Size Determinantal Point Processes Sampling For Species Phylogeny

Jean-Baptiste Bellet

Gérard Berginc

Heuristic imaging from generic projections: backprojection outside the range of the Radon transform

Simon Garnotel

Stéphanie Salmon

Olivier Balédent

Numerical Modeling of the Intracranial Pressure using Windkessel Models

Not a member yet