포토리소그래피 검사 시스템의 이미지 분할을 위한 새로운 깊은 아키텍처

학위논문(박사) -- 서울대학교대학원 : 융합과학기술대학원 융합과학부(지능형융합시스템전공), 2021.8. 홍성수.In semiconductor manufacturing, defect detection is critical to maintain high yield. Typically, the defects of semiconductor wafer may be generated from the manufacturing process. Most computer vision systems used in semiconductor photolithography process inspection still have adopt to image processing algorithm, which often occur inspection faults due to sensitivity to external environment changes. Therefore, we intend to tackle this problem by means of converging the advantages of image processing algorithm and deep learning. In this dissertation, we propose Image Segmentation Detector (ISD) to extract the enhanced feature-maps under the situations where training dataset is limited in the specific industry domain, such as semiconductor photolithography inspection. ISD is used as a novel backbone network of state-of-the-art Mask R-CNN framework for image segmentation. ISD consists of four dense blocks and four transition layers. Especially, each dense block in ISD has the shortcut connection and the concatenation of the feature-maps produced in layer with dynamic growth rate for more compactness. ISD is trained from scratch without using recently approached transfer learning method. Additionally, ISD is trained with image dataset pre-processed by means of our designed image filter to extract the better enhanced feature map of Convolutional Neural Network (CNN). In ISD, one of the key design principles is the compactness, plays a critical role for addressing real-time problem and for application on resource bounded devices. To empirically demonstrate the model, this dissertation uses the existing image obtained from the computer vision system embedded in the currently operating semiconductor manufacturing equipment. ISD achieves consistently better results than state-of-the-art methods at the standard mean average precision which is the most common metric used to measure the accuracy of the instance detection. Significantly, our ISD outperforms baseline method DenseNet, while requiring only 1/4 parameters. We also observe that ISD can achieve comparable better results in performance than ResNet, with only much smaller 1/268 parameters, using no extra data or pre-trained models. Our experimental results show that ISD can be useful to many future image segmentation research efforts in diverse fields of semiconductor industry which is requiring real-time and good performance with only limited training dataset.반도체 제조에서 결함 검출은 높은 수율을 유지하는데 중요합니다. 전형적으로, 반도체 웨이퍼의 결함은 제조 공정에서 발생하고 있습니다. 반도체 포토리소그래피 공정 검사에 사용되는 대부분의 컴퓨터 비전 시스템들은 여전히 외부 환경 변화에 민감한 이미지 처리 알고리즘을 사용하고 있어서 검사 오류가 자주 발생하고 있습니다. 따라서, 이미지 처리 알고리즘의 장점과 딥 러닝의 장점을 융합하여 이 문제를 해결하려고 합니다. 이 논문에서 우리는 반도체 포토리소그래피 검사와 같이 훈련 데이터 세트가 제한된 상황에서 향상된 기능 맵을 추출하기 위해 이미지 분할 검출기(Image Segmentation Detector, 이하 ISD)를 제안합니다. ISD는 이미지 분할을 위한 최신 Mask R-CNN 프레임 워크의 새로운 백본 네트워크로 사용합니다. ISD는 4 개의 조밀한 블록과 4 개의 전환 레이어로 구성합니다. 특히, ISD의 각 조밀한 블록은 보다 컴팩트함을 위해 단축 연결 및 동적 성장률을 가지고 레이어에서 생성된 피쳐 맵을 결합하고 있습니다. ISD는 최근 적용하고 있는 전이 학습 방법을 사용하지 않고 처음부터 훈련합니다. 또한, ISD는 합성곱 신경망(Convolutional Neural Network, 이하 CNN)의 향상된 기능 맵을 추출하기 위해 우리가 설계한 이미지 필터를 통해 사전 처리된 이미지 데이터 세트로 훈련을 합니다. ISD의 설계 핵심 원칙 중 하나는 소형화로 실시간 문제를 해결하고 리소스에 제한이 있는 장치에 적용하는데 중요한 역할을 하게 합니다. 모델을 실증적으로 입증하기 위해 이 논문에서는 현재 운영 중인 반도체 제조 장비에 내장된 컴퓨터 비전 시스템에서 획득한 실제 이미지를 사용합니다. ISD는 가장 일반적인 성능 측정 지표인 평균 정밀도에서 최첨단 백본 네트워크 보다 일관되게 더 나은 성능을 얻습니다. 특히, ISD는 베이스 라인으로 삼은 DenseNet 보다 파라미터들이 4배 더 적지만, 성능이 우수 합니다. 우리는 또한 ISD가 Mask R-CNN 백본 네트워크로 주로 사용하는 ResNet 보다 268배 훨씬 더 적은 파라미터들을 가지고, 추가 데이터 또는 사전 훈련된 모델을 사용하지 않고, 성능에서 비슷하거나 더 나은 결과를 얻을 수 있음을 관찰합니다. 우리의 실험 결과들은 ISD가 제한된 훈련 데이터 세트만으로 실시간 및 우수한 성능을 요구하는 반도체 산업의 다양한 분야들에서 많은 미래의 이미지 분할 연구 노력에 유용할 수 있음을 보여줍니다.Chapter 1. Introduction １ 1.1. Background and Motivation ４ Chapter 2. Related Work １２ 2.1. Inspection Method １２ 2.2. Instance Segmentation １６ 2.3. Backbone Structure ２４ 2.4. Enhanced Feature Map ３５ 2.5. Detection Performance Evaluation ４７ 2.6. Learning Network Model from Scratch ５０ Chapter 3. Proposed Method ５２ 3.1. ISD Architecture ５２ 3.2. Pre-processing ６３ 3.3. Model Training ７１ 3.4. Training Objective ７３ 3.5. Setting and Configurations ７５ Chapter 4. Experimental Evaluation ７８ 4.1. Classification Results on ISD ８１ 4.2. Comparison with Pre-processing ８５ 4.3. Image Segmentation Results on ISD ９４ 4.3.1. Results on Suck-back State ９４ 4.3.2. Results on Dispensing State １０４ 4.4. Comparison with State-of-the-art Methods １１３ Chapter 5. Conclusion １２１ Bibliography １２７ 초록 １４６박

Intelligent strategies for sheep monitoring and management

With the growth in world population, there is an increasing demand for food resources and better land utilisation, e.g., domesticated animals and land management, which in turn brought about developments in intelligent farming. Modern farms rely upon intelligent sensors and advanced software solutions, to optimally manage pasture and support animal welfare. A very significant aspect in domesticated animal farms is monitoring and understanding of animal activity, which provides vital insight into animal well-being and the environment they live in. Moreover, “virtual” fencing systems provide an alternative to managing farmland by replacing traditional boundaries. This thesis proposes novel solutions to animal activity recognition based on accelerometer data using machine learning strategies, and supports the development of virtual fencing systems via animal behaviour management using audio stimuli. The first contribution of this work is four datasets comprising accelerometer gait signals. The first dataset consisted of accelerometer and gyroscope measurements, which were obtained using a Samsung smartphone on seven animals. Next, a dataset of accelerometer measurements was collected using the MetamotionR device on 8 Hebridean ewes. Finally, two datasets of nine Hebridean ewes were collected from two sensors (MetamotionR and Raspberry Pi) comprising of accelerometer signals describing active, inactive and grazing activity of the animal. These datasets will be made publicly available as there is limited availability of such datasets. In respect to activity recognition, a systematic study of the experimental setup, associated signal features and machine learning methods was performed. It was found that Random Forest using accelerometer measurements and a sample rate of 12.5Hz with a sliding window of 5 seconds provides an accuracy of above 96% when discriminating animal activity. The problem of sensor heterogeneity was addressed with transfer learning of Convolutional Neural Networks, which has been used for the first time in this problem, and resulted to an accuracy of 98.55%, and 96.59%, respectively, in the two experimental datasets. Next, the feasibility of using only audio stimuli in the context of a virtual fencing system was explored. Specifically, a systematic evaluation of the parameters of audio stimuli, e.g., frequency and duration, was performed on two sheep breeds, Hebridean and Greyface Dartmoor ewes, in the context of controlling animal position and keeping them away from a designated area. It worth noting that the use of sounds is different to existing approaches, which utilize electric shocks to train animals to adhere within the boundaries of a virtual fence. It was found that audio signals in the frequencies of 125Hz-440Hz, 10kHz-17kHz and white noise are able to control animal activity with accuracies of 89.88%, and 95.93%, for Hebridean and Greyface Dartmoor ewes, respectively. Last but not least, the thesis proposes a multifunctional system that identifies whether the animal is active or inactive, using transfer learning, and manipulates its position using the optimized sound settings achieving a classification accuracy of over 99.95%