약한 지도학습 기반의 물체 탐지에서의 학습 부담을 줄이기 위한 연구

학위논문(박사) -- 서울대학교대학원 : 자연과학대학 수리과학부, 2023. 2. 강명주.In this thesis, we propose two models for weakly supervised object localization (WSOL). Many existing WSOL models have various burdens of learning, e.g., the nonnegligible cost of hyperparameter search for loss function. Thus, we first propose a model called SFPN to reduce the cost of hyperparameter search for loss function. SFPN enhances the information of the feature maps by exploiting the structure of feature pyramid network. Then these feature maps are engaged in the prediction of the bounding box. This process helps us use only cross-entropy loss as well as improving performance. Furthermore, we propose the second model named A2E Net to enjoy a smaller number of parameters. A2E Net consists of spatial attention branch and refinement branch. Spatial attention branch heightens the spatial information using few parameters. Also, refinement branch is composed of attention module and erasing module, and these modules have no trainable parameters. With the output feature map of spatial attention branch, attention module makes the feature map with more accurate information by using a connection between pixels. Also, erasing module erases the most discriminative region to make the network take account of the less discriminative region. Moreover, we boost the performance with multiple sizes of erasing. Finally, we sum up two output feature maps from attention module and erasing module to utilize information from these two modules. Extensive experiments on CUB-200-2011 and ILSVRC show the great performance of SFPN and A2E Net compared to other existing WSOL models.본 학위논문에서 우리는 약한 지도 기반의 물체탐지를 위한 두 가지 모델을 제안한다. 기존의 많은 약한 지도 기반의 물체탐지를 위한 모델들은 손실함수의 하이퍼파라미터 찾기에 들어가는 비용이 무시하기 어렵다는 등의 한계점이 있다. 그래서 우리는 먼저 이 손실함수의 하이퍼파라미터 찾기에 들어가는 비용을 줄이기 위해서 SFPN이라는 이름을 가진 모델을 제안한다. SFPN은 특징 피라미드 네트워크의 구조를 활용하여 특징 맵들의 정보를 강화시켰다. 이후에 이 특징 맵들은 경계 상자의 예측에 참여한다. 이 과정은 성능 향상뿐만 아니라 오직 교차 엔트로피 함수만을 사용할 수 있게 하는 효과를 가져왔다. 뿐만 아니라 우리는 좀 더 적은 개수의 파라미터를 활용하기 위하여 두 번째 모델인 A2E Net을 제안한다. 이 모델은 공간 집중 분기, 정제 분기로 구성된다. 우선, 공간 집중 분기는 적은 개수의 파라미터를 사용하여 공간 정보를 강화시킨다. 그리고 정제 분기는 집중 모듈과 지우기 모듈로 구성되고, 이 모듈들은 모두 학습 가능한 파라미터가 없다. 공간 집중 분기의 결과를 입력으로 사용하여, 집중 모듈은 픽셀 간의 관계를 고려하여 특징 맵의 정보를 좀 더 정교하게 만든다. 또한, 지우기 모듈은 공간 집중 분기의 출력 특징 맵의 가장 구별되는 영역을 지워서 네트워크가 덜 구별되는 영역도 고려할 수 있도록 한다. 더욱이 지우는 영역의 크기를 다양하게 사용할 수 있게 하여 성능을 더 향상시켰다. 마지막으로, 집중과 지우기에서 나오는 정보를 모두 활용하기 위하여 이 두 모듈의 출력 특징 맵들을 더한다. 이렇게 제안된 SFPN과 A2E Net은 CUB-200-2011과 ILSVRC 에서의 실험을 통해 기존의 약지도 물체 탐지 기법들보다 좋은 성능을 가짐을 보였다.1 Introduction 1 2 Preliminaries 5 2.1 Convolutional Neural Networks 5 2.1.1 Convolution Operation 5 2.1.2 Some Convolutional Neural Networks 7 3 SFPN: Simple Feature Pyramid Network for Weakly Supervised Object Localization 12 3.1 Introduction 12 3.2 Related works 14 3.2.1 Some Object Detection Methods 14 3.2.2 Existing Methods for Weakly Supervised Object Localization 18 3.3 Proposed Method 23 3.4 Experiment 26 3.4.1 Datasets 26 3.4.2 Evaluation Metrics 27 3.4.3 Implementation Details 28 3.4.4 Result 28 3.4.5 Ablation Study 30 4 A2E Net: Aggregation of Attention and Erasing for Weakly Supervised Object Localization 33 4.1 Introduction 33 4.2 Related Works 35 4.2.1 Attention Mechanism 35 4.2.2 Erasing Methods 40 4.2.3 Existing Methods for Weakly Supervised Object Localization 43 4.3 Proposed Method 48 4.3.1 Spatial Attention Branch 48 4.3.2 Refinement Branch 49 4.4 Experiment 56 4.4.1 Implementation Details 56 4.4.2 Result 57 4.4.3 Ablation Study 60 5 Conclusion 67 The bibliography 70 Abstract (in Korean) 78박

Semantic-Constraint Matching Transformer for Weakly Supervised Object Localization

Weakly supervised object localization (WSOL) strives to learn to localize objects with only image-level supervision. Due to the local receptive fields generated by convolution operations, previous CNN-based methods suffer from partial activation issues, concentrating on the object's discriminative part instead of the entire entity scope. Benefiting from the capability of the self-attention mechanism to acquire long-range feature dependencies, Vision Transformer has been recently applied to alleviate the local activation drawbacks. However, since the transformer lacks the inductive localization bias that are inherent in CNNs, it may cause a divergent activation problem resulting in an uncertain distinction between foreground and background. In this work, we proposed a novel Semantic-Constraint Matching Network (SCMN) via a transformer to converge on the divergent activation. Specifically, we first propose a local patch shuffle strategy to construct the image pairs, disrupting local patches while guaranteeing global consistency. The paired images that contain the common object in spatial are then fed into the Siamese network encoder. We further design a semantic-constraint matching module, which aims to mine the co-object part by matching the coarse class activation maps (CAMs) extracted from the pair images, thus implicitly guiding and calibrating the transformer network to alleviate the divergent activation. Extensive experimental results conducted on two challenging benchmarks, including CUB-200-2011 and ILSVRC datasets show that our method can achieve the new state-of-the-art performance and outperform the previous method by a large margin

Novel meta-learning approaches for few-shot image classification

In recent years, there has been rapid progress in computing performance and communication techniques, leading to a surging interest in artificial intelligence. Artificial intelligence aims to achieve human intelligence by making a machine think. However, current machine learning and optimisation techniques are far from fully accomplishing this, suffering from several limitations. For example, humans can learn a new concept quickly from very few examples, while artificial intelligence algorithms usually require a large number of examples to extract useful patterns. To tackle this issue, the computer science community has recently delved into the challenge of learning from very limited data, also known as few-shot learning. Few-shot image classification is the most studied research field of few-shot learning, which attempts to learn a new visual concept from limited labelled images. The conventional deep learning techniques cannot be simply applied to solve the problem, hindered by two core issues of few-shot learning, namely lack of information and intrinsic uncertainties. Lack of information is related to the insufficient visual patterns in limited training data, and intrinsic uncertainties are reflected by unrepresentative examples and background clutters. To tackle the problems, recent approaches mostly incorporate meta-learning methods which learn the general knowledge about how to make a few-shot learning process easier and quicker from a collection of learning tasks. However, existing meta-learning approaches mostly focus on either of the two key problems of few-shot image classification. Very few existing works consider both of them at the same time. Therefore, there is a need for developing novel meta-learning approaches that take into account both problems simultaneously for few-shot image classification. The thesis focuses on developing novel strategies of meta-learning approaches for few-shot image classification through three progressive stages, with the goal of addressing the aforementioned two core issues concurrently from different perspectives. In the first stage, we tackle the two main problems from the viewpoint of maximising the use of limited training data. Concretely, we propose learning to aggregate embeddings based on a channel-wise attention module. In this stage, we assume the embeddings after feature extraction consists of sufficient useful features. However, a feature extraction process could also lose relevant features. Hence, in the second stage, we target making sure as many useful features as possible can be extracted during a feature extraction process. Specifically, we design a spatial attention-based adaptive pooling module, in which a learnable pooling weight generation block is trained to assign different pooling weights to the features at different spatial locations. To further improve the classification performance, in the third stage, we leverage auxiliary information, such as saliency maps which can highlight the target object in an image, to compensate for the lack of information and mitigate background clutters. A comprehensive exploration of the suitable auxiliary information and how to effectively use it is provided. In summary, the research presented here introduces novel strategies of meta-learning approaches for few-shot image classification, addressing its two core issues from three different perspectives. The conducted works provide insights and solutions about how to effectively overcome the lack of information and intrinsic uncertainties on few-shot image classification. Our proposed methods lead to competitive results on various few-shot learning benchmarks with respect to the state-of-the-art. Besides, they contribute new meta-learning strategies that deal with the two main problems of few-shot image classification simultaneously to the few-shot learning research community