Incremental Learning Techniques for Part-Based Semantic Segmentation

In this work we use an Incremental Learning approach to try to develop a model for the Part-Based Semantic Segmentation. Our framework uses two networks where the second one is fed with the output of the first. The first network performs the Semantic Segmentation task with 21 classes from the Pascal-VOC, while the second one performs the Part-Based Semantic Segmentation task with the Part-Based Semantic Segmentation with 108 classes/parts on the Pascal-Part dataset

Deep Adversarial Attention Alignment for Unsupervised Domain Adaptation: The Benefit of Target Expectation Maximization

© 2018, Springer Nature Switzerland AG. In this paper, we make two contributions to unsupervised domain adaptation (UDA) using the convolutional neural network (CNN). First, our approach transfers knowledge in all the convolutional layers through attention alignment. Most previous methods align high-level representations, e.g., activations of the fully connected (FC) layers. In these methods, however, the convolutional layers which underpin critical low-level domain knowledge cannot be updated directly towards reducing domain discrepancy. Specifically, we assume that the discriminative regions in an image are relatively invariant to image style changes. Based on this assumption, we propose an attention alignment scheme on all the target convolutional layers to uncover the knowledge shared by the source domain. Second, we estimate the posterior label distribution of the unlabeled data for target network training. Previous methods, which iteratively update the pseudo labels by the target network and refine the target network by the updated pseudo labels, are vulnerable to label estimation errors. Instead, our approach uses category distribution to calculate the cross-entropy loss for training, thereby ameliorating the error accumulation of the estimated labels. The two contributions allow our approach to outperform the state-of-the-art methods by +2.6% on the Office-31 dataset

Does a Neural Network Really Encode Symbolic Concepts?

Recently, a series of studies have tried to extract interactions between input variables modeled by a DNN and define such interactions as concepts encoded by the DNN. However, strictly speaking, there still lacks a solid guarantee whether such interactions indeed represent meaningful concepts. Therefore, in this paper, we examine the trustworthiness of interaction concepts from four perspectives. Extensive empirical studies have verified that a well-trained DNN usually encodes sparse, transferable, and discriminative concepts, which is partially aligned with human intuition

변형된 FusionNet을 이용한 회색조 이미지의 자연스러운 채색

학위논문 (석사) -- 서울대학교 대학원 : 자연과학대학 협동과정 계산과학전공, 2021. 2. 강명주.In this paper, we propose a grayscale image colorizing technique. The colorization task can be divided into three main ways, the Scribble-based method, Exemplar-based method and Fully automatic method. Our proposed method is included in the third one. We use a deep learning model that is widely used in the colorization eld recently. We propose Encoder-Docoder model using Convolutional Neural Networks. In particular, we modify the FusionNet with good performance to suit this purpose. Also, in order to get better results, we do not use MSE loss function. Instead, we use the loss function suitable for the colorizing purpose. We use a subset of the ImageNet dataset as the training, validation and test dataset. We take some existing methods from Fully automatic Deep Learning method and compared them with our models. Our algorithm is evaluated using a quantitative metric called PSNR (Peak Signal-to-Noise Ratio). In addition, in order to evaluate the results qualitatively, our model was applied to the test dataset and compared with various other models. Our model has better performance both quantitatively and qualitatively than other models. Finally, we apply our model to old black and white photographs.본 논문에서는 회색조 이미지들에 대한 채색 기법을 제안한다. 채색 작업은 크게 Scribble 기반 방법, Exemplar 기반 방법, 완전 자동 방법의 세 가지로 나눌 수 있다. 본 논문에서는 세 번째 방법을 사용했다. 최근에 채색 분야에서 널리 사용되는 딥 러닝 모델을 사용한다. Convolutional Neural Networks를 이용한 Encoder-Docoder 모델을 제안한다. 특히 기존에 image segmetation 분야에서 좋은 성능을 보이는 FusionNet을 자동 채색 목적에 맞게 다양한 방법으로 수정했다. 또한 더 나은 결과를 얻기 위해 MSE 손실 함수를 사용하지 않았다. 대신, 우리는 자동 채색 목적에 적합한 손실 함수를 사용하였다. ImageNet 데이터셋의 부분 집합을 훈련, 검증 및 테스트 데이터셋으로 사용했다. 우리는 완전 자동 딥 러닝 방법에서 기존 방법을 가져와 우리의 모델과 비교했다. 우리의 알고리즘은 PSNR (Peak Signal-to-Noise Ratio)이라는 정량적 지표를 사용하여 평가되었다. 또한 결과를 정성적으로 평가하기 위해 테스트 데이터셋에 모델을 적용하여 다양한 모델과 비교했다. 그 결과 다른 모델에 비해 정성적으로도, 정량적으로도 좋은 성능을 보였다. 마지막으로 오래된 흑백 사진과 같은 다양한 유형의 이미지에 적용한 결과를 제시했다.Abstract i 1 Introduction 1 2 Related Works 4 2.1 Scribble-based method . . . . . . . . . . . . . . . . . . . . . . 4 2.2 Exemplar-based method . . . . . . . . . . . . . . . . . . . . . 5 2.3 Fully automatic method . . . . . . . . . . . . . . . . . . . . . 6 3 Proposed Method 8 3.1 Method Overview . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.2 Loss Function . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.3 Architecture detail . . . . . . . . . . . . . . . . . . . . . . . . 10 3.3.1 Encoder . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.3.2 Decoder . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3.3.3 Bridge . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 4 Experiments 14 4.1 CIE Lab Color Space . . . . . . . . . . . . . . . . . . . . . . . 15 4.2 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 4.3 Qualitative Evaluation . . . . . . . . . . . . . . . . . . . . . . 17 4.4 Quantitative Evaluation . . . . . . . . . . . . . . . . . . . . . 18 4.5 Legacy Old image Colorization . . . . . . . . . . . . . . . . . . 20 5 Conclusion 23 The bibliography 24 Abstract (in Korean) 28Maste

Deep learning for remote sensing image classification:A survey

Remote sensing (RS) image classification plays an important role in the earth observation technology using RS data, having been widely exploited in both military and civil fields. However, due to the characteristics of RS data such as high dimensionality and relatively small amounts of labeled samples available, performing RS image classification faces great scientific and practical challenges. In recent years, as new deep learning (DL) techniques emerge, approaches to RS image classification with DL have achieved significant breakthroughs, offering novel opportunities for the research and development of RS image classification. In this paper, a brief overview of typical DL models is presented first. This is followed by a systematic review of pixel?wise and scene?wise RS image classification approaches that are based on the use of DL. A comparative analysis regarding the performances of typical DL?based RS methods is also provided. Finally, the challenges and potential directions for further research are discussedpublishersversionPeer reviewe

Short-term motion prediction of autonomous vehicles in complex environments: A Deep Learning approach

Complex environments manifest a high level of complexity and it is of critical importance that the safety systems embedded within autonomous vehicles (AVs) are able to accurately anticipate short-term future motion of agents in close proximity. This problem can be further understood as generating a sequence of coordinates describing the plausible future motion of the tracked agent. Number of recently proposed techniques that present satisfactory performance exploit the learning capabilities of novel deep learning (DL) architectures to tackle the discussed task. Nonetheless, there still exists a vast number of challenging issues that must be resolved to further advance capabilities of motion prediction models.This thesis explores novel deep learning techniques within the area of short-term motion prediction of on-road participants, specifically other vehicles from a points of autonomous vehicles. First and foremost, various approaches in the literature demonstrate significant benefits of using a rasterised top-down image of the road to encode the context of tracked vehicle’s surroundings which generally encapsulates a large, global portion of the environment. This work on the other hand explores a use of local regions of the rasterised map to more explicitly focus on the encoding of the tracked vehicle’s state. The proposed technique demonstrates plausible results against several baseline models and in addition outperforms the same model that instead uses global maps. Next, the typical method for extracting features from rasterised maps involves employing one of the popular vision models (e.g. ResNet-50) that has been previously pre-trained on a distinct task such as image classification. Recently however, it has been demonstrated that this approach can be sub-optimal for tasks that strongly rely on precise localisation of features and it can be more advantageous to train the model from scratch directly on the task at hand. In contrast, the subsequent part of this thesis investigates an alternative method for processing and encoding of spatial data based on the capsule networks in order to eradicate several issues that standard vision models exhibit. Through several experiments it is established that the novel capsule based motion predictor that is trained from scratch is able to achieve competitive results against numerous popular vision models. Finally, the proposed model is further extended with the use of generative framework to account for the fact that the space of possible movements of the tracked vehicle is not strictly limited to single trajectory. More specifically, to account for the multi-modality of the problem a conditional variational auto-encoder (CVAE) is employed which enables to sample an arbitrary amount of diverse trajectories. The final model is examined against methods from literature on a publicly available dataset and as presented it significantly outperforms other models whilst drastically reducing the number of trainable parameters

Logging Trail Segmentation via a Novel U-Net Convolutional Neural Network and High-Density Laser Scanning Data

Logging trails are one of the main components of modern forestry. However, spotting the accurate locations of old logging trails through common approaches is challenging and time consuming. This study was established to develop an approach, using cutting-edge deep-learning convolutional neural networks and high-density laser scanning data, to detect logging trails in different stages of commercial thinning, in Southern Finland. We constructed a U-Net architecture, consisting of encoder and decoder paths with several convolutional layers, pooling and non-linear operations. The canopy height model (CHM), digital surface model (DSM), and digital elevation models (DEMs) were derived from the laser scanning data and were used as image datasets for training the model. The labeled dataset for the logging trails was generated from different references as well. Three forest areas were selected to test the efficiency of the algorithm that was developed for detecting logging trails. We designed 21 routes, including 390 samples of the logging trails and non-logging trails, covering all logging trails inside the stands. The results indicated that the trained U-Net using DSM (k = 0.846 and IoU = 0.867) shows superior performance over the trained model using CHM (k = 0.734 and IoU = 0.782), DEMavg (k = 0.542 and IoU = 0.667), and DEMmin (k = 0.136 and IoU = 0.155) in distinguishing logging trails from non-logging trails. Although the efficiency of the developed approach in young and mature stands that had undergone the commercial thinning is approximately perfect, it needs to be improved in old stands that have not received the second or third commercial thinning

Logging Trail Segmentation via a Novel U-Net Convolutional Neural Network and High-Density Laser Scanning Data

Logging trails are one of the main components of modern forestry. However, spotting the accurate locations of old logging trails through common approaches is challenging and time consuming. This study was established to develop an approach, using cutting-edge deep-learning convolutional neural networks and high-density laser scanning data, to detect logging trails in different stages of commercial thinning, in Southern Finland. We constructed a U-Net architecture, consisting of encoder and decoder paths with several convolutional layers, pooling and non-linear operations. The canopy height model (CHM), digital surface model (DSM), and digital elevation models (DEMs) were derived from the laser scanning data and were used as image datasets for training the model. The labeled dataset for the logging trails was generated from different references as well. Three forest areas were selected to test the efficiency of the algorithm that was developed for detecting logging trails. We designed 21 routes, including 390 samples of the logging trails and non-logging trails, covering all logging trails inside the stands. The results indicated that the trained U-Net using DSM (k = 0.846 and IoU = 0.867) shows superior performance over the trained model using CHM (k = 0.734 and IoU = 0.782), DEMavg (k = 0.542 and IoU = 0.667), and DEMmin (k = 0.136 and IoU = 0.155) in distinguishing logging trails from non-logging trails. Although the efficiency of the developed approach in young and mature stands that had undergone the commercial thinning is approximately perfect, it needs to be improved in old stands that have not received the second or third commercial thinning