2,910 research outputs found
Deep learning in remote sensing: a review
Standing at the paradigm shift towards data-intensive science, machine
learning techniques are becoming increasingly important. In particular, as a
major breakthrough in the field, deep learning has proven as an extremely
powerful tool in many fields. Shall we embrace deep learning as the key to all?
Or, should we resist a 'black-box' solution? There are controversial opinions
in the remote sensing community. In this article, we analyze the challenges of
using deep learning for remote sensing data analysis, review the recent
advances, and provide resources to make deep learning in remote sensing
ridiculously simple to start with. More importantly, we advocate remote sensing
scientists to bring their expertise into deep learning, and use it as an
implicit general model to tackle unprecedented large-scale influential
challenges, such as climate change and urbanization.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin
DAugNet: Unsupervised, Multi-source, Multi-target, and Life-long Domain Adaptation for Semantic Segmentation of Satellite Images
The domain adaptation of satellite images has recently gained an increasing
attention to overcome the limited generalization abilities of machine learning
models when segmenting large-scale satellite images. Most of the existing
approaches seek for adapting the model from one domain to another. However,
such single-source and single-target setting prevents the methods from being
scalable solutions, since nowadays multiple source and target domains having
different data distributions are usually available. Besides, the continuous
proliferation of satellite images necessitates the classifiers to adapt to
continuously increasing data. We propose a novel approach, coined DAugNet, for
unsupervised, multi-source, multi-target, and life-long domain adaptation of
satellite images. It consists of a classifier and a data augmentor. The data
augmentor, which is a shallow network, is able to perform style transfer
between multiple satellite images in an unsupervised manner, even when new data
are added over the time. In each training iteration, it provides the classifier
with diversified data, which makes the classifier robust to large data
distribution difference between the domains. Our extensive experiments prove
that DAugNet significantly better generalizes to new geographic locations than
the existing approaches
μ΄κ³ ν΄μλ μμ λΆλ₯λ₯Ό μν μν μ λμ μμ± μ κ²½λ§ κΈ°λ°μ μ€μ§λ νμ΅ νλ μμν¬
νμλ
Όλ¬Έ(μμ¬) -- μμΈλνκ΅λνμ : 곡과λν 건μ€ν경곡νλΆ, 2021.8. κΉμ©μΌ.κ³ ν΄μλ μμ λΆλ₯λ ν μ§νΌλ³΅μ§λ μ μ, μμ λΆλ₯, λμ κ³ν λ±μμ λ€μνκ² νμ©λλ λνμ μΈ μμ λΆμ κΈ°μ μ΄λ€. μ΅κ·Ό, μ¬μΈ΅ ν©μ±κ³± μ κ²½λ§ (deep convolutional neural network)μ μμ λΆλ₯ λΆμΌμμ λκ°μ 보μ¬μλ€. νΉν, μ¬μΈ΅ ν©μ±κ³± μ κ²½λ§ κΈ°λ°μ μλ―Έλ‘ μ μμ λΆν (semantic segmentation) κΈ°λ²μ μ°μ° λΉμ©μ λ§€μ° κ°μμν€λ©°, μ΄λ¬ν μ μ μ§μμ μΌλ‘ κ³ ν΄μλ λ°μ΄ν°κ° μΆμ λκ³ μλ κ³ ν΄μλ μμμ λΆμν λ μ€μνκ² μμ©λλ€.
μ¬μΈ΅ νμ΅ (deep learning) κΈ°λ° κΈ°λ²μ΄ μμ μ μΈ μ±λ₯μ λ¬μ±νκΈ° μν΄μλ μΌλ°μ μΌλ‘ μΆ©λΆν μμ λΌλ²¨λ§λ λ°μ΄ν° (labeled data)κ° ν보λμ΄μΌ νλ€. κ·Έλ¬λ, μ격νμ¬ λΆμΌμμ κ³ ν΄μλ μμμ λν μ°Έμ‘°λ°μ΄ν°λ₯Ό μ»λ κ²μ λΉμ©μ μΌλ‘ μ νμ μΈ κ²½μ°κ° λ§λ€. μ΄λ¬ν λ¬Έμ λ₯Ό ν΄κ²°νκΈ° μν΄ λ³Έ λ
Όλ¬Έμμλ λΌλ²¨λ§λ μμκ³Ό λΌλ²¨λ§λμ§ μμ μμ (unlabeled image)μ ν¨κ» μ¬μ©νλ μ€μ§λ νμ΅ νλ μμν¬λ₯Ό μ μνμμΌλ©°, μ΄λ₯Ό ν΅ν΄ κ³ ν΄μλ μμ λΆλ₯λ₯Ό μννμλ€. λ³Έ λ
Όλ¬Έμμλ λΌλ²¨λ§λμ§ μμ μμμ μ¬μ©νκΈ° μν΄μ κ°μ λ μν μ λμ μμ± μ κ²½λ§ (CycleGAN) λ°©λ²μ μ μνμλ€.
μν μ λμ μμ± μ κ²½λ§μ μμ λ³ν λͺ¨λΈ (image translation model)λ‘ μ²μ μ μλμμΌλ©°, νΉν μν μΌκ΄μ± μμ€ ν¨μ (cycle consistency loss function)λ₯Ό ν΅ν΄ νμ΄λ§λμ§ μμ μμ (unpaired image)μ λͺ¨λΈ νμ΅μ νμ©ν μ°κ΅¬μ΄λ€. μ΄λ¬ν μν μΌκ΄μ± μμ€ ν¨μμ μκ°μ λ°μ, λ³Έ λ
Όλ¬Έμμλ λΌλ²¨λ§λμ§ μμ μμμ μ°Έμ‘°λ°μ΄ν°μ νμ΄λ§λμ§ μμ λ°μ΄ν°λ‘ κ°μ£ΌνμμΌλ©°, μ΄λ₯Ό ν΅ν΄ λΌλ²¨λ§λμ§ μμ μμμΌλ‘ λΆλ₯ λͺ¨λΈμ ν¨κ» νμ΅μμΌ°λ€.
μλ§μ λΌλ²¨λ§λμ§ μμ λ°μ΄ν°μ μλμ μΌλ‘ μ μ λΌλ²¨λ§λ λ°μ΄ν°λ₯Ό ν¨κ» νμ©νκΈ° μν΄, λ³Έ λ
Όλ¬Έμ μ§λ νμ΅κ³Ό κ°μ λ μ€μ§λ νμ΅ κΈ°λ°μ μν μ λμ μμ± μ κ²½λ§μ κ²°ν©νμλ€. μ μλ νλ μμν¬λ μν κ³Όμ (cyclic phase), μ λμ κ³Όμ (adversarial phase), μ§λ νμ΅ κ³Όμ (supervised learning phase), μΈ λΆλΆμ ν¬ν¨νκ³ μλ€. λΌλ²¨λ§λ μμμ μ§λ νμ΅ κ³Όμ μμ λΆλ₯ λͺ¨λΈμ νμ΅μν€λ λ°μ μ¬μ©λλ€. μ λμ κ³Όμ κ³Ό μ§λ νμ΅ κ³Όμ μμλ λΌλ²¨λ§λμ§ μμ λ°μ΄ν°κ° μ¬μ©λ μ μμΌλ©°, μ΄λ₯Ό ν΅ν΄ μ μ μμ μ°Έμ‘°λ°μ΄ν°λ‘ μΈν΄ μΆ©λΆν νμ΅λμ§ λͺ»ν λΆλ₯ λͺ¨λΈμ μΆκ°μ μΌλ‘ νμ΅μν¨λ€.
μ μλ νλ μμν¬μ κ²°κ³Όλ 곡곡 λ°μ΄ν°μΈ ISPRS Vaihingen Datasetμ ν΅ν΄ νκ°λμλ€. μ νλ κ²μ¦μ μν΄, μ μλ νλ μμν¬μ κ²°κ³Όλ 5κ°μ λ²€μΉλ§ν¬λ€ (benchmarks)κ³Ό λΉκ΅λμμΌλ©°, μ΄λ μ¬μ©λ λ²€μΉλ§ν¬ λͺ¨λΈλ€μ μ§λ νμ΅κ³Ό μ€μ§λ νμ΅ λ°©λ² λͺ¨λλ₯Ό ν¬ν¨νλ€. μ΄μ λν΄, λ³Έ λ
Όλ¬Έμμλ λΌλ²¨λ§λ λ°μ΄ν°μ λΌλ²¨λ§λμ§ μμ λ°μ΄ν°μ ꡬμ±μ λ°λ₯Έ μν₯μ νμΈνμμΌλ©°, λ€λ₯Έ λΆλ₯ λͺ¨λΈμ λν λ³Έ νλ μμν¬μ μ μ©κ°λ₯μ±μ λν μΆκ°μ μΈ μ€νλ μννμλ€.
μ μλ νλ μμν¬λ λ€λ₯Έ λ²€μΉλ§ν¬λ€κ³Ό λΉκ΅ν΄μ κ°μ₯ λμ μ νλ (μΈ μ€ν μ§μμ λν΄ 0.796, 0.786, 0.784μ μ 체 μ νλ)λ₯Ό λ¬μ±νμλ€. νΉν, κ°μ²΄μ ν¬κΈ°λ λͺ¨μκ³Ό κ°μ νΉμ±μ΄ λ€λ₯Έ μ€ν μ§μμμ κ°μ₯ ν° μ νλ μμΉμ νμΈνμμΌλ©°, μ΄λ¬ν κ²°κ³Όλ₯Ό ν΅ν΄ μ μλ μ€μ§λ νμ΅μ΄ λͺ¨λΈμ μ°μνκ² μ κ·ν(regularization)ν¨μ νμΈνμλ€. λν, μ€μ§λ νμ΅μ ν΅ν΄ ν₯μλλ μ νλλ λΌλ²¨λ§λ λ°μ΄ν°μ λΉν΄ λΌλ²¨λ§λμ§ μμ λ°μ΄ν°κ° μλμ μΌλ‘ λ§μμ λ κ·Έ μ¦κ° νμ΄ λμ± μ»€μ‘λ€. λ§μ§λ§μΌλ‘, μ μλ μ€μ§λ νμ΅ κΈ°λ°μ μν μ λμ μμ± μ κ²½λ§ κΈ°λ²μ΄ UNet μΈμλ FPNκ³Ό PSPNetμ΄λΌλ λ€λ₯Έ λΆλ₯ λͺ¨λΈμμλ μ μλ―Έν μ νλ μμΉμ 보μλ€. μ΄λ₯Ό ν΅ν΄ λ€λ₯Έ λΆλ₯ λͺ¨λΈμ λν μ μλ νλ μμν¬μ μ μ©κ°λ₯μ±μ νμΈνμλ€Image classification of Very High Resolution (VHR) images is a fundamental task in the remote sensing domain for various applications such as land cover mapping, vegetation mapping, and urban planning. In recent years, deep convolutional neural networks have shown promising performance in image classification studies. In particular, semantic segmentation models with fully convolutional architecture-based networks demonstrated great improvements in terms of computational cost, which has become especially important with the large accumulation of VHR images in recent years.
However, deep learning-based approaches are generally limited by the need of a sufficient amount of labeled data to obtain stable accuracy, and acquiring reference labels of remotely-sensed VHR images is very labor-extensive and expensive. To overcome this problem, this thesis proposed a semi-supervised learning framework for VHR image classification. Semi-supervised learning uses both labeled and unlabeled data together, thus reducing the modelβs dependency on data labels. To address this issue, this thesis employed a modified CycleGAN model to utilize large amounts of unlabeled images.
CycleGAN is an image translation model which was developed from Generative Adversarial Networks (GAN) for image generation. CycleGAN trains unpaired dataset by using cycle consistency loss with two generators and two discriminators. Inspired by the concept of cycle consistency, this thesis modified CycleGAN to enable the use of unlabeled VHR data in model training by considering the unlabeled images as images unpaired with their corresponding ground truth maps.
To utilize a large amount of unlabeled VHR data and a relatively small amount of labeled VHR data, this thesis combined a supervised learning classification model with the modified CycleGAN architecture. The proposed framework contains three phases: cyclic phase, adversarial phase, and supervised learning phase. Through the three phase, both labeled and unlabeled data can be utilized simultaneously to train the model in an end-to-end manner.
The result of the proposed framework was evaluated by using an open-source VHR image dataset, referred to as the International Society for Photogrammetry and Remote Sensing (ISPRS) Vaihingen dataset. To validate the accuracy of the proposed framework, benchmark models including both supervised and semi-supervised learning methods were compared on the same dataset. Furthermore, two additional experiments were conducted to confirm the impact of labeled and unlabeled data on classification accuracy and adaptation of the CycleGAN model for other classification models. These results were evaluated by the popular three metrics for image classification: Overall Accuracy (OA), F1-score, and mean Intersection over Union (mIoU).
The proposed framework achieved the highest accuracy (OA: 0.796, 0.786, and 0.784, respectively in three test sites) in comparison to the other five benchmarks. In particular, in a test site containing numerous objects with various properties, the largest increase in accuracy was observed due to the regularization effect from the semi-supervised method using unlabeled data with the modified CycleGAN. Moreover, by controlling the amount of labeled and unlabeled data, results indicated that a relatively sufficient amount of unlabeled and labeled data is required to increase the accuracy when using the semi-supervised CycleGAN. Lastly, this thesis applied the proposed CycleGAN method to other classification models such as the feature pyramid network (FPN) and the pyramid scene parsing network (PSPNet), in place of UNet. In all cases, the proposed framework returned significantly improved results, displaying the frameworkβs applicability for semi-supervised image classification on remotely-sensed VHR images.1. Introduction 1
2. Background and Related Works 6
2.1. Deep Learning for Image Classification 6
2.1.1. Image-level Classifiaction 6
2.1.2. Fully Convolutional Architectures 7
2.1.3. Semantic Segmentation for Remote Sensing Images 9
2.2. Generative Adversarial Networks (GAN) 12
2.2.1. Introduction to GAN 12
2.2.2. Image Translation 14
2.2.3. GAN for Semantic Segmentation 16
3. Proposed Framework 20
3.1. Modification of CycleGAN 22
3.2. Feed-forward Path of the Proposed Framework 23
3.2.1. Cyclic Phase 23
3.2.2. Adversarial Phase 23
3.2.3. Supervised Learning Phase 24
3.3. Loss Function for Back-propagation 25
3.4. Proposed Network Architecture 28
3.4.1. Generator Architecture 28
3.4.2. Discriminator Architecture 29
4. Experimental Design 31
4.1. Overall Workflow 33
4.2. Vaihingen Dataset 38
4.3. Implementation Details 40
4.4. Metrics for Quantitative Evaluation 41
5. Results and Discussion 42
5.1. Performance Evaluation of the Proposed Feamwork 42
5.2. Comparison of Classification Performance in the Proposed Framework and Benchmarks 45
5.3. Impact of labeled and Unlabeled Data for Semi-supervised Learning 52
5.4. Cycle Consistency in Semi-supervised Learning 55
5.5. Adaptation of the GAN Framework for Other Classification Models 59
6. Conclusion 62
Reference 65
κ΅λ¬Έ μ΄λ‘ 69μ
- β¦