2,262 research outputs found

    SC-Fuse: A Feature Fusion Approach for Unpaved Road Detection from Remotely Sensed Images

    Get PDF
    Road network extraction from remote sensing imagery is crucial for numerous applications, ranging from autonomous navigation to urban and rural planning. A particularly challenging aspect is the detection of unpaved roads, often underrepresented in research and data. These roads display variability in texture, width, shape, and surroundings, making their detection quite complex. This thesis addresses these challenges by creating a specialized dataset and introducing the SC-Fuse model. Our custom dataset comprises high resolution remote sensing imagery which primarily targets unpaved roads of the American Midwest. To capture the diverse seasonal variation and their impact, the dataset includes images from different times of the year, capturing various weather conditions and offering a comprehensive view of these changing conditions. To detect roads from our custom dataset we developed SC-Fuse model, a novel deep learning architecture designed to extract unpaved road networks from satellite imagery. This model leverages the strengths of dual feature extractors: the Swin Transformer and a Residual CNN. By combining features from these, SC-fuse captures the local as well as the global context of the images. The fusion of these features is done by a Feature Fusion Module which uses Linear Attention Mechanism, to optimize the computational efficiency. A LinkNet based decoder is used to ensure precise road network reconstruction. The evaluation of SC-Fuse model is done using various metrics, including qualitative visual assessments, to test its effectiveness in unpaved road detection. Advisors: Ashok Samal and Cody Stoll

    Geoscience-aware deep learning:A new paradigm for remote sensing

    Get PDF
    Information extraction is a key activity for remote sensing images. A common distinction exists between knowledge-driven and data-driven methods. Knowledge-driven methods have advanced reasoning ability and interpretability, but have difficulty in handling complicated tasks since prior knowledge is usually limited when facing the highly complex spatial patterns and geoscience phenomena found in reality. Data-driven models, especially those emerging in machine learning (ML) and deep learning (DL), have achieved substantial progress in geoscience and remote sensing applications. Although DL models have powerful feature learning and representation capabilities, traditional DL has inherent problems including working as a black box and generally requiring a large number of labeled training data. The focus of this paper is on methods that integrate domain knowledge, such as geoscience knowledge and geoscience features (GK/GFs), into the design of DL models. The paper introduces the new paradigm of geoscience-aware deep learning (GADL), in which GK/GFs and DL models are combined deeply to extract information from remote sensing data. It first provides a comprehensive summary of GK/GFs used in GADL, which forms the basis for subsequent integration of GK/GFs with DL models. This is followed by a taxonomy of approaches for integrating GK/GFs with DL models. Several approaches are detailed using illustrative examples. Challenges and research prospects in GADL are then discussed. Developing more novel and advanced methods in GADL is expected to become the prevailing trend in advancing remotely sensed information extraction in the future.</p

    A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

    Full text link
    In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

    Classification using semantic feature and machine learning: Land-use case application

    Get PDF
    Land cover classification has interested recent works especially for deforestation, urban are monitoring and agricultural land use. Traditional classification approaches have limited accuracy especially for non-heterogeneous land cover. Thus, using machine may improve the classification accuracy. The presented paper deals with the land-use scene recognition on very high-resolution remote sensing imagery. We proposed a new framework based on semantic features, handcrafted features and machine learning classifiers decisions. The method starts by semantic feature extraction using a convolutional neural network. Handcraft features are also extracted based on color and multi-resolution characteristics. Then, the classification stage is processed by three learning machine algorithms. The final classification result performed by majority vote algorithm. The idea behind is to take advantages from semantic features and handcrafted features. The second scope is to use the decision fusion to enhance the classification result. Experimentation results show that the proposed method provides good accuracy and trustable tool for land use image identification

    μ΄ˆκ³ ν•΄μƒλ„ μ˜μƒ λΆ„λ₯˜λ₯Ό μœ„ν•œ μˆœν™˜ μ λŒ€μ  생성 신경망 기반의 쀀지도 ν•™μŠ΅ ν”„λ ˆμž„μ›Œν¬

    Get PDF
    ν•™μœ„λ…Όλ¬Έ(석사) -- μ„œμšΈλŒ€ν•™κ΅λŒ€ν•™μ› : κ³΅κ³ΌλŒ€ν•™ κ±΄μ„€ν™˜κ²½κ³΅ν•™λΆ€, 2021.8. κΉ€μš©μΌ.고해상도 μ˜μƒ λΆ„λ₯˜λŠ” 토지피볡지도 μ œμž‘, 식생 λΆ„λ₯˜, λ„μ‹œ κ³„νš λ“±μ—μ„œ λ‹€μ–‘ν•˜κ²Œ ν™œμš©λ˜λŠ” λŒ€ν‘œμ μΈ μ˜μƒ 뢄석 κΈ°μˆ μ΄λ‹€. 졜근, 심측 ν•©μ„±κ³± 신경망 (deep convolutional neural network)은 μ˜μƒ λΆ„λ₯˜ λΆ„μ•Όμ—μ„œ 두각을 보여왔닀. 특히, 심측 ν•©μ„±κ³± 신경망 기반의 의미둠적 μ˜μƒ λΆ„ν•  (semantic segmentation) 기법은 μ—°μ‚° λΉ„μš©μ„ 맀우 κ°μ†Œμ‹œν‚€λ©°, μ΄λŸ¬ν•œ 점은 μ§€μ†μ μœΌλ‘œ 고해상도 데이터가 μΆ•μ λ˜κ³  μžˆλŠ” 고해상도 μ˜μƒμ„ 뢄석할 λ•Œ μ€‘μš”ν•˜κ²Œ μž‘μš©λœλ‹€. 심측 ν•™μŠ΅ (deep learning) 기반 기법이 μ•ˆμ •μ μΈ μ„±λŠ₯을 λ‹¬μ„±ν•˜κΈ° μœ„ν•΄μ„œλŠ” 일반적으둜 μΆ©λΆ„ν•œ μ–‘μ˜ 라벨링된 데이터 (labeled data)κ°€ ν™•λ³΄λ˜μ–΄μ•Ό ν•œλ‹€. κ·ΈλŸ¬λ‚˜, 원격탐사 λΆ„μ•Όμ—μ„œ 고해상도 μ˜μƒμ— λŒ€ν•œ 참쑰데이터λ₯Ό μ–»λŠ” 것은 λΉ„μš©μ μœΌλ‘œ μ œν•œμ μΈ κ²½μš°κ°€ λ§Žλ‹€. μ΄λŸ¬ν•œ 문제λ₯Ό ν•΄κ²°ν•˜κΈ° μœ„ν•΄ λ³Έ λ…Όλ¬Έμ—μ„œλŠ” 라벨링된 μ˜μƒκ³Ό λΌλ²¨λ§λ˜μ§€ μ•Šμ€ μ˜μƒ (unlabeled image)을 ν•¨κ»˜ μ‚¬μš©ν•˜λŠ” 쀀지도 ν•™μŠ΅ ν”„λ ˆμž„μ›Œν¬λ₯Ό μ œμ•ˆν•˜μ˜€μœΌλ©°, 이λ₯Ό 톡해 고해상도 μ˜μƒ λΆ„λ₯˜λ₯Ό μˆ˜ν–‰ν•˜μ˜€λ‹€. λ³Έ λ…Όλ¬Έμ—μ„œλŠ” λΌλ²¨λ§λ˜μ§€ μ•Šμ€ μ˜μƒμ„ μ‚¬μš©ν•˜κΈ° μœ„ν•΄μ„œ κ°œμ„ λœ μˆœν™˜ μ λŒ€μ  생성 신경망 (CycleGAN) 방법을 μ œμ•ˆν•˜μ˜€λ‹€. μˆœν™˜ μ λŒ€μ  생성 신경망은 μ˜μƒ λ³€ν™˜ λͺ¨λΈ (image translation model)둜 처음 μ œμ•ˆλ˜μ—ˆμœΌλ©°, 특히 μˆœν™˜ 일관성 손싀 ν•¨μˆ˜ (cycle consistency loss function)λ₯Ό 톡해 νŽ˜μ–΄λ§λ˜μ§€ μ•Šμ€ μ˜μƒ (unpaired image)을 λͺ¨λΈ ν•™μŠ΅μ— ν™œμš©ν•œ 연ꡬ이닀. μ΄λŸ¬ν•œ μˆœν™˜ 일관성 손싀 ν•¨μˆ˜μ— μ˜κ°μ„ λ°›μ•„, λ³Έ λ…Όλ¬Έμ—μ„œλŠ” λΌλ²¨λ§λ˜μ§€ μ•Šμ€ μ˜μƒμ„ 참쑰데이터와 νŽ˜μ–΄λ§λ˜μ§€ μ•Šμ€ λ°μ΄ν„°λ‘œ κ°„μ£Όν•˜μ˜€μœΌλ©°, 이λ₯Ό 톡해 λΌλ²¨λ§λ˜μ§€ μ•Šμ€ μ˜μƒμœΌλ‘œ λΆ„λ₯˜ λͺ¨λΈμ„ ν•¨κ»˜ ν•™μŠ΅μ‹œμΌ°λ‹€. μˆ˜λ§Žμ€ λΌλ²¨λ§λ˜μ§€ μ•Šμ€ 데이터와 μƒλŒ€μ μœΌλ‘œ 적은 라벨링된 데이터λ₯Ό ν•¨κ»˜ ν™œμš©ν•˜κΈ° μœ„ν•΄, λ³Έ 논문은 지도 ν•™μŠ΅κ³Ό κ°œμ„ λœ 쀀지도 ν•™μŠ΅ 기반의 μˆœν™˜ μ λŒ€μ  생성 신경망을 κ²°ν•©ν•˜μ˜€λ‹€. μ œμ•ˆλœ ν”„λ ˆμž„μ›Œν¬λŠ” μˆœν™˜ κ³Όμ •(cyclic phase), μ λŒ€μ  κ³Όμ •(adversarial phase), 지도 ν•™μŠ΅ κ³Όμ •(supervised learning phase), μ„Έ 뢀뢄을 ν¬ν•¨ν•˜κ³  μžˆλ‹€. 라벨링된 μ˜μƒμ€ 지도 ν•™μŠ΅ κ³Όμ •μ—μ„œ λΆ„λ₯˜ λͺ¨λΈμ„ ν•™μŠ΅μ‹œν‚€λŠ” 데에 μ‚¬μš©λœλ‹€. μ λŒ€μ  κ³Όμ •κ³Ό 지도 ν•™μŠ΅ κ³Όμ •μ—μ„œλŠ” λΌλ²¨λ§λ˜μ§€ μ•Šμ€ 데이터가 μ‚¬μš©λ  수 있으며, 이λ₯Ό 톡해 적은 μ–‘μ˜ μ°Έμ‘°λ°μ΄ν„°λ‘œ 인해 μΆ©λΆ„νžˆ ν•™μŠ΅λ˜μ§€ λͺ»ν•œ λΆ„λ₯˜ λͺ¨λΈμ„ μΆ”κ°€μ μœΌλ‘œ ν•™μŠ΅μ‹œν‚¨λ‹€. μ œμ•ˆλœ ν”„λ ˆμž„μ›Œν¬μ˜ κ²°κ³ΌλŠ” 곡곡 데이터인 ISPRS Vaihingen Dataset을 톡해 ν‰κ°€λ˜μ—ˆλ‹€. 정확도 검증을 μœ„ν•΄, μ œμ•ˆλœ ν”„λ ˆμž„μ›Œν¬μ˜ κ²°κ³ΌλŠ” 5개의 λ²€μΉ˜λ§ˆν¬λ“€ (benchmarks)κ³Ό λΉ„κ΅λ˜μ—ˆμœΌλ©°, μ΄λ•Œ μ‚¬μš©λœ 벀치마크 λͺ¨λΈλ“€μ€ 지도 ν•™μŠ΅κ³Ό 쀀지도 ν•™μŠ΅ 방법 λͺ¨λ‘λ₯Ό ν¬ν•¨ν•œλ‹€. 이에 더해, λ³Έ λ…Όλ¬Έμ—μ„œλŠ” 라벨링된 데이터와 λΌλ²¨λ§λ˜μ§€ μ•Šμ€ λ°μ΄ν„°μ˜ ꡬ성에 λ”°λ₯Έ 영ν–₯을 ν™•μΈν•˜μ˜€μœΌλ©°, λ‹€λ₯Έ λΆ„λ₯˜ λͺ¨λΈμ— λŒ€ν•œ λ³Έ ν”„λ ˆμž„μ›Œν¬μ˜ μ μš©κ°€λŠ₯성에 λŒ€ν•œ 좔가적인 μ‹€ν—˜λ„ μˆ˜ν–‰ν•˜μ˜€λ‹€. μ œμ•ˆλœ ν”„λ ˆμž„μ›Œν¬λŠ” λ‹€λ₯Έ λ²€μΉ˜λ§ˆν¬λ“€κ³Ό λΉ„κ΅ν•΄μ„œ κ°€μž₯ 높은 정확도 (μ„Έ μ‹€ν—˜ 지역에 λŒ€ν•΄ 0.796, 0.786, 0.784의 전체 정확도)λ₯Ό λ‹¬μ„±ν•˜μ˜€λ‹€. 특히, 객체의 ν¬κΈ°λ‚˜ λͺ¨μ–‘κ³Ό 같은 νŠΉμ„±μ΄ λ‹€λ₯Έ μ‹€ν—˜ μ§€μ—­μ—μ„œ κ°€μž₯ 큰 정확도 μƒμŠΉμ„ ν™•μΈν•˜μ˜€μœΌλ©°, μ΄λŸ¬ν•œ κ²°κ³Όλ₯Ό 톡해 μ œμ•ˆλœ 쀀지도 ν•™μŠ΅μ΄ λͺ¨λΈμ„ μš°μˆ˜ν•˜κ²Œ μ •κ·œν™”(regularization)함을 ν™•μΈν•˜μ˜€λ‹€. λ˜ν•œ, 쀀지도 ν•™μŠ΅μ„ 톡해 ν–₯μƒλ˜λŠ” μ •ν™•λ„λŠ” 라벨링된 데이터에 λΉ„ν•΄ λΌλ²¨λ§λ˜μ§€ μ•Šμ€ 데이터가 μƒλŒ€μ μœΌλ‘œ λ§Žμ•˜μ„ λ•Œ κ·Έ 증가 폭이 λ”μš± μ»€μ‘Œλ‹€. λ§ˆμ§€λ§‰μœΌλ‘œ, μ œμ•ˆλœ 쀀지도 ν•™μŠ΅ 기반의 μˆœν™˜ μ λŒ€μ  생성 신경망 기법이 UNet 외에도 FPNκ³Ό PSPNetμ΄λΌλŠ” λ‹€λ₯Έ λΆ„λ₯˜ λͺ¨λΈμ—μ„œλ„ μœ μ˜λ―Έν•œ 정확도 μƒμŠΉμ„ λ³΄μ˜€λ‹€. 이λ₯Ό 톡해 λ‹€λ₯Έ λΆ„λ₯˜ λͺ¨λΈμ— λŒ€ν•œ μ œμ•ˆλœ ν”„λ ˆμž„μ›Œν¬μ˜ μ μš©κ°€λŠ₯성을 ν™•μΈν•˜μ˜€λ‹€Image classification of Very High Resolution (VHR) images is a fundamental task in the remote sensing domain for various applications such as land cover mapping, vegetation mapping, and urban planning. In recent years, deep convolutional neural networks have shown promising performance in image classification studies. In particular, semantic segmentation models with fully convolutional architecture-based networks demonstrated great improvements in terms of computational cost, which has become especially important with the large accumulation of VHR images in recent years. However, deep learning-based approaches are generally limited by the need of a sufficient amount of labeled data to obtain stable accuracy, and acquiring reference labels of remotely-sensed VHR images is very labor-extensive and expensive. To overcome this problem, this thesis proposed a semi-supervised learning framework for VHR image classification. Semi-supervised learning uses both labeled and unlabeled data together, thus reducing the model’s dependency on data labels. To address this issue, this thesis employed a modified CycleGAN model to utilize large amounts of unlabeled images. CycleGAN is an image translation model which was developed from Generative Adversarial Networks (GAN) for image generation. CycleGAN trains unpaired dataset by using cycle consistency loss with two generators and two discriminators. Inspired by the concept of cycle consistency, this thesis modified CycleGAN to enable the use of unlabeled VHR data in model training by considering the unlabeled images as images unpaired with their corresponding ground truth maps. To utilize a large amount of unlabeled VHR data and a relatively small amount of labeled VHR data, this thesis combined a supervised learning classification model with the modified CycleGAN architecture. The proposed framework contains three phases: cyclic phase, adversarial phase, and supervised learning phase. Through the three phase, both labeled and unlabeled data can be utilized simultaneously to train the model in an end-to-end manner. The result of the proposed framework was evaluated by using an open-source VHR image dataset, referred to as the International Society for Photogrammetry and Remote Sensing (ISPRS) Vaihingen dataset. To validate the accuracy of the proposed framework, benchmark models including both supervised and semi-supervised learning methods were compared on the same dataset. Furthermore, two additional experiments were conducted to confirm the impact of labeled and unlabeled data on classification accuracy and adaptation of the CycleGAN model for other classification models. These results were evaluated by the popular three metrics for image classification: Overall Accuracy (OA), F1-score, and mean Intersection over Union (mIoU). The proposed framework achieved the highest accuracy (OA: 0.796, 0.786, and 0.784, respectively in three test sites) in comparison to the other five benchmarks. In particular, in a test site containing numerous objects with various properties, the largest increase in accuracy was observed due to the regularization effect from the semi-supervised method using unlabeled data with the modified CycleGAN. Moreover, by controlling the amount of labeled and unlabeled data, results indicated that a relatively sufficient amount of unlabeled and labeled data is required to increase the accuracy when using the semi-supervised CycleGAN. Lastly, this thesis applied the proposed CycleGAN method to other classification models such as the feature pyramid network (FPN) and the pyramid scene parsing network (PSPNet), in place of UNet. In all cases, the proposed framework returned significantly improved results, displaying the framework’s applicability for semi-supervised image classification on remotely-sensed VHR images.1. Introduction 1 2. Background and Related Works 6 2.1. Deep Learning for Image Classification 6 2.1.1. Image-level Classifiaction 6 2.1.2. Fully Convolutional Architectures 7 2.1.3. Semantic Segmentation for Remote Sensing Images 9 2.2. Generative Adversarial Networks (GAN) 12 2.2.1. Introduction to GAN 12 2.2.2. Image Translation 14 2.2.3. GAN for Semantic Segmentation 16 3. Proposed Framework 20 3.1. Modification of CycleGAN 22 3.2. Feed-forward Path of the Proposed Framework 23 3.2.1. Cyclic Phase 23 3.2.2. Adversarial Phase 23 3.2.3. Supervised Learning Phase 24 3.3. Loss Function for Back-propagation 25 3.4. Proposed Network Architecture 28 3.4.1. Generator Architecture 28 3.4.2. Discriminator Architecture 29 4. Experimental Design 31 4.1. Overall Workflow 33 4.2. Vaihingen Dataset 38 4.3. Implementation Details 40 4.4. Metrics for Quantitative Evaluation 41 5. Results and Discussion 42 5.1. Performance Evaluation of the Proposed Feamwork 42 5.2. Comparison of Classification Performance in the Proposed Framework and Benchmarks 45 5.3. Impact of labeled and Unlabeled Data for Semi-supervised Learning 52 5.4. Cycle Consistency in Semi-supervised Learning 55 5.5. Adaptation of the GAN Framework for Other Classification Models 59 6. Conclusion 62 Reference 65 κ΅­λ¬Έ 초둝 69석
    • …
    corecore