234 research outputs found
Sea Ice Classification of SAR Imagery Based on Convolution Neural Networks
We explore new and existing convolutional neural network (CNN) architectures for sea ice classification using Sentinel-1 (S1) synthetic aperture radar (SAR) data by investigating two key challenges: binary sea ice versus open-water classification, and a multi-class sea ice type classification. The analysis of sea ice in SAR images is challenging because of the thermal noise effects and ambiguities in the radar backscatter for certain conditions that include the reflection of complex information from sea ice surfaces. We use manually annotated SAR images containing various sea ice types to construct a dataset for our Deep Learning (DL) analysis. To avoid contamination between classes we use a combination of near-simultaneous SAR images from S1 and fine resolution cloud-free optical data from Sentinel-2 (S2). For the classification, we use data augmentation to adjust for the imbalance of sea ice type classes in the training data. The SAR images are divided into small patches which are processed one at a time. We demonstrate that the combination of data augmentation and training of a proposed modified Visual Geometric Group 16-layer (VGG-16) network, trained from scratch, significantly improves the classification performance, compared to the original VGG-16 model and an ad hoc CNN model. The experimental results show both qualitatively and quantitatively that our models produce accurate classification results
Scalable computing for earth observation - Application on Sea Ice analysis
In recent years, Deep learning (DL) networks have shown considerable improvements and have become a preferred methodology in many different applications. These networks have outperformed other classical techniques, particularly in large data settings. In earth observation from the satellite field, for example, DL algorithms have demonstrated the ability to learn complicated nonlinear relationships in input data accurately. Thus, it contributed to advancement in this field. However, the training process of these networks has heavy computational overheads. The reason is two-fold: The sizable complexity of these networks and the high number of training samples needed to learn all parameters comprising these architectures. Although the quantity of training data enhances the accuracy of the trained models in general, the computational cost may restrict the amount of analysis that can be done. This issue is particularly critical in satellite remote sensing, where a myriad of satellites generate an enormous amount of data daily, and acquiring in-situ ground truth for building a large training dataset is a fundamental prerequisite.
This dissertation considers various aspects of deep learning based sea ice monitoring from SAR data. In this application, labeling data is very costly and time-consuming. Also, in some cases, it is not even achievable due to challenges in establishing the required domain knowledge, specifically when it comes to monitoring Arctic Sea ice with Synthetic Aperture Radar (SAR), which is the application domain of this thesis. Because the Arctic is remote, has long dark seasons, and has a very dynamic weather system, the collection of reliable in-situ data is very demanding. In addition to the challenges of interpreting SAR data of sea ice, this issue makes SAR-based sea ice analysis with DL networks a complicated process.
We propose novel DL methods to cope with the problems of scarce training data and address the computational cost of the training process. We analyze DL network capabilities based on self-designed architectures and learn strategies, such as transfer learning for sea ice classification. We also address the scarcity of training data by proposing a novel deep semi-supervised learning method based on SAR data for incorporating unlabeled data information into the training process. Finally, a new distributed DL method that can be used in a semi-supervised manner is proposed to address the computational complexity of deep neural network training
HED-UNet: Combined Segmentation and Edge Detection for Monitoring the Antarctic Coastline
Deep learning-based coastline detection algorithms have begun to outshine
traditional statistical methods in recent years. However, they are usually
trained only as single-purpose models to either segment land and water or
delineate the coastline. In contrast to this, a human annotator will usually
keep a mental map of both segmentation and delineation when performing manual
coastline detection. To take into account this task duality, we therefore
devise a new model to unite these two approaches in a deep learning model. By
taking inspiration from the main building blocks of a semantic segmentation
framework (UNet) and an edge detection framework (HED), both tasks are combined
in a natural way. Training is made efficient by employing deep supervision on
side predictions at multiple resolutions. Finally, a hierarchical attention
mechanism is introduced to adaptively merge these multiscale predictions into
the final model output. The advantages of this approach over other traditional
and deep learning-based methods for coastline detection are demonstrated on a
dataset of Sentinel-1 imagery covering parts of the Antarctic coast, where
coastline detection is notoriously difficult. An implementation of our method
is available at \url{https://github.com/khdlr/HED-UNet}.Comment: This work has been accepted by IEEE TGRS for publication. Copyright
may be transferred without notice, after which this version may no longer be
accessibl
Sea Ice Extraction via Remote Sensed Imagery: Algorithms, Datasets, Applications and Challenges
The deep learning, which is a dominating technique in artificial
intelligence, has completely changed the image understanding over the past
decade. As a consequence, the sea ice extraction (SIE) problem has reached a
new era. We present a comprehensive review of four important aspects of SIE,
including algorithms, datasets, applications, and the future trends. Our review
focuses on researches published from 2016 to the present, with a specific focus
on deep learning-based approaches in the last five years. We divided all
relegated algorithms into 3 categories, including classical image segmentation
approach, machine learning-based approach and deep learning-based methods. We
reviewed the accessible ice datasets including SAR-based datasets, the
optical-based datasets and others. The applications are presented in 4 aspects
including climate research, navigation, geographic information systems (GIS)
production and others. It also provides insightful observations and inspiring
future research directions.Comment: 24 pages, 6 figure
Enhancing sea ice segmentation in Sentinel-1 images with atrous convolutions
Due to the growing volume of remote sensing data and the low latency required
for safe marine navigation, machine learning (ML) algorithms are being
developed to accelerate sea ice chart generation, currently a manual
interpretation task. However, the low signal-to-noise ratio of the freely
available Sentinel-1 Synthetic Aperture Radar (SAR) imagery, the ambiguity of
backscatter signals for ice types, and the scarcity of open-source
high-resolution labelled data makes automating sea ice mapping challenging. We
use Extreme Earth version 2, a high-resolution benchmark dataset generated for
ML training and evaluation, to investigate the effectiveness of ML for
automated sea ice mapping. Our customized pipeline combines ResNets and Atrous
Spatial Pyramid Pooling for SAR image segmentation. We investigate the
performance of our model for: i) binary classification of sea ice and open
water in a segmentation framework; and ii) a multiclass segmentation of five
sea ice types. For binary ice-water classification, models trained with our
largest training set have weighted F1 scores all greater than 0.95 for January
and July test scenes. Specifically, the median weighted F1 score was 0.98,
indicating high performance for both months. By comparison, a competitive
baseline U-Net has a weighted average F1 score of ranging from 0.92 to 0.94
(median 0.93) for July, and 0.97 to 0.98 (median 0.97) for January. Multiclass
ice type classification is more challenging, and even though our models achieve
2% improvement in weighted F1 average compared to the baseline U-Net, test
weighted F1 is generally between 0.6 and 0.80. Our approach can efficiently
segment full SAR scenes in one run, is faster than the baseline U-Net, retains
spatial resolution and dimension, and is more robust against noise compared to
approaches that rely on patch classification
Country-wide retrieval of forest structure from optical and SAR satellite imagery with deep ensembles
Monitoring and managing Earth’s forests in an informed manner is an important requirement for addressing challenges like biodiversity loss and climate change. While traditional in situ or aerial campaigns for forest assessments provide accurate data for analysis at regional level, scaling them to entire countries and beyond with high temporal resolution is hardly possible. In this work, we propose a method based on deep ensembles that densely estimates forest structure variables at country-scale with 10-m resolution, using freely available satellite imagery as input. Our method jointly transforms Sentinel-2 optical images and Sentinel-1 syntheticaperture radar images into maps of five different forest structure variables: 95th height percentile, mean height, density, Gini coefficient, and fractional cover. We train and test our model on reference data from 41 airborne laser scanning missions across Norway and demonstrate that it is able to generalize to unseen test regions, achieving normalized mean absolute errors between 11% and 15%, depending on the variable. Our work is also the first to propose a variant of so-called Bayesian deep learning to densely predict multiple forest structure variables with well-calibrated uncertainty estimates from satellite imagery. The uncertainty information increases the trustworthiness of the model and its suitability for downstream tasks that require reliable confidence estimates as a basis for decision making. We present an extensive set of experiments to validate the accuracy of the predicted maps as well as the quality of the predicted uncertainties. To demonstrate scalability, we provide Norway-wide maps for the five forest structure variables.publishedVersio
A review of technical factors to consider when designing neural networks for semantic segmentation of Earth Observation imagery
Semantic segmentation (classification) of Earth Observation imagery is a
crucial task in remote sensing. This paper presents a comprehensive review of
technical factors to consider when designing neural networks for this purpose.
The review focuses on Convolutional Neural Networks (CNNs), Recurrent Neural
Networks (RNNs), Generative Adversarial Networks (GANs), and transformer
models, discussing prominent design patterns for these ANN families and their
implications for semantic segmentation. Common pre-processing techniques for
ensuring optimal data preparation are also covered. These include methods for
image normalization and chipping, as well as strategies for addressing data
imbalance in training samples, and techniques for overcoming limited data,
including augmentation techniques, transfer learning, and domain adaptation. By
encompassing both the technical aspects of neural network design and the
data-related considerations, this review provides researchers and practitioners
with a comprehensive and up-to-date understanding of the factors involved in
designing effective neural networks for semantic segmentation of Earth
Observation imagery.Comment: 145 pages with 32 figure
Guided Unsupervised Learning by Subaperture Decomposition for Ocean SAR Image Retrieval
Spaceborne synthetic aperture radar (SAR) can provide accurate images of the
ocean surface roughness day-or-night in nearly all weather conditions, being an
unique asset for many geophysical applications. Considering the huge amount of
data daily acquired by satellites, automated techniques for physical features
extraction are needed. Even if supervised deep learning methods attain
state-of-the-art results, they require great amount of labeled data, which are
difficult and excessively expensive to acquire for ocean SAR imagery. To this
end, we use the subaperture decomposition (SD) algorithm to enhance the
unsupervised learning retrieval on the ocean surface, empowering ocean
researchers to search into large ocean databases. We empirically prove that SD
improve the retrieval precision with over 20% for an unsupervised transformer
auto-encoder network. Moreover, we show that SD brings important performance
boost when Doppler centroid images are used as input data, leading the way to
new unsupervised physics guided retrieval algorithms
- …