Sea ice segmentation in SAR images using Deep Learning

Sea ice covers over seven percent of the world's oceans. Due to the effect of global warming, Arctic's ice extent has decreased significantly in the past decades. This reduction in sea ice cover is opening new pathways for the international shipping community through the Arctic. Due to the lengthening of the open water season, the Canadian Arctic has also observed a three-fold increase in the shipping traffic in the past few years. Although the ice extent has reduced, the risks and hazards involved in shipping through these regions are still significant. To promote safe and efficient maritime activities in the Canadian Arctic, Canadian Ice Service (CIS) provides information about ice in Canada's navigable waters. CIS uses Synthetic Aperture Radar (SAR) images as one of the prominent sources to gain insights about the ice conditions in Canadian waters. Automated SAR image interpretation is a complex task and requires algorithms to learn complex and rich features. Convolutional neural networks (CNNs) have demonstrated their ability to learn such features and have been used in various image classification, segmentation and object detection tasks. In this thesis, we first propose a method to detect marginal ice zones (MIZs) in SAR images. This method uses transfer learning combined with a multi-scale patch technique to detect the MIZs. The multi-scale patch technique involves generating the segmentation masks over different patch sizes for the same image. These masks are later stacked together and thresholded to generate the final MIZ prediction mask for an image. Later we dive deep into the MIZs and focus on segmenting sea ice floes. We propose a segmentation model optimized for the task of ice floe segmentation in SAR images. The model is based on a fully convolutional architecture with residual connections. In addition to this, a conditional random field is also used as a post-processing step. The whole network is trained end-to-end using a dual loss function. Qualitative and quantitative analysis suggests that our model beats the conventional segmentation architectures for the task of ice floe detection

HED-UNet: Combined Segmentation and Edge Detection for Monitoring the Antarctic Coastline

Deep learning-based coastline detection algorithms have begun to outshine traditional statistical methods in recent years. However, they are usually trained only as single-purpose models to either segment land and water or delineate the coastline. In contrast to this, a human annotator will usually keep a mental map of both segmentation and delineation when performing manual coastline detection. To take into account this task duality, we therefore devise a new model to unite these two approaches in a deep learning model. By taking inspiration from the main building blocks of a semantic segmentation framework (UNet) and an edge detection framework (HED), both tasks are combined in a natural way. Training is made efficient by employing deep supervision on side predictions at multiple resolutions. Finally, a hierarchical attention mechanism is introduced to adaptively merge these multiscale predictions into the final model output. The advantages of this approach over other traditional and deep learning-based methods for coastline detection are demonstrated on a dataset of Sentinel-1 imagery covering parts of the Antarctic coast, where coastline detection is notoriously difficult. An implementation of our method is available at \url{https://github.com/khdlr/HED-UNet}.Comment: This work has been accepted by IEEE TGRS for publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

Sea Ice Extraction via Remote Sensed Imagery: Algorithms, Datasets, Applications and Challenges

The deep learning, which is a dominating technique in artificial intelligence, has completely changed the image understanding over the past decade. As a consequence, the sea ice extraction (SIE) problem has reached a new era. We present a comprehensive review of four important aspects of SIE, including algorithms, datasets, applications, and the future trends. Our review focuses on researches published from 2016 to the present, with a specific focus on deep learning-based approaches in the last five years. We divided all relegated algorithms into 3 categories, including classical image segmentation approach, machine learning-based approach and deep learning-based methods. We reviewed the accessible ice datasets including SAR-based datasets, the optical-based datasets and others. The applications are presented in 4 aspects including climate research, navigation, geographic information systems (GIS) production and others. It also provides insightful observations and inspiring future research directions.Comment: 24 pages, 6 figure

A review of technical factors to consider when designing neural networks for semantic segmentation of Earth Observation imagery

Semantic segmentation (classification) of Earth Observation imagery is a crucial task in remote sensing. This paper presents a comprehensive review of technical factors to consider when designing neural networks for this purpose. The review focuses on Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Generative Adversarial Networks (GANs), and transformer models, discussing prominent design patterns for these ANN families and their implications for semantic segmentation. Common pre-processing techniques for ensuring optimal data preparation are also covered. These include methods for image normalization and chipping, as well as strategies for addressing data imbalance in training samples, and techniques for overcoming limited data, including augmentation techniques, transfer learning, and domain adaptation. By encompassing both the technical aspects of neural network design and the data-related considerations, this review provides researchers and practitioners with a comprehensive and up-to-date understanding of the factors involved in designing effective neural networks for semantic segmentation of Earth Observation imagery.Comment: 145 pages with 32 figure

Supervised Burned Areas delineation by means of Sentinel-2 imagery and Convolutional Neural Networks

Wildfire events are increasingly threatening our lands, cities, and lives. To contrast this phenomenon and to limit its damages, governments around the globe are trying to find proper counter-measures, identifying prevention and monitoring as two key factors to reduce wildfires impact worldwide. In this work, we propose two deep convolutional neural networks to automatically detect and delineate burned areas from satellite acquisitions, assessing their performances at scale using validated maps of burned areas of historical wildfires. We demonstrate that the proposed networks substantially improve the burned area delineation accuracy over conventional methods

Artificial Neural Networks and Evolutionary Computation in Remote Sensing

Artificial neural networks (ANNs) and evolutionary computation methods have been successfully applied in remote sensing applications since they offer unique advantages for the analysis of remotely-sensed images. ANNs are effective in finding underlying relationships and structures within multidimensional datasets. Thanks to new sensors, we have images with more spectral bands at higher spatial resolutions, which clearly recall big data problems. For this purpose, evolutionary algorithms become the best solution for analysis. This book includes eleven high-quality papers, selected after a careful reviewing process, addressing current remote sensing problems. In the chapters of the book, superstructural optimization was suggested for the optimal design of feedforward neural networks, CNN networks were deployed for a nanosatellite payload to select images eligible for transmission to ground, a new weight feature value convolutional neural network (WFCNN) was applied for fine remote sensing image segmentation and extracting improved land-use information, mask regional-convolutional neural networks (Mask R-CNN) was employed for extracting valley fill faces, state-of-the-art convolutional neural network (CNN)-based object detection models were applied to automatically detect airplanes and ships in VHR satellite images, a coarse-to-fine detection strategy was employed to detect ships at different sizes, and a deep quadruplet network (DQN) was proposed for hyperspectral image classification

Sky and ground segmentation in the navigation visions of the planetary rovers

Sky and ground are two essential semantic components in computer vision, robotics, and remote sensing. The sky and ground segmentation has become increasingly popular. This research proposes a sky and ground segmentation framework for the rover navigation visions by adopting weak supervision and transfer learning technologies. A new sky and ground segmentation neural network (network in U-shaped network (NI-U-Net)) and a conservative annotation method have been proposed. The pre-trained process achieves the best results on a popular open benchmark (the Skyfinder dataset) by evaluating seven metrics compared to the state-of-the-art. These seven metrics achieve 99.232%, 99.211%, 99.221%, 99.104%, 0.0077, 0.0427, and 98.223% on accuracy, precision, recall, dice score (F1), misclassification rate (MCR), root mean squared error (RMSE), and intersection over union (IoU), respectively. The conservative annotation method achieves superior performance with limited manual intervention. The NI-U-Net can operate with 40 frames per second (FPS) to maintain the real-time property. The proposed framework successfully fills the gap between the laboratory results (with rich idea data) and the practical application (in the wild). The achievement can provide essential semantic information (sky and ground) for the rover navigation vision

Image segmentation methods with limited data sets

Cieľom tejto práce bolo navrhnúť metódu segmentácie obrazu, ktorá dokáže efektívne pracovať aj s dátovými množinami malej veľkosti. Bola k tomu využitá nedávno publikovaná neurónová sieť ODE, ktorá by vďaka svojim vlastnostiam mala dosahovať lepšej generalizácie v prípade riešenia úlohy s malým množstvom trénovacích dát. Navrhnutá sieť ODE-UNet vznikla kombináciou architektúry UNet a siete ODE, pričom využíva výhod oboch sietí. Na ISBI dátovej sade bolo dosiahnutých presností Rand: 0,950272 a Info: 0,978061 na testovacích snímkoch, čím bola prekonaná presnosť siete UNet, ktorá bola takisto v tejto práci testovaná. Týmito výsledkami bolo ale preukázané, že pomocou ODE sietí nie je možné prekonať stav vedy a techniky. Avšak preukázaných bolo aj niekoľko výhod oproti testovanej UNet architektúre a aktuálne používaným metódam a v rámci ďalšieho rozširovania by bolo možné výsledky stále zlepšovať.The goal of this thesis was to propose an image segmentation method, which is capable of effective segmentation process with small datasets. Recently published ODE neural network was used for this method, because its features should provide better generalization in case of tasks with only small datasets available. The proposed ODE-UNet network was created by combining UNet architecture with ODE neural network, while using benefits of both networks. ODE-UNet reached following results on ISBI dataset: Rand: 0,950272 and Info: 0,978061. These results are better than the ones received from UNet model, which was also tested in this thesis, but it has been proven that state of the art can not be outperformed using ODE neural networks. However, the advantages of ODE neural network over tested UNet architecture and other methods were confirmed, and there is still a room for improvement by extending this method.

Deep learning-based change detection in remote sensing images:a review

Images gathered from different satellites are vastly available these days due to the fast development of remote sensing (RS) technology. These images significantly enhance the data sources of change detection (CD). CD is a technique of recognizing the dissimilarities in the images acquired at distinct intervals and are used for numerous applications, such as urban area development, disaster management, land cover object identification, etc. In recent years, deep learning (DL) techniques have been used tremendously in change detection processes, where it has achieved great success because of their practical applications. Some researchers have even claimed that DL approaches outperform traditional approaches and enhance change detection accuracy. Therefore, this review focuses on deep learning techniques, such as supervised, unsupervised, and semi-supervised for different change detection datasets, such as SAR, multispectral, hyperspectral, VHR, and heterogeneous images, and their advantages and disadvantages will be highlighted. In the end, some significant challenges are discussed to understand the context of improvements in change detection datasets and deep learning models. Overall, this review will be beneficial for the future development of CD methods