Sea Ice Extraction via Remote Sensed Imagery: Algorithms, Datasets, Applications and Challenges

The deep learning, which is a dominating technique in artificial intelligence, has completely changed the image understanding over the past decade. As a consequence, the sea ice extraction (SIE) problem has reached a new era. We present a comprehensive review of four important aspects of SIE, including algorithms, datasets, applications, and the future trends. Our review focuses on researches published from 2016 to the present, with a specific focus on deep learning-based approaches in the last five years. We divided all relegated algorithms into 3 categories, including classical image segmentation approach, machine learning-based approach and deep learning-based methods. We reviewed the accessible ice datasets including SAR-based datasets, the optical-based datasets and others. The applications are presented in 4 aspects including climate research, navigation, geographic information systems (GIS) production and others. It also provides insightful observations and inspiring future research directions.Comment: 24 pages, 6 figure

Synthetic Aperture Radar (SAR) Meets Deep Learning

This reprint focuses on the application of the combination of synthetic aperture radars and depth learning technology. It aims to further promote the development of SAR image intelligent interpretation technology. A synthetic aperture radar (SAR) is an important active microwave imaging sensor, whose all-day and all-weather working capacity give it an important place in the remote sensing community. Since the United States launched the first SAR satellite, SAR has received much attention in the remote sensing community, e.g., in geological exploration, topographic mapping, disaster forecast, and traffic monitoring. It is valuable and meaningful, therefore, to study SAR-based remote sensing applications. In recent years, deep learning represented by convolution neural networks has promoted significant progress in the computer vision community, e.g., in face recognition, the driverless field and Internet of things (IoT). Deep learning can enable computational models with multiple processing layers to learn data representations with multiple-level abstractions. This can greatly improve the performance of various applications. This reprint provides a platform for researchers to handle the above significant challenges and present their innovative and cutting-edge research results when applying deep learning to SAR in various manuscript types, e.g., articles, letters, reviews and technical reports

Automatic near real-time flood detection in high resolution X-band synthetic aperture radar satellite data using context-based classification on irregular graphs

This thesis is an outcome of the project “Flood and damage assessment using very high resolution SAR data” (SAR-HQ), which is embedded in the interdisciplinary oriented RIMAX (Risk Management of Extreme Flood Events) programme, funded by the Federal Ministry of Education and Research (BMBF). It comprises the results of three scientific papers on automatic near real-time flood detection in high resolution X-band synthetic aperture radar (SAR) satellite data for operational rapid mapping activities in terms of disaster and crisis-management support. Flood situations seem to become more frequent and destructive in many regions of the world. A rising awareness of the availability of satellite based cartographic information has led to an increase in requests to corresponding mapping services to support civil-protection and relief organizations with disaster-related mapping and analysis activities. Due to the rising number of satellite systems with high revisit frequencies, a strengthened pool of SAR data is available during operational flood mapping activities. This offers the possibility to observe the whole extent of even large-scale flood events and their spatio-temporal evolution, but also calls for computationally efficient and automatic flood detection methods, which should drastically reduce the user input required by an active image interpreter. This thesis provides solutions for the near real-time derivation of detailed flood parameters such as flood extent, flood-related backscatter changes as well as flood classification probabilities from the new generation of high resolution X-band SAR satellite imagery in a completely unsupervised way. These data are, in comparison to images from conventional medium-resolution SAR sensors, characterized by an increased intra-class and decreased inter-class variability due to the reduced mixed pixel phenomenon. This problem is addressed by utilizing multi-contextual models on irregular hierarchical graphs, which consider that semantic image information is less represented in single pixels but in homogeneous image objects and their mutual relation. A hybrid Markov random field (MRF) model is developed, which integrates scale-dependent as well as spatio-temporal contextual information into the classification process by combining hierarchical causal Markov image modeling on automatically generated irregular hierarchical graphs with noncausal Markov modeling related to planar MRFs. This model is initialized in an unsupervised manner by an automatic tile-based thresholding approach, which solves the flood detection problem in large-size SAR data with small a priori class probabilities by statistical parameterization of local bi-modal class-conditional density functions in a time efficient manner. Experiments performed on TerraSAR-X StripMap data of Southwest England and ScanSAR data of north-eastern Namibia during large-scale flooding show the effectiveness of the proposed methods in terms of classification accuracy, computational performance, and transferability. It is further demonstrated that hierarchical causal Markov models such as hierarchical maximum a posteriori (HMAP) and hierarchical marginal posterior mode (HMPM) estimation can be effectively used for modeling the inter-spatial context of X-band SAR data in terms of flood and change detection purposes. Although the HMPM estimator is computationally more demanding than the HMAP estimator, it is found to be more suitable in terms of classification accuracy. Further, it offers the possibility to compute marginal posterior entropy-based confidence maps, which are used for the generation of flood possibility maps that express that the uncertainty in labeling of each image element. The supplementary integration of intra-spatial and, optionally, temporal contextual information into the Markov model results in a reduction of classification errors. It is observed that the application of the hybrid multi-contextual Markov model on irregular graphs is able to enhance classification results in comparison to modeling on regular structures of quadtrees, which is the hierarchical representation of images usually used in MRF-based image analysis. X-band SAR systems are generally not suited for detecting flooding under dense vegetation canopies such as forests due to the low capability of the X-band signal to penetrate into media. Within this thesis a method is proposed for the automatic derivation of flood areas beneath shrubs and grasses from TerraSAR-X data. Furthermore, an approach is developed, which combines high resolution topographic information with multi-scale image segmentation to enhance the mapping accuracy in areas consisting of flooded vegetation and anthropogenic objects as well as to remove non-water look-alike areas

Density Estimates as Representations of Agricultural Fields for Remote Sensing-Based Monitoring of Tillage and Vegetation Cover

We consider the use of remote sensing for large-scale monitoring of agricultural land use, focusing on classification of tillage and vegetation cover for individual field parcels across large spatial areas. From the perspective of remote sensing and modelling, field parcels are challenging as objects of interest due to highly varying shape and size but relatively uniform pixel content and texture. To model such areas we need representations that can be reliably estimated already for small parcels and that are invariant to the size of the parcel. We propose representing the parcels using density estimates of remote imaging pixels and provide a computational pipeline that combines the representation with arbitrary supervised learning algorithms, while allowing easy integration of multiple imaging sources. We demonstrate the method in the task of the automatic monitoring of autumn tillage method and vegetation cover of Finnish crop fields, based on the integrated analysis of intensity of Synthetic Aperture Radar (SAR) polarity bands of the Sentinel-1 satellite and spectral indices calculated from Sentinel-2 multispectral image data. We use a collection of 127,757 field parcels monitored in April 2018 and annotated to six tillage method and vegetation cover classes, reaching 70% classification accuracy for test parcels when using both SAR and multispectral data. Besides this task, the method could also directly be applied for other agricultural monitoring tasks, such as crop yield prediction

Density Estimates as Representations of Agricultural Fields for Remote Sensing-Based Monitoring of Tillage and Vegetation Cover

We consider the use of remote sensing for large-scale monitoring of agricultural land use, focusing on classification of tillage and vegetation cover for individual field parcels across large spatial areas. From the perspective of remote sensing and modelling, field parcels are challenging as objects of interest due to highly varying shape and size but relatively uniform pixel content and texture. To model such areas we need representations that can be reliably estimated already for small parcels and that are invariant to the size of the parcel. We propose representing the parcels using density estimates of remote imaging pixels and provide a computational pipeline that combines the representation with arbitrary supervised learning algorithms, while allowing easy integration of multiple imaging sources. We demonstrate the method in the task of the automatic monitoring of autumn tillage method and vegetation cover of Finnish crop fields, based on the integrated analysis of intensity of Synthetic Aperture Radar (SAR) polarity bands of the Sentinel-1 satellite and spectral indices calculated from Sentinel-2 multispectral image data. We use a collection of 127,757 field parcels monitored in April 2018 and annotated to six tillage method and vegetation cover classes, reaching 70% classification accuracy for test parcels when using both SAR and multispectral data. Besides this task, the method could also directly be applied for other agricultural monitoring tasks, such as crop yield prediction.Peer reviewe

Medical image analysis methods in MR/CT-imaged acute-subacute ischemic stroke lesion:Segmentation, prediction and insights into dynamic evolution simulation models. A critical appraisal

AbstractOver the last 15years, basic thresholding techniques in combination with standard statistical correlation-based data analysis tools have been widely used to investigate different aspects of evolution of acute or subacute to late stage ischemic stroke in both human and animal data. Yet, a wave of biology-dependent and imaging-dependent issues is still untackled pointing towards the key question: “how does an ischemic stroke evolve?” Paving the way for potential answers to this question, both magnetic resonance (MRI) and CT (computed tomography) images have been used to visualize the lesion extent, either with or without spatial distinction between dead and salvageable tissue. Combining diffusion and perfusion imaging modalities may provide the possibility of predicting further tissue recovery or eventual necrosis. Going beyond these basic thresholding techniques, in this critical appraisal, we explore different semi-automatic or fully automatic 2D/3D medical image analysis methods and mathematical models applied to human, animal (rats/rodents) and/or synthetic ischemic stroke to tackle one of the following three problems: (1) segmentation of infarcted and/or salvageable (also called penumbral) tissue, (2) prediction of final ischemic tissue fate (death or recovery) and (3) dynamic simulation of the lesion core and/or penumbra evolution. To highlight the key features in the reviewed segmentation and prediction methods, we propose a common categorization pattern. We also emphasize some key aspects of the methods such as the imaging modalities required to build and test the presented approach, the number of patients/animals or synthetic samples, the use of external user interaction and the methods of assessment (clinical or imaging-based). Furthermore, we investigate how any key difficulties, posed by the evolution of stroke such as swelling or reperfusion, were detected (or not) by each method. In the absence of any imaging-based macroscopic dynamic model applied to ischemic stroke, we have insights into relevant microscopic dynamic models simulating the evolution of brain ischemia in the hope to further promising and challenging 4D imaging-based dynamic models. By depicting the major pitfalls and the advanced aspects of the different reviewed methods, we present an overall critique of their performances and concluded our discussion by suggesting some recommendations for future research work focusing on one or more of the three addressed problems