A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Remote sensing satellite image processing techniques for image classification: a comprehensive survey

This paper is a brief survey of advance technological aspects of Digital Image Processing which are applied to remote sensing images obtained from various satellite sensors. In remote sensing, the image processing techniques can be categories in to four main processing stages: Image preprocessing, Enhancement, Transformation and Classification. Image pre-processing is the initial processing which deals with correcting radiometric distortions, atmospheric distortion and geometric distortions present in the raw image data. Enhancement techniques are applied to preprocessed data in order to effectively display the image for visual interpretation. It includes techniques to effectively distinguish surface features for visual interpretation. Transformation aims to identify particular feature of earth’s surface and classification is a process of grouping the pixels, that produces effective thematic map of particular land use and land cover

Identification of storm eye from Satellite image data using fuzzy logic with machine learning

This research presents a study of a unique technique for identifying storm eye that is based on fuzzy logic and image processing with the help of cloud images. Fuzzy logic is a term that refers to complicated systems with unclear behaviour caused by a number of different circumstances. It provides the ability to model the dynamic behavior of the storm and determines the location of the best eye in an area of interest. After that, image processing is applied to enable accurate eye positioning based on the search results. The experimental results are analyzing the storm eye position with approxiamtely $98\%$ accurate compared to the India meteorological department provided best track data and Cooperative Institute for Meteorological Satellite Studies provided Advances Dvorak Technique data. As a result, the identification of storm's eye location using this technique can be found to improve significantly. Using the present technique, it is possible to determine the eye entirely automatically, which replacing the manual method that has been employed in the past

Semantic segmentation on small datasets of satellite images using convolutional neural networks

This is the final version. Available from SPIE via the DOI in this recordSemantic segmentation is one of the most popular and challenging applications of deep learning. It refers to the process of dividing a digital image into semantically homogeneous areas with similar properties. We employ the use of deep learning techniques to perform semantic segmentation on high-resolution satellite images representing urban scenes to identify roads, vegetation, and buildings. A SegNet-based neural network with an encoder–decoder architecture is employed. Despite the small size of the dataset, the results are promising. We show that the network is able to accurately distinguish between these groups for different test images, when using a network with four convolutional layers

The utility of complex soil reflectance image properties for soil mapping

This investigation is concerned with the application of complex quantitative analysis to remotely sensed data for mapping soils. The major aim of this thesis is to examine, by means of illustrative examples, the utility of complex image metrics in the detection, differentiation, and partitioning of satellite images of soil landscapes. Satellite images have been widely used for soil mapping. In order to realise the maximum potential of satellite imagery, improvements are needed both in visual presentation of such images, and in their automatic classification, in order to reveal the complex properties of soil landscape. A Landsat TM image of the Al-Ahsa area of Saudi Arabia was used in the investigation. It presents an ideal region for remote sensing studies due to the absence of vegetation cover and the existence of different type of landforms in a region of low topography. Three techniques for modelling complex elements of images were used and evaluated; Fast Fourier Transform (FFT), Artificial Neural Network Analysis (ANN), Fractal and Multifractal Analysis. The FFT technique developed in this thesis isolates spatial frequency components in specific wavebands. The inverse FFT images are enhanced to (i) display optimised zoning of the image, and (ii) to display specific features. This technique partitions images into major zones that are different zones from the standard soil maps. The ANN technique developed is a non-linear measure of image texture. It shows difference within an image. The texture model is trained on areas selected on the basis of the existing soil map. Substitution analysis of training areas allows an assessment of image zones and boundaries. The texture image is displayed by linear contrast stretch. Zonation does not correspond with published maps or with FFT zonation. The fractal method is based on the local fractal dimension that is used as a texture measure based on a moving pre-set size filter over the entire image. The resulting images do not give zones but shows clear patterns of complexity such as spatial transitions. It is possible to derive areas of similar patterns of transition in complexity. There are implications of these results for soil mapping at the theoretical and practical levels. The implications of the theoretical level are about the existences of soil units defined following the classical approach. In the practical level, the classical approach would be abandoned. There is at present nowhere near the same support of the ideas to complement the traditional mapping approach and raise awareness that soils are inherently complex. The study has important implications for classical theory and practice of soil mapping

Artificial Intelligence Based Classification for Urban Surface Water Modelling

Estimations and predictions of surface water runoff can provide very useful insights, regarding flood risks in urban areas. To automatically predict the flow behaviour of the rainfall-runoff water, in real-world satellite images, it is important to precisely identify permeable and impermeable areas. This identification indicates and helps to calculate the amount of surface water, by taking into account the amount of water being absorbed in a permeable area and what remains on the impermeable area. In this research, a model of surface water has been established, to predict the behavioural flow of rainfall-runoff water. This study employs a combination of image processing, artificial intelligence and machine learning techniques, for automatic segmentation and classification of permeable and impermeable areas, in satellite images. These techniques investigate the image classification approaches for classifying three land-use categories (roofs, roads, and pervious areas), commonly found in satellite images of the earth’s surface. Three different classification scenarios are investigated, to select the best classification model. The first scenario involves pixel by pixel classification of images, using Classification Tree and Random Forest classification techniques, in 2 different settings of sequential and parallel execution of algorithms. In the second classification scenario, the image is divided into objects, by using Superpixels (SLIC) segmentation method, while three kinds of feature sets are extracted from the segmented objects. The performance of eight different supervised machine learning classifiers is probed, using 5-fold cross-validation, for multiple SLIC values, while detailed performance comparisons lead to conclusions about the classification into different classes, regarding Object-based and Pixel-based classification schemes. Pareto analysis and Knee point selection are used to select SLIC value and the suitable type of classification, among the aforementioned two. Furthermore, a new diversity and weighted sum-based ensemble classification model, called ParetoEnsemble, is proposed, in this classification scenario. The weights are applied to selected component classifiers of an ensemble, creating a strong classifier, where classification is done based on multiple votes from candidate classifiers of the ensemble, as opposed to individual classifiers, where classification is done based on a single vote, from only one classifier. Unbalanced and balanced data-based classification results are also evaluated, to determine the most suitable mode, for satellite image classifications, in this study. Convolutional Neural Networks, based on semantic segmentation, are also employed in the classification phase, as a third scenario, to evaluate the strength of deep learning model SegNet, in the classification of satellite imaging. The best results, from the three classification scenarios, are compared and the best classification method, among the three scenarios, is used in the next phase of water modelling, with the InfoWorks ICM software, to explore the potential of modelling process, regarding a partially automated surface water network. By using the parameter settings, with a specified amount of simulated rain falling, onto the imaged area, the amount of surface water flow is estimated, to get predictions about runoff situations in urban areas, since runoff, in such a situation, can be high enough to pose a dangerous flood risk. The area of Feock, in Cornwall, is used as a simulation area of study, in this research, where some promising results have been derived, regarding classification and modelling of runoff. The correlation coefficient estimation, between classification and runoff accuracy, provides useful insight, regarding the dependence of runoff performance on classification performance. The trained system was tested on some unknown area images as well, demonstrating a reasonable performance, considering the training and classification limitations and conditions. Furthermore, in these unknown area images, reasonable estimations were derived, regarding surface water runoff. An analysis of unbalanced and balanced data-based classification and runoff estimations, for multiple parameter configurations, provides aid to the selection of classification and modelling parameter values, to be used in future unknown data predictions. This research is founded on the incorporation of satellite imaging into water modelling, using selective images for analysis and assessment of results. This system can be further improved, and runoff predictions of high precision can be better achieved, by adding more high-resolution images to the classifiers training. The added variety, to the trained model, can lead to an even better classification of any unknown image, which could eventually provide better modelling and better insights into surface water modelling. Moreover, the modelling phase can be extended, in future research, to deal with real-time parameters, by calibrating the model, after the classification phase, in order to observe the impact of classification on the actual calibration

Historical Land use/Land cover classification and its change detection mapping using Different Remotely Sensed Data from LANDSAT (MSS, TM and ETM+) and Terra (ASTER) sensors: a case study of the Euphrates River Basin in Syria with focus on agricultural irrigation projects

This thesis deals spatially and regionally with the natural boundaries of the Euphrates River Basin (ERB) in Syria. Scientifically, the research covers the application of remote sensing science (optical remote sensing: LANDSAT-MSS, TM, and ETM+; and TERRA: ASTER); and methodologically, in Land Use/Land Cover (LULC) classification and mapping, automatically and/or semi-automatically; in LULC-change detection; and finally in the mapping of historical irrigation and agricultural projects for the extraction of differing crop types and the estimation of their areas. With regard to time, the work is based on the years 1975, 1987, 2005 and 2007. Initially, preprocessing of the satellite data (geometric- and radiometric- processing, image enhancement, best bands composite selection, transformation, mosaicing and finally subsetting) was carried out. Then, the Land Use/Land Cover Classification System (LCCS) of the Food and Agriculture Organization (FAO) was chosen. The following steps were followed in LULC- classification and change detection mapping: visual interpretation in addition to digital image processing techniques; pixel-based classification methods; unsupervised classification: ISODATA-method; and supervised classification and multistage supervised approaches using the algorithms: Maximum Likelihood Classifier (MLC), Neural Network classifier (NN) and Support Vector Machines (SVM). These were trialed on a test area to determine the optimized classification approach/algorithm for application on the whole study area (ERB) based on the available imagery. Pre- and post- classification change detection methods (comparison approaches) were used to detect changes in land use/land cover-classes (for the years 1975, 1987 and 2007) in the study area. The remote sensing methods show a high potential in mapping historical and present land use/land cover classes and its changes over time. Significant results are also possible for agricultural crop classification in relatively large regional areas (the ERB in Syria is almost 50,335 km²). Change trends in the study area and period was characterized by land-intensive agricultural expansion. The rapid, more labor- and capital- intensive growth in the agricultural sector was enabled by the introduction of fertilizer, improved access to rural roads and markets, and the expansion of the government irrigation projects. Irrigated areas increased 148 % in the past 32 years from 249,681 ha in 1975 to 596,612 ha in 2007

Deep Learning Methods for Remote Sensing

Remote sensing is a field where important physical characteristics of an area are exacted using emitted radiation generally captured by satellite cameras, sensors onboard aerial vehicles, etc. Captured data help researchers develop solutions to sense and detect various characteristics such as forest fires, flooding, changes in urban areas, crop diseases, soil moisture, etc. The recent impressive progress in artificial intelligence (AI) and deep learning has sparked innovations in technologies, algorithms, and approaches and led to results that were unachievable until recently in multiple areas, among them remote sensing. This book consists of sixteen peer-reviewed papers covering new advances in the use of AI for remote sensing

Application of Geospatial Technologies for Land Use Analysis and Soil Science Education

This research is composed of three parts: 1) Adaptation of Soil Judging to Libya, 2) Predicting the classes and distribution of salt-affected soils in Northwest Libya, and 3) Geospatial analysis of land use change in the Savannah River Basin using Google Earth. Soil judging (Evaluation) plays an important role in soil science education. Libya has six soil orders according to the U.S. Soil Taxonomy (Entisols, Aridisols, Alfisols, Inceptisols, Vertisols, and Mollisols) and the most common soil orders are Entisols and Aridisols. A Soil judging (Evaluation) scorecard was tested at two different universities in Libya: The University of Tripoli and University of Zawia. Eighty-two percent of Libyan users were not aware of Soil Judging prior to this study. After completing Soil Judging trials in various locations in Libya, ninety-five percent of those surveyed indicated that Soil Judging is useful to the natural science education. Libya is mostly a dry and arid country, where sodicity and salinity problems are often accelerated by the prevailing climatic condition and geographical setting of the area. A framework was identified for classifying and mapping salt-affected soils in northwest Libya using field measurements (ECe, soil pH, and SAR) and Geographic Information Systems (GIS). The majority of soils in this region of Libya are normal (slight degree of limitation). Twenty percent of the topsoil is saline-sodic (extreme degree of limitation). Land use change and the loss of wildlife habitats are serious issues facing the Southeastern United States. Across the Savannah River basin, the major change of land use was deforestation and reforestation during the entire study period with most of the changes located near lakes and water tributaries