A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Art Neural Networks for Remote Sensing: Vegetation Classification from Landsat TM and Terrain Data

A new methodology for automatic mapping from Landsat Thematic Mapper (TM) and terrain data, based on the fuzzy ARTMAP neural network, is developed. System capabilities are tested on a challenging remote sensing classification problem, using spectral and terrain features for vegetation classification in the Cleveland National Forest. After training at the pixel level, system performance is tested at the stand level, using sites not seen during training. Results are compared to those of maximum likelihood classifiers, as well as back propagation neural networks and K Nearest Neighbor algorithms. ARTMAP dynamics are fast, stable, and scalable, overcoming common limitations of back propagation, which did not give satisfactory performance. Best results are obtained using a hybrid system based on a convex combination of fuzzy ARTMAP and maximum likelihood predictions. A prototype remote sensing example introduces each aspect of data processing and fuzzy ARTMAP classification. The example shows how the network automatically constructs a minimal number of recognition categories to meet accuracy criteria. A voting strategy improves prediction and assigns confidence estimates by training the system several times on different orderings of an input set.National Science Foundation (IRI 94-01659, SBR 93-00633); Office of Naval Research (N00014-95-l-0409, N00014-95-0657

Fuzzy spectral and spatial feature integration for classification of nonferrous materials in hyperspectral data

Hyperspectral data allows the construction of more elaborate models to sample the properties of the nonferrous materials than the standard RGB color representation. In this paper, the nonferrous waste materials are studied as they cannot be sorted by classical procedures due to their color, weight and shape similarities. The experimental results presented in this paper reveal that factors such as the various levels of oxidization of the waste materials and the slight differences in their chemical composition preclude the use of the spectral features in a simplistic manner for robust material classification. To address these problems, the proposed FUSSER (fuzzy spectral and spatial classifier) algorithm detailed in this paper merges the spectral and spatial features to obtain a combined feature vector that is able to better sample the properties of the nonferrous materials than the single pixel spectral features when applied to the construction of multivariate Gaussian distributions. This approach allows the implementation of statistical region merging techniques in order to increase the performance of the classification process. To achieve an efficient implementation, the dimensionality of the hyperspectral data is reduced by constructing bio-inspired spectral fuzzy sets that minimize the amount of redundant information contained in adjacent hyperspectral bands. The experimental results indicate that the proposed algorithm increased the overall classification rate from 44% using RGB data up to 98% when the spectral-spatial features are used for nonferrous material classification

Land Cover/Land Use Mapping Using Soft Computing Techniques with Optimized Features

The chapter discusses soft computing techniques for solving complex computational tasks. It highlights some of the soft computing techniques like fuzzy logic, genetic algorithm, artificial neural network, and machine learning. The classification of the remotely sensed images is always a tedious task. So, here we explain how these soft computing techniques could be used for image classification. Image classification mainly concentrates on the feature’s extraction process. The features extracted in an efficient manner improve classification accuracy. Hence, the different kinds of features and different methods for these extractions are explained. The best extracted features are selected using genetic algorithm. Various algorithms are shown and comparisons are made. Finally, the results are verified using a hypothetical case study

The applications of neural network in mapping, modeling and change detection using remotely sensed data

Thesis (Ph.D.)--Boston UniversityAdvances in remote sensing and associated capabilities are expected to proceed in a number of ways in the era of the Earth Observing System (EOS). More complex multitemporal, multi-source data sets will become available, requiring more sophisticated analysis methods. This research explores the applications of artificial neural networks in land-cover mapping, forward and inverse canopy modeling and change detection. For land-cover mapping a multi-layer feed-forward neural network produced 89% classification accuracy using a single band of multi-angle data from the Advanced Solidstate Array Spectroradiometer (ASAS). The principal results include the following: directional radiance measurements contain much useful information for discrimination among land-cover classes; the combination of multi-angle and multi-spectral data improves the overall classification accuracy compared with a single multi-angle band; and neural networks can successfully learn class discrimination from directional data or multi-domain data. Forward canopy modeling shows that a multi-layer feed-forward neural network is able to predict the bidirectional reflectance distribution function (BRDF) of different canopy sites with 90% accuracy. Analysis of the signal captured by the network indicates that the canopy structural parameters, and illumination and viewing geometry, are essential for predicting the BRDF of vegetated surfaces. The inverse neural network model shows that the R2 between the network-predicted canopy parameters and the actual canopy parameters is 0.85 for canopy density and 0.75 for both the crown shape and the height parameters. [TRUNCATED

Fused LISS IV Image Classification using Deep Convolution Neural Networks

These days, earth observation frameworks give a large number of heterogeneous remote sensing information. The most effective method to oversee such fulsomeness in utilizing its reciprocity is a vital test in current remote sensing investigation. Considering optical Very High Spatial Resolution (VHSR) images, satellites acquire both Multi Spectral (MS) and panchromatic (PAN) images at various spatial goals. Information fusion procedures manage this by proposing a technique to consolidate reciprocity among the various information sensors. Classification of remote sensing image by Deep learning techniques using Convolutional Neural Networks (CNN) is increasing a solid decent footing because of promising outcomes. The most significant attribute of CNN-based strategies is that earlier element extraction is not required which prompts great speculation capacities. In this article, we are proposing a novel Deep learning based SMDTR-CNN (Same Model with Different Training Round with Convolution Neural Network) approach for classifying fused (LISS IV + PAN) image next to image fusion. The fusion of remote sensing images from CARTOSAT-1 (PAN image) and IRS P6 (LISS IV image) sensor is obtained by Quantization Index Modulation with Discrete Contourlet Transform (QIM-DCT). For enhancing the image fusion execution, we remove specific commotions utilizing Bayesian channel by Adaptive Type-2 Fuzzy System. The outcomes of the proposed procedures are evaluated with respect to precision, classification accuracy and kappa coefficient. The results revealed that SMDTR-CNN with Deep Learning got the best all-around precision and kappa coefficient. Likewise, the accuracy of each class of fused images in LISS IV + PAN dataset is improved by 2% and 5%, respectively

GETNET: A General End-to-end Two-dimensional CNN Framework for Hyperspectral Image Change Detection

Change detection (CD) is an important application of remote sensing, which provides timely change information about large-scale Earth surface. With the emergence of hyperspectral imagery, CD technology has been greatly promoted, as hyperspectral data with the highspectral resolution are capable of detecting finer changes than using the traditional multispectral imagery. Nevertheless, the high dimension of hyperspectral data makes it difficult to implement traditional CD algorithms. Besides, endmember abundance information at subpixel level is often not fully utilized. In order to better handle high dimension problem and explore abundance information, this paper presents a General End-to-end Two-dimensional CNN (GETNET) framework for hyperspectral image change detection (HSI-CD). The main contributions of this work are threefold: 1) Mixed-affinity matrix that integrates subpixel representation is introduced to mine more cross-channel gradient features and fuse multi-source information; 2) 2-D CNN is designed to learn the discriminative features effectively from multi-source data at a higher level and enhance the generalization ability of the proposed CD algorithm; 3) A new HSI-CD data set is designed for the objective comparison of different methods. Experimental results on real hyperspectral data sets demonstrate the proposed method outperforms most of the state-of-the-arts

Artificial neural networks to detect forest fire prone areas in the southeast fire district of Mississippi

An analysis of the fire occurrences parameters is essential to save human lives, property, timber resources and conservation of biodiversity. Data conversion formats such as raster to ASCII facilitate the integration of various GIS software’s in the context of RS and GIS modeling. This research explores fire occurrences in relation to human interaction, fuel density interaction, euclidean distance from the perennial streams and slope using artificial neural networks. The human interaction (ignition source) and density of fuels is assessed by Newton’s Gravitational theory. Euclidean distance to perennial streams and slope that do posses a significant role were derived using GIS tools. All the four non linear predictor variables were modeled using the inductive nature of neural networks. The Self organizing feature map (SOM) utilized for fire size risk classification produced an overall classification accuracy of 62% and an overall kappa coefficient of 0.52 that is moderate (fair) for annual fires

Spectral and spatial methods for the classification of urban remote sensing data

Lors de ces travaux, nous nous sommes intéressés au problème de la classification supervisée d'images satellitaires de zones urbaines. Les données traitées sont des images optiques à très hautes résolutions spatiales: données panchromatiques à très haute résolution spatiale (IKONOS, QUICKBIRD, simulations PLEIADES) et des images hyperspectrales (DAIS, ROSIS). Deux stratégies ont été proposées. La première stratégie consiste en une phase d'extraction de caractéristiques spatiales et spectrales suivie d'une phase de classification. Ces caractéristiques sont extraites par filtrages morphologiques : ouvertures et fermetures géodésiques et filtrages surfaciques auto-complémentaires. La classification est réalisée avec les machines à vecteurs supports (SVM) non linéaires. Nous proposons la définition d'un noyau spatio-spectral utilisant de manière conjointe l'information spatiale et l'information spectrale extraites lors de la première phase. La seconde stratégie consiste en une phase de fusion de données pre- ou post-classification. Lors de la fusion postclassification, divers classifieurs sont appliqués, éventuellement sur plusieurs données issues d'une même scène (image panchromat ique, image multi-spectrale). Pour chaque pixel, l'appartenance à chaque classe est estimée à l'aide des classifieurs. Un schéma de fusion adaptatif permettant d'utiliser l'information sur la fiabilité locale de chaque classifieur, mais aussi l'information globale disponible a priori sur les performances de chaque algorithme pour les différentes classes, est proposé. Les différents résultats sont fusionnés à l'aide d'opérateurs flous. Les méthodes ont été validées sur des images réelles. Des améliorations significatives sont obtenues par rapport aux méthodes publiées dans la litterature

Estimating the concentration of physico chemical parameters in hydroelectric power plant reservoir

The United Nations Educational, Scientific and Cultural Organization (UNESCO) defines the amazon region and adjacent areas, such as the Pantanal, as world heritage territories, since they possess unique flora and fauna and great biodiversity. Unfortunately, these regions have increasingly been suffering from anthropogenic impacts. One of the main anthropogenic impacts in the last decades has been the construction of hydroelectric power plants. As a result, dramatic altering of these ecosystems has been observed, including changes in water levels, decreased oxygenation and loss of downstream organic matter, with consequent intense land use and population influxes after the filling and operation of these reservoirs. This, in turn, leads to extreme loss of biodiversity in these areas, due to the large-scale deforestation. The fishing industry in place before construction of dams and reservoirs, for example, has become much more intense, attracting large populations in search of work, employment and income. Environmental monitoring is fundamental for reservoir management, and several studies around the world have been performed in order to evaluate the water quality of these ecosystems. The Brazilian Amazon, in particular, goes through well defined annual hydrological cycles, which are very importante since their study aids in monitoring anthropogenic environmental impacts and can lead to policy and decision making with regard to environmental management of this area. The water quality of amazon reservoirs is greatly influenced by this defined hydrological cycle, which, in turn, causes variations of microbiological, physical and chemical characteristics. Eutrophication, one of the main processes leading to water deterioration in lentic environments, is mostly caused by anthropogenic activities, such as the releases of industrial and domestic effluents into water bodies. Physico-chemical water parameters typically related to eutrophication are, among others, chlorophyll-a levels, transparency and total suspended solids, which can, thus, be used to assess the eutrophic state of water bodies. Usually, these parameters must be investigated by going out to the field and manually measuring water transparency with the use of a Secchi disk, and taking water samples to the laboratory in order to obtain chlorophyll-a and total suspended solid concentrations. These processes are time- consuming and require trained personnel. However, we have proposed other techniques to environmental monitoring studies which do not require fieldwork, such as remote sensing and computational intelligence. Simulations in different reservoirs were performed to determine a relationship between these physico-chemical parameters and the spectral response. Based on the in situ measurements, empirical models were established to relate the reflectance of the reservoir measured by the satellites. The images were calibrated and corrected atmospherically. Statistical analysis using error estimation was used to evaluate the most accurate methodology. The Neural Networks were trained by hydrological cycle, and were useful to estimate the physicalchemical parameters of the water from the reflectance of visible bands and NIR of satellite images, with better results for the period with few clouds in the regions analyzed. The present study shows the application of wavelet neural network to estimate water quality parameters using concentration of the water samples collected in the Amazon reservoir and Cefni reservoir, UK. Sattelite imagens from Landsats and Sentinel-2 were used to train the ANN by hydrological cycle. The trained ANNs demonstrated good results between observed and estimated after Atmospheric corrections in satellites images. The ANNs showed in the results are useful to estimate these concentrations using remote sensing and wavelet transform for image processing. Therefore, the techniques proposed and applied in the present study are noteworthy since they can aid in evaluating important physico-chemical parameters, which, in turn, allows for identification of possible anthropogenic impacts, being relevant in environmental management and policy decision-making processes. The tests results showed that the predicted values have good accurate. Improving efficiency to monitor water quality parameters and confirm the reliability and accuracy of the approaches proposed for monitoring water reservoirs. This thesis contributes to the evaluation of the accuracy of different methods in the estimation of physical-chemical parameters, from satellite images and artificial neural networks. For future work, the accuracy of the results can be improved by adding more satellite images and testing new neural networks with applications in new water reservoirs