A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Clarification of Water Stress in Apple Seedlings Using HSI Texture with Machine Learning Technique

Apples are known for their nutrition and economic value. Accurate and rapid diagnosis of water status in apple seedlings on an individual rootstock basis is a prerequisite for precision water management. This study presents a rapid and non-destructive approach for estimating water content in apple seedlings at leaf levels. A PIKA L system collects hyperspectral images(400-1000nm) of apple leaves. To the author's knowledge, no prior work was conducted using the spectral-texture approach in plant water stress. Our research extracts spatial information, gray-level co-occurrence matrix (GLCM), from feature wavelength images of hypercubes. Machine learning methods are applied to these spatial feature matrixs to identify apple leaves under different water stresses. In addition, differences in spectral responses were analysed using machine learning techniques for sorting apple seedlings with varying water treatments (dry, normal, and overwatering). Also, we measure chlorophyll to determine the relationship between hyperspectral characteristics and physiological changes. The achievements of the research indicate that the fusion of texture and hyperspectral imaging coupled with machine learning techniques is promising and presents a powerful potential to determine the water stress in the leaves of apple seedlings

An Approach for the Customized High-Dimensional Segmentation of Remote Sensing Hyperspectral Images

Abstract: This paper addresses three problems in the field of hyperspectral image segmentation: the fact that the way an image must be segmented is related to what the user requires and the application; the lack and cost of appropriately labeled reference images; and, finally, the information loss problem that arises in many algorithms when high dimensional images are projected onto lower dimensional spaces before starting the segmentation process. To address these issues, the Multi-Gradient based Cellular Automaton (MGCA) structure is proposed to segment multidimensional images without projecting them to lower dimensional spaces. The MGCA structure is coupled with an evolutionary algorithm (ECAS-II) in order to produce the transition rule sets required by MGCA segmenters. These sets are customized to specific segmentation needs as a function of a set of low dimensional training images in which the user expresses his segmentation requirements. Constructing high dimensional image segmenters from low dimensional training sets alleviates the problem of lack of labeled training images. These can be generated online based on a parametrization of the desired segmentation extracted from a set of examples. The strategy has been tested in experiments carried out using synthetic and real hyperspectral images, and it has been compared to state-of-the-art segmentation approaches over benchmark images in the area of remote sensing hyperspectral imaging.Ministerio de Economía y competitividad; TIN2015-63646-C5-1-RMinisterio de Economía y competitividad; RTI2018-101114-B-I00Xunta de Galicia: ED431C 2017/1

Highlighting Water Stress in Apple Seedlings Using HSI Texture with Machine Learning Technique

Apples are known for their nutrition and economic value. Accurate and rapid diagnosis of water status in apple seedlings on an individual rootstock basis is a prerequisite for precision water management. This study presents a rapid and non-destructive approach for estimating water content in apple seedlings at leaf levels. A PIKA L system collects hyperspectral images (400-1000nm) of apple leaves. Our research extracts spatial information, gray-level co-occurrence matrix (GLCM), from feature wavelength images of hypercubes. Machine learning methods are applied to these spatial feature matrixs to identify apple leaves under different water stresses. In addition, differences in spectral responses were analysed using machine learning techniques for sorting apple seedlings with varying water treatments (dry, normal, and overwatering). Also, we measure chlorophyll to determine the relationship between hyperspectral characteristics and physiological changes. The achievements of the research indicate that the fusion of texture and hyperspectral imaging coupled with machine learning techniques is promising and presents a powerful potential to determine the water stress in the leaves of apple seedlings

Classification of Hyperspectral and LiDAR Data Using Coupled CNNs

In this paper, we propose an efficient and effective framework to fuse hyperspectral and Light Detection And Ranging (LiDAR) data using two coupled convolutional neural networks (CNNs). One CNN is designed to learn spectral-spatial features from hyperspectral data, and the other one is used to capture the elevation information from LiDAR data. Both of them consist of three convolutional layers, and the last two convolutional layers are coupled together via a parameter sharing strategy. In the fusion phase, feature-level and decision-level fusion methods are simultaneously used to integrate these heterogeneous features sufficiently. For the feature-level fusion, three different fusion strategies are evaluated, including the concatenation strategy, the maximization strategy, and the summation strategy. For the decision-level fusion, a weighted summation strategy is adopted, where the weights are determined by the classification accuracy of each output. The proposed model is evaluated on an urban data set acquired over Houston, USA, and a rural one captured over Trento, Italy. On the Houston data, our model can achieve a new record overall accuracy of 96.03%. On the Trento data, it achieves an overall accuracy of 99.12%. These results sufficiently certify the effectiveness of our proposed model

Extreme Learning Machine combined with a Differential Evolution algorithm for lithology identification

Lithology identification, obtained through the analysis of several geophysical properties, has an important role in the process of characterization of oil reservoirs. The identification can be accomplished by direct and indirect methods, but these methods are not always feasible because of the cost or imprecision of the results generated. Consequently, there is a need to automate the procedure of reservoir characterization and, in this context, computational intelligence techniques appear as an alternative to lithology identification. However, to acquire proper performance, usually some parameters should be adjusted and this can become a hard task depending on the complexity of the underlying problem. This paper aims to apply an Extreme Learning Machine (ELM) adjusted with a Differential Evolution (DE) to classify data from the South Provence Basin, using a previously published paper as a baseline reference. The paper contributions include the use of an evolutionary algorithm as a tool for search on the hyperparameters of the ELM. In addition, an  activation function recently proposed in the literature is implemented and tested. The  computational approach developed here has the potential to assist in petrographic data classification and helps to improve the process of reservoir characterization and the production development planning