Deep learning in remote sensing: a review

Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin

Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery

Change detection is one of the central problems in earth observation and was extensively investigated over recent decades. In this paper, we propose a novel recurrent convolutional neural network (ReCNN) architecture, which is trained to learn a joint spectral-spatial-temporal feature representation in a unified framework for change detection in multispectral images. To this end, we bring together a convolutional neural network (CNN) and a recurrent neural network (RNN) into one end-to-end network. The former is able to generate rich spectral-spatial feature representations, while the latter effectively analyzes temporal dependency in bi-temporal images. In comparison with previous approaches to change detection, the proposed network architecture possesses three distinctive properties: 1) It is end-to-end trainable, in contrast to most existing methods whose components are separately trained or computed; 2) it naturally harnesses spatial information that has been proven to be beneficial to change detection task; 3) it is capable of adaptively learning the temporal dependency between multitemporal images, unlike most of algorithms that use fairly simple operation like image differencing or stacking. As far as we know, this is the first time that a recurrent convolutional network architecture has been proposed for multitemporal remote sensing image analysis. The proposed network is validated on real multispectral data sets. Both visual and quantitative analysis of experimental results demonstrates competitive performance in the proposed mode

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Automated machine learning driven stacked ensemble modelling for forest aboveground biomass prediction using multitemporal sentinel-2 data

Modelling and large-scale mapping of forest aboveground biomass (AGB) is a complicated, challenging and expensive task. There are considerable variations in forest characteristics that creates functional disparity for different models and needs comprehensive evaluation. Moreover, the human-bias involved in the process of modelling and evaluation affects the generalization of models at larger scales. In this paper, we present an automated machine learning (AutoML) framework for modelling, evaluation and stacking of multiple base models for AGB prediction. We incorporate a hyperparameter optimization procedure for automatic extraction of targeted features from multitemporal Sentinel-2 data that minimizes human-bias in the proposed modelling pipeline. We integrate the two independent frameworks for automatic feature extraction and automatic model ensembling and evaluation. The results suggest that the extracted target-oriented features have excessive contribution of red-edge and short-wave infrared spectrum. The feature importance scale indicates a dominant role of summer based features as compared to other seasons. The automated ensembling and evaluation framework produced a stacked ensemble of base models that outperformed individual base models in accurately predicting forest AGB. The stacked ensemble model delivered the best scores of R2 cv = 0.71 and RMSE = 74.44 Mgha-1 . The other base models delivered R2 cv and RMSE ranging between 0.38–0.66 and 81.27– 109.44 Mg ha-1 respectively. The model evaluation metrics indicated that the stacked ensemble model was more resistant to outliers and achieved a better generalization. Thus, the proposed study demonstrated an effective automated modelling pipeline for predicting AGB by minimizing human-bias and deployable over large and diverse forest area

A multi-temporal phenology based classification approach for Crop Monitoring in Kenya

The SBAM (Satellite Based Agricultural Monitoring) project, funded by the Italian Space Agency aims at: developing a validated satellite imagery based method for estimating and updating the agricultural areas in the region of Central-Africa; implementing an automated process chain capable of providing periodical agricultural land cover maps of the area of interest and, possibly, an estimate of the crop yield. The project aims at filling the gap existing in the availability of high spatial resolution maps of the agricultural areas of Kenya. A high spatial resolution land cover map of Central-Eastern Africa including Kenya was compiled in the year 2000 in the framework of the Africover project using Landsat images acquired, mostly, in 1995. We investigated the use of phenological information in supporting the use of remotely sensed images for crop classification and monitoring based on Landsat 8 and, in the near future, Sentinel 2 imagery. Phenological information on crop condition was collected using time series of NDVI (Normalized Difference Vegetation Index) based on Landsat 8 images. Kenyan countryside is mainly characterized by a high number of fragmented small and medium size farmlands that dramatically increase the difficulty in classification; 30 m spatial resolution images are not enough for a proper classification of such areas. So, a pan-sharpening FIHS (Fast Intensity Hue Saturation) technique was implemented to increase image resolution from 30 m to 15 m. Ground test sites were selected, searching for agricultural vegetated areas from which phenological information was extracted. Therefore, the classification of agricultural areas is based on crop phenology, vegetation index behaviour retrieved from a time series of satellite images and on AEZ (Agro Ecological Zones) information made available by FAO (FAO, 1996) for the area of interest. This paper presents the results of the proposed classification procedure in comparison with land cover maps produced in the past years by other projects. The results refer to the Nakuru County and they were validated using field campaigns data. It showed a satisfactory overall accuracy of 92.66 % which is a significant improvement with respect to previous land cover maps

Deep learning detection of types of water-bodies using optical variables and ensembling

Water features are one of the most crucial environmental elements for strengthening climate-change adaptation. Remote sensing (RS) technologies driven by artificial intelligence (AI) have emerged as one of the most sought-after approaches for automating water information extraction and indeed. In this paper, a stacked ensemble model approach is proposed on AquaSat dataset (more than 500,000 images collection via satellite and Google Earth Engine). A one-way Analysis of variance (ANOVA) test and the Kruskal Wallis test are conducted for various optical-based variables at 99% significance level to understand how these vary for different water bodies. An oversampling is done on the training data using Synthetic Minority Oversampling Technique (SMOTE) to solve the problem of class imbalance while the model is tested on an imbalanced data, replicating the real-life situation. To enhance state-of-the-art, the pros of standalone machine learning classifiers and neural networks have been utilized. The stacked model obtained 100% accuracy on the testing data when using the decision tree classifier as the meta model. This study has been cross validated five-fold and will help researchers working in in-situ water bodies detection with the use of stacked model classification

Leveraging colour-based pseudo-labels to supervise saliency detection in hyperspectral image datasets

Saliency detection mimics the natural visual attention mechanism that identifies an imagery region to be salient when it attracts visual attention more than the background. This image analysis task covers many important applications in several fields such as military science, ocean research, resources exploration, disaster and land-use monitoring tasks. Despite hundreds of models have been proposed for saliency detection in colour images, there is still a large room for improving saliency detection performances in hyperspectral imaging analysis. In the present study, an ensemble learning methodology for saliency detection in hyperspectral imagery datasets is presented. It enhances saliency assignments yielded through a robust colour-based technique with new saliency information extracted by taking advantage of the abundance of spectral information on multiple hyperspectral images. The experiments performed with the proposed methodology provide encouraging results, also compared to several competitors