A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Exploiting Deep Features for Remote Sensing Image Retrieval: A Systematic Investigation

Remote sensing (RS) image retrieval is of great significant for geological information mining. Over the past two decades, a large amount of research on this task has been carried out, which mainly focuses on the following three core issues: feature extraction, similarity metric and relevance feedback. Due to the complexity and multiformity of ground objects in high-resolution remote sensing (HRRS) images, there is still room for improvement in the current retrieval approaches. In this paper, we analyze the three core issues of RS image retrieval and provide a comprehensive review on existing methods. Furthermore, for the goal to advance the state-of-the-art in HRRS image retrieval, we focus on the feature extraction issue and delve how to use powerful deep representations to address this task. We conduct systematic investigation on evaluating correlative factors that may affect the performance of deep features. By optimizing each factor, we acquire remarkable retrieval results on publicly available HRRS datasets. Finally, we explain the experimental phenomenon in detail and draw conclusions according to our analysis. Our work can serve as a guiding role for the research of content-based RS image retrieval

Hybrid Image Classification Technique for Land-Cover Mapping in the Arctic Tundra, North Slope, Alaska

Remotely sensed image classification techniques are very useful to understand vegetation patterns and species combination in the vast and mostly inaccessible arctic region. Previous researches that were done for mapping of land cover and vegetation in the remote areas of northern Alaska have considerably low accuracies compared to other biomes. The unique arctic tundra environment with short growing season length, cloud cover, low sun angles, snow and ice cover hinders the effectiveness of remote sensing studies. The majority of image classification research done in this area as reported in the literature used traditional unsupervised clustering technique with Landsat MSS data. It was also emphasized by previous researchers that SPOT/HRV-XS data lacked the spectral resolution to identify the small arctic tundra vegetation parcels. Thus, there is a motivation and research need to apply a new classification technique to develop an updated, detailed and accurate vegetation map at a higher spatial resolution i.e. SPOT-5 data. Traditional classification techniques in remotely sensed image interpretation are based on spectral reflectance values with an assumption of the training data being normally distributed. Hence it is difficult to add ancillary data in classification procedures to improve accuracy. The purpose of this dissertation was to develop a hybrid image classification approach that effectively integrates ancillary information into the classification process and combines ISODATA clustering, rule-based classifier and the Multilayer Perceptron (MLP) classifier which uses artificial neural network (ANN). The main goal was to find out the best possible combination or sequence of classifiers for typically classifying tundra type vegetation that yields higher accuracy than the existing classified vegetation map from SPOT data. Unsupervised ISODATA clustering and rule-based classification techniques were combined to produce an intermediate classified map which was used as an input to a Multilayer Perceptron (MLP) classifier. The result from the MLP classifier was compared to the previous classified map and for the pixels where there was a disagreement for the class allocations, the class having a higher kappa value was assigned to the pixel in the final classified map. The results were compared to standard classification techniques: simple unsupervised clustering technique and supervised classification with Feature Analyst. The results indicated higher classification accuracy (75.6%, with kappa value of .6840) for the proposed hybrid classification method than the standard classification techniques: unsupervised clustering technique (68.3%, with kappa value of 0.5904) and supervised classification with Feature Analyst (62.44%, with kappa value of 0.5418). The results were statistically significant at 95% confidence level

Manifold learning based spectral unmixing of hyperspectral remote sensing data

Nonlinear mixing effects inherent in hyperspectral data are not properly represented in linear spectral unmixing models. Although direct nonlinear unmixing models provide capability to capture nonlinear phenomena, they are difficult to formulate and the results are not always generalizable. Manifold learning based spectral unmixing accommodates nonlinearity in the data in the feature extraction stage followed by linear mixing, thereby incorporating some characteristics of nonlinearity while retaining advantages of linear unmixing approaches. Since endmember selection is critical to successful spectral unmixing, it is important to select proper endmembers from the manifold space. However, excessive computational burden hinders development of manifolds for large-scale remote sensing datasets. This dissertation addresses issues related to high computational overhead requirements of manifold learning for developing representative manifolds for the spectral unmixing task. Manifold approximations using landmarks are popular for mitigating the computational complexity of manifold learning. A new computationally effective landmark selection method that exploits spatial redundancy in the imagery is proposed. A robust, less costly landmark set with low spectral and spatial redundancy is successfully incorporated with a hybrid manifold which shares properties of both global and local manifolds. While landmark methods reduce computational demand, the resulting manifolds may not represent subtle features of the manifold adequately. Active learning heuristics are introduced to increase the number of landmarks, with the goal of developing more representative manifolds for spectral unmixing. By communicating between the landmark set and the query criteria relative to spectral unmixing, more representative and stable manifolds with less spectrally and spatially redundant landmarks are developed. A new ranking method based on the pixels with locally high spectral variability within image subsets and convex-geometry finds a solution more quickly and precisely. Experiments were conducted to evaluate the proposed methods using the AVIRIS Cuprite hyperspectral reference dataset. A case study of manifold learning based spectral unmixing in agricultural areas is included in the dissertation.Remotely sensed data collected by airborne or spaceborne sensors are utilized to quantify crop residue cover over an extensive area. Although remote sensing indices are popular for characterizing residue amounts, they are not effective with noisy Hyperion data because the effect of residual striping artifacts is amplified in ratios involving band differences. In this case study, spectral unmixing techniques are investigated for estimating crop residue as an alternative approach to empirical models developed using band based indices. The spectral unmixing techniques, and especially the manifold learning approaches, provide more robust, lower RMSE estimates for crop residue cover than the hyperspectral index based method for Hyperion data

Classification using semantic feature and machine learning: Land-use case application

Land cover classification has interested recent works especially for deforestation, urban are monitoring and agricultural land use. Traditional classification approaches have limited accuracy especially for non-heterogeneous land cover. Thus, using machine may improve the classification accuracy. The presented paper deals with the land-use scene recognition on very high-resolution remote sensing imagery. We proposed a new framework based on semantic features, handcrafted features and machine learning classifiers decisions. The method starts by semantic feature extraction using a convolutional neural network. Handcraft features are also extracted based on color and multi-resolution characteristics. Then, the classification stage is processed by three learning machine algorithms. The final classification result performed by majority vote algorithm. The idea behind is to take advantages from semantic features and handcrafted features. The second scope is to use the decision fusion to enhance the classification result. Experimentation results show that the proposed method provides good accuracy and trustable tool for land use image identification

A Survey on Hybrid Techniques Using SVM

Support Vector Machines (SVM) with linear or nonlinear kernels has become one of the most promising learning algorithms for classification as well as for regression. All the multilayer perceptron (MLP),Radial Basic Function(RBF) and Learning Polynomials are also worked efficiently with SVM. SVM is basically derived from statistical Learning Theory and it is very powerful statistical tool. The basic principal for the SVM is structural risk minimization and closely related to regularization theory. SVM is a group of supervised learning techniques or methods, which is used to do for classification or regression. In this paper discussed the importance of Support Vector Machines in various areas. This paper discussing the efficiency of SVM with the combination of other classification techniques

Remote sensing and GIS in support of sustainable agricultural development

Over the coming decades it is expected that the vast amounts of area currently in agricultural production will face growing pressure to intensify as world populations continue to grow, and the demand for a more Western-based diet increases. Coupled with the potential consequences of climate change, and the increasing costs involved with current energy-intensive agricultural production methods, meeting goals of environmental and socioeconomic sustainability will become ever more challenging. At a minimum, meeting such goals will require a greater understanding of rates of change, both over time and space, to properly assess how present demand may affect the needs of future generations. As agriculture represents a fundamental component of modern society, and the most ubiquitous form of human induced landscape change on the planet, it follows that mapping and tracking changes in such environments represents a crucial first step towards meeting the goal of sustainability. In anticipation of the mounting need for consistent and timely information related to agricultural development, this thesis proposes several advances in the field of geomatics, with specific contributions in the areas of remote sensing and spatial analysis: First, the relative strengths of several supervised machine learning algorithms used to classify remotely sensed imagery were assessed using two image analysis approaches: pixel-based and object-based. Second, a feature selection process, based on a Random Forest classifier, was applied to a large data set to reduce the overall number of object-based predictor variables used by a classification model without sacrificing overall classification accuracy. Third, a hybrid object-based change detection method was introduced with the ability to handle disparate image sources, generate per-class change thresholds, and minimize map updating errors. Fourth, a spatial disaggregation procedure was performed on coarse scale agricultural census data to render an indicator of agricultural development in a spatially explicit manner across a 9,000 km2 watershed in southwest Saskatchewan for three time periods spanning several decades. The combination of methodologies introduced represents an overall analytical framework suitable for supporting the sustainable development of agricultural environments