A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Adaptive Markov random fields for joint unmixing and segmentation of hyperspectral image

Linear spectral unmixing is a challenging problem in hyperspectral imaging that consists of decomposing an observed pixel into a linear combination of pure spectra (or endmembers) with their corresponding proportions (or abundances). Endmember extraction algorithms can be employed for recovering the spectral signatures while abundances are estimated using an inversion step. Recent works have shown that exploiting spatial dependencies between image pixels can improve spectral unmixing. Markov random fields (MRF) are classically used to model these spatial correlations and partition the image into multiple classes with homogeneous abundances. This paper proposes to define the MRF sites using similarity regions. These regions are built using a self-complementary area filter that stems from the morphological theory. This kind of filter divides the original image into flat zones where the underlying pixels have the same spectral values. Once the MRF has been clearly established, a hierarchical Bayesian algorithm is proposed to estimate the abundances, the class labels, the noise variance, and the corresponding hyperparameters. A hybrid Gibbs sampler is constructed to generate samples according to the corresponding posterior distribution of the unknown parameters and hyperparameters. Simulations conducted on synthetic and real AVIRIS data demonstrate the good performance of the algorithm

Deep learning in remote sensing: a review

Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin

Using Unmanned Aerial Systems for Deriving Forest Stand Characteristics in Mixed Hardwoods of West Virginia

Forest inventory information is a principle driver for forest management decisions. Information gathered through these inventories provides a summary of the condition of forested stands. The method by which remote sensing aids land managers is changing rapidly. Imagery produced from unmanned aerial systems (UAS) offer high temporal and spatial resolutions to small-scale forest management. UAS imagery is less expensive and easier to coordinate to meet project needs compared to traditional manned aerial imagery. This study focused on producing an efficient and approachable work flow for producing forest stand board volume estimates from UAS imagery in mixed hardwood stands of West Virginia. A supplementary aim of this project was to evaluate which season was best to collect imagery for forest inventory. True color imagery was collected with a DJI Phantom 3 Professional UAS and was processed in Agisoft Photoscan Professional. Automated tree crown segmentation was performed with Trimble eCognition Developer’s multi-resolution segmentation function with manual optimization of parameters through an iterative process. Individual tree volume metrics were derived from field data relationships and volume estimates were processed in EZ CRUZ forest inventory software. The software, at best, correctly segmented 43% of the individual tree crowns. No correlation between season of imagery acquisition and quality of segmentation was shown. Volume and other stand characteristics were not accurately estimated and were faulted by poor segmentation. However, the imagery was able to capture gaps consistently and provide a visualization of forest health. Difficulties, successes and time required for these procedures were thoroughly noted