A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Satellite image classification and spatial analysis of agricultural areas for land cover mapping of grizzly bear habitat

Habitat loss and human-caused mortality are the most serious threats facing grizzly bear (Ursus arctosi L.) populations in Alberta, with conflicts between people and bears in agricultural areas being especially important. For this reason, information is needed about grizzly bears in agricultural areas. The objectives of this research were to find the best possible classification approach for determining multiple classes of agricultural and herbaceous land cover for the purpose of grizzly bear habitat mapping, and to determine what, if any, spatial and compositional components of the landscape affected the bears in these agricultural areas. Spectral and environmental data for five different land-cover types of interest were acquired in late July, 2007, from Landsat Thematic Mapper satellite imagery and field data collection in two study areas in Alberta. Three different classification methods were analyzed, the best method being the Supervised Sequential Masking (SSM) technique, which gave an overall accuracy of 88% and a Kappa Index of Agreement (KIA) of 83%. The SSM classification was then expanded to cover 6 more Landsat scenes, and combined with bear GPS location data. Analysis of this data revealed that bears in agricultural areas were found in grasses / forage crops 77% of the time, with small grains and bare soil / fallow fields making up the rest of the visited land-cover. Locational data for 8 bears were examined in an area southwest of Calgary, Alberta. The 4494 km2 study area was divided into 107 sub-landscapes of 42 km2. Five-meter spatial resolution IRS panchromatic imagery was used to classify the area and derive compositional and configurational metrics for each sub-landscape. It was found that the amount of agricultural land did not explain grizzly bear use; however, secondary effects of agriculture on landscape configuration did. High patch density and variation in distances between neighboring similar patch types were seen as the most significant metrics in the abundance models; higher variation in patch shape, greater contiguity between patches, and lower average distances between neighboring similar patches were the most consistently significant predictors in the bear presence / absence models. Grizzly bears appeared to prefer areas that were structurally correlated to natural areas, and avoided areas that were structurally correlated to agricultural areas. Grizzly bear presence could be predicted in a particular sub-landscape with 87% accuracy using a logistic regression model. Between 30% and 35% of the grizzlies‟ landscape scale habitat selection was explained

Large Area Land Cover Mapping Using Deep Neural Networks and Landsat Time-Series Observations

This dissertation focuses on analysis and implementation of deep learning methodologies in the field of remote sensing to enhance land cover classification accuracy, which has important applications in many areas of environmental planning and natural resources management. The first manuscript conducted a land cover analysis on 26 Landsat scenes in the United States by considering six classifier variants. An extensive grid search was conducted to optimize classifier parameters using only the spectral components of each pixel. Results showed no gain in using deep networks by using only spectral components over conventional classifiers, possibly due to the small reference sample size and richness of features. The effect of changing training data size, class distribution, or scene heterogeneity were also studied and we found all of them having significant effect on classifier accuracy. The second manuscript reviewed 103 research papers on the application of deep learning methodologies in remote sensing, with emphasis on per-pixel classification of mono-temporal data and utilizing spectral and spatial data dimensions. A meta-analysis quantified deep network architecture improvement over selected convolutional classifiers. The effect of network size, learning methodology, input data dimensionality and training data size were also studied, with deep models providing enhanced performance over conventional one using spectral and spatial data. The analysis found that input dataset was a major limitation and available datasets have already been utilized to their maximum capacity. The third manuscript described the steps to build the full environment for dataset generation based on Landsat time-series data using spectral, spatial, and temporal information available for each pixel. A large dataset containing one sample block from each of 84 ecoregions in the conterminous United States (CONUS) was created and then processed by a hybrid convolutional+recurrent deep network, and the network structure was optimized with thousands of simulations. The developed model achieved an overall accuracy of 98% on the test dataset. Also, the model was evaluated for its overall and per-class performance under different conditions, including individual blocks, individual or combined Landsat sensors, and different sequence lengths. The analysis found that although the deep model performance per each block is superior to other candidates, the per block performance still varies considerably from block to block. This suggests extending the work by model fine-tuning for local areas. The analysis also found that including more time stamps or combining different Landsat sensor observations in the model input significantly enhances the model performance

Brain-Computer Interface

Brain-computer interfacing (BCI) with the use of advanced artificial intelligence identification is a rapidly growing new technology that allows a silently commanding brain to manipulate devices ranging from smartphones to advanced articulated robotic arms when physical control is not possible. BCI can be viewed as a collaboration between the brain and a device via the direct passage of electrical signals from neurons to an external system. The book provides a comprehensive summary of conventional and novel methods for processing brain signals. The chapters cover a range of topics including noninvasive and invasive signal acquisition, signal processing methods, deep learning approaches, and implementation of BCI in experimental problems