A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Multi-scalar remote sensing of the northern mixed prairie vegetation

Optimal scale of study and scaling are fundamental to ecological research, and have been made easier with remotely sensed (RS) data. With access to RS data at multiple scales, it is important to identify how they compare and how effectively information at a specific scale will potentially transfer between scales. Therefore, my research compared the spatial, spectral, and temporal aspects of scale of RS data to study biophysical properties and spatio-temporal dynamics of the northern mixed prairie vegetation. I collected ground cover, dominant species, aboveground biomass, and leaf area index (LAI) from 41 sites and along 3 transects in the West Block of Grasslands National Park of Canada (GNPC; +49°, -107°) between June-July of 2006 and 2007. Narrowband (VIn) and broadband vegetation indices (VIb) were derived from RS data at multiple scales acquired through field spectroradiometry (1 m) and satellite imagery (10, 20, 30 m). VIs were upscaled from their native scales to coarser scales for spatial comparison, and time-series imagery at ~5-year intervals was used for temporal comparison. Results showed VIn, VIb, and LAI captured the spatial variation of plant biophysical properties along topographical gradients and their spatial scales ranged from 35-200 m. Among the scales compared, RS data at finer scales showed stronger ability than coarser scales to estimate ground vegetation. VIn were found to be better predictors than VIb in estimating LAI. Upscaling at all spatial scales showed similar weakening trends for LAI prediction using VIb, however spatial regression methods were necessary to minimize spatial effects in the RS data sets and to improve the prediction results. Multiple endmember spectral mixture analysis (MESMA) successfully captured the spatial heterogeneity of vegetation and effective modeling of sub-pixel spectral variability to produce improved vegetation maps. However, the efficiency of spectral unmixing was found to be highly dependent on the identification of optimal type and number of region-specific endmembers, and comparison of spectral unmixing on imagery at different scales showed spectral resolution to be important over spatial resolution. With the development of a comprehensive endmember library, MESMA may be used as a standard tool for identifying spatio-temporal changes in time-series imagery. Climatic variables were found to affect the success of unmixing, with lower success for years of climatic extremes. Change-detection analysis showed the success of biodiversity conservation practices of GNPC since establishment of the park and suggests that its management strategies are effective in maintaining vegetation heterogeneity in the region. Overall, my research has advanced the understanding of RS of the northern mixed prairie vegetation, especially in the context of effects of scale and scaling. From an eco-management perspective, this research has provided cost- and time-effective methods for vegetation mapping and monitoring. Data and techniques tested in this study will be even more useful with hyperspectral imagery should they become available for the northern mixed prairie

Application of advanced techniques for the remote detection, modelling and spatial analysis of mesquite (prosopis spp.) invasion in Western Australia

Invasive plants pose serious threats to economic, social and environmental interests throughout the world. Developing strategies for their management requires a range of information that is often impractical to collect from ground based surveys. In other cases, such as retrospective analyses of historical invasion rates and patterns, data is rarely, if ever, available from such surveys. Instead, historical archives of remotely sensed imagery provides one of the only existing records, and are used in this research to determine invasion rates and reconstruct invasion patterns of a ca 70 year old exotic mesquite population (Leguminoseae: Prosopis spp.) in the Pilbara Region of Western Australia, thereby helping to identify ways to reduce spread and infill. A model was then developed using this, and other, information to predict which parts of the Pilbara are most a risk. This information can assist in identifying areas requiring the most vigilant intervention and pre-emptive measures. Precise information of the location and areal extent of an invasive species is also crucial for land managers and policy makers for crafting management strategies aimed at control, confinement or eradication of some or all of the population. Therefore, the third component of this research was to develop and test high spectral and spatial resolution airborne imagery as a potential monitoring tool for tracking changes at various intervals and quantifying the effectiveness of management strategies adopted. To this end, high spatial resolution digital multispectral imagery (4 channels, 1 m spatial resolution) and hyperspectral imagery (126 channels, 3 m spatial resolution) was acquired and compared for its potential for distinguishing mesquite from coexisting species and land covers.These three modules of research are summarised hereafter. To examine the rates and patterns of mesquite invasion through space and time, canopies were extracted from a temporal series of panchromatic aerial photography over an area of 450 ha using unsupervised classification. Non-mesquite trees and shrubs were not discernible from mesquite using this imagery (or technique) and so were masked out using an image acquired prior to invasion. The accuracy of the mesquite extractions were corroborated in the field and found to be high (R2 = 0.98, P36 m2 (66-94%) with both approaches and image types. However, both approaches used on the hyperspectral imagery were more reliable at capturing patches >36 m2 than the DMSI using either approach. The lowest omission and commission rates were obtained using pairwise separation on the hyperspectral imagery, which was significantly more accurate than DMSI using an overall separation approach (Z=2.78, P36 m2. However, hyperspectral imagery processed using pairwise separation appears to be superior, even though not statistically different to hyperspectral imagery processed using overall separation or DMSI processed using pairwise separation at the 95% confidence level. Mapping smaller patches may require the use of very high spatial resolution imagery, such as that achievable from unmanned airborne vehicles, coupled with a hyperspectral instrument. Alternatively, management may continue to rely on visual airborne surveys flown at low altitude and speed, which have proven to be capable at mapping small and isolated mesquite shrubs in the study area used in this research

Rolling Guidance Based Scale-Aware Spatial Sparse Unmixing for Hyperspectral Remote Sensing Imagery

Spatial regularization based sparse unmixing has attracted much attention in the hyperspectral remote sensing image processing field, which combines spatial information consideration with a sparse unmixing model, and has achieved improved fractional abundance results. However, the traditional spatial sparse unmixing approaches can suppress discrete wrong unmixing points and smooth an abundance map with low-contrast changes, and it has no concept of scale difference. In this paper, to better extract the different levels of spatial details, rolling guidance based scale-aware spatial sparse unmixing (namely, Rolling Guidance Sparse Unmixing (RGSU)) is proposed to extract and recover the different levels of important structures and details in the hyperspectral remote sensing image unmixing procedure, as the different levels of structures and edges in remote sensing imagery have different meanings and importance. Differing from the existing spatial regularization based sparse unmixing approaches, the proposed method considers the different levels of edges by combining a Gaussian filter-like method to realize small-scale structure removal with a joint bilateral filtering process to account for the spatial domain and range domain correlations. The proposed method is based on rolling guidance spatial regularization in a traditional spatial regularization sparse unmixing framework, and it accomplishes scale-aware sparse unmixing. The experimental results obtained with both simulated and real hyperspectral images show that the proposed method achieves visual effects better and produces higher quantitative results (i.e., higher SRE values) when compared to the current state-of-the-art sparse unmixing algorithms, which illustrates the effectiveness of the rolling guidance based scale aware method. In the future work, adaptive scale-aware spatial sparse unmixing framework will be studied and developed to enhance the current idea

UAVs for the Environmental Sciences

This book gives an overview of the usage of UAVs in environmental sciences covering technical basics, data acquisition with different sensors, data processing schemes and illustrating various examples of application

Semantic location extraction from crowdsourced data

Crowdsourced Data (CSD) has recently received increased attention in many application areas including disaster management. Convenience of production and use, data currency and abundancy are some of the key reasons for attracting this high interest. Conversely, quality issues like incompleteness, credibility and relevancy prevent the direct use of such data in important applications like disaster management. Moreover, location information availability of CSD is problematic as it remains very low in many crowd sourced platforms such as Twitter. Also, this recorded location is mostly related to the mobile device or user location and often does not represent the event location. In CSD, event location is discussed descriptively in the comments in addition to the recorded location (which is generated by means of mobile device's GPS or mobile communication network). This study attempts to semantically extract the CSD location information with the help of an ontological Gazetteer and other available resources. 2011 Queensland flood tweets and Ushahidi Crowd Map data were semantically analysed to extract the location information with the support of Queensland Gazetteer which is converted to an ontological gazetteer and a global gazetteer. Some preliminary results show that the use of ontologies and semantics can improve the accuracy of place name identification of CSD and the process of location information extraction

Air Force Institute of Technology Research Report 2013

This report summarizes the research activities of the Air Force Institute of Technology’s Graduate School of Engineering and Management. It describes research interests and faculty expertise; lists student theses/dissertations; identifies research sponsors and contributions; and outlines the procedures for contacting the school. Included in the report are: faculty publications, conference presentations, consultations, and funded research projects. Research was conducted in the areas of Aeronautical and Astronautical Engineering, Electrical Engineering and Electro-Optics, Computer Engineering and Computer Science, Systems Engineering and Management, Operational Sciences, Mathematics, Statistics and Engineering Physics

Precision Agriculture Technology for Crop Farming

This book provides a review of precision agriculture technology development, followed by a presentation of the state-of-the-art and future requirements of precision agriculture technology. It presents different styles of precision agriculture technologies suitable for large scale mechanized farming; highly automated community-based mechanized production; and fully mechanized farming practices commonly seen in emerging economic regions. The book emphasizes the introduction of core technical features of sensing, data processing and interpretation technologies, crop modeling and production control theory, intelligent machinery and field robots for precision agriculture production