A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Mapping Crop Cycles in China Using MODIS-EVI Time Series

As the Earth’s population continues to grow and demand for food increases, the need for improved and timely information related to the properties and dynamics of global agricultural systems is becoming increasingly important. Global land cover maps derived from satellite data provide indispensable information regarding the geographic distribution and areal extent of global croplands. However, land use information, such as cropping intensity (defined here as the number of cropping cycles per year), is not routinely available over large areas because mapping this information from remote sensing is challenging. In this study, we present a simple but efficient algorithm for automated mapping of cropping intensity based on data from NASA’s (NASA: The National Aeronautics and Space Administration) MODerate Resolution Imaging Spectroradiometer (MODIS). The proposed algorithm first applies an adaptive Savitzky-Golay filter to smooth Enhanced Vegetation Index (EVI) time series derived from MODIS surface reflectance data. It then uses an iterative moving-window methodology to identify cropping cycles from the smoothed EVI time series. Comparison of results from our algorithm with national survey data at both the provincial and prefectural level in China show that the algorithm provides estimates of gross sown area that agree well with inventory data. Accuracy assessment comparing visually interpreted time series with algorithm results for a random sample of agricultural areas in China indicates an overall accuracy of 91.0% for three classes defined based on the number of cycles observed in EVI time series. The algorithm therefore appears to provide a straightforward and efficient method for mapping cropping intensity from MODIS time series data

Tracing The Origins And Propagation Of African Easterly Waves And Mesoscale Convective Systems Using Pattern Recognition And Data Fusion

This thesis is focus on developing pattern recognition techniques to trace the origins and propagation of the Pre-Tropical Storm Debby (2006) African easterly waves (AEWs) and mesoscale convective systems (MCSs) using satellite imagery. The results are used to verify a numerical weather prediction (NWP) model. The pre-Debby MCSs’ movement and formation needs to be precisely and objectively tracked. These MCSs could be generated over mountains in North Africa, such as Ethiopian Highlands (EH), Darfur Mountains (DF), and Asir Mountains (AS), going through complicated splitting and merging processes. Thus, an objectively analyzed MCS movement is essential not only to help provide data to verify numerical modeling results, but also to help understand the formation and propagation of the African easterly waves and MCSs. The technique used could be applied to other AEWs and MCSs leading to tropical cyclogenesis. This, in turn, will improve the NWP over the data sparse areas, such as over eastern and central North Africa. The accuracy of numerical simulations of pre-tropical cyclone (TC) AEWs and MCSs is improved by fusing data with different data fusion techniques. In order to provide enhanced information to help predict the weather, this thesis investigates various techniques of data fusion. In many cases, fusing data provides more accurate and complete initial data for the NWP models which will then reduce the errors in weather prediction

Recent Advances in Image Restoration with Applications to Real World Problems

In the past few decades, imaging hardware has improved tremendously in terms of resolution, making widespread usage of images in many diverse applications on Earth and planetary missions. However, practical issues associated with image acquisition are still affecting image quality. Some of these issues such as blurring, measurement noise, mosaicing artifacts, low spatial or spectral resolution, etc. can seriously affect the accuracy of the aforementioned applications. This book intends to provide the reader with a glimpse of the latest developments and recent advances in image restoration, which includes image super-resolution, image fusion to enhance spatial, spectral resolution, and temporal resolutions, and the generation of synthetic images using deep learning techniques. Some practical applications are also included

Urban tree classification using discrete-return LiDAR and an object-level local binary pattern algorithm

© 2019, © 2019 Informa UK Limited, trading as Taylor & Francis Group. Urban trees have the potential to mitigate some of the harm brought about by rapid urbanization and population growth, as well as serious environmental degradation (e.g. soil erosion, carbon pollution and species extirpation), in cities. This paper presents a novel urban tree extraction modelling approach that uses discrete laser scanning point clouds and object-based textural analysis to (1) develop a model characterised by four sub-models, including (a) height-based split segmentation, (b) feature extraction, (c) texture analysis and (d) classification, and (2) apply this model to classify urban trees. The canopy height model is integrated with the object-level local binary pattern algorithm (LBP) to achieve high classification accuracy. The results of each sub-model reveal that the classification of urban trees based on the height at 47.14 (high) and 2.12 m (low), respectively, while based on crown widths were highest and lowest at 22.5 and 2.55 m, respectively. Results also indicate that the proposed algorithm of urban tree modelling is effective for practical use

Clearing the Clouds: Extracting 3D information from amongst the noise

Advancements permitting the rapid extraction of 3D point clouds from a variety of imaging modalities across the global landscape have provided a vast collection of high fidelity digital surface models. This has created a situation with unprecedented overabundance of 3D observations which greatly outstrips our current capacity to manage and infer actionable information. While years of research have removed some of the manual analysis burden for many tasks, human analysis is still a cornerstone of 3D scene exploitation. This is especially true for complex tasks which necessitate comprehension of scale, texture and contextual learning. In order to ameliorate the interpretation burden and enable scientific discovery from this volume of data, new processing paradigms are necessary to keep pace. With this context, this dissertation advances fundamental and applied research in 3D point cloud data pre-processing and deep learning from a variety of platforms. We show that the representation of 3D point data is often not ideal and sacrifices fidelity, context or scalability. First ground scanning terrestrial LIght Detection And Ranging (LiDAR) models are shown to have an inherent statistical bias, and present a state of the art method for correcting this, while preserving data fidelity and maintaining semantic structure. This technique is assessed in the dense canopy of Micronesia, with our technique being the best at retaining high levels of detail under extreme down-sampling (\u3c 1%). Airborne systems are then explored with a method which is presented to pre-process data to preserve a global contrast and semantic content in deep learners. This approach is validated with a building footprint detection task from airborne imagery captured in Eastern TN from the 3D Elevation Program (3DEP), our approach was found to achieve significant accuracy improvements over traditional techniques. Finally, topography data spanning the globe is used to assess past and previous global land cover change. Utilizing Shuttle Radar Topography Mission (SRTM) and Moderate Resolution Imaging Spectroradiometer (MODIS) data, paired with the airborne preprocessing technique described previously, a model for predicting land-cover change from topography observations is described. The culmination of these efforts have the potential to enhance the capabilities of automated 3D geospatial processing, substantially lightening the burden of analysts, with implications improving our responses to global security, disaster response, climate change, structural design and extraplanetary exploration

Computational intelligence approaches to robotics, automation, and control [Volume guest editors]

No abstract available