A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

The integration of freely available medium resolution optical sensors with Synthetic Aperture Radar (SAR) imagery capabilities for American bramble (Rubus cuneifolius) invasion detection and mapping.

Doctoral Degree. University of KwaZulu- Natal, Pietermaritzburg.The emergence of American bramble (Rubus cuneifolius) across South Africa has caused severe ecological and economic damage. To date, most of the efforts to mitigate its effects have been largely unsuccessful due to its prolific growth and widespread distribution. Accurate and timeous detection and mapping of Bramble is therefore critical to the development of effective eradication management plans. Hence, this study sought to determine the potential of freely available, new generation medium spatial resolution satellite imagery for the detection and mapping of American Bramble infestations within the UNESCO world heritage site of the uKhahlamba Drakensberg Park (UDP). The first part of the thesis determined the potential of conventional freely available remote sensing imagery for the detection and mapping of Bramble. Utilizing the Support Vector Machine (SVM) learning algorithm, it was established that Bramble could be detected with limited users (45%) and reasonable producers (80%) accuracies. Much of the confusion occurred between the grassland land cover class and Bramble. The second part of the study focused on fusing the new age optical imagery and Synthetic Aperture Radar (SAR) imagery for Bramble detection and mapping. The synergistic potential of fused imagery was evaluated using multiclass SVM classification algorithm. Feature level image fusion of optical imagery and SAR resulted in an overall classification accuracy of 76%, with increased users and producers’ accuracies for Bramble. These positive results offered an opportunity to explore the polarization variables associated with SAR imagery for improved classification accuracies. The final section of the study dwelt on the use of Vegetation Indices (VIs) derived from new age satellite imagery, in concert with SAR to improve Bramble classification accuracies. Whereas improvement in classification accuracies were minimal, the potential of stand-alone VIs to detect and map Bramble (80%) was noteworthy. Lastly, dual-polarized SAR was fused with new age optical imagery to determine the synergistic potential of dual-polarized SAR to increase Bramble mapping accuracies. Results indicated a marked increase in overall Bramble classification accuracy (85%), suggesting improved potential of dual-polarized SAR and optical imagery in invasive species detection and mapping. Overall, this study provides sufficient evidence of the complimentary and synergistic potential of active and passive remote sensing imagery for invasive alien species detection and mapping. Results of this study are important for supporting contemporary decision making relating to invasive species management and eradication in order to safeguard ecological biodiversity and pristine status of nationally protected areas

JERS-1 SAR and LANDSAT-5 TM image data fusion: An application approach for lithological mapping

Satellite image data fusion is an image processing set of procedures utilise either for image optimisation for visual photointerpretation, or for automated thematic classification with low error rate and high accuracy. Lithological mapping using remote sensing image data relies on the spectral and textural information of the rock units of the area to be mapped. These pieces of information can be derived from Landsat optical TM and JERS-1 SAR images respectively. Prior to extracting such information (spectral and textural) and fusing them together, geometric image co-registration between TM and the SAR, atmospheric correction of the TM, and SAR despeckling are required. In this thesis, an appropriate atmospheric model is developed and implemented utilising the dark pixel subtraction method for atmospheric correction. For SAR despeckling, an efficient new method is also developed to test whether the SAR filter used remove the textural information or not. For image optimisation for visual photointerpretation, a new method of spectral coding of the six bands of the optical TM data is developed. The new spectral coding method is used to produce efficient colour composite with high separability between the spectral classes similar to that if the whole six optical TM bands are used together. This spectral coded colour composite is used as a spectral component, which is then fused with the textural component represented by the despeckled JERS-1 SAR using the fusion tools, including the colour transform and the PCT. The Grey Level Cooccurrence Matrix (GLCM) technique is used to build the textural data set using the speckle filtered JERS-1 SAR data making seven textural GLCM measures. For automated thematic mapping and by the use of both the six TM spectral data and the seven textural GLCM measures, a new method of classification has been developed using the Maximum Likelihood Classifier (MLC). The method is named the sequential maximum likelihood classification and works efficiently by comparison the classified textural pixels, the classified spectral pixels, and the classified textural-spectral pixels, and gives the means of utilising the textural and spectral information for automated lithological mapping

A comparison of multisensor integration methods for land cover classification in the Brazilian Amazon.

Many data fusion methods are available, but it is poorly understood which fusion method is suitable for integrating Landsat Thematic Mapper (TM) and radar data for land cover classification. This research explores the integration of Landsat TM and radar images (i.e., ALOS PALSAR L-band and RADARSAT-2 C-band) for land cover classification in a moist tropical region of the Brazilian Amazon. Different data fusion methods?principal component analysis (PCA), wavelet-merging technique (Wavelet), high-pass filter resolution-merging (HPF), and normalized multiplication (NMM)?were explored. Land cover classification was conducted with maximum likelihood classification based on different scenarios. This research indicates that individual radar data yield much poorer land cover classifications than TM data, and PALSAR L-band data perform relatively better than RADARSAT-2 C-band data. Compared to the TM data, the Wavelet multisensor fusion improved overall classification by 3.3%?5.7%, HPF performed similarly, but PCA and NMM reduced overall classification accuracy by 5.1%?6.1% and 7.6% ?12.7%, respectively. Different polarization options, such as HH and HV, work similarly when used in data fusion. This research underscores the importance of selecting a suitable data fusion method that can preserve spectral fidelity while improving spatial resolution

A Multi Views Approach for Remote Sensing Fusion Based on Spectral, Spatial and Temporal Information

The objectives of this chapter are to contribute to the apprehension of image fusion approaches including concepts definition, techniques ethics and results assessment. It is structured in five sections. Following this introduction, a definition of image fusion provides involved fundamental concepts. Respectively, we explain cases in which image fusion might be useful. Most existing techniques and architectures are reviewed and classified in the third section. In fourth section, we focuses heavily on algorithms based on multi-views approach, we compares and analyses the process model and algorithms including advantages, limitations and applicability of each view. The last part of the chapter summarized the benefits and limitations of a multi-view approach image fusion; it gives some recommendations on the effectiveness and the performance of these methods. These recommendations, based on a comprehensive study and meaningful quantitative metrics, evaluate various proposed views by applying them to various environmental applications with different remotely sensed images coming from different sensors. In the concluding section, we fence the chapter with a summary and recommendations for future researches

Advanced signal processing solutions for ATR and spectrum sharing in distributed radar systems

Previously held under moratorium from 11 September 2017 until 16 February 2022This Thesis presents advanced signal processing solutions for Automatic Target Recognition (ATR) operations and for spectrum sharing in distributed radar systems. Two Synthetic Aperture Radar (SAR) ATR algorithms are described for full- and single-polarimetric images, and tested on the GOTCHA and the MSTAR datasets. The first one exploits the Krogager polarimetric decomposition in order to enhance peculiar scattering mechanisms from manmade targets, used in combination with the pseudo-Zernike image moments. The second algorithm employs the Krawtchouk image moments, that, being discrete defined, provide better representations of targets’ details. The proposed image moments based framework can be extended to the availability of several images from multiple sensors through the implementation of a simple fusion rule. A model-based micro-Doppler algorithm is developed for the identification of helicopters. The approach relies on the proposed sparse representation of the signal scattered from the helicopter’s rotor and received by the radar. Such a sparse representation is obtained through the application of a greedy sparse recovery framework, with the goal of estimating the number, the length and the rotation speed of the blades, parameters that are peculiar for each helicopter’s model. The algorithm is extended to deal with the identification of multiple helicopters flying in formation that cannot be resolved in another domain. Moreover, a fusion rule is presented to integrate the results of the identification performed from several sensors in a distributed radar system. Tests performed both on simulated signals and on real signals acquired from a scale model of a helicopter, confirm the validity of the algorithm. Finally, a waveform design framework for joint radar-communication systems is presented. The waveform is composed by quasi-orthogonal chirp sub-carriers generated through the Fractional Fourier Transform (FrFT), with the aim of preserving the radar performance of a typical Linear Frequency Modulated (LFM) pulse while embedding data to be sent to a cooperative system. Techniques aimed at optimise the design parameters and mitigate the Inter-Carrier Interference (ICI) caused by the quasiorthogonality of the chirp sub-carriers are also described. The FrFT based waveform is extensively tested and compared with Orthogonal Frequency Division Multiplexing (OFDM) and LFM waveforms, in order to assess both its radar and communication performance.This Thesis presents advanced signal processing solutions for Automatic Target Recognition (ATR) operations and for spectrum sharing in distributed radar systems. Two Synthetic Aperture Radar (SAR) ATR algorithms are described for full- and single-polarimetric images, and tested on the GOTCHA and the MSTAR datasets. The first one exploits the Krogager polarimetric decomposition in order to enhance peculiar scattering mechanisms from manmade targets, used in combination with the pseudo-Zernike image moments. The second algorithm employs the Krawtchouk image moments, that, being discrete defined, provide better representations of targets’ details. The proposed image moments based framework can be extended to the availability of several images from multiple sensors through the implementation of a simple fusion rule. A model-based micro-Doppler algorithm is developed for the identification of helicopters. The approach relies on the proposed sparse representation of the signal scattered from the helicopter’s rotor and received by the radar. Such a sparse representation is obtained through the application of a greedy sparse recovery framework, with the goal of estimating the number, the length and the rotation speed of the blades, parameters that are peculiar for each helicopter’s model. The algorithm is extended to deal with the identification of multiple helicopters flying in formation that cannot be resolved in another domain. Moreover, a fusion rule is presented to integrate the results of the identification performed from several sensors in a distributed radar system. Tests performed both on simulated signals and on real signals acquired from a scale model of a helicopter, confirm the validity of the algorithm. Finally, a waveform design framework for joint radar-communication systems is presented. The waveform is composed by quasi-orthogonal chirp sub-carriers generated through the Fractional Fourier Transform (FrFT), with the aim of preserving the radar performance of a typical Linear Frequency Modulated (LFM) pulse while embedding data to be sent to a cooperative system. Techniques aimed at optimise the design parameters and mitigate the Inter-Carrier Interference (ICI) caused by the quasiorthogonality of the chirp sub-carriers are also described. The FrFT based waveform is extensively tested and compared with Orthogonal Frequency Division Multiplexing (OFDM) and LFM waveforms, in order to assess both its radar and communication performance