A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Design and implementation of an SDR-based multi-frequency ground-based SAR system

Synthetic Aperture Radar (SAR) has proven a valuable tool in the monitoring of the Earth, either at a global or local scales. SAR is a coherent radar system able to image extended areas with high resolution, and finds applications in many areas such as forestry, agriculture, mining, structure inspection or security operations. Although space-borne SAR systems can image extended areas, their main limitation is the long revisit times, which are not suitable for applications where the target experiments rapid changes, in the scale of minutes to few days. GBSAR systems have proven useful to fill this revisit time gap by imaging relatively small areas continuously, with extensions usually smaller than a few square kilometers. Ground Based SAR (GBSAR) systems have been used extensively for the monitoring of slope instability, and are a common tool in the mining sector. The development of the GBSAR is relatively recent, and various developments have taken place since the 2000s, transitioning from the usage of Vector Network Analyzers (VNAs) to custom radar cores tailored for this application. This transition is accompanied by a reduction in cost, but at the same time is accompanied by a loss of operational flexibility. Specifically, most GBSAR sensors now operate at a single frequency, losing the value of the multi-band operation that VNAs provided. This work is motivated by the idea that it is worth to use the value of multi-frequency GBSAR measurements, while maintaining a limited system cost. In order to implement a GBSAR with these characteristics, it is realized that Software Defined Radio (SDR) devices are a good option for fast and flexible implementation of broadband transceivers. This thesis details the design and implementation process of an SDR-based Frequency Modulated Continuous Wave (FMCW) GBSAR system from the ground up, presenting the main issues related with the usage of the most common SDR analog architecture, the Zero-IF transceiver. The main problem is determined to be the behavior of spurs related to IQ imbalances of the analog transceiver with the FMCW demodulation process. Two effective techniques to overcome these issues, the Super Spatial Variant Apodization (SSVA) and the Short Time Fourier Transform (STFT) signal reconstruction techniques, are implemented and tested. The thesis also deals with the digital implementation of the signal generator and digital receiver, which are implemented on top of an RF Network-on-Chip (RFNoC) architecture in the SDR Field Programmable Gate Array (FPGA). Another important aspect of this work is the development of an radiofrequency front-end that extends the capabilities of the SDR, implementing filtering, amplification, leakage mitigation and up-conversion to X-band. Finally, a set of test campaigns is described, in which the operation of the system is verified and the value of multi-frequency GBSAR observations is shown.El radar d'obertura sintètica (SAR) ha demostrat ser una eina valuosa en el monitoratge de la Terra, sigui a escala global o local. El SAR és un sistema de radar coherent capaç d’obtenir imatges de zones extenses amb alta resolució i té aplicacions en moltes àrees com la silvicultura, l’agricultura, la mineria, la inspecció d’estructures o les operacions de seguretat. Tot i que els sistemes SAR embarcats en plataformes orbitals poden obtenir imatges d'àrees extenses, la seva principal limitació és el temps de revisita, que no són adequats per a aplicacions on l'objectiu experimenta canvis ràpids, en una escala de minuts a pocs dies. Els sistemes GBSAR han demostrat ser útils per omplir aquesta bretxa de temps, obtenint imatges d'àrees relativament petites de manera contínua, amb extensions generalment inferiors a uns pocs quilòmetres quadrats. Els sistemes SAR terrestres (GBSAR) s’han utilitzat àmpliament per al control de la inestabilitat de talussos i esllavissades i són una eina comuna al sector miner. El desenvolupament del GBSAR és relativament recent i s’han produït diversos desenvolupaments des de la dècada de 2000, passant de l’ús d’analitzadors de xarxes vectorials (VNA) a nuclis de radar personalitzats i adaptats a aquesta aplicació. Aquesta transició s’acompanya d’una reducció del cost, però al mateix temps d’una pèrdua de flexibilitat operativa. Concretament, la majoria dels sensors GBSAR funcionen a una única freqüència, perdent el valor de l’operació en múltiples bandes que proporcionaven els VNA. Aquesta tesi està motivada per la idea de recuperar el valor de les mesures GBSAR multifreqüència, mantenint un cost del sistema limitat. Per tal d’implementar un GBSAR amb aquestes característiques, s’adona que els dispositius de ràdio definida per software (SDR) són una bona opció per a la implementació ràpida i flexible dels transceptors de banda ampla. Aquesta tesi detalla el procés de disseny i implementació d’un sistema GBSAR d’ona contínua modulada en freqüència (FMCW) basat en la tecnologia SDR, presentant els principals problemes relacionats amb l’ús de l’arquitectura analògica de SDR més comuna, el transceptor Zero-IF. Es determina que el problema principal és el comportament dels espuris relacionats amb el balanç de les cadenes de fase i quadratura del transceptor analògic amb el procés de desmodulació FMCW. S’implementen i comproven dues tècniques efectives per minimitzar aquests problemes basades en la reconstrucció de la senyal contaminada per espuris: la tècnica anomenada Super Spatial Variant Apodization (SSVA) i una tècnica basada en la transformada de Fourier amb finestra (STFT). La tesi també tracta la implementació digital del generador de senyal i del receptor digital, que s’implementen sobre una arquitectura RF Network-on-Chip (RFNoC). Un altre aspecte important d’aquesta tesi és el desenvolupament d’un front-end de radiofreqüència que amplia les capacitats de la SDR, implementant filtratge, amplificació, millora de l'aïllament entre transmissió i recepció i conversió a banda X. Finalment, es descriu un conjunt de campanyes de prova en què es verifica el funcionament del sistema i es mostra el valor de les observacions GBSAR multifreqüència

Efficient Algorithms for Large-Scale Image Analysis

This work develops highly efficient algorithms for analyzing large images. Applications include object-based change detection and screening. The algorithms are 10-100 times as fast as existing software, sometimes even outperforming FGPA/GPU hardware, because they are designed to suit the computer architecture. This thesis describes the implementation details and the underlying algorithm engineering methodology, so that both may also be applied to other applications

Synthetic Aperture Radar (SAR) Meets Deep Learning

This reprint focuses on the application of the combination of synthetic aperture radars and depth learning technology. It aims to further promote the development of SAR image intelligent interpretation technology. A synthetic aperture radar (SAR) is an important active microwave imaging sensor, whose all-day and all-weather working capacity give it an important place in the remote sensing community. Since the United States launched the first SAR satellite, SAR has received much attention in the remote sensing community, e.g., in geological exploration, topographic mapping, disaster forecast, and traffic monitoring. It is valuable and meaningful, therefore, to study SAR-based remote sensing applications. In recent years, deep learning represented by convolution neural networks has promoted significant progress in the computer vision community, e.g., in face recognition, the driverless field and Internet of things (IoT). Deep learning can enable computational models with multiple processing layers to learn data representations with multiple-level abstractions. This can greatly improve the performance of various applications. This reprint provides a platform for researchers to handle the above significant challenges and present their innovative and cutting-edge research results when applying deep learning to SAR in various manuscript types, e.g., articles, letters, reviews and technical reports