A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Sparsity enhanced MRF algorithm for automatic object detection in GPR imagery

This study addressed the problem of automated object detection from ground penetrating radar imaging (GPR), using the concept of sparse representation. The detection task is first formulated as a Markov random field (MRF) process. Then, we propose a novel detection algorithm by introducing the sparsity constraint to the standard MRF model. Specifically, the traditional approach finds it difficult to determine the central target due to the influence of different neighbors from the imaging area. As such, we introduce a domain search algorithm to overcome this issue and increase the accuracy of target detection. Additionally, in the standard MRF model, the Gibbs parameters are empirically predetermined and fixed during the detection process, yet those hyperparameters may have a significant effect on the performance of the detection. Accordingly, in this paper, Gibbs parameters are self-adaptive and fine-tuned using an iterative updating strategy followed the concept of sparse representation. Furthermore, the proposed algorithm has then been proven to have a strong convergence property theoretically. Finally, we verify the proposed method using a real-world dataset, with a set of ground penetrating radar antennas in three different transmitted frequencies (50 MHz, 200 MHz and 300 MHz). Experimental evaluations demonstrate the advantages of utilizing the proposed algorithm to detect objects in ground penetrating radar imagery, in comparison with four traditional detection algorithms

Online change detection techniques in time series: an overview

Time-series change detection has been studied in several fields. From sensor data, engineering systems, medical diagnosis, and financial markets to user actions on a network, huge amounts of temporal data are generated. There is a need for a clear separation between normal and abnormal behaviour of the system in order to investigate causes or forecast change. Characteristics include irregularities, deviations, anomalies, outliers, novelties or surprising patterns. The efficient detection of such patterns is challenging, especially when constraints need to be taken into account, such as the data velocity, volume, limited time for reacting to events, and the details of the temporal sequence.This paper reviews the main techniques for time series change point detection, focusing on online methods. Performance criteria including complexity, time granularity, and robustness is used to compare techniques, followed by a discussion about current challenges and open issue

Multidimensional Balance-Based Cluster Boundary Detection for High-Dimensional Data

© 2018 IEEE. The balance of neighborhood space around a central point is an important concept in cluster analysis. It can be used to effectively detect cluster boundary objects. The existing neighborhood analysis methods focus on the distribution of data, i.e., analyzing the characteristic of the neighborhood space from a single perspective, and could not obtain rich data characteristics. In this paper, we analyze the high-dimensional neighborhood space from multiple perspectives. By simulating each dimension of a data point's k nearest neighbors space (k NNs) as a lever, we apply the lever principle to compute the balance fulcrum of each dimension after proving its inevitability and uniqueness. Then, we model the distance between the projected coordinate of the data point and the balance fulcrum on each dimension and construct the DHBlan coefficient to measure the balance of the neighborhood space. Based on this theoretical model, we propose a simple yet effective cluster boundary detection algorithm called Lever. Experiments on both low- and high-dimensional data sets validate the effectiveness and efficiency of our proposed algorithm

Automatic Autism Spectrum Disorder Detection Using Artificial Intelligence Methods with MRI Neuroimaging: A Review

Autism spectrum disorder (ASD) is a brain condition characterized by diverse signs and symptoms that appear in early childhood. ASD is also associated with communication deficits and repetitive behavior in affected individuals. Various ASD detection methods have been developed, including neuroimaging modalities and psychological tests. Among these methods, magnetic resonance imaging (MRI) imaging modalities are of paramount importance to physicians. Clinicians rely on MRI modalities to diagnose ASD accurately. The MRI modalities are non-invasive methods that include functional (fMRI) and structural (sMRI) neuroimaging methods. However, the process of diagnosing ASD with fMRI and sMRI for specialists is often laborious and time-consuming; therefore, several computer-aided design systems (CADS) based on artificial intelligence (AI) have been developed to assist the specialist physicians. Conventional machine learning (ML) and deep learning (DL) are the most popular schemes of AI used for diagnosing ASD. This study aims to review the automated detection of ASD using AI. We review several CADS that have been developed using ML techniques for the automated diagnosis of ASD using MRI modalities. There has been very limited work on the use of DL techniques to develop automated diagnostic models for ASD. A summary of the studies developed using DL is provided in the appendix. Then, the challenges encountered during the automated diagnosis of ASD using MRI and AI techniques are described in detail. Additionally, a graphical comparison of studies using ML and DL to diagnose ASD automatically is discussed. We conclude by suggesting future approaches to detecting ASDs using AI techniques and MRI neuroimaging

Détection de changement par fusion d'images de télédétection de résolutions et modalités différentes

La détection de changements dans une scène est l’un des problèmes les plus complexes en télédétection. Il s’agit de détecter des modifications survenues dans une zone géographique donnée par comparaison d’images de cette zone acquises à différents instants. La comparaison est facilitée lorsque les images sont issues du même type de capteur c’est-à-dire correspondent à la même modalité (le plus souvent optique multi-bandes) et possèdent des résolutions spatiales et spectrales identiques. Les techniques de détection de changements non supervisées sont, pour la plupart, conçues spécifiquement pour ce scénario. Il est, dans ce cas, possible de comparer directement les images en calculant la différence de pixels homologues, c’est-à-dire correspondant au même emplacement au sol. Cependant, dans certains cas spécifiques tels que les situations d’urgence, les missions ponctuelles, la défense et la sécurité, il peut s’avérer nécessaire d’exploiter des images de modalités et de résolutions différentes. Cette hétérogénéité dans les images traitées introduit des problèmes supplémentaires pour la mise en œuvre de la détection de changements. Ces problèmes ne sont pas traités par la plupart des méthodes de l’état de l’art. Lorsque la modalité est identique mais les résolutions différentes, il est possible de se ramener au scénario favorable en appliquant des prétraitements tels que des opérations de rééchantillonnage destinées à atteindre les mêmes résolutions spatiales et spectrales. Néanmoins, ces prétraitements peuvent conduire à une perte d’informations pertinentes pour la détection de changements. En particulier, ils sont appliqués indépendamment sur les deux images et donc ne tiennent pas compte des relations fortes existant entre les deux images. L’objectif de cette thèse est de développer des méthodes de détection de changements qui exploitent au mieux l’information contenue dans une paire d’images observées, sans condition sur leur modalité et leurs résolutions spatiale et spectrale. Les restrictions classiquement imposées dans l’état de l’art sont levées grâce à une approche utilisant la fusion des deux images observées. La première stratégie proposée s’applique au cas d’images de modalités identiques mais de résolutions différentes. Elle se décompose en trois étapes. La première étape consiste à fusionner les deux images observées ce qui conduit à une image de la scène à haute résolution portant l’information des changements éventuels. La deuxième étape réalise la prédiction de deux images non observées possédant des résolutions identiques à celles des images observées par dégradation spatiale et spectrale de l’image fusionnée. Enfin, la troisième étape consiste en une détection de changements classique entre images observées et prédites de mêmes résolutions. Une deuxième stratégie modélise les images observées comme des versions dégradées de deux images non observées caractérisées par des résolutions spectrales et spatiales identiques et élevées. Elle met en œuvre une étape de fusion robuste qui exploite un a priori de parcimonie des changements observés. Enfin, le principe de la fusion est étendu à des images de modalités différentes. Dans ce cas où les pixels ne sont pas directement comparables, car correspondant à des grandeurs physiques différentes, la comparaison est réalisée dans un domaine transformé. Les deux images sont représentées par des combinaisons linéaires parcimonieuses des éléments de deux dictionnaires couplés, appris à partir des données. La détection de changements est réalisée à partir de l’estimation d’un code couplé sous condition de parcimonie spatiale de la différence des codes estimés pour chaque image. L’expérimentation de ces différentes méthodes, conduite sur des changements simulés de manière réaliste ou sur des changements réels, démontre les avantages des méthodes développées et plus généralement de l’apport de la fusion pour la détection de changement

Panchromatic and multispectral image fusion for remote sensing and earth observation: Concepts, taxonomy, literature review, evaluation methodologies and challenges ahead

Panchromatic and multispectral image fusion, termed pan-sharpening, is to merge the spatial and spectral information of the source images into a fused one, which has a higher spatial and spectral resolution and is more reliable for downstream tasks compared with any of the source images. It has been widely applied to image interpretation and pre-processing of various applications. A large number of methods have been proposed to achieve better fusion results by considering the spatial and spectral relationships among panchromatic and multispectral images. In recent years, the fast development of artificial intelligence (AI) and deep learning (DL) has significantly enhanced the development of pan-sharpening techniques. However, this field lacks a comprehensive overview of recent advances boosted by the rise of AI and DL. This paper provides a comprehensive review of a variety of pan-sharpening methods that adopt four different paradigms, i.e., component substitution, multiresolution analysis, degradation model, and deep neural networks. As an important aspect of pan-sharpening, the evaluation of the fused image is also outlined to present various assessment methods in terms of reduced-resolution and full-resolution quality measurement. Then, we conclude this paper by discussing the existing limitations, difficulties, and challenges of pan-sharpening techniques, datasets, and quality assessment. In addition, the survey summarizes the development trends in these areas, which provide useful methodological practices for researchers and professionals. Finally, the developments in pan-sharpening are summarized in the conclusion part. The aim of the survey is to serve as a referential starting point for newcomers and a common point of agreement around the research directions to be followed in this exciting area