Deep learning in remote sensing: a review

Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin

Target recognition for synthetic aperture radar imagery based on convolutional neural network feature fusion

Driven by the great success of deep convolutional neural networks (CNNs) that are currently used by quite a few computer vision applications, we extend the usability of visual-based CNNs into the synthetic aperture radar (SAR) data domain without employing transfer learning. Our SAR automatic target recognition (ATR) architecture efficiently extends the pretrained Visual Geometry Group CNN from the visual domain into the X-band SAR data domain by clustering its neuron layers, bridging the visual—SAR modality gap by fusing the features extracted from the hidden layers, and by employing a local feature matching scheme. Trials on the moving and stationary target acquisition dataset under various setups and nuisances demonstrate a highly appealing ATR performance gaining 100% and 99.79% in the 3-class and 10-class ATR problem, respectively. We also confirm the validity, robustness, and conceptual coherence of the proposed method by extending it to several state-of-the-art CNNs and commonly used local feature similarity/match metrics

Fusing Deep Learning and Sparse Coding for SAR ATR

We propose a multimodal and multidiscipline data fusion strategy appropriate for automatic target recognition (ATR) on synthetic aperture radar imagery. Our architecture fuses a proposed clustered version of the AlexNet convolutional neural network with sparse coding theory that is extended to facilitate an adaptive elastic net optimization concept. Evaluation on the MSTAR dataset yields the highest ATR performance reported yet, which is 99.33% and 99.86% for the three- and ten-class problems, respectively

Exploring scatterer anisotrophy in synthetic aperture radar via sub-aperture analysis

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2001.Includes bibliographical references (p. 189-193).Scattering from man-made objects in SAR imagery exhibits aspect and frequency dependencies which are not always well modeled by standard SAR imaging techniques based on the ideal point scattering model. This is particularly the case for highresolution wide-band and wide-aperture data where model deviations are even more pronounced. If ignored, these deviations will reduce recognition performance due to the model mismatch, but when appropriately accounted for, these deviations from the ideal point scattering model can be exploited as attributes to better distinguish scatterers and their respective targets. With this in mind, this thesis develops an efficient modeling framework based on a sub-aperture pyramid to utilize scatterer anisotropy for the purpose of target classification. Two approaches are presented to exploit scatterer anisotropy using the sub-aperture pyramid. The first is a nonparametric classifier that learns the azimuthal dependencies within an image and makes a classification decision based on the learned dependencies. The second approach is a parametric attribution of the observed anisotropy characterizing the azimuthal location and concentration of the scattering response. Working from the sub-aperture scattering model, we develop a hypothesis test to characterize anisotropy. We start with an isolated scatterer model which produces a test with an intuitive interpretation. We then address the problem of robustness to interfering scatterers by extending the model to account for neighboring scatterers which corrupt the anisotropy attribution.(cont.) The development of the anisotropy attribution culminates with an iterative attribution approach that identifies and compensates for neighboring scatterers. In the course of the development of the anisotropy attribution, we also study the relationship between scatterer phenomenology and our anisotropy attribution. This analysis reveals the information provided by the anisotropy attribution for two common sources of anisotropy. Furthermore, the analysis explicitly demonstrates the benefit of using wide-aperture data to produce more stable and more descriptive characterizations of scatterer anisotropy.y Andrew J. Kim.Ph.D

Classification of Radar Targets Using Invariant Features

Automatic target recognition ATR using radar commonly relies on modeling a target as a collection of point scattering centers, Features extracted from these scattering centers for input to a target classifier may be constructed that are invariant to translation and rotation, i.e., they are independent of the position and aspect angle of the target in the radar scene. Here an iterative approach for building effective scattering center models is developed, and the shape space of these models is investigated. Experimental results are obtained for three-dimensional scattering centers compressed to nineteen-dimensional feature sets, each consisting of the singular values of the matrix of scattering center locations augmented with the singular values of its second and third order monomial expansions. These feature sets are invariant to translation and rotation and permit the comparison of targets modeled by different numbers of scattering centers. A metric distance metric is used that effectively identifies targets under real world conditions that include noise and obscuration

Automatic target recognition based on cross-plot

Automatic target recognition that relies on rapid feature extraction of real-time target from photo-realistic imaging will enable efficient identification of target patterns. To achieve this objective, Cross-plots of binary patterns are explored as potential signatures for the observed target by high-speed capture of the crucial spatial features using minimal computational resources. Target recognition was implemented based on the proposed pattern recognition concept and tested rigorously for its precision and recall performance. We conclude that Cross-plotting is able to produce a digital fingerprint of a target that correlates efficiently and effectively to signatures of patterns having its identity in a target repository.Kelvin Kian Loong Wong and Derek Abbot

Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

Central to the looming paradigm shift toward data-intensive science, machine-learning techniques are becoming increasingly important. In particular, deep learning has proven to be both a major breakthrough and an extremely powerful tool in many fields. Shall we embrace deep learning as the key to everything? Or should we resist a black-box solution? These are controversial issues within the remote-sensing community. In this article, we analyze the challenges of using deep learning for remote-sensing data analysis, review recent advances, and provide resources we hope will make deep learning in remote sensing seem ridiculously simple. More importantly, we encourage remote-sensing scientists to bring their expertise into deep learning and use it as an implicit general model to tackle unprecedented, large-scale, influential challenges, such as climate change and urbanization

A Few-Shot Learning Method for SAR Images Based on Weighted Distance and Feature Fusion

Convolutional Neural Network (CNN) has been widely applied in the field of synthetic aperture radar (SAR) image recognition. Nevertheless, CNN-based recognition methods usually encounter the problem of poor feature representation ability due to insufficient labeled SAR images. In addition, the large inner-class variety and high cross-class similarity of SAR images pose a challenge for classification. To alleviate the problems mentioned above, we propose a novel few-shot learning (FSL) method for SAR image recognition, which is composed of the multi-feature fusion network (MFFN) and the weighted distance classifier (WDC). The MFFN is utilized to extract input images’ features, and the WDC outputs the classification results based on these features. The MFFN is constructed by adding a multi-scale feature fusion module (MsFFM) and a hand-crafted feature insertion module (HcFIM) to a standard CNN. The feature extraction and representation capability can be enhanced by inserting the traditional hand-crafted features as auxiliary features. With the aid of information from different scales of features, targets of the same class can be more easily aggregated. The weight generation module in WDC is designed to generate category-specific weights for query images. The WDC distributes these weights along the corresponding Euclidean distance to tackle the high cross-class similarity problem. In addition, weight generation loss is proposed to improve recognition performance by guiding the weight generation module. Experimental results on the Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset and the Vehicle and Aircraft (VA) dataset demonstrate that our proposed method surpasses several typical FSL methods