Deep learning in remote sensing: a review

Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin

Oil spill detection using optical sensors: a multi-temporal approach

Oil pollution is one of the most destructive consequences due to human activities in the marine environment. Oil wastes come from many sources and take decades to be disposed of. Satellite based remote sensing systems can be implemented into a surveillance and monitoring network. In this study, a multi-temporal approach to the oil spill detection problem is investigated. Change Detection (CD) analysis was applied to MODIS/Terra and Aqua and OLI/Landsat 8 images of several reported oil spill events, characterized by different geographic location, sea conditions, source and extension of the spill. Toward the development of an automatic detection algorithm, a Change Vector Analysis (CVA) technique was implemented to carry out the comparison between the current image of the area of interest and a dataset of reference image, statistically analyzed to reduce the sea spectral variability between different dates. The proposed approach highlights the optical sensors’ capabilities in detecting oil spills at sea. The effectiveness of different sensors’ resolution towards the detection of spills of different size, and the relevance of the sensors’ revisiting time to track and monitor the evolution of the event is also investigated

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Classification accuracy increase using multisensor data fusion

The practical use of very high resolution visible and near-infrared (VNIR) data is still growing (IKONOS, Quickbird, GeoEye-1, etc.) but for classification purposes the number of bands is limited in comparison to full spectral imaging. These limitations may lead to the confusion of materials such as different roofs, pavements, roads, etc. and therefore may provide wrong interpretation and use of classification products. Employment of hyperspectral data is another solution, but their low spatial resolution (comparing to multispectral data) restrict their usage for many applications. Another improvement can be achieved by fusion approaches of multisensory data since this may increase the quality of scene classification. Integration of Synthetic Aperture Radar (SAR) and optical data is widely performed for automatic classification, interpretation, and change detection. In this paper we present an approach for very high resolution SAR and multispectral data fusion for automatic classification in urban areas. Single polarization TerraSAR-X (SpotLight mode) and multispectral data are integrated using the INFOFUSE framework, consisting of feature extraction (information fission), unsupervised clustering (data representation on a finite domain and dimensionality reduction), and data aggregation (Bayesian or neural network). This framework allows a relevant way of multisource data combination following consensus theory. The classification is not influenced by the limitations of dimensionality, and the calculation complexity primarily depends on the step of dimensionality reduction. Fusion of single polarization TerraSAR-X, WorldView-2 (VNIR or full set), and Digital Surface Model (DSM) data allow for different types of urban objects to be classified into predefined classes of interest with increased accuracy. The comparison to classification results of WorldView-2 multispectral data (8 spectral bands) is provided and the numerical evaluation of the method in comparison to other established methods illustrates the advantage in the classification accuracy for many classes such as buildings, low vegetation, sport objects, forest, roads, rail roads, etc

Digital image correlation (DIC) analysis of the 3 December 2013 Montescaglioso landslide (Basilicata, Southern Italy). Results from a multi-dataset investigation

Image correlation remote sensing monitoring techniques are becoming key tools for providing effective qualitative and quantitative information suitable for natural hazard assessments, specifically for landslide investigation and monitoring. In recent years, these techniques have been successfully integrated and shown to be complementary and competitive with more standard remote sensing techniques, such as satellite or terrestrial Synthetic Aperture Radar interferometry. The objective of this article is to apply the proposed in-depth calibration and validation analysis, referred to as the Digital Image Correlation technique, to measure landslide displacement. The availability of a multi-dataset for the 3 December 2013 Montescaglioso landslide, characterized by different types of imagery, such as LANDSAT 8 OLI (Operational Land Imager) and TIRS (Thermal Infrared Sensor), high-resolution airborne optical orthophotos, Digital Terrain Models and COSMO-SkyMed Synthetic Aperture Radar, allows for the retrieval of the actual landslide displacement field at values ranging from a few meters (2–3 m in the north-eastern sector of the landslide) to 20–21 m (local peaks on the central body of the landslide). Furthermore, comprehensive sensitivity analyses and statistics-based processing approaches are used to identify the role of the background noise that affects the whole dataset. This noise has a directly proportional relationship to the different geometric and temporal resolutions of the processed imagery. Moreover, the accuracy of the environmental-instrumental background noise evaluation allowed the actual displacement measurements to be correctly calibrated and validated, thereby leading to a better definition of the threshold values of the maximum Digital Image Correlation sub-pixel accuracy and reliability (ranging from 1/10 to 8/10 pixel) for each processed dataset

Real-time multiple-look synthetic aperture radar processor for spacecraft applications

A spaceborne synthetic aperture radar (SAR) having pipeline multiple-look data processing is described which makes use of excessive azimuth bandwidth in radar echo signals to produce multiple-looking images. Time multiplexed single-look image lines from an azimuth correlator go through an energy analyzer which analyzes the mean energy in each separate look to determine the radar antenna electric boresight for use in generating the correct reference functions for the production of high quality SAR images. The multiplexed single look image lines also go through a registration delay to produce multi-look images

High-resolution optical and SAR image fusion for building database updating

This paper addresses the issue of cartographic database (DB) creation or updating using high-resolution synthetic aperture radar and optical images. In cartographic applications, objects of interest are mainly buildings and roads. This paper proposes a processing chain to create or update building DBs. The approach is composed of two steps. First, if a DB is available, the presence of each DB object is checked in the images. Then, we verify if objects coming from an image segmentation should be included in the DB. To do those two steps, relevant features are extracted from images in the neighborhood of the considered object. The object removal/inclusion in the DB is based on a score obtained by the fusion of features in the framework of Dempster–Shafer evidence theory