Deep learning in remote sensing: a review

Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin

Towards Automatic SAR-Optical Stereogrammetry over Urban Areas using Very High Resolution Imagery

In this paper we discuss the potential and challenges regarding SAR-optical stereogrammetry for urban areas, using very-high-resolution (VHR) remote sensing imagery. Since we do this mainly from a geometrical point of view, we first analyze the height reconstruction accuracy to be expected for different stereogrammetric configurations. Then, we propose a strategy for simultaneous tie point matching and 3D reconstruction, which exploits an epipolar-like search window constraint. To drive the matching and ensure some robustness, we combine different established handcrafted similarity measures. For the experiments, we use real test data acquired by the Worldview-2, TerraSAR-X and MEMPHIS sensors. Our results show that SAR-optical stereogrammetry using VHR imagery is generally feasible with 3D positioning accuracies in the meter-domain, although the matching of these strongly hetereogeneous multi-sensor data remains very challenging. Keywords: Synthetic Aperture Radar (SAR), optical images, remote sensing, data fusion, stereogrammetr

Open source tool for DSMs generation from high resolution optical satellite imagery. Development and testing of an OSSIM plug-in

The fully automatic generation of digital surface models (DSMs) is still an open research issue. From recent years, computer vision algorithms have been introduced in photogrammetry in order to exploit their capabilities and efficiency in three-dimensional modelling. In this article, a new tool for fully automatic DSMs generation from high resolution satellite optical imagery is presented. In particular, a new iterative approach in order to obtain the quasi-epipolar images from the original stereopairs has been defined and deployed. This approach is implemented in a new Free and Open Source Software (FOSS) named Digital Automatic Terrain Extractor (DATE) developed at the Geodesy and Geomatics Division, University of Rome ‘La Sapienza’, and conceived as an Open Source Software Image Map (OSSIM) plug-in. DATE key features include: the epipolarity achievement in the object space, thanks to the images ground projection (Ground quasi-Epipolar Imagery (GrEI)) and the coarse-to-fine pyramidal scheme adopted; the use of computer vision algorithms in order to improve the processing efficiency and make the DSMs generation process fully automatic; the free and open source aspect of the developed code. The implemented plug-in was validated through two optical datasets, GeoEye-1 and the newest Pléiades-high resolution (HR) imagery, on Trento (Northern Italy) test site. The DSMs, generated on the basis of the metadata rational polynomial coefficients only, without any ground control point, are compared to a reference lidar in areas with different land use/land cover and morphology. The results obtained thanks to the developed workflow are good in terms of statistical parameters (root mean square error around 5 m for GeoEye-1 and around 4 m for Pléiades-HR imagery) and comparable with the results obtained through different software by other authors on the same test site, whereas in terms of efficiency DATE outperforms most of the available commercial software. These first achievements indicate good potential for the developed plug-in, which in a very near future will be also upgraded for synthetic aperture radar and tri-stereo optical imagery processing

Urban building detection from optical and insar features exploiting context

We investigate the potential of combined features of aerial images and high-resolution interferometric SAR (InSAR) data for building detection in urban areas. It is shown that completeness and correctness may be increased if we integrate both InSAR double-bounce lines and 3D lines of stereo data in addition to building hints of a single optical orthophoto. In order to exploit context information, which is crucial for object detection in urban areas, we use a Conditional Random Field approach. It proves to be a valuable method for context-based building detection with multi-sensor features

Buildings Detection in VHR SAR Images Using Fully Convolution Neural Networks

This paper addresses the highly challenging problem of automatically detecting man-made structures especially buildings in very high resolution (VHR) synthetic aperture radar (SAR) images. In this context, the paper has two major contributions: Firstly, it presents a novel and generic workflow that initially classifies the spaceborne TomoSAR point clouds $-$ generated by processing VHR SAR image stacks using advanced interferometric techniques known as SAR tomography (TomoSAR) $-$ into buildings and non-buildings with the aid of auxiliary information (i.e., either using openly available 2-D building footprints or adopting an optical image classification scheme) and later back project the extracted building points onto the SAR imaging coordinates to produce automatic large-scale benchmark labelled (buildings/non-buildings) SAR datasets. Secondly, these labelled datasets (i.e., building masks) have been utilized to construct and train the state-of-the-art deep Fully Convolution Neural Networks with an additional Conditional Random Field represented as a Recurrent Neural Network to detect building regions in a single VHR SAR image. Such a cascaded formation has been successfully employed in computer vision and remote sensing fields for optical image classification but, to our knowledge, has not been applied to SAR images. The results of the building detection are illustrated and validated over a TerraSAR-X VHR spotlight SAR image covering approximately 39 km$^2$ $-$ almost the whole city of Berlin $-$ with mean pixel accuracies of around 93.84%Comment: Accepted publication in IEEE TGR

Fusion of Interferometric SAR and Photogrammetric Elevation Data

Neurology & clinical neurophysiolog

Airborne and Terrestrial Laser Scanning Data for the Assessment of Standing and Lying Deadwood: Current Situation and New Perspectives

LiDAR technology is finding uses in the forest sector, not only for surveys in producing forests but also as a tool to gain a deeper understanding of the importance of the three-dimensional component of forest environments. Developments of platforms and sensors in the last decades have highlighted the capacity of this technology to catch relevant details, even at finer scales. This drives its usage towards more ecological topics and applications for forest management. In recent years, nature protection policies have been focusing on deadwood as a key element for the health of forest ecosystems and wide-scale assessments are necessary for the planning process on a landscape scale. Initial studies showed promising results in the identification of bigger deadwood components (e.g., snags, logs, stumps), employing data not specifically collected for the purpose. Nevertheless, many efforts should still be made to transfer the available methodologies to an operational level. Newly available platforms (e.g., Mobile Laser Scanner) and sensors (e.g., Multispectral Laser Scanner) might provide new opportunities for this field of study in the near future

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

ARTMAP-FTR: A Neural Network For Fusion Target Recognition, With Application To Sonar Classification

ART (Adaptive Resonance Theory) neural networks for fast, stable learning and prediction have been applied in a variety of areas. Applications include automatic mapping from satellite remote sensing data, machine tool monitoring, medical prediction, digital circuit design, chemical analysis, and robot vision. Supervised ART architectures, called ARTMAP systems, feature internal control mechanisms that create stable recognition categories of optimal size by maximizing code compression while minimizing predictive error in an on-line setting. Special-purpose requirements of various application domains have led to a number of ARTMAP variants, including fuzzy ARTMAP, ART-EMAP, ARTMAP-IC, Gaussian ARTMAP, and distributed ARTMAP. A new ARTMAP variant, called ARTMAP-FTR (fusion target recognition), has been developed for the problem of multi-ping sonar target classification. The development data set, which lists sonar returns from underwater objects, was provided by the Naval Surface Warfare Center (NSWC) Coastal Systems Station (CSS), Dahlgren Division. The ARTMAP-FTR network has proven to be an effective tool for classifying objects from sonar returns. The system also provides a procedure for solving more general sensor fusion problems.Office of Naval Research (N00014-95-I-0409, N00014-95-I-0657