Deep learning in remote sensing: a review

Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin

Continual Adaptation for Deep Stereo

Depth estimation from stereo images is carried out with unmatched results by convolutional neural networks trained end-to-end to regress dense disparities. Like for most tasks, this is possible if large amounts of labelled samples are available for training, possibly covering the whole data distribution encountered at deployment time. Being such an assumption systematically unmet in real applications, the capacity of adapting to any unseen setting becomes of paramount importance. Purposely, we propose a continual adaptation paradigm for deep stereo networks designed to deal with challenging and ever-changing environments. We design a lightweight and modular architecture, Modularly ADaptive Network (MADNet), and formulate Modular ADaptation algorithms (MAD, MAD++) which permit efficient optimization of independent sub-portions of the entire network. In our paradigm, the learning signals needed to continuously adapt models online can be sourced from self-supervision via right-to-left image warping or from traditional stereo algorithms. With both sources, no other data than the input images being gathered at deployment time are needed. Thus, our network architecture and adaptation algorithms realize the first real-time self-adaptive deep stereo system and pave the way for a new paradigm that can facilitate practical deployment of end-to-end architectures for dense disparity regression

Near-infrared modeling and enhanced visualization, as a novel approach for 3D decay mapping of stone sculptures

Representation of the surface pathology of heritage objects imposes a problematic task. It usually involves the implementation of on-site visual inspections, and diagnostic procedures on-site, and after sampling, through minimally destructive laboratory tests, to produce area-specific results or two-dimensional mapping visualizations. Mapping of stone weathering is usually performed manually with time-consuming two-dimensional approaches, thus losing the importance of topology and, in general, its threedimensional metric quality. The recent introduction of modified cameras to heritage science has enabled enhanced observation at higher resolutions, concomitantly having the capacity to produce datasets that can be used for direct image-based three-dimensional reconstruction. With this article, we present a novel work combining near-infrared imaging using a modified sensor, and contemporary dense multiple-image reconstruction software, to produce spectral models of historical stone sculptures. This combined approach enables the simultaneous capturing of the shape of the historical stone surfaces and the different responses of deteriorated materials in the near-infrared spectrum. Thus, we investigate the capacity of the suggested method to assist threedimensional diagnosis and mapping of stone weathering. We explore the usability of produced spectral textures via classification and three-dimensional segmentation techniques to obtain and assess different types of visualization. We additionally evaluate the produced models for their metric and radiometric properties, by comparing them with models produced with visible spectrum imagery, acquired with similar capturing parameters