Challenges and Opportunities of Multimodality and Data Fusion in Remote Sensing

International audience—Remote sensing is one of the most common ways to extract relevant information about the Earth and our environment. Remote sensing acquisitions can be done by both active (synthetic aperture radar, LiDAR) and passive (optical and thermal range, multispectral and hyperspectral) devices. According to the sensor, a variety of information about the Earth's surface can be obtained. The data acquired by these sensors can provide information about the structure (optical, synthetic aperture radar), elevation (LiDAR) and material content (multi and hyperspectral) of the objects in the image. Once considered together their comple-mentarity can be helpful for characterizing land use (urban analysis, precision agriculture), damage detection (e.g., in natural disasters such as floods, hurricanes, earthquakes, oil-spills in seas), and give insights to potential exploitation of resources (oil fields, minerals). In addition, repeated acquisitions of a scene at different times allows one to monitor natural resources and environmental variables (vegetation phenology, snow cover), anthropological effects (urban sprawl, deforestation), climate changes (desertification, coastal erosion) among others. In this paper, we sketch the current opportunities and challenges related to the exploitation of multimodal data for Earth observation. This is done by leveraging the outcomes of the Data Fusion contests, organized by the IEEE Geoscience and Remote Sensing Society since 2006. We will report on the outcomes of these contests, presenting the multimodal sets of data made available to the community each year, the targeted applications and an analysis of the submitted methods and results: How was multimodality considered and integrated in the processing chain? What were the improvements/new opportunities offered by the fusion? What were the objectives to be addressed and the reported solutions? And from this, what will be the next challenges

Extraction des informations sur la morphologie des milieux urbains par analyse des images satellites radars interférométriques

Aujourd’hui, les villes connaissent une croissance exponentielle de leur population. Le suivi de cette croissance est essentiel pour garantir le bien-être des citadins. Cependant, ce suivi nécessite des bases de données cartographiques sur les différents aspects de la morphologie urbaine. Bien que l’interférométrie satellite radar à synthèse d’ouverture (RSO) soit largement exploitée pour la création de modèles numériques de terrain (MNT) et le calcul de la déformation du terrain, son usage en milieu urbain est bien plus complexe, notamment en raison des multiples zones d’inversion, d’occlusion et d’ombre présentes dans ces milieux. Tout d’abord, des algorithmes d’extraction de l’information 2D sur la morphologie urbaine (emprise au sol des bâtiments, occupation du sol et réseau routier), s’appuyant uniquement sur des données satellites RSO mono-polarisées, ont été implémentés. L’accent a été mis sur le caractère robuste, automatique et rapide de ces algorithmes. Les résultats obtenus sont comparables à ceux présentés à partir d’images aéroportées. Après avoir testé les algorithmes à partir des images satellites en amplitude, l’apport des produits interférométriques (interférogramme et cohérence) a été évalué. Il résulte de cette approche en deux étapes que les produits interférométriques, en raison de leur faible résolution, ont un réel impact uniquement sur la segmentation des éléments de tailles importantes. En ce qui concerne l’extraction de l’information 3D sur la hauteur des bâtiments, une procédure s’appuyant sur deux interférogrammes, l’un possédant une petite ligne de base, et l’autre une grande ligne de base, a été développée. L’utilisation de ces deux interférogrammes permet de détecter la majorité des sauts de phase, tout en conservant une précision convenable. Toutefois, cette procédure n’aurait pas été optimale sans l’apport des informations 2D extraites ci-dessus, tant pour le calcul de la hauteur des bâtiments que pour la génération du MNT. L’apport de ces informations a, notamment, permis d’exclure les zones d’inversion, d’occlusion et d’ombre, qui génèrent une valeur aléatoire pour la phase.Nowadays, towns are undergoing exponential growth. The monitoring of their expansion is essential to guarantee the welfare of citizens. To do that, cartographic databases of multiple aspects of urban morphology are required. Satellite imaging using interferometric synthetic aperture radar (SAR) is widely applied to generate digital terrain models (DTM) and calculate ground deformations. However, satellite interferometric SAR in urban zones is much more complex, due in part to numerous reversal, occlusion and shaded areas. First of all, algorithms to extract the 2D information on urban morphology (building footprints, land cover and road network) have been implemented. These algorithms are based only on single-polarized satellite SAR data. The decision on the type of approach was driven by robustness, automatic and speed criteria. Achieved results are comparable to results presented with aircraft images. Once algorithms have been tested on satellite intensity images, the contribution of interferometric products (interferogram and coherence) have been evaluated. Thanks to this two-step approach, we found that interferometric products have a significant contribution to segment big size objects only. Concerning the extraction of the 3D information on building heights, a method based on two interferograms, with a short and a long baseline respectively, has been developed. This approach allows to detect a large number of phase jumps while preserving a reasonable accuracy. However, this method would not have been possible without the contribution of the 2D information extracted earlier, whether for the calculation building height or for the generation of DTM. Among other things, this additional information allows to resolve the phase disturbance generated by reversal, occlusion and shaded areas

Statistical Fusion of Multi-aspect Synthetic Aperture Radar Data for Automatic Road Extraction

In this dissertation, a new statistical fusion for automatic road extraction from SAR images taken from different looking angles (i.e. multi-aspect SAR data) was presented. The main input to the fusion is extracted line features. The fusion is carried out on decision-level and is based on Bayesian network theory