Urban Land Cover Classification with Missing Data Modalities Using Deep Convolutional Neural Networks

Automatic urban land cover classification is a fundamental problem in remote sensing, e.g. for environmental monitoring. The problem is highly challenging, as classes generally have high inter-class and low intra-class variance. Techniques to improve urban land cover classification performance in remote sensing include fusion of data from different sensors with different data modalities. However, such techniques require all modalities to be available to the classifier in the decision-making process, i.e. at test time, as well as in training. If a data modality is missing at test time, current state-of-the-art approaches have in general no procedure available for exploiting information from these modalities. This represents a waste of potentially useful information. We propose as a remedy a convolutional neural network (CNN) architecture for urban land cover classification which is able to embed all available training modalities in a so-called hallucination network. The network will in effect replace missing data modalities in the test phase, enabling fusion capabilities even when data modalities are missing in testing. We demonstrate the method using two datasets consisting of optical and digital surface model (DSM) images. We simulate missing modalities by assuming that DSM images are missing during testing. Our method outperforms both standard CNNs trained only on optical images as well as an ensemble of two standard CNNs. We further evaluate the potential of our method to handle situations where only some DSM images are missing during testing. Overall, we show that we can clearly exploit training time information of the missing modality during testing

Image fusion techniqes for remote sensing applications

Image fusion refers to the acquisition, processing and synergistic combination of information provided by various sensors or by the same sensor in many measuring contexts. The aim of this survey paper is to describe three typical applications of data fusion in remote sensing. The first study case considers the problem of the Synthetic Aperture Radar (SAR) Interferometry, where a pair of antennas are used to obtain an elevation map of the observed scene; the second one refers to the fusion of multisensor and multitemporal (Landsat Thematic Mapper and SAR) images of the same site acquired at different times, by using neural networks; the third one presents a processor to fuse multifrequency, multipolarization and mutiresolution SAR images, based on wavelet transform and multiscale Kalman filter. Each study case presents also results achieved by the proposed techniques applied to real data

Polarimetric Synthetic Aperture Radar (SAR) Application for Geological Mapping and Resource Exploration in the Canadian Arctic

The role of remote sensing in geological mapping has been rapidly growing by providing predictive maps in advance of field surveys. Remote predictive maps with broad spatial coverage have been produced for northern Canada and the Canadian Arctic which are typically very difficult to access. Multi and hyperspectral airborne and spaceborne sensors are widely used for geological mapping as spectral characteristics are able to constrain the minerals and rocks that are present in a target region. Rock surfaces in the Canadian Arctic are altered by extensive glacial activity and freeze-thaw weathering, and form different surface roughnesses depending on rock type. Different physical surface properties, such as surface roughness and soil moisture, can be revealed by distinct radar backscattering signatures at different polarizations. This thesis aims to provide a multidisciplinary approach for remote predictive mapping that integrates the lithological and physical surface properties of target rocks. This work investigates the physical surface properties of geological units in the Tunnunik and Haughton impact structures in the Canadian Arctic characterized by polarimetric synthetic aperture radar (SAR). It relates the radar scattering mechanisms of target surfaces to their lithological compositions from multispectral analysis for remote predictive geological mapping in the Canadian Arctic. This work quantitatively estimates the surface roughness relative to the transmitted radar wavelength and volumetric soil moisture by radar scattering model inversion. The SAR polarization signatures of different geological units were also characterized, which showed a significant correlation with their surface roughness. This work presents a modified radar scattering model for weathered rock surfaces. More broadly, it presents an integrative remote predictive mapping algorithm by combining multispectral and polarimetric SAR parameters

Land-cover classification in the Brazilian Amazon with the integration of Landsat ETM + and Radarsat data.

Land-cover classification with remotely sensed data in moist tropical regions in a challenge due to the complex biophysical conditions. This paper explores techniques to improve land-cover classification accuracy through a comparative analysis of different combinations of spectral signatures and textures from Landsat Enhanced Thematic Mapper Plus (ETM +) and Radarsat data. A wavelet-merging technique was used to integrate Landsat ETM + multispectral and panchromatic data or Radarsat data. Grey-level co-occurrence matrix (GLCM) textures based on Landsat ETM + panchromatic of Radarsat data and different sizes of moving windows were examined. A maximum-likelihood classifier was used to implement image classification for different combinations. This research indicates the important role of textures in improving land-cover classification accuracies in Amazonian environments. ..