Unsupervised Domain Adaptation Using Generative Adversarial Networks for Semantic Segmentation of Aerial Images

Segmenting aerial images is of great potential in surveillance and scene understanding of urban areas. It provides a mean for automatic reporting of the different events that happen in inhabited areas. This remarkably promotes public safety and traffic management applications. After the wide adoption of convolutional neural networks methods, the accuracy of semantic segmentation algorithms could easily surpass 80% if a robust dataset is provided. Despite this success, the deployment of a pretrained segmentation model to survey a new city that is not included in the training set significantly decreases accuracy. This is due to the domain shift between the source dataset on which the model is trained and the new target domain of the new city images. In this paper, we address this issue and consider the challenge of domain adaptation in semantic segmentation of aerial images. We designed an algorithm that reduces the domain shift impact using generative adversarial networks (GANs). In the experiments, we tested the proposed methodology on the International Society for Photogrammetry and Remote Sensing (ISPRS) semantic segmentation dataset and found that our method improves overall accuracy from 35% to 52% when passing from the Potsdam domain (considered as source domain) to the Vaihingen domain (considered as target domain). In addition, the method allows efficiently recovering the inverted classes due to sensor variation. In particular, it improves the average segmentation accuracy of the inverted classes due to sensor variation from 14% to 61%.info:eu-repo/semantics/publishedVersio

One-dimensional convolutional neural networks for spectroscopic signal regression

This paper proposes a novel approach for driving chemometric analyses from spectroscopic data and based on a convolutional neural network (CNN) architecture. For such purpose, the well-known 2-D CNN is adapted to the monodimensional nature of spectroscopic data. In particular, filtering and pooling operations as well as equations for training are revisited. We also propose an alternative to train the resulting 1D-CNN by means of particle swarm optimization. The resulting trained CNN architecture is successively exploited to extract features from a given 1D spectral signature to feed any regression method. In this work, we resorted to 2 advanced and effective methods, which are support vector machine regression and Gaussian process regression. Experimental results conducted on 3 real spectroscopic datasets show the interesting capabilities of the proposed 1D-CNN methods

Support vector machine active learning through significance space construction

Active learning is showing to be a useful approach to improve the efficiency of the classification process for remote sensing images. This letter introduces a new active learning strategy specifically developed for support vector machine (SVM) classification. It relies on the idea of the following: 1) reformulating the original classification problem into a new problem where it is needed to discriminate between significant and nonsignificant samples, according to a concept of significance which is proper to the SVM theory; and 2) constructing the corresponding significance space to suitably guide the selection of the samples potentially useful to better deal with the original classification problem. Experiments were conducted on both multi- and hyperspectral images. Results show interesting advantages of the proposed method in terms of convergence speed, stability, and sparseness

Multimodal Approach for Enhancing Biometric Authentication

Unimodal biometric systems rely on a single source or unique individual biological trait for measurement and examination. Fingerprint-based biometric systems are the most common, but they are vulnerable to presentation attacks or spoofing when a fake fingerprint is presented to the sensor. To address this issue, we propose an enhanced biometric system based on a multimodal approach using two types of biological traits. We propose to combine fingerprint and Electrocardiogram (ECG) signals to mitigate spoofing attacks. Specifically, we design a multimodal deep learning architecture that accepts fingerprints and ECG as inputs and fuses the feature vectors using stacking and channel-wise approaches. The feature extraction backbone of the architecture is based on data-efficient transformers. The experimental results demonstrate the promising capabilities of the proposed approach in enhancing the robustness of the system to presentation attacks

An automatic approach for palm tree counting in UAV images

In this paper, we develop an automatic method for counting palm trees in UAV images. First we extract a set of keypoints using the Scale Invariant Feature Transform (SIFT). Then, we analyze these keypoints with an Extreme Learning Machine (ELM) classifier a priori trained on a set of palm and no-palm keypoints. As output, the ELM classifier will mark each detected palm tree by several keypoints. Then, in order to capture the shape of each tree, we propose to merge these keypoints with an active contour method based on level-sets (LS). Finally, we further analyze the texture of the regions obtained by LS with local binary patterns (LBPs) to distinguish palm trees from other vegetations. Experimental results obtained on a UAV image acquired over a palm farm are reported and discussed

Reconstructing Cloud-Contaminated Multispectral Images With Contextualized Autoencoder Neural Networks

The accurate reconstruction of areas obscured by clouds is among the most challenging topics for the remote sensing community since a significant percentage of images archived throughout the world are affected by cloud covers which make them not fully exploitable. The purpose of this paper is to propose new methods to recover missing data in multispectral images due to the presence of clouds by relying on a formulation based on an autoencoder (AE) neural network. We suppose that clouds are opaque and their detection is performed by dedicated algorithms. The AE in our methods aims at modeling the relationship between a given cloud-free image (source image) and a cloud-contaminated image (target image). In particular, two strategies are developed: the first one performs the mapping at a pixel level while the second one at a patch level to take profit from spatial contextual information. Moreover, in order to fix the problem of the hidden layer size, a new solution combining the minimum descriptive length criterion and a Pareto-like selection procedure is introduced. The results of experiments conducted on three different data sets are reported and discussed together with a comparison with reference techniques

Learning from Data for Remote Sensing Image Analysis

Recent advances in satellite technology have led to a regular, frequent and high- resolution monitoring of Earth at the global scale, providing an unprecedented amount of Earth observation (EO) data. The growing operational capability of global Earth monitoring from space provides a wealth of information on the state of our planet Earth that waits to be mined for several different EO applications, e.g. climate change analysis, urban area studies, forestry applications, risk and damage assessment, water quality assessment, crop monitoring and so on. Recent studies in machine learning have triggered substantial performance gain for the above-mentioned tasks. Advanced machine learning models such as deep convolutional neural networks (CNNs), recursive neural networks and transformers have recently made great progress in a wide range of remote sensing (RS) tasks, such as object detection, RS image classification, image captioning and so on. The study of Bai et al. (2021) analyzes the research progress, hotspots, trends and methods in the field of deep learning in remote sensing, and deep learning is becoming an important tool for remote sensing and has been widely used in numerous remote sensing tasks related to image processing and analysis. In this context, the present special issue aims at gathering a collection of papers in the most advanced and trendy areas dealing with learning from data and with applications to remote sensing image analysis. The manuscripts can be subdivided into five groups depending mainly on the processing or learning task. A specific collection for hyperspectral imagery has been included given the special attention by the remote sensing community to this kind of data

SSDAN: Multi-Source Semi-Supervised Domain Adaptation Network for Remote Sensing Scene Classification

We present a new method for multi-source semi-supervised domain adaptation in remote sensing scene classification. The method consists of a pre-trained convolutional neural network (CNN) model, namely EfficientNet-B3, for the extraction of highly discriminative features, followed by a classification module that learns feature prototypes for each class. Then, the classification module computes a cosine distance between feature vectors of target data samples and the feature prototypes. Finally, the proposed method ends with a Softmax activation function that converts the distances into class probabilities. The feature prototypes are also divided by a temperature parameter to normalize and control the classification module. The whole model is trained on both the unlabeled and labeled target samples. It is trained to predict the correct classes utilizing the standard cross-entropy loss computed over the labeled source and target samples. At the same time, the model is trained to learn domain invariant features using another loss function based on entropy computed over the unlabeled target samples. Unlike the standard cross-entropy loss, the new entropy loss function is computed on the model’s predicted probabilities and does not need the true labels. This entropy loss, called minimax loss, needs to be maximized with respect to the classification module to learn features that are domain-invariant (hence removing the data shift), and at the same time, it should be minimized with respect to the CNN feature extractor to learn discriminative features that are clustered around the class prototypes (in other words reducing intra-class variance). To accomplish these maximization and minimization processes at the same time, we use an adversarial training approach, where we alternate between the two processes. The model combines the standard cross-entropy loss and the new minimax entropy loss and optimizes them jointly. The proposed method is tested on four RS scene datasets, namely UC Merced, AID, RESISC45, and PatternNet, using two-source and three-source domain adaptation scenarios. The experimental results demonstrate the strong capability of the proposed method to achieve impressive performance despite using only a few (six in our case) labeled target samples per class. Its performance is already better than several state-of-the-art methods, including RevGrad, ADDA, Siamese-GAN, and MSCN