Supervised learning on graphs of spatio-temporal similarity in satellite image sequences

High resolution satellite image sequences are multidimensional signals composed of spatio-temporal patterns associated to numerous and various phenomena. Bayesian methods have been previously proposed in (Heas and Datcu, 2005) to code the information contained in satellite image sequences in a graph representation using Bayesian methods. Based on such a representation, this paper further presents a supervised learning methodology of semantics associated to spatio-temporal patterns occurring in satellite image sequences. It enables the recognition and the probabilistic retrieval of similar events. Indeed, graphs are attached to statistical models for spatio-temporal processes, which at their turn describe physical changes in the observed scene. Therefore, we adjust a parametric model evaluating similarity types between graph patterns in order to represent user-specific semantics attached to spatio-temporal phenomena. The learning step is performed by the incremental definition of similarity types via user-provided spatio-temporal pattern examples attached to positive or/and negative semantics. From these examples, probabilities are inferred using a Bayesian network and a Dirichlet model. This enables to links user interest to a specific similarity model between graph patterns. According to the current state of learning, semantic posterior probabilities are updated for all possible graph patterns so that similar spatio-temporal phenomena can be recognized and retrieved from the image sequence. Few experiments performed on a multi-spectral SPOT image sequence illustrate the proposed spatio-temporal recognition method

Further results on dissimilarity spaces for hyperspectral images RF-CBIR

Content-Based Image Retrieval (CBIR) systems are powerful search tools in image databases that have been little applied to hyperspectral images. Relevance feedback (RF) is an iterative process that uses machine learning techniques and user's feedback to improve the CBIR systems performance. We pursued to expand previous research in hyperspectral CBIR systems built on dissimilarity functions defined either on spectral and spatial features extracted by spectral unmixing techniques, or on dictionaries extracted by dictionary-based compressors. These dissimilarity functions were not suitable for direct application in common machine learning techniques. We propose to use a RF general approach based on dissimilarity spaces which is more appropriate for the application of machine learning algorithms to the hyperspectral RF-CBIR. We validate the proposed RF method for hyperspectral CBIR systems over a real hyperspectral dataset.Comment: In Pattern Recognition Letters (2013

CMIR-NET : A Deep Learning Based Model For Cross-Modal Retrieval In Remote Sensing

We address the problem of cross-modal information retrieval in the domain of remote sensing. In particular, we are interested in two application scenarios: i) cross-modal retrieval between panchromatic (PAN) and multi-spectral imagery, and ii) multi-label image retrieval between very high resolution (VHR) images and speech based label annotations. Notice that these multi-modal retrieval scenarios are more challenging than the traditional uni-modal retrieval approaches given the inherent differences in distributions between the modalities. However, with the growing availability of multi-source remote sensing data and the scarcity of enough semantic annotations, the task of multi-modal retrieval has recently become extremely important. In this regard, we propose a novel deep neural network based architecture which is considered to learn a discriminative shared feature space for all the input modalities, suitable for semantically coherent information retrieval. Extensive experiments are carried out on the benchmark large-scale PAN - multi-spectral DSRSID dataset and the multi-label UC-Merced dataset. Together with the Merced dataset, we generate a corpus of speech signals corresponding to the labels. Superior performance with respect to the current state-of-the-art is observed in all the cases

Sentinel 1 and 2: Searching for Physical Image Content

ESA’s upcoming Sentinel-1 and Sentinel-2 missions open new perspectives for the application-oriented use of SAR and/or optical multispectral images. We expect short and regular revisit times as well as easily available and well documented products with attractive features such as cross-polarized SAR images and optical images delivered, for instance, as spectral reflectance data. Thus, users do not have to live any longer with simple digital units or detector counts; instead, the data provided as Sentinel products can be understood as samples of calibrated and validated physical quantities. As a consequence, users can concentrate immediately on the physics and quantitative details of the observable phenomena. This also affects content-based image retrieval, where a user searches for images containing phenomena being similar to given examples. While retrieval systems based on visible image data can only exploit characteristic shapes or patterns, the use of Sentinel data will address the determination of real physical relationships. In particular, this allows a physics-based analysis of image time series data, where one analyzes spatio-temporal phenomena. This physics-based approach will allow us to employ content-based image retrieval as an attractive tool for the analysis of SAR and optical images

Expanding the Algorithmic Information Theory Frame for Applications to Earth Observation

Recent years have witnessed an increased interest towards compression-based methods and their applications to remote sensing, as these have a data-driven and parameter-free approach and can be thus succesfully employed in several applications, especially in image information mining. This paper expands the algorithmic information theory frame, on which these methods are based. On the one hand, algorithms originally defined in the pattern matching domain are reformulated, allowing a better understanding of the available compression-based tools for remote sensing applications. On the other hand, the use of existing compression algorithms is proposed to store satellite images with added semantic value

Accelerated Probabilistic Learning Concept for Mining Heterogeneous Earth Observation Images

We present an accelerated probabilistic learning concept and its prototype implementation for mining heterogeneous Earth observation images, e.g., multispectral images, synthetic aperture radar (SAR) images, image time series, or geographical information systems (GIS) maps. The system prototype combines, at pixel level, the unsupervised clustering results of different features, extracted from heterogeneous satellite images and geographical information resources, with user-defined semantic annotations in order to calculate the posterior probabilities that allow the final probabilistic searches. The system is able to learn different semantic labels based on a newly developed Bayesian networks algorithm and allows different probabilistic retrieval methods of all semantically related images with only a few user interactions. The new algorithm reduces the computational cost, overperforming existing conventional systems, under certain conditions, by several orders of magnitude. The achieved speed-up allows the introduction of new feature models improving the learning capabilities of knowledge-driven image information mining systems and opening them to Big Data environment

Automatic Change Analysis in Satellite Images Using Binary Descriptors and Lloyd–Max Quantization

In this letter, we present a novel technique for unsupervised change analysis that leads to a method of ranking the changes that occur between two satellite images acquired at different moments of time. The proposed change analysis is based on binary descriptors and uses the Hamming distance as a similarity metric. In order to render a completely unsupervised solution, the obtained distances are further classified using vector quantization methods (i.e., Lloyd's algorithm for optimal quantization). The ultimate goal in the change analysis chain is to build change intensity maps that provide an overview of the severeness of changes in the area under analysis. In addition, the proposed analysis technique can be easily adapted for change detection by selecting only two levels for quantization. This discriminative method (i.e., between changed/unchanged zones) is compared with other previously developed techniques that use principal component analysis or Bayes theory as starting points for their analysis. The experiments are carried on Landsat images at a 30-m spatial resolution, covering an area of approximately 59×51 km2 over the surroundings of Bucharest, Romania, and containing multispectral information

A fast compression-based similarity measure with applications to content-based image retrieval

Compression-based similarity measures are effectively employed in applications on diverse data types with a basically parameter-free approach. Nevertheless, there are problems in applying these techniques to medium-to-large datasets which have been seldom addressed. This paper proposes a similarity measure based on compression with dictionaries, the Fast Compression Distance (FCD), which reduces the complexity of these methods, without degradations in performance. On its basis a content-based color image retrieval system is defined, which can be compared to state-of-the-art methods based on invariant color features. Through the FCD a better understanding of compression-based techniques is achieved, by performing experiments on datasets which are larger than the ones analyzed so far in literature

FrFT-Based Scene Classification of Phase-Gradient InSAR Images and Effective Baseline Dependence

In the literature, scene recognition from interferometric synthetic aperture radar (InSAR) images has been mainly focused on the joint use of the backscatter intensity and the coherence between interferometric image pairs. However, the terrain height information residing in the interferometric phase requires further exploration for classification purposes. In this letter, taking the interferometric phase information into account together with the backscatter intensity, the whole complex- valued InSAR image is exploited for feature extraction. In addition, a new complex-valued phase-gradient InSAR (PGInSAR) image is defined. A fractional-Fourier-transform-based feature ex traction, which was proposed for the classification of single-look complex (SLC) SAR images, is adopted for InSAR and PGInSAR images. For patch-based classification, an image database is generated from bistatic pairs acquired from the same terrain with three different effective baselines. The supervised κ-nearest neighbor classification results show that InSAR outperforms SLC by 15%, whereas PGInSAR introduces further 10% improvement over InSAR or a total improvement of 27% over SLC. Moreover, PGInSAR is found to be more robust to effective baseline changes than InSAR, which makes PGInSAR a better candidate for feature extraction

AFFECTIVE COMPUTING AND AUGMENTED REALITY FOR CAR DRIVING SIMULATORS

Car simulators are essential for training and for analyzing the behavior, the responses and the performance of the driver. Augmented Reality (AR) is the technology that enables virtual images to be overlaid on views of the real world. Affective Computing (AC) is the technology that helps reading emotions by means of computer systems, by analyzing body gestures, facial expressions, speech and physiological signals. The key aspect of the research relies on investigating novel interfaces that help building situational awareness and emotional awareness, to enable affect-driven remote collaboration in AR for car driving simulators. The problem addressed relates to the question about how to build situational awareness (using AR technology) and emotional awareness (by AC technology), and how to integrate these two distinct technologies [4], into a unique affective framework for training, in a car driving simulator