Graph-based Data Modeling and Analysis for Data Fusion in Remote Sensing

Hyperspectral imaging provides the capability of increased sensitivity and discrimination over traditional imaging methods by combining standard digital imaging with spectroscopic methods. For each individual pixel in a hyperspectral image (HSI), a continuous spectrum is sampled as the spectral reflectance/radiance signature to facilitate identification of ground cover and surface material. The abundant spectrum knowledge allows all available information from the data to be mined. The superior qualities within hyperspectral imaging allow wide applications such as mineral exploration, agriculture monitoring, and ecological surveillance, etc. The processing of massive high-dimensional HSI datasets is a challenge since many data processing techniques have a computational complexity that grows exponentially with the dimension. Besides, a HSI dataset may contain a limited number of degrees of freedom due to the high correlations between data points and among the spectra. On the other hand, merely taking advantage of the sampled spectrum of individual HSI data point may produce inaccurate results due to the mixed nature of raw HSI data, such as mixed pixels, optical interferences and etc. Fusion strategies are widely adopted in data processing to achieve better performance, especially in the field of classification and clustering. There are mainly three types of fusion strategies, namely low-level data fusion, intermediate-level feature fusion, and high-level decision fusion. Low-level data fusion combines multi-source data that is expected to be complementary or cooperative. Intermediate-level feature fusion aims at selection and combination of features to remove redundant information. Decision level fusion exploits a set of classifiers to provide more accurate results. The fusion strategies have wide applications including HSI data processing. With the fast development of multiple remote sensing modalities, e.g. Very High Resolution (VHR) optical sensors, LiDAR, etc., fusion of multi-source data can in principal produce more detailed information than each single source. On the other hand, besides the abundant spectral information contained in HSI data, features such as texture and shape may be employed to represent data points from a spatial perspective. Furthermore, feature fusion also includes the strategy of removing redundant and noisy features in the dataset. One of the major problems in machine learning and pattern recognition is to develop appropriate representations for complex nonlinear data. In HSI processing, a particular data point is usually described as a vector with coordinates corresponding to the intensities measured in the spectral bands. This vector representation permits the application of linear and nonlinear transformations with linear algebra to find an alternative representation of the data. More generally, HSI is multi-dimensional in nature and the vector representation may lose the contextual correlations. Tensor representation provides a more sophisticated modeling technique and a higher-order generalization to linear subspace analysis. In graph theory, data points can be generalized as nodes with connectivities measured from the proximity of a local neighborhood. The graph-based framework efficiently characterizes the relationships among the data and allows for convenient mathematical manipulation in many applications, such as data clustering, feature extraction, feature selection and data alignment. In this thesis, graph-based approaches applied in the field of multi-source feature and data fusion in remote sensing area are explored. We will mainly investigate the fusion of spatial, spectral and LiDAR information with linear and multilinear algebra under graph-based framework for data clustering and classification problems

REMOTELY SENSED IMAGE FAST CLASSIFICATION AND SMART THEMATIC MAP PRODUCTION

Abstract. Apps available for Smartphone, as well as software for GNSS/GIS devices, permit to easily mapping the localization and shape of an area by acquiring the vertices coordinates of its contour. This option is useful for remote sensing classification, supporting the detection of representative sample sites of a known cover type to use for algorithm training or to test classification results. This article aims to analyse the possibility to produce smart maps from remotely sensed image classification in rapid way: the attention is focalized on different methods that are compared to identify fast and accurate procedure for producing up-to-date and reliable maps. Landsat 8 OLI multispectral images of northern Sicily (Italy) are submitted to various classification algorithms to distinguish water, bare soil and vegetation. The resulting map is useful for many purposes: appropriately inserted in a larger database aimed at representing the situation in a space-time evolutionary scenario, it is suitable whenever you want to capture the variation induced in a scene, e.g. burnt areas identification, vegetated areas definition for tourist-recreational purposes, etc. Particularly, pixel-based classification approaches are preferred, and experiments are carried out using unsupervised (k-means), vegetation index (NDVI, Normalized Difference Vegetation Index), supervised (minimum distance, maximum likelihood) methods. Using test sites, confusion matrix is built for each method, and quality indices are calculated to compare the results. Experiments demonstrate that NDVI submitted to k-means algorithm allows the best performance for distinguishing not only vegetation areas but also water bodies and bare soils. The resulting thematic map is converted for web publishing

Remotely sensed image fast classification and smart thematic map production

Apps available for Smartphone, as well as software for GNSS/GIS devices, permit to easily mapping the localization and shape of an area by acquiring the vertices coordinates of its contour. This option is useful for remote sensing classification, supporting the detection of representative sample sites of a known cover type to use for algorithm training or to test classification results. This article aims to analyse the possibility to produce smart maps from remotely sensed image classification in rapid way: The attention is focalized on different methods that are compared to identify fast and accurate procedure for producing up-To-date and reliable maps. Landsat 8 OLI multispectral images of northern Sicily (Italy) are submitted to various classification algorithms to distinguish water, bare soil and vegetation. The resulting map is useful for many purposes: Appropriately inserted in a larger database aimed at representing the situation in a space-Time evolutionary scenario, it is suitable whenever you want to capture the variation induced in a scene, e.g. burnt areas identification, vegetated areas definition for tourist-recreational purposes, etc. Particularly, pixel-based classification approaches are preferred, and experiments are carried out using unsupervised (k-means), vegetation index (NDVI, Normalized Difference Vegetation Index), supervised (minimum distance, maximum likelihood) methods. Using test sites, confusion matrix is built for each method, and quality indices are calculated to compare the results. Experiments demonstrate that NDVI submitted to k-means algorithm allows the best performance for distinguishing not only vegetation areas but also water bodies and bare soils. The resulting thematic map is converted for web publishing

Urban land cover mapping using medium spatial resolution satellite imageries: effectiveness of Decision Tree Classifier

The study is inserted in the framework of information extraction from satellite imageries for supporting rapid mapping activities, where information need to be extracted quickly and the elimination, also if partially, of manual digitalization procedures, can be considered a great breakthrough. The main aim of this study was therefore to develop algorithms for the extraction of urban layer by means of medium spatial resolution Landsat data processing; Decision Tree classifier was investigated as classification techniques, thus it allows to extract rules that can be later applied to different scenes. In particular, the aim was to evaluate which steps to perform in order to obtain a good classification procedure, mainly focusing on processing that can be applied to images and on training set features. The training set was evaluated on the basis of the number of classes to use for its creation, together with the temporal extension of the training set and input attributes, while images were submitted to different kind of radiometric pre and post-processing. The aim was the evaluation of the best variables to set for the creation of the training set, to be used for the classifier generation. Above-mentioned variables were compared and results evaluated on the basis of reached accuracies. Data used for the validation were derived from the Digital Regional Technical Ma

Deep learning-based change detection in remote sensing images:a review

Images gathered from different satellites are vastly available these days due to the fast development of remote sensing (RS) technology. These images significantly enhance the data sources of change detection (CD). CD is a technique of recognizing the dissimilarities in the images acquired at distinct intervals and are used for numerous applications, such as urban area development, disaster management, land cover object identification, etc. In recent years, deep learning (DL) techniques have been used tremendously in change detection processes, where it has achieved great success because of their practical applications. Some researchers have even claimed that DL approaches outperform traditional approaches and enhance change detection accuracy. Therefore, this review focuses on deep learning techniques, such as supervised, unsupervised, and semi-supervised for different change detection datasets, such as SAR, multispectral, hyperspectral, VHR, and heterogeneous images, and their advantages and disadvantages will be highlighted. In the end, some significant challenges are discussed to understand the context of improvements in change detection datasets and deep learning models. Overall, this review will be beneficial for the future development of CD methods

Understanding Heterogeneous EO Datasets: A Framework for Semantic Representations

Earth observation (EO) has become a valuable source of comprehensive, reliable, and persistent information for a wide number of applications. However, dealing with the complexity of land cover is sometimes difficult, as the variety of EO sensors reflects in the multitude of details recorded in several types of image data. Their properties dictate the category and nature of the perceptible land structures. The data heterogeneity hampers proper understanding, preventing the definition of universal procedures for content exploitation. The main shortcomings are due to the different human and sensor perception on objects, as well as to the lack of coincidence between visual elements and similarities obtained by computation. In order to bridge these sensory and semantic gaps, the paper presents a compound framework for EO image information extraction. The proposed approach acts like a common ground between the user's understanding, who is visually shortsighted to the visible domain, and the machines numerical interpretation of a much wider information. A hierarchical data representation is considered. At first, basic elements are automatically computed. Then, users can enforce their judgement on the data processing results until semantic structures are revealed. This procedure completes a user-machine knowledge transfer. The interaction is formalized as a dialogue, where communication is determined by a set of parameters guiding the computational process at each level of representation. The purpose is to maintain the data-driven observable connected to the level of semantics and to human awareness. The proposed concept offers flexibility and interoperability to users, allowing them to generate those results that best fit their application scenario. The experiments performed on different satellite images demonstrate the ability to increase the performances in case of semantic annotation by adjusting a set of parameters to the particularities of the analyzed data

Object Detection in High Resolution Aerial Images and Hyperspectral Remote Sensing Images

With rapid developments in satellite and sensor technologies, there has been a dramatic increase in the availability of remotely sensed images. However, the exploration of these images still involves a tremendous amount of human interventions, which are tedious, time-consuming, and inefficient. To help imaging experts gain a complete understanding of the images and locate the objects of interest in a more accurate and efficient way, there is always an urgent need for developing automatic detection algorithms. In this work, we delve into the object detection problems in remote sensing applications, exploring the detection algorithms for both hyperspectral images (HSIs) and high resolution aerial images. In the first part, we focus on the subpixel target detection problem in HSIs with low spatial resolutions, where the objects of interest are much smaller than the image pixel spatial resolution. To this end, we explore the detection frameworks that integrate image segmentation techniques in designing the matched filters (MFs). In particular, we propose a novel image segmentation algorithm to identify the spatial-spectral coherent image regions, from which the background statistics were estimated for deriving the MFs. Extensive experimental studies were carried out to demonstrate the advantages of the proposed subpixel target detection framework. Our studies show the superiority of the approach when comparing to state-of-the-art methods. The second part of the thesis explores the object based image analysis (OBIA) framework for geospatial object detection in high resolution aerial images. Specifically, we generate a tree representation of the aerial images from the output of hierarchical image segmentation algorithms and reformulate the object detection problem into a tree matching task. We then proposed two tree-matching algorithms for the object detection framework. We demonstrate the efficiency and effectiveness of the proposed tree-matching based object detection framework. In the third part, we study object detection in high resolution aerial images from a machine learning perspective. We investigate both traditional machine learning based framework and end-to-end convolutional neural network (CNN) based approach for various object detection tasks. In the traditional detection framework, we propose to apply the Gaussian process classifier (GPC) to train an object detector and demonstrate the advantages of the probabilistic classification algorithm. In the CNN based approach, we proposed a novel scale transfer module that generates enhanced feature maps for object detection. Our results show the efficiency and competitiveness of the proposed algorithms when compared to state-of-the-art counterparts

GETNET: A General End-to-end Two-dimensional CNN Framework for Hyperspectral Image Change Detection

Change detection (CD) is an important application of remote sensing, which provides timely change information about large-scale Earth surface. With the emergence of hyperspectral imagery, CD technology has been greatly promoted, as hyperspectral data with the highspectral resolution are capable of detecting finer changes than using the traditional multispectral imagery. Nevertheless, the high dimension of hyperspectral data makes it difficult to implement traditional CD algorithms. Besides, endmember abundance information at subpixel level is often not fully utilized. In order to better handle high dimension problem and explore abundance information, this paper presents a General End-to-end Two-dimensional CNN (GETNET) framework for hyperspectral image change detection (HSI-CD). The main contributions of this work are threefold: 1) Mixed-affinity matrix that integrates subpixel representation is introduced to mine more cross-channel gradient features and fuse multi-source information; 2) 2-D CNN is designed to learn the discriminative features effectively from multi-source data at a higher level and enhance the generalization ability of the proposed CD algorithm; 3) A new HSI-CD data set is designed for the objective comparison of different methods. Experimental results on real hyperspectral data sets demonstrate the proposed method outperforms most of the state-of-the-arts