Exploiting Deep Features for Remote Sensing Image Retrieval: A Systematic Investigation

Remote sensing (RS) image retrieval is of great significant for geological information mining. Over the past two decades, a large amount of research on this task has been carried out, which mainly focuses on the following three core issues: feature extraction, similarity metric and relevance feedback. Due to the complexity and multiformity of ground objects in high-resolution remote sensing (HRRS) images, there is still room for improvement in the current retrieval approaches. In this paper, we analyze the three core issues of RS image retrieval and provide a comprehensive review on existing methods. Furthermore, for the goal to advance the state-of-the-art in HRRS image retrieval, we focus on the feature extraction issue and delve how to use powerful deep representations to address this task. We conduct systematic investigation on evaluating correlative factors that may affect the performance of deep features. By optimizing each factor, we acquire remarkable retrieval results on publicly available HRRS datasets. Finally, we explain the experimental phenomenon in detail and draw conclusions according to our analysis. Our work can serve as a guiding role for the research of content-based RS image retrieval

Integration of geographic information system and RADARSAT synthetic aperture radar data using a self-organizing map network as compensation for realtime ground data in automatic image classification

The paper presents results of using advanced techniques such as Self-Organizing feature Map (SOM) to incorporate a GIS data layer to compensate for the limited amount of real-time ground-truth data available for land-use and land-cover mapping in wet-season conditions in Bangladesh based on multi-temporal RADARSAT-1 SAR images. The experimental results were compared with those of traditional statistical classifiers such as Maximum Likelihood, Mahalanobis Distance, and Minimum Distance, which are not suitable for incorporating low-level GIS data in the image classification process. The performances of the classifiers were evaluated in terms of the classification accuracy with respect to the collected real-time ground truth data. The SOM neural network provided the highest overall accuracy when a GIS layer of land type classification with respect to the depth and duration of regular flooding was used in the network. Using this method, the overall accuracy was around 15% higher than the previously mentioned traditional classifiers at 79.6% where the training data covered only 0.53% of the total image. It also achieved higher accuracies for more classes in comparison to the other classifiers

Uncertainty Analysis for the Classification of Multispectral Satellite Images Using SVMs and SOMs

Abstract: Classification of multispectral remotely sensed data with textural features is investigated with a special focus on uncertainty analysis in the produced land-cover maps. Much effort has already been directed into the research of satisfactory accuracy-assessment techniques in image classification, but a common approach is not yet universally adopted. We look at the relationship between hard accuracy and the uncertainty on the produced answers, introducing two measures based on maximum probability and a quadratic entropy. Their impact differs depending on the type of classifier. In this paper, we deal with two different classification strategies, based on support vector machines (SVMs) and Kohonen's self-organizingmaps (SOMs), both suitably modified to give soft answers. Once the multiclass probability answer vector is available for each pixel in the image, we studied the behavior of the overall classification accuracy as a function of the uncertainty associated with each vector, given a hard-labeled test set. The experimental results show that the SVM with one-versus-one architecture and linear kernel clearly outperforms the other supervised approaches in terms of overall accuracy. On the other hand, our analysis reveals that the proposed SOM-based classifier, despite its unsupervised learning procedure, is able to provide soft answers which are the best candidates for a fusion with supervised results

Multi-Decadal Changes in Mangrove Extent, Age and Species in the Red River Estuaries of Viet Nam

This research investigated the performance of four different machine learning supervised image classifiers: artificial neural network (ANN), decision tree (DT), random forest (RF), and support vector machine (SVM) using SPOT-7 and Sentinel-1 images to classify mangrove age and species in 2019 in a Red River estuary, typical of others found in northern Viet Nam. The four classifiers were chosen because they are considered to have high accuracy, however, their use in mangrove age and species classifications has thus far been limited. A time-series of Landsat images from 1975 to 2019 was used to map mangrove extent changes using the unsupervised classification method of iterative self-organizing data analysis technique (ISODATA) and a comparison with accuracy of K-means classification, which found that mangrove extent has increased, despite a fall in the 1980s, indicating the success of mangrove plantation and forest protection efforts by local people in the study area. To evaluate the supervised image classifiers, 183 in situ training plots were assessed, 70% of them were used to train the supervised algorithms, with 30% of them employed to validate the results. In order to improve mangrove species separations, Gram–Schmidt and principal component analysis image fusion techniques were applied to generate better quality images. All supervised and unsupervised (2019) results of mangrove age, species, and extent were mapped and accuracy was evaluated. Confusion matrices were calculated showing that the classified layers agreed with the ground-truth data where most producer and user accuracies were greater than 80%. The overall accuracy and Kappa coefficients (around 0.9) indicated that the image classifications were very good. The test showed that SVM was the most accurate, followed by DT, ANN, and RF in this case study. The changes in mangrove extent identified in this study and the methods tested for using remotely sensed data will be valuable to monitoring and evaluation assessments of mangrove plantation projects

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

The contribution of multitemporal information from multispectral satellite images for automatic land cover classification at the national scale

Thesis submitted to the Instituto Superior de Estatística e Gestão de Informação da Universidade Nova de Lisboa in partial fulfillment of the requirements for the Degree of Doctor of Philosophy in Information Management – Geographic Information SystemsImaging and sensing technologies are constantly evolving so that, now, the latest generations of satellites commonly provide with Earth’s surface snapshots at very short sampling periods (i.e. daily images). It is unquestionable that this tendency towards continuous time observation will broaden up the scope of remotely sensed activities. Inevitable also, such increasing amount of information will prompt methodological approaches that combine digital image processing techniques with time series analysis for the characterization of land cover distribution and monitoring of its dynamics on a frequent basis. Nonetheless, quantitative analyses that convey the proficiency of three-dimensional satellite images data sets (i.e. spatial, spectral and temporal) for the automatic mapping of land cover and land cover time evolution have not been thoroughly explored. In this dissertation, we investigate the usefulness of multispectral time series sets of medium spatial resolution satellite images for the regular land cover characterization at the national scale. This study is carried out on the territory of Continental Portugal and exploits satellite images acquired by the Moderate Resolution Imaging Spectroradiometer (MODIS) and MEdium Resolution Imaging Spectrometer (MERIS). In detail, we first focus on the analysis of the contribution of multitemporal information from multispectral satellite images for the automatic land cover classes’ discrimination. The outcomes show that multispectral information contributes more significantly than multitemporal information for the automatic classification of land cover types. In the sequence, we review some of the most important steps that constitute a standard protocol for the automatic land cover mapping from satellite images. Moreover, we delineate a methodological approach for the production and assessment of land cover maps from multitemporal satellite images that guides us in the production of a land cover map with high thematic accuracy for the study area. Finally, we develop a nonlinear harmonic model for fitting multispectral reflectances and vegetation indices time series from satellite images for numerous land cover classes. The simplified multitemporal information retrieved with the model proves adequate to describe the main land cover classes’ characteristics and to predict the time evolution of land cover classes’individuals