Retrieving Images with Generated Textual Descriptions

© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.This paper presents a novel remote sensing (RS) image retrieval system that is defined based on generation and exploitation of textual descriptions that model the content of RS images. The proposed RS image retrieval system is composed of three main steps. The first one generates textual descriptions of the content of the RS images combining a convolutional neural network (CNN) and a recurrent neural network (RNN) to extract the features of the images and to generate the descriptions of their content, respectively. The second step encodes the semantic content of the generated descriptions using word embedding techniques able to produce semantically rich word vectors. The third step retrieves the most similar images with respect to the query image by measuring the similarity between the encoded generated textual descriptions of the query image and those of the archive. Experimental results on RS image archive composed of RS images acquired by unmanned aerial vehicles (UAVs) are reported and discussed

Characterizing the spatiotemporal distribution of meteorological drought as a response to climate variability: The case of rift valley lakes basin of Ethiopia

Climate variability and recurrent meteorological droughts frequently affect the rain-dependent Ethiopian agriculture, where the rift valley lakes basin is one of the most drought-prone regions in the country. The aim of this study was to evaluate climate variability and characterize the spatiotemporal distribution of meteorological droughts using a merged satellite-gauge rainfall across the major agroecological zones (AEZs) of the rift valley lakes basin. To this end, coefficient of variation (CV) and standardized rainfall anomaly (SRA) were used to evaluate rainfall variability; Mann-Kendell test was used to examine trends of temperature and rainfall; and a grid-rainfall based standardized precipitation index (SPI) was used to assess the spatiotemporal distribution and severity of meteorological droughts. The SPI was computed for 37 years over 1981–2017 at 3-month and 4-month timescales for the bimodal rainy seasons. Finally, a higher inter-annual and spatial variability of rainfall and frequent meteorological droughts were found across the basin. Compared to the nationally documented historical drought years in the country, more frequent drought events were found in this basin, signifying its higher vulnerability to climate variability. As a result, between 1981 and 2017, the basin has partially experienced at least a moderate drought intensity on average every 1.68 and 1.76 years during the 'Belg' and 'Kiremt' season, respectively. Drought frequency was higher at the 'Kolla' AEZ, characterized by the highest CV of rainfall. Furthermore, these frequent droughts were accompanied by significant rising trends in monthly temperature. Such a warming trend, in this inherently warm area, coupled with expected global climate change scenarios could further aggravate drought conditions in the future. Moreover, the spatiotemporal distribution of drought events was found to be variable between and within AEZs in the basin so that more localized drought adaptation strategies could help to alleviate potential impacts. Thus, the drought history of each agroecological zone and the spatiotemporal distributions of recent droughts, this study has delivered, could enhance the awareness of concerned decision makers in tracing frequently affected locations, which could in turn enable them to design and implement improved water management techniques as a means of drought mitigation strategy. Keywords: Climate variability, Drought, Mann-Kendall test, Merged satellite-gauge rainfall, Rift valley lakes basin, SP

MaxEnt-based modeling of suitable habitat for rehabilitation of Podocarpus forest at landscape-scale

AbstractModeling the current distribution and predicting suitable habitats of threatened species support proper planning processes for conservation and restoration. The aim of this study was thus to model the actual distribution and predict environmentally suitable habitats for Podocarpus falcatus, a locally threatened native tree species in Ethiopia. To realize this objective, species' presence samples, BIOCLIMATIC, and topographic predictors were combined to run a MaxEnt model. Finally, a model-generated habitat suitability map was produced with AUC accuracy of 0.783. Among the variables used for modeling, elevation range was found to be a key predictor of Podocarpus distribution, followed by precipitation of the driest quarter and isothermality. An extensive area (> 48%) of the studied landscape has been predicted to be environmentally suitable for the target species. However, only a small portion open-land area is practically available for rehabilitation since the area has been intensively cultivated to support the densely inhabited population. Therefore, potential areas for a small-scale plantation of Podocarpus trees remain to be pocket sites in religious places and around farmers' homesteads. So far, many farmers in this area have demonstrated a successful experience of growing this degraded native tree species. Thus, encouraging privately owned small-scale plantations could enhance rehabilitation and more sustainable conservation of the locally threatened native tree species

Capsule Networks for Object Detection in UAV Imagery

Recent advances in Convolutional Neural Networks (CNNs) have attracted great attention in remote sensing due to their high capability to model high-level semantic content of Remote Sensing (RS) images. However, CNNs do not explicitly retain the relative position of objects in an image and, thus, the effectiveness of the obtained features is limited in the framework of the complex object detection problems. To address this problem, in this paper we introduce Capsule Networks (CapsNets) for object detection in Unmanned Aerial Vehicle-acquired images. Unlike CNNs, CapsNets extract and exploit the information content about objects’ relative position across several layers, which enables parsing crowded scenes with overlapping objects. Experimental results obtained on two datasets for car and solar panel detection problems show that CapsNets provide similar object detection accuracies when compared to state-of-the-art deep models with significantly reduced computational time. This is due to the fact that CapsNets emphasize dynamic routine instead of the depth.EC/H2020/759764/EU/Accurate and Scalable Processing of Big Data in Earth Observation/BigEart

Detecting premature ventricular contractions in ECG signals with Gaussian processes

The aim of this work is twofold. First, we propose to investigate the capabilities of a new Bayesian approach for detecting premature ventricular contractions (PVCs), namely the Gaussian process (GP) approach. Second, we report an experimental comparison of different kinds of ECG signal representations, which are the standard temporal signal morphology, the discrete wavelet transform domain, the S-transform characteristics and the high-order statistics. In general, the obtained classification results show that the GP detector can compete seriously with state-of-the-art methods since it allows to yield better overall accuracy as well as better sensitivity. In addition, among the different kinds of features explored, those based on high-order statistics appear to be the best compromise between accuracy and computational time for PVC detection. 1

Toward Remote Sensing Image Retrieval Under a Deep Image Captioning Perspective

The performance of remote sensing image retrieval (RSIR) systems depends on the capability of the extracted features in characterizing the semantic content of images. Existing RSIR systems describe images by visual descriptors that model the primitives (such as different land-cover classes) present in the images. However, the visual descriptors may not be sufficient to describe the high-level complex content of RS images (e.g., attributes and relationships among different land-cover classes). To address this issue, in this article, we present an RSIR system that aims at generating and exploiting textual descriptions to accurately describe the relationships between the objects and their attributes present in RS images with captions (i.e., sentences). To this end, the proposed retrieval system consists of three main steps. The first step aims to encode the image visual features and then translate the encoded features into a textual description that summarizes the content of the image with captions. This is achieved based on the combination of a convolutional neural network with a recurrent neural network. The second step aims to convert the generated textual descriptions into semantically meaningful feature vectors. This is achieved by using the recent word embedding techniques. Finally, the last step estimates the similarity between the vectors of the textual descriptions of the query image and those of the archive images, and then retrieve the most similar images to the query image. Experimental results obtained on two different datasets show that the description of the image content with captions in the framework of RSIR leads to an accurate retrieval performance.EC/H2020/759764/EU/Accurate and Scalable Processing of Big Data in Earth Observation/BigEart

A scalable dataflow accelerator for real time onboard hyperspectral image classification

© Springer International Publishing Switzerland 2016.Real-time hyperspectral image classification is a necessary primitive in many remotely sensed image analysis applications. Previous work has shown that Support Vector Machines (SVMs) can achieve high classification accuracy, but unfortunately it is very computationally expensive. This paper presents a scalable dataflow accelerator on FPGA for real-time SVM classification of hyperspectral images.To address data dependencies, we adapt multi-class classifier based on Hamming distance. The architecture is scalable to high problem dimensionality and available hardware resources. Implementation results show that the FPGA design achieves speedups of 26x, 1335x, 66x and 14x compared with implementations on ZYNQ, ARM, DSP and Xeon processors. Moreover, one to two orders of magnitude reduction in power consumption is achieved for the AVRIS hyperspectral image datasets

Multilayer Markov Random Field Models for Change Detection in Optical Remote Sensing Images

In this paper, we give a comparative study on three Multilayer Markov Random Field (MRF) based solutions proposed for change detection in optical remote sensing images, called Multicue MRF, Conditional Mixed Markov model, and Fusion MRF. Our purposes are twofold. On one hand, we highlight the significance of the focused model family and we set them against various state-of-the-art approaches through a thematic analysis and quantitative tests. We discuss the advantages and drawbacks of class comparison vs. direct approaches, usage of training data, various targeted application fields and different ways of ground truth generation, meantime informing the Reader in which roles the Multilayer MRFs can be efficiently applied. On the other hand we also emphasize the differences between the three focused models at various levels, considering the model structures, feature extraction, layer interpretation, change concept definition, parameter tuning and performance. We provide qualitative and quantitative comparison results using principally a publicly available change detection database which contains aerial image pairs and Ground Truth change masks. We conclude that the discussed models are competitive against alternative state-of-the-art solutions, if one uses them as pre-processing filters in multitemporal optical image analysis. In addition, they cover together a large range of applications, considering the different usage options of the three approaches

Spatially adaptive semi‐supervised learning with Gaussian processes for hyperspectral data analysis

This paper presents a semi‐supervised learning algorithm called Gaussian process expectation‐maximization (GP‐EM), for classification of landcover based on hyperspectral data analysis. Model parameters for each land cover class are first estimated by a supervised algorithm using Gaussian process regressions to find spatially adaptive parameters, and the estimated parameters are then used to initialize a spatially adaptive mixture‐of‐Gaussians model. The mixture model is updated by expectation‐maximization iterations using the unlabeled data, and the spatially adaptive parameters for unlabeled instances are obtained by Gaussian process regressions with soft assignments. Spatially and temporally distant hyperspectral images taken from the Botswana area by the NASA EO‐1 satellite are used for experiments. Detailed empirical evaluations show that the proposed framework performs significantly better than all previously reported results by a wide variety of alternative approaches and algorithms on the same datasets. © 2011 Wiley Periodicals, Inc. Statistical Analysis and Data Mining 4: 358–371, 2011Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/87150/1/10119_ftp.pd