289 research outputs found
CMIR-NET : A Deep Learning Based Model For Cross-Modal Retrieval In Remote Sensing
We address the problem of cross-modal information retrieval in the domain of
remote sensing. In particular, we are interested in two application scenarios:
i) cross-modal retrieval between panchromatic (PAN) and multi-spectral imagery,
and ii) multi-label image retrieval between very high resolution (VHR) images
and speech based label annotations. Notice that these multi-modal retrieval
scenarios are more challenging than the traditional uni-modal retrieval
approaches given the inherent differences in distributions between the
modalities. However, with the growing availability of multi-source remote
sensing data and the scarcity of enough semantic annotations, the task of
multi-modal retrieval has recently become extremely important. In this regard,
we propose a novel deep neural network based architecture which is considered
to learn a discriminative shared feature space for all the input modalities,
suitable for semantically coherent information retrieval. Extensive experiments
are carried out on the benchmark large-scale PAN - multi-spectral DSRSID
dataset and the multi-label UC-Merced dataset. Together with the Merced
dataset, we generate a corpus of speech signals corresponding to the labels.
Superior performance with respect to the current state-of-the-art is observed
in all the cases
Aggregated Deep Local Features for Remote Sensing Image Retrieval
Remote Sensing Image Retrieval remains a challenging topic due to the special
nature of Remote Sensing Imagery. Such images contain various different
semantic objects, which clearly complicates the retrieval task. In this paper,
we present an image retrieval pipeline that uses attentive, local convolutional
features and aggregates them using the Vector of Locally Aggregated Descriptors
(VLAD) to produce a global descriptor. We study various system parameters such
as the multiplicative and additive attention mechanisms and descriptor
dimensionality. We propose a query expansion method that requires no external
inputs. Experiments demonstrate that even without training, the local
convolutional features and global representation outperform other systems.
After system tuning, we can achieve state-of-the-art or competitive results.
Furthermore, we observe that our query expansion method increases overall
system performance by about 3%, using only the top-three retrieved images.
Finally, we show how dimensionality reduction produces compact descriptors with
increased retrieval performance and fast retrieval computation times, e.g. 50%
faster than the current systems.Comment: Published in Remote Sensing. The first two authors have equal
contributio
Exploiting Deep Features for Remote Sensing Image Retrieval: A Systematic Investigation
Remote sensing (RS) image retrieval is of great significant for geological
information mining. Over the past two decades, a large amount of research on
this task has been carried out, which mainly focuses on the following three
core issues: feature extraction, similarity metric and relevance feedback. Due
to the complexity and multiformity of ground objects in high-resolution remote
sensing (HRRS) images, there is still room for improvement in the current
retrieval approaches. In this paper, we analyze the three core issues of RS
image retrieval and provide a comprehensive review on existing methods.
Furthermore, for the goal to advance the state-of-the-art in HRRS image
retrieval, we focus on the feature extraction issue and delve how to use
powerful deep representations to address this task. We conduct systematic
investigation on evaluating correlative factors that may affect the performance
of deep features. By optimizing each factor, we acquire remarkable retrieval
results on publicly available HRRS datasets. Finally, we explain the
experimental phenomenon in detail and draw conclusions according to our
analysis. Our work can serve as a guiding role for the research of
content-based RS image retrieval
A Rotation Meanout Network with Invariance for Dermoscopy Image Classification and Retrieval
The computer-aided diagnosis (CAD) system can provide a reference basis for
the clinical diagnosis of skin diseases. Convolutional neural networks (CNNs)
can not only extract visual elements such as colors and shapes but also
semantic features. As such they have made great improvements in many tasks of
dermoscopy images. The imaging of dermoscopy has no principal orientation,
indicating that there are a large number of skin lesion rotations in the
datasets. However, CNNs lack rotation invariance, which is bound to affect the
robustness of CNNs against rotations. To tackle this issue, we propose a
rotation meanout (RM) network to extract rotation-invariant features from
dermoscopy images. In RM, each set of rotated feature maps corresponds to a set
of outputs of the weight-sharing convolutions and they are fused using meanout
strategy to obtain the final feature maps. Through theoretical derivation, the
proposed RM network is rotation-equivariant and can extract rotation-invariant
features when followed by the global average pooling (GAP) operation. The
extracted rotation-invariant features can better represent the original data in
classification and retrieval tasks for dermoscopy images. The RM is a general
operation, which does not change the network structure or increase any
parameter, and can be flexibly embedded in any part of CNNs. Extensive
experiments are conducted on a dermoscopy image dataset. The results show our
method outperforms other anti-rotation methods and achieves great improvements
in dermoscopy image classification and retrieval tasks, indicating the
potential of rotation invariance in the field of dermoscopy images
- …