168,236 research outputs found
CMIR-NET : A Deep Learning Based Model For Cross-Modal Retrieval In Remote Sensing
We address the problem of cross-modal information retrieval in the domain of
remote sensing. In particular, we are interested in two application scenarios:
i) cross-modal retrieval between panchromatic (PAN) and multi-spectral imagery,
and ii) multi-label image retrieval between very high resolution (VHR) images
and speech based label annotations. Notice that these multi-modal retrieval
scenarios are more challenging than the traditional uni-modal retrieval
approaches given the inherent differences in distributions between the
modalities. However, with the growing availability of multi-source remote
sensing data and the scarcity of enough semantic annotations, the task of
multi-modal retrieval has recently become extremely important. In this regard,
we propose a novel deep neural network based architecture which is considered
to learn a discriminative shared feature space for all the input modalities,
suitable for semantically coherent information retrieval. Extensive experiments
are carried out on the benchmark large-scale PAN - multi-spectral DSRSID
dataset and the multi-label UC-Merced dataset. Together with the Merced
dataset, we generate a corpus of speech signals corresponding to the labels.
Superior performance with respect to the current state-of-the-art is observed
in all the cases
Evaluating Text-to-Image Matching using Binary Image Selection (BISON)
Providing systems the ability to relate linguistic and visual content is one
of the hallmarks of computer vision. Tasks such as text-based image retrieval
and image captioning were designed to test this ability but come with
evaluation measures that have a high variance or are difficult to interpret. We
study an alternative task for systems that match text and images: given a text
query, the system is asked to select the image that best matches the query from
a pair of semantically similar images. The system's accuracy on this Binary
Image SelectiON (BISON) task is interpretable, eliminates the reliability
problems of retrieval evaluations, and focuses on the system's ability to
understand fine-grained visual structure. We gather a BISON dataset that
complements the COCO dataset and use it to evaluate modern text-based image
retrieval and image captioning systems. Our results provide novel insights into
the performance of these systems. The COCO-BISON dataset and corresponding
evaluation code are publicly available from \url{http://hexianghu.com/bison/}
One-Shot Fine-Grained Instance Retrieval
Fine-Grained Visual Categorization (FGVC) has achieved significant progress
recently. However, the number of fine-grained species could be huge and
dynamically increasing in real scenarios, making it difficult to recognize
unseen objects under the current FGVC framework. This raises an open issue to
perform large-scale fine-grained identification without a complete training
set. Aiming to conquer this issue, we propose a retrieval task named One-Shot
Fine-Grained Instance Retrieval (OSFGIR). "One-Shot" denotes the ability of
identifying unseen objects through a fine-grained retrieval task assisted with
an incomplete auxiliary training set. This paper first presents the detailed
description to OSFGIR task and our collected OSFGIR-378K dataset. Next, we
propose the Convolutional and Normalization Networks (CN-Nets) learned on the
auxiliary dataset to generate a concise and discriminative representation.
Finally, we present a coarse-to-fine retrieval framework consisting of three
components, i.e., coarse retrieval, fine-grained retrieval, and query
expansion, respectively. The framework progressively retrieves images with
similar semantics, and performs fine-grained identification. Experiments show
our OSFGIR framework achieves significantly better accuracy and efficiency than
existing FGVC and image retrieval methods, thus could be a better solution for
large-scale fine-grained object identification.Comment: Accepted by MM2017, 9 pages, 7 figure
Exploring EEG for Object Detection and Retrieval
This paper explores the potential for using Brain Computer Interfaces (BCI)
as a relevance feedback mechanism in content-based image retrieval. We
investigate if it is possible to capture useful EEG signals to detect if
relevant objects are present in a dataset of realistic and complex images. We
perform several experiments using a rapid serial visual presentation (RSVP) of
images at different rates (5Hz and 10Hz) on 8 users with different degrees of
familiarization with BCI and the dataset. We then use the feedback from the BCI
and mouse-based interfaces to retrieve localized objects in a subset of TRECVid
images. We show that it is indeed possible to detect such objects in complex
images and, also, that users with previous knowledge on the dataset or
experience with the RSVP outperform others. When the users have limited time to
annotate the images (100 seconds in our experiments) both interfaces are
comparable in performance. Comparing our best users in a retrieval task, we
found that EEG-based relevance feedback outperforms mouse-based feedback. The
realistic and complex image dataset differentiates our work from previous
studies on EEG for image retrieval.Comment: This preprint is the full version of a short paper accepted in the
ACM International Conference on Multimedia Retrieval (ICMR) 2015 (Shanghai,
China
Information-theoretic measures of music listening behaviour
We present an information-theoretic approach to the mea-
surement of users’ music listening behaviour and selection of music features. Existing
ethnographic studies of mu- sic use have guided the design of music retrieval systems however are
typically qualitative and exploratory in nature. We introduce the SPUD dataset, comprising 10, 000
hand- made playlists, with user and audio stream metadata. With this, we illustrate the use of
entropy for analysing music listening behaviour, e.g. identifying when a user changed music
retrieval system. We then develop an approach to identifying music features that reflect users’
criteria for playlist curation, rejecting features that are independent of user behaviour. The
dataset and the code used to produce it are made available. The techniques described support a
quantitative yet user-centred approach to the evaluation of music features and retrieval systems,
without assuming objective ground truth labels
Improving Entity Retrieval on Structured Data
The increasing amount of data on the Web, in particular of Linked Data, has
led to a diverse landscape of datasets, which make entity retrieval a
challenging task. Explicit cross-dataset links, for instance to indicate
co-references or related entities can significantly improve entity retrieval.
However, only a small fraction of entities are interlinked through explicit
statements. In this paper, we propose a two-fold entity retrieval approach. In
a first, offline preprocessing step, we cluster entities based on the
\emph{x--means} and \emph{spectral} clustering algorithms. In the second step,
we propose an optimized retrieval model which takes advantage of our
precomputed clusters. For a given set of entities retrieved by the BM25F
retrieval approach and a given user query, we further expand the result set
with relevant entities by considering features of the queries, entities and the
precomputed clusters. Finally, we re-rank the expanded result set with respect
to the relevance to the query. We perform a thorough experimental evaluation on
the Billions Triple Challenge (BTC12) dataset. The proposed approach shows
significant improvements compared to the baseline and state of the art
approaches
- …