221 research outputs found
Exploiting Local Features from Deep Networks for Image Retrieval
Deep convolutional neural networks have been successfully applied to image
classification tasks. When these same networks have been applied to image
retrieval, the assumption has been made that the last layers would give the
best performance, as they do in classification. We show that for instance-level
image retrieval, lower layers often perform better than the last layers in
convolutional neural networks. We present an approach for extracting
convolutional features from different layers of the networks, and adopt VLAD
encoding to encode features into a single vector for each image. We investigate
the effect of different layers and scales of input images on the performance of
convolutional features using the recent deep networks OxfordNet and GoogLeNet.
Experiments demonstrate that intermediate layers or higher layers with finer
scales produce better results for image retrieval, compared to the last layer.
When using compressed 128-D VLAD descriptors, our method obtains
state-of-the-art results and outperforms other VLAD and CNN based approaches on
two out of three test datasets. Our work provides guidance for transferring
deep networks trained on image classification to image retrieval tasks.Comment: CVPR DeepVision Workshop 201
Image Retrieval with Mixed Initiative and Multimodal Feedback
How would you search for a unique, fashionable shoe that a friend wore and
you want to buy, but you didn't take a picture? Existing approaches propose
interactive image search as a promising venue. However, they either entrust the
user with taking the initiative to provide informative feedback, or give all
control to the system which determines informative questions to ask. Instead,
we propose a mixed-initiative framework where both the user and system can be
active participants, depending on whose initiative will be more beneficial for
obtaining high-quality search results. We develop a reinforcement learning
approach which dynamically decides which of three interaction opportunities to
give to the user: drawing a sketch, providing free-form attribute feedback, or
answering attribute-based questions. By allowing these three options, our
system optimizes both the informativeness and exploration capabilities allowing
faster image retrieval. We outperform three baselines on three datasets and
extensive experimental settings.Comment: In submission to BMVC 201
- …