3 research outputs found
Give me a hint! Navigating Image Databases using Human-in-the-loop Feedback
In this paper, we introduce an attribute-based interactive image search which
can leverage human-in-the-loop feedback to iteratively refine image search
results. We study active image search where human feedback is solicited
exclusively in visual form, without using relative attribute annotations used
by prior work which are not typically found in many datasets. In order to
optimize the image selection strategy, a deep reinforcement model is trained to
learn what images are informative rather than rely on hand-crafted measures
typically leveraged in prior work. Additionally, we extend the recently
introduced Conditional Similarity Network to incorporate global similarity in
training visual embeddings, which results in more natural transitions as the
user explores the learned similarity embeddings. Our experiments demonstrate
the effectiveness of our approach, producing compelling results on both active
image search and image attribute representation tasks
Semi-supervised Feature-Level Attribute Manipulation for Fashion Image Retrieval
With a growing demand for the search by image, many works have studied the
task of fashion instance-level image retrieval (FIR). Furthermore, the recent
works introduce a concept of fashion attribute manipulation (FAM) which
manipulates a specific attribute (e.g color) of a fashion item while
maintaining the rest of the attributes (e.g shape, and pattern). In this way,
users can search not only "the same" items but also "similar" items with the
desired attributes. FAM is a challenging task in that the attributes are hard
to define, and the unique characteristics of a query are hard to be preserved.
Although both FIR and FAM are important in real-life applications, most of the
previous studies have focused on only one of these problem. In this study, we
aim to achieve competitive performance on both FIR and FAM. To do so, we
propose a novel method that converts a query into a representation with the
desired attributes. We introduce a new idea of attribute manipulation at the
feature level, by matching the distribution of manipulated features with real
features. In this fashion, the attribute manipulation can be done independently
from learning a representation from the image. By introducing the feature-level
attribute manipulation, the previous methods for FIR can perform attribute
manipulation without sacrificing their retrieval performance.Comment: Accepted to BMVC 201
Learning Similarity Conditions Without Explicit Supervision
Many real-world tasks require models to compare images along multiple
similarity conditions (e.g. similarity in color, category or shape). Existing
methods often reason about these complex similarity relationships by learning
condition-aware embeddings. While such embeddings aid models in learning
different notions of similarity, they also limit their capability to generalize
to unseen categories since they require explicit labels at test time. To
address this deficiency, we propose an approach that jointly learns
representations for the different similarity conditions and their contributions
as a latent variable without explicit supervision. Comprehensive experiments
across three datasets, Polyvore-Outfits, Maryland-Polyvore and UT-Zappos50k,
demonstrate the effectiveness of our approach: our model outperforms the
state-of-the-art methods, even those that are strongly supervised with
pre-defined similarity conditions, on fill-in-the-blank, outfit compatibility
prediction and triplet prediction tasks. Finally, we show that our model learns
different visually-relevant semantic sub-spaces that allow it to generalize
well to unseen categories.Comment: Accepted at ICCV 201