2 research outputs found
Creating a test collection to evaluate diversity in image retrieval
This paper describes the adaptation of an existing test collection
for image retrieval to enable diversity in the results set to be
measured. Previous research has shown that a more diverse set of
results often satisfies the needs of more users better than standard
document rankings. To enable diversity to be quantified, it is
necessary to classify images relevant to a given theme to one or
more sub-topics or clusters. We describe the challenges in
building (as far as we are aware) the first test collection for
evaluating diversity in image retrieval. This includes selecting
appropriate topics, creating sub-topics, and quantifying the overall
effectiveness of a retrieval system. A total of 39 topics were
augmented for cluster-based relevance and we also provide an
initial analysis of assessor agreement for grouping relevant
images into sub-topics or clusters
Mining photographic collections to enhance the precision and recall of search results using semantically controlled query expansion
Driven by a larger and more diverse user-base and datasets, modern Information Retrieval techniques are
striving to become contextually-aware in order to provide users with a more satisfactory search experience.
While text-only retrieval methods are significantly more accurate and faster to render results than purely
visual retrieval methods, these latter provide a rich complementary medium which can be used to obtain
relevant and different results from those obtained using text-only retrieval. Moreover, the visual retrieval
methods can be used to learn the user’s context and preferences, in particular the user’s relevance feedback,
and exploit them to narrow down the search to more accurate results. Despite the overall deficiency in
precision of visual retrieval result, the top results are accurate enough to be used for query expansion, when
expanded in a controlled manner.
The method we propose overcomes the usual pitfalls of visual retrieval:
1. The hardware barrier giving rise to prohibitively slow systems.
2. Results dominated by noise.
3. A significant gap between the low-level features and the semantics of the query.
In our thesis, the first barrier is overcome by employing a simple block-based visual features which
outperforms a method based on MPEG-7 features specially at early precision (precision of the top results).
For the second obstacle, lists from words semantically weighted according to their degree of relation to
the original query or to relevance feedback from example images are formed. These lists provide filters
through which the confidence in the candidate results is assessed for inclusion in the results. This allows
for more reliable Pseudo-Relevance Feedback (PRF). This technique is then used to bridge the third barrier;
the semantic gap. It consists of a second step query, re-querying the data set with an query expanded with
weighted words obtained from the initial query, and semantically filtered (SF) without human intervention.
We developed our PRF-SF method on the IAPR TC-12 benchmark dataset of 20,000 tourist images, obtaining
promising results, and tested it on the different and much larger Belga benchmark dataset of approximately
500,000 news images originating from a different source. Our experiments confirmed the potential of
the method in improving the overall Mean Average Precision, recall, as well as the level of diversity of the
results measured using cluster recall