Vocabulary-supported Image Retrieval

Today's content-based image retrieval systems (CBIR) mostly rely on a predefined set of low-level image features and incorporate user-interactions using techniques such as relevance feedback. These systems however do not take advantage of the fact that in many applications queries can be formulated using a vocabulary. In this paper we propose a general framework which allows to use vocabulary at several levels. The framework should be seen as an extension of today's CBIR systems enabling the use of vocabulary as well as online learning techniques such as relevance feedback. The image detectors supporting the vocabulary can be either implemented directly or learned offline from examples and user-interactions

Performance Prediction for Vocabulary-Supported Image Retrieval

The majority of today's content based image retrieval systems rely on low-level image descriptors which limit their capability to support meaningful interactions with the users. Even though relevance feedback helps, most of the current interaction paradigms are far from the semantic representations which most people use to categorize and describe image content. Therefore we propose a concept called ``vocabu-la-ry-suppor-ted image retrieval'' which aims to enable the user to access an image database in a more natural way. In particular this paper develops a technique to predict the system's performance with respect to the user query. This allows the system to translate the user query into an internal query which may satisfy predefined criteria such as precision and recall rates. In addition, given the performance parameters of the system's sub-components, the feasibility and the success of the retrieval process can be evaluated beforehand and optimized dynamically online

Query-Dependent Performance Optimization for Vocabulary-Supported Image Retrieval

In [1], we proposed a two-stage retrieval framework which makes not only performance characterization but also performance optimization manageable. There, the performance optimization focused on the second stage of the retrieval framework. In this paper, we extend the method to a full two-stage performance characterization and optimization. In our retrieval framework, the user specifies a high-level concept to be searched for, the size of the image region to be covered by the concept (e.g.“Search images with 30–50% of sky”) and an optimization option (e.g. “maximum recall”, “maximum precision” or “joint maximization of precision and recall”). For the detection of each concept such as “sky”, a multitude of concept detectors exist that perform differently. In order to reach optimum retrieval performance, the detector best satisfying the user query is selected and the information of the corresponding concept detector is processed and optimized. Besides the optimization procedure itself the paper discusses the generation of multiple detectors per semantic concept. In experiments, the advantage of joint compared to individual optimization of first and second stage is shown