Using Visual Linkages for Multilingual Image Retrieval

Abstract. The use of visual features in text-based ad-hoc image retrieval is challenging. Visual and textual information have so far been treated equally in terms of their properties and combined with weighting mechanisms for balancing their contributions to the ranking. The use of visual and textual information in a single retrieval system sometimes limits its applicability due to the lack of modularity. In this paper, we propose an image retrieval method that separates the usage of visual information from that of textual information. Visual clustering establishes linkages between images and the relationships are later used for the reranking. By applying clustering on visual features prior to the ranking, the main retrieval process becomes purely text-based. Experimental results on the ImageCLEFphoto ad-hoc task show this scheme is suitable for querying multilingual collections on some search topics.