Combining Re-ranking And Rank Aggregation Methods For Image Retrieval

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)This paper presents novel approaches for combining re-ranking and rank aggregation methods aiming at improving the effectiveness of Content-Based Image Retrieval (CBIR) systems. Given a query image as input, CBIR systems retrieve the most similar images in a collection by taking into account image visual properties. In this scenario, accurately ranking collection images is of great relevance. Aiming at improving the effectiveness of CBIR systems, re-ranking and rank aggregation algorithms have been proposed. However, different re-ranking and rank aggregation approaches, applied to different image descriptors, may produce different and complementary image rankings. In this paper, we present four novel approaches for combining these rankings aiming at obtaining more effective results. Several experiments were conducted involving shape, color, and texture descriptors. The proposed approaches are also evaluated on multimodal retrieval tasks, considering visual and textual descriptors. Experimental results demonstrate that our approaches can improve significantly the effectiveness of image retrieval systems.751591219144Sao Paulo Research Foundation - FAPESP [2013/08645-0]CNPq [306580/2012-8, 484254/2012-0]CAPESAMDMicrosoft ResearchFundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES

Unsupervised manifold learning using Reciprocal kNN Graphs in image re-ranking and rank aggregation tasks

In this paper, we present an unsupervised distance learning approach for improving the effectiveness of image retrieval tasks. We propose a Reciprocal kNN Graph algorithm that considers the relationships among ranked lists in the context of a k-reciprocal neighborhood. The similarity is propagated among neighbors considering the geometry of the dataset manifold. The proposed method can be used both for re-ranking and rank aggregation tasks. Unlike traditional diffusion process methods, which require matrix multiplication operations, our algorithm takes only a subset of ranked lists as input, presenting linear complexity in terms of computational and storage requirements. We conducted a large evaluation protocol involving shape, color, and texture descriptors, various datasets, and comparisons with other post-processing approaches. The re-ranking and rank aggregation algorithms yield better results in terms of effectiveness performance than various state-of-the-art algorithms recently proposed in the literature, achieving bull's eye and MAP scores of 100% on the well-known MPEG-7 shape dataset (C) 2013 Elsevier B.V. All rights reserved.Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq

Semi-supervised learning for relevance feedback on image retrieval tasks

Relevance feedback approaches have been established as an important tool for interactive search, enabling users to express their needs. However, in view of the growth of multimedia collections available, the user efforts required by these methods tend to increase as well, demanding approaches for reducing the need of user interactions. In this context, this paper proposes a semi-supervised learning algorithm for relevance feedback to be used in image retrieval tasks. The proposed semi-supervised algorithm aims at using both supervised and unsupervised approaches simultaneously. While a supervised step is performed using the information collected from the user feedback, an unsupervised step exploits the intrinsic dataset structure, which is represented in terms of ranked lists of images. Several experiments were conducted for different image retrieval tasks involving shape, color, and texture descriptors and different datasets. The proposed approach was also evaluated on multimodal retrieval tasks, considering visual and textual descriptors. Experimental results demonstrate the effectiveness of the proposed approach

Image Re-Ranking Acceleration on GPUs

Huge image collections are becoming available lately. In this scenario, the use of Content-Based Image Retrieval (CBIR) systems has emerged as a promising approach to support image searches. The objective of CBIR systems is to retrieve the most similar images in a collection, given a query image, by taking into account image visual properties such as texture, color, and shape. In these systems, the effectiveness of the retrieval process depends heavily on the accuracy of ranking approaches. Recently, re-ranking approaches have been proposed to improve the effectiveness of CBIR systems by taking into account the relationships among images. The re-ranking approaches consider the relationships among all images in a given dataset. These approaches typically demands a huge amount of computational power, which hampers its use in practical situations. On the other hand, these methods can be massively parallelized. In this paper, we propose to speedup the computation of the RL-Sim algorithm, a recently proposed image re-ranking approach, by using the computational power of Graphics Processing Units (GPU). GPUs are emerging as relatively inexpensive parallel processors that are becoming available on a wide range of computer systems. We address the image re-ranking performance challenges by proposing a parallel solution designed to fit the computational model of GPUs. We conducted an experimental evaluation considering different implementations and devices. Experimental results demonstrate that significant performance gains can be obtained. Our approach achieves speedups of 7x from serial implementation considering the overall algorithm and up to 36x on its core steps

Multimedia retrieval through unsupervised hypergraph-based manifold ranking

Accurately ranking images and multimedia objects are of paramount relevance in many retrieval and learning tasks. Manifold learning methods have been investigated for ranking mainly due to their capacity of taking into account the intrinsic global manifold structure. In this paper, a novel manifold ranking algorithm is proposed based on the hypergraphs for unsupervised multimedia retrieval tasks. Different from traditional graph-based approaches, which represent only pairwise relationships, hypergraphs are capable of modeling similarity relationships among a set of objects. The proposed approach uses the hyperedges for constructing a contextual representation of data samples and exploits the encoded information for deriving a more effective similarity function. An extensive experimental evaluation was conducted on nine public datasets including diverse retrieval scenarios and multimedia content. Experimental results demonstrate that high effectiveness gains can be obtained in comparison with the state-of-the-art methods281258245838CONSELHO NACIONAL DE DESENVOLVIMENTO CIENTÍFICO E TECNOLÓGICO - CNPQCOORDENAÇÃO DE APERFEIÇOAMENTO DE PESSOAL DE NÍVEL SUPERIOR - CAPESFUNDAÇÃO DE AMPARO À PESQUISA DO ESTADO DE SÃO PAULO - FAPESP423228/2016-1; 307560/2016-3; 308194/2017-9; 313122/2017-2Sem informação2018/15597-6; 2017/25908-6; 2017/02091-4; 2017/20945-0; 2016/06441-7; 2015/24494-8; 2016/50250-1; 2013/50155-0; 2014/12236-1; 2014/50715-