Semantic Image Search From Multiple Query Images

This paper presents a novel search paradigm that uses mul-tiple images as input to perform semantic search of images. While earlier focuses on using single or multiple query im-ages to retrieve images with views of the same instance, the proposed paradigm uses each query image to discover com-mon concepts that are implicitly shared by all of the query images and retrieves images considering the found concepts. Our implementation uses high level visual features extracted from a deep convolutional network to retrieve images simi-lar to each query input. These images have associated text previously generated by implicit crowdsourcing. A Bag of Words (BoW) textual representation of each query image is built from the associated text of the retrieved similar images. A learned vector space representation of English words ex-tracted from a corpus of 100 billion words allows computing the conceptual similarity of words. The words that repre-sent the input images are used to nd new words that share conceptual similarity across all the input images. These new words are combined with the representations of the input im-ages to obtain a BoW textual representation of the search, which is used to perform image retrieval. The retrieved im-ages are re-ranked to enhance visual similarity with respect to any of the input images. Our experiments show that the concepts found are meaningful and that they retrieve cor-rectly 72.43 % of the images from the top 25, along with user ratings performed in the cases of study. 1

Semantic Image Search From Multiple Query Images

This paper presents a novel search paradigm that uses mul-tiple images as input to perform semantic search of images. While earlier focuses on using single or multiple query im-ages to retrieve images with views of the same instance, the proposed paradigm uses each query image to discover com-mon concepts that are implicitly shared by all of the query images and retrieves images considering the found concepts. Our implementation uses high level visual features extracted from a deep convolutional network to retrieve images simi-lar to each query input. These images have associated text previously generated by implicit crowdsourcing. A Bag of Words (BoW) textual representation of each query image is built from the associated text of the retrieved similar images. A learned vector space representation of English words ex-tracted from a corpus of 100 billion words allows computing the conceptual similarity of words. The words that repre-sent the input images are used to find new words that share conceptual similarity across all the input images. These new words are combined with the representations of the input im-ages to obtain a BoW textual representation of the search, which is used to perform image retrieval. The retrieved im-ages are re-ranked to enhance visual similarity with respect to any of the input images. Our experiments show that the concepts found are meaningful and that they retrieve cor-rectly 72.43% of the images from the top 25, along with user ratings performed in the cases of study