Large Scale Retrieval and Generation of Image Descriptions

What is the story of an image? What is the relationship between pictures, language, and information we can extract using state of the art computational recognition systems? In an attempt to address both of these questions, we explore methods for retrieving and generating natural language descriptions for images. Ideally, we would like our generated textual descriptions (captions) to both sound like a person wrote them, and also remain true to the image content. To do this we develop data-driven approaches for image description generation, using retrieval-based techniques to gather either: (a) whole captions associated with a visually similar image, or (b) relevant bits of text (phrases) from a large collection of image + description pairs. In the case of (b), we develop optimization algorithms to merge the retrieved phrases into valid natural language sentences. The end result is two simple, but effective, methods for harnessing the power of big data to produce image captions that are altogether more general, relevant, and human-like than previous attempts

A Review on Attribute Based Image Search Reranking

Image search reranking is one of the effective approach to refine the text-based image search result. Text-based image retrieval suffers from essential problems that are lead to the incapability of the associated text to appropriately evoke the image content. In this paper, reranking methods are put forward to address this drawback in scalable fashion. Based on the classifiers for each and every predefined attributes,each and every  image is represented by an attribute feature consisting of the responses from these classifiers. This hypergraph can be used to model the relationship between images by integration of low-level visual features and attribute features. Hypergraph ranking is then performed to order the images. Its basic principle is that visually close images should have identical ranking scores. It improves the performance over the text-based image search engin

Image Based Model for Document Search and Re-ranking

Traditional Web search engines do not use the images in the web pages to search relevant documents for a given query. Instead, they are typically operated by computing a measure of agreement between the keywords provided by the user and only the text portion of each web page. This project describes whether the image content appearing in a Web page can be used to enhance the semantic description of Web page and accordingly improve the performance of a keyword-based search engine. A Web-scalable system is presented in such a way that exploits a pure text-based search engine that finds an initial set of candidate documents as per given query. Then, by using visual information extracted from the images contained in the pages, the candidate set will be re-ranked. The computational efficiency of traditional text-based search engines will be maintained by the resulting system with only a small additional storage cost that will be needed to predetermine the visual information

Multimedia question answering

Ph.DDOCTOR OF PHILOSOPH

Separability versus Prototypicality in Handwritten Word Retrieval

User appreciation of a word-image retrieval system is based on the quality ofa hit list for a query. Using support vector machines for ranking in largescale, handwritten document collections, we observed that many hit listssuffered from bad instances in the top ranks. An analysis of this problemrevealed that two functions needed to be optimised concerning bothseparability and prototypicality. By ranking images in two stages, the numberof distracting images is reduced, making the method very convenient formassive scale, continuously trainable retrieval engines. Instead of cumbersomeSVM training, we present a nearest-centroid method and show that precisionimprovements of up to 35 percentage points can be achieved, yielding up to100% precision in data sets with a large amount of instances, whilemaintaining high recall performances.<br/

Image based Search Engine for Online Shopping

This paper presents a method based on principle of content based image retrieval for online shopping based on color, HSV aiming at efficient retrieval of images from the large database for online shopping specially for fashion shopping. Here, HSV modeling is used for creating our application with a huge image database, which compares image source with the destination components. In this paper, a technique is used for finding items by image search, which is convenient for buyers in order to allow them to see the products. The reason for using image search for items instead of text searches is that item searching by keywords or text has some issues such as errors in search items, expansion in search and inaccuracy in search results. This paper is an attempt to help users to choose the best options among many products and decide exactly what they want with the fast and easy search by image retrieval. This technology is providing a new search mode, searching by image, which will help buyers for finding the same or similar image retrieval in the database store. The image searching results have been made customers buy products quickly. This feature is implemented to identify and extract features of prominent object present in an image. Using different statistical measures, similarity measures are calculated and evaluated. Image retrieval based on color is a trivial task. Identifying objects of prominence in an image and retrieving image with similar features is a complex task. Finding prominent object in an image is difficult in a background image and is the challenging task in retrieving images. We calculated and change the region of interest in order to increase speed of operation as well as accuracy by masking the background content. The Implementation results proved that proposed method is effective in recalling the images of same pattern or texture

Towards Content-based Pixel Retrieval in Revisited Oxford and Paris

This paper introduces the first two pixel retrieval benchmarks. Pixel retrieval is segmented instance retrieval. Like semantic segmentation extends classification to the pixel level, pixel retrieval is an extension of image retrieval and offers information about which pixels are related to the query object. In addition to retrieving images for the given query, it helps users quickly identify the query object in true positive images and exclude false positive images by denoting the correlated pixels. Our user study results show pixel-level annotation can significantly improve the user experience. Compared with semantic and instance segmentation, pixel retrieval requires a fine-grained recognition capability for variable-granularity targets. To this end, we propose pixel retrieval benchmarks named PROxford and PRParis, which are based on the widely used image retrieval datasets, ROxford and RParis. Three professional annotators label 5,942 images with two rounds of double-checking and refinement. Furthermore, we conduct extensive experiments and analysis on the SOTA methods in image search, image matching, detection, segmentation, and dense matching using our pixel retrieval benchmarks. Results show that the pixel retrieval task is challenging to these approaches and distinctive from existing problems, suggesting that further research can advance the content-based pixel-retrieval and thus user search experience. The datasets can be downloaded from \href{https://github.com/anguoyuan/Pixel_retrieval-Segmented_instance_retrieval}{this link}