Evaluation of pointer click relevance feedback in PicSOM : deliverable D1.2 of FP7 project nº 216529 PinView

This report presents the results of a series of experiments where knowledge of the most relevant part of images is given as additional information to a content-based image retrieval system. The most relevant parts have been identified by search-task-dependent pointer clicks on the images. As such they provide a rudimentary form of explicit enriched relevance feedback and to some extent mimic genuine implicit eye movement measurements which are essential ingredients of the PinView project

Intelligent Image Retrieval Techniques: A Survey

AbstractIn the current era of digital communication, the use of digital images has increased for expressing, sharing and interpreting information. While working with digital images, quite often it is necessary to search for a specific image for a particular situation based on the visual contents of the image. This task looks easy if you are dealing with tens of images but it gets more difficult when the number of images goes from tens to hundreds and thousands, and the same content-based searching task becomes extremely complex when the number of images is in the millions. To deal with the situation, some intelligent way of content-based searching is required to fulfill the searching request with right visual contents in a reasonable amount of time. There are some really smart techniques proposed by researchers for efficient and robust content-based image retrieval. In this research, the aim is to highlight the efforts of researchers who conducted some brilliant work and to provide a proof of concept for intelligent content-based image retrieval techniques

Definition of enriched relevance feedback in PicSOM : deliverable D1.3.1 of FP7 project nº 216529 PinView

This report defines and implements communication principles and data formats for transferring enriched relevance feedback to the PicSOM content-based image retrieval system used in the PinView project. The modalities of enriched relevance feedback include recorded eye movements, pointer and keyboard events and audio including speech. The communication is based on the AJAX technology, where the client and server exchange XML formatted content by using the XMLHttpRequest method

Using biased support vector machine in image retrieval with self-organizing map.

Chan Chi Hang.Thesis submitted in: August 2004.Thesis (M.Phil.)--Chinese University of Hong Kong, 2005.Includes bibliographical references (leaves 105-114).Abstracts in English and Chinese.Abstract --- p.iAcknowledgement --- p.ivChapter 1 --- Introduction --- p.1Chapter 1.1 --- Problem Statement --- p.3Chapter 1.2 --- Major Contributions --- p.5Chapter 1.3 --- Publication List --- p.6Chapter 1.4 --- Thesis Organization --- p.7Chapter 2 --- Background Survey --- p.9Chapter 2.1 --- Relevance Feedback Framework --- p.9Chapter 2.1.1 --- Relevance Feedback Types --- p.11Chapter 2.1.2 --- Data Distribution --- p.12Chapter 2.1.3 --- Training Set Size --- p.14Chapter 2.1.4 --- Inter-Query Learning and Intra-Query Learning --- p.15Chapter 2.2 --- History of Relevance Feedback Techniques --- p.16Chapter 2.3 --- Relevance Feedback Approaches --- p.19Chapter 2.3.1 --- Vector Space Model --- p.19Chapter 2.3.2 --- Ad-hoc Re-weighting --- p.26Chapter 2.3.3 --- Distance Optimization Approach --- p.29Chapter 2.3.4 --- Probabilistic Model --- p.33Chapter 2.3.5 --- Bayesian Approach --- p.39Chapter 2.3.6 --- Density Estimation Approach --- p.42Chapter 2.3.7 --- Support Vector Machine --- p.48Chapter 2.4 --- Presentation Set Selection --- p.52Chapter 2.4.1 --- Most-probable strategy --- p.52Chapter 2.4.2 --- Most-informative strategy --- p.52Chapter 3 --- Biased Support Vector Machine for Content-Based Image Retrieval --- p.57Chapter 3.1 --- Motivation --- p.57Chapter 3.2 --- Background --- p.58Chapter 3.2.1 --- Regular Support Vector Machine --- p.59Chapter 3.2.2 --- One-class Support Vector Machine --- p.61Chapter 3.3 --- Biased Support Vector Machine --- p.63Chapter 3.4 --- Interpretation of parameters in BSVM --- p.67Chapter 3.5 --- Soft Label Biased Support Vector Machine --- p.69Chapter 3.6 --- Interpretation of parameters in Soft Label BSVM --- p.73Chapter 3.7 --- Relevance Feedback Using Biased Support Vector Machine --- p.74Chapter 3.7.1 --- Advantages of BSVM in Relevance Feedback . . --- p.74Chapter 3.7.2 --- Relevance Feedback Algorithm By BSVM --- p.75Chapter 3.8 --- Experiments --- p.78Chapter 3.8.1 --- Synthetic Dataset --- p.80Chapter 3.8.2 --- Real-World Dataset --- p.81Chapter 3.8.3 --- Experimental Results --- p.83Chapter 3.9 --- Conclusion --- p.86Chapter 4 --- Self-Organizing Map-based Inter-Query Learning --- p.88Chapter 4.1 --- Motivation --- p.88Chapter 4.2 --- Algorithm --- p.89Chapter 4.2.1 --- Initialization and Replication of SOM --- p.89Chapter 4.2.2 --- SOM Training for Inter-Query Learning --- p.90Chapter 4.2.3 --- Incorporate with Intra-Query Learning --- p.92Chapter 4.3 --- Experiments --- p.93Chapter 4.3.1 --- Synthetic Dataset --- p.95Chapter 4.3.2 --- Real-World Dataset --- p.95Chapter 4.3.3 --- Experimental Results --- p.97Chapter 4.4 --- Conclusion --- p.98Chapter 5 --- Conclusion --- p.102Bibliography --- p.10

Discriminative learning with application to interactive facial image retrieval

The amount of digital images is growing drastically and advanced tools for searching in large image collections are therefore becoming urgently needed. Content-based image retrieval is advantageous for such a task in terms of automatic feature extraction and indexing without human labor and subjectivity in image annotations. The semantic gap between high-level semantics and low-level visual features can be reduced by the relevance feedback technique. However, most existing interactive content-based image retrieval (ICBIR) systems require a substantial amount of human evaluation labor, which leads to the evaluation fatigue problem that heavily restricts the application of ICBIR. In this thesis a solution based on discriminative learning is presented. It extends an existing ICBIR system, PicSOM, towards practical applications. The enhanced ICBIR system allows users to input partial relevance which includes not only relevance extent but also relevance reason. A multi-phase retrieval with partial relevance can adapt to the user's searching intention in a from-coarse-to-fine manner. The retrieval performance can be improved by employing supervised learning as a preprocessing step before unsupervised content-based indexing. In this work, Parzen Discriminant Analysis (PDA) is proposed to extract discriminative components from images. PDA regularizes the Informative Discriminant Analysis (IDA) objective with a greatly accelerated optimization algorithm. Moreover, discriminative Self-Organizing Maps trained with resulting features can easily handle fuzzy categorizations. The proposed techniques have been applied to interactive facial image retrieval. Both a query example and a benchmark simulation study are presented, which indicate that the first image depicting the target subject can be retrieved in a small number of rounds

Biased classification for relevance feedback in content-based image retrieval.

Peng, Xiang.Thesis (M.Phil.)--Chinese University of Hong Kong, 2007.Includes bibliographical references (leaves 98-115).Abstracts in English and Chinese.Abstract --- p.iAcknowledgement --- p.ivChapter 1 --- Introduction --- p.1Chapter 1.1 --- Problem Statement --- p.3Chapter 1.2 --- Major Contributions --- p.6Chapter 1.3 --- Thesis Outline --- p.7Chapter 2 --- Background Study --- p.9Chapter 2.1 --- Content-based Image Retrieval --- p.9Chapter 2.1.1 --- Image Representation --- p.11Chapter 2.1.2 --- High Dimensional Indexing --- p.15Chapter 2.1.3 --- Image Retrieval Systems Design --- p.16Chapter 2.2 --- Relevance Feedback --- p.19Chapter 2.2.1 --- Self-Organizing Map in Relevance Feedback --- p.20Chapter 2.2.2 --- Decision Tree in Relevance Feedback --- p.22Chapter 2.2.3 --- Bayesian Classifier in Relevance Feedback --- p.24Chapter 2.2.4 --- Nearest Neighbor Search in Relevance Feedback --- p.25Chapter 2.2.5 --- Support Vector Machines in Relevance Feedback --- p.26Chapter 2.3 --- Imbalanced Classification --- p.29Chapter 2.4 --- Active Learning --- p.31Chapter 2.4.1 --- Uncertainly-based Sampling --- p.33Chapter 2.4.2 --- Error Reduction --- p.34Chapter 2.4.3 --- Batch Selection --- p.35Chapter 2.5 --- Convex Optimization --- p.35Chapter 2.5.1 --- Overview of Convex Optimization --- p.35Chapter 2.5.2 --- Linear Program --- p.37Chapter 2.5.3 --- Quadratic Program --- p.37Chapter 2.5.4 --- Quadratically Constrained Quadratic Program --- p.37Chapter 2.5.5 --- Cone Program --- p.38Chapter 2.5.6 --- Semi-definite Program --- p.39Chapter 3 --- Imbalanced Learning with BMPM for CBIR --- p.40Chapter 3.1 --- Research Motivation --- p.41Chapter 3.2 --- Background Review --- p.42Chapter 3.2.1 --- Relevance Feedback for CBIR --- p.42Chapter 3.2.2 --- Minimax Probability Machine --- p.42Chapter 3.2.3 --- Extensions of Minimax Probability Machine --- p.44Chapter 3.3 --- Relevance Feedback using BMPM --- p.45Chapter 3.3.1 --- Model Definition --- p.45Chapter 3.3.2 --- Advantages of BMPM in Relevance Feedback --- p.46Chapter 3.3.3 --- Relevance Feedback Framework by BMPM --- p.47Chapter 3.4 --- Experimental Results --- p.47Chapter 3.4.1 --- Experiment Datasets --- p.48Chapter 3.4.2 --- Performance Evaluation --- p.50Chapter 3.4.3 --- Discussions --- p.53Chapter 3.5 --- Summary --- p.53Chapter 4 --- BMPM Active Learning for CBIR --- p.55Chapter 4.1 --- Problem Statement and Motivation --- p.55Chapter 4.2 --- Background Review --- p.57Chapter 4.3 --- Relevance Feedback by BMPM Active Learning . --- p.58Chapter 4.3.1 --- Active Learning Concept --- p.58Chapter 4.3.2 --- General Approaches for Active Learning . --- p.59Chapter 4.3.3 --- Biased Minimax Probability Machine --- p.60Chapter 4.3.4 --- Proposed Framework --- p.61Chapter 4.4 --- Experimental Results --- p.63Chapter 4.4.1 --- Experiment Setup --- p.64Chapter 4.4.2 --- Performance Evaluation --- p.66Chapter 4.5 --- Summary --- p.68Chapter 5 --- Large Scale Learning with BMPM --- p.70Chapter 5.1 --- Introduction --- p.71Chapter 5.1.1 --- Motivation --- p.71Chapter 5.1.2 --- Contribution --- p.72Chapter 5.2 --- Background Review --- p.72Chapter 5.2.1 --- Second Order Cone Program --- p.72Chapter 5.2.2 --- General Methods for Large Scale Problems --- p.73Chapter 5.2.3 --- Biased Minimax Probability Machine --- p.75Chapter 5.3 --- Efficient BMPM Training --- p.78Chapter 5.3.1 --- Proposed Strategy --- p.78Chapter 5.3.2 --- Kernelized BMPM and Its Solution --- p.81Chapter 5.4 --- Experimental Results --- p.82Chapter 5.4.1 --- Experimental Testbeds --- p.83Chapter 5.4.2 --- Experimental Settings --- p.85Chapter 5.4.3 --- Performance Evaluation --- p.87Chapter 5.5 --- Summary --- p.92Chapter 6 --- Conclusion and Future Work --- p.93Chapter 6.1 --- Conclusion --- p.93Chapter 6.2 --- Future Work --- p.94Chapter A --- List of Symbols and Notations --- p.96Chapter B --- List of Publications --- p.98Bibliography --- p.10

Visual thesaurus for color image retrieval using SOM.

Yip King-Fung.Thesis (M.Phil.)--Chinese University of Hong Kong, 2003.Includes bibliographical references (leaves 84-89).Abstracts in English and Chinese.Abstract --- p.i論文摘要 --- p.iiiTable of Contents --- p.ivList of Abbreviations --- p.viAcknowledgements --- p.viiChapter 1. --- Introduction --- p.1Chapter 1.1. --- Background --- p.1Chapter 1.2. --- Motivation --- p.3Chapter 1.3. --- Thesis Organization --- p.4Chapter 2. --- A Survey of Content-based Image Retrieval --- p.5Chapter 2.1. --- Text-based Image Retrieval --- p.5Chapter 2.2. --- Content-Based Image Retrieval --- p.7Chapter 2.2.1. --- Content-Based Image Retrieval Systems --- p.7Chapter 2.2.2. --- Query Methods --- p.9Chapter 2.2.3. --- Image Features --- p.11Chapter 2.2.4. --- Summary --- p.16Chapter 3. --- Visual Thesaurus using SOM --- p.17Chapter 3.1. --- Algorithm --- p.17Chapter 3.1.1. --- Image Representation --- p.17Chapter 3.1.2. --- Self-Organizing Map --- p.21Chapter 3.2. --- Preliminary Experiment --- p.27Chapter 3.2.1. --- Feature differences --- p.27Chapter 3.2.2. --- Labeling differences --- p.30Chapter 4. --- Experiment --- p.33Chapter 4.1. --- Subjects --- p.33Chapter 4.2. --- Apparatus --- p.33Chapter 4.2.1. --- Systems --- p.33Chapter 4.2.2. --- Test Databases --- p.33Chapter 4.3. --- Procedure --- p.34Chapter 4.3.1. --- Description --- p.35Chapter 4.3.2. --- SOM (text) --- p.36Chapter 4.3.3. --- SOM (image) --- p.38Chapter 4.3.4. --- QBE (text) --- p.40Chapter 4.3.5. --- QBE (image) --- p.42Chapter 4.3.6. --- Questionnaire --- p.44Chapter 4.3.7. --- Experiment Flow --- p.45Chapter 4.4. --- Results --- p.46Chapter 4.5. --- Discussion --- p.51Chapter 5. --- Quantizing Color Histogram --- p.55Chapter 5.1. --- Algorithm --- p.56Chapter 5.1.1. --- Codebook Generation Phrase --- p.57Chapter 5.1.2. --- Histogram Generation Phrase --- p.66Chapter 5.2. --- Experiment --- p.67Chapter 5.2.1. --- Test Database --- p.67Chapter 5.2.2. --- Evaluation Methods --- p.67Chapter 5.2.3. --- Results and Discussion --- p.69Chapter 5.2.4. --- Summary --- p.74Chapter 6. --- Relevance Feedback --- p.75Chapter 6.1. --- Relevance Feedback in Text Information Retrieval --- p.75Chapter 6.2. --- Relevance Feedback in Multimedia Information Retrieval --- p.76Chapter 6.3. --- Relevance Feedback in Visual Thesaurus --- p.76Chapter 7. --- Conclusions --- p.80Chapter 7.1. --- Applications --- p.81Chapter 7.2. --- Future Directions --- p.81Chapter 7.2.1. --- SOM Generation --- p.81Chapter 7.2.2. --- Hybrid Architecture --- p.82References --- p.8

Object identification within images

Mestrado em Engenharia de Computadores e TelemáticaO aumento de conteúdo digital armazenado em bases de dados é acompanhado por uma elevada importância atribuída à disponibilização de métodos eficientes para a sua pesquisa. No caso da pesquisa de imagens, esta é, normalmente, realizada através de “keywords”, o que, nem sempre garante resultados satisfatórios, uma vez que as “imagens estão para além das palavras”. Para melhorar este tema é necessário avaliar o conteúdo de cada imagem. Este trabalho propõem-se a divulgar um sistema que, inicialmente, de todas as imagens presentes numa base de dados, obtenha um conjunto de elevada qualidade para posterior processamento. Este método baseia-se na analise do histograma de cada imagem e respectiva distribuição dos contornos de cada objecto presente na mesma. A este conjunto de imagens obtido, para cada instância, são extraídas características que a identifiquem. Este passo, baseia-se na segmentação de imagens e classificação de características através de uma rede neuronal. Para testar a eficiência do método apresentado nesta tese, é feita a comparação entre as características de cada imagem com as restantes, e respectiva devolução de uma lista de imagens, ordenada por ordem decrescente de semelhança. Os nossos resultados provam que o nosso sistema pode produzir melhores resultados do que alguns sistemas existentes. ABSTRACT: The rise of digital content stored in large databases increased the importance of efficient algorithms for information retrieval. These algorithms are, usually, based on keywords which, for image retrieval, do not work properly, since “images are beyond words”. In order to improve image retrieval it is necessary to analyze the contents of each image. This work proposes a system that, firstly, will get a subset of high quality images from the entire database, which will help in further processing. This first method is based in the histogram and edge analysis. In a next method, for each element of the image set obtained, features are extracted. These features will identify each image in the database. In this step, an image segmentation technique and a classification with a neural network are used. This feature extraction process is tested doing comparison between each image features and all the target ones. Each image is associated with a list of images ordered by a similarity level, which allows us to conclude that our system produces better results than some other systems available

ImSe : Instant Interactive Image Retrieval System with Exploration/Exploitation trade-off

Imagine a journalist looking for an illustration to his article about patriotism in a database of unannotated images. The idea of a suitable image is very vague and the best way to navigate through the database is to provide feedback to the images proposed by an Image Retrieval system in order to enable the system to learn what the ideal target image of the user is. Thus, at each search iteration a set of n images is displayed and the user must indicate how relevant they are to his/her target. When considering real-life problems we must also take into account the system's time-complexity and scalability to work with Big Data. To tackle this issue we utilize hierarchical Gaussian Process Bandits with visual Self-Organizing Map as a preprocessing technique. A prototype system called ImSe was developed and tested in experiments with real users in different types of tasks. The experiments show favorable results and indicate the benefits of proposed algorithms in different types of tasks