Research on Image Retrieval Optimization Based on Eye Movement Experiment Data

Satisfying a user's actual underlying needs in the image retrieval process is a difficult challenge facing image retrieval technology. The aim of this study is to improve the performance of a retrieval system and provide users with optimized search results using the feedback of eye movement. We analyzed the eye movement signals of the user’s image retrieval process from cognitive and mathematical perspectives. Data collected for 25 designers in eye tracking experiments were used to train and evaluate the model. In statistical analysis, eight eye movement features were statistically significantly different between selected and unselected groups of images (p < 0.05). An optimal selection of input features resulted in overall accuracy of the support vector machine prediction model of 87.16%. Judging the user’s requirements in the image retrieval process through eye movement behaviors was shown to be effective

Inter-query Learning in Content-based Image Retrieval

Computer Scienc

Intelligent Image Retrieval Techniques: A Survey

AbstractIn the current era of digital communication, the use of digital images has increased for expressing, sharing and interpreting information. While working with digital images, quite often it is necessary to search for a specific image for a particular situation based on the visual contents of the image. This task looks easy if you are dealing with tens of images but it gets more difficult when the number of images goes from tens to hundreds and thousands, and the same content-based searching task becomes extremely complex when the number of images is in the millions. To deal with the situation, some intelligent way of content-based searching is required to fulfill the searching request with right visual contents in a reasonable amount of time. There are some really smart techniques proposed by researchers for efficient and robust content-based image retrieval. In this research, the aim is to highlight the efforts of researchers who conducted some brilliant work and to provide a proof of concept for intelligent content-based image retrieval techniques

CHORUS Deliverable 2.1: State of the Art on Multimedia Search Engines

Based on the information provided by European projects and national initiatives related to multimedia search as well as domains experts that participated in the CHORUS Think-thanks and workshops, this document reports on the state of the art related to multimedia content search from, a technical, and socio-economic perspective. The technical perspective includes an up to date view on content based indexing and retrieval technologies, multimedia search in the context of mobile devices and peer-to-peer networks, and an overview of current evaluation and benchmark inititiatives to measure the performance of multimedia search engines. From a socio-economic perspective we inventorize the impact and legal consequences of these technical advances and point out future directions of research

Learning on relevance feedback in content-based image retrieval.

Hoi, Chu-Hong.Thesis (M.Phil.)--Chinese University of Hong Kong, 2004.Includes bibliographical references (leaves 89-103).Abstracts in English and Chinese.Abstract --- p.iAcknowledgement --- p.ivChapter 1 --- Introduction --- p.1Chapter 1.1 --- Content-based Image Retrieval --- p.1Chapter 1.2 --- Relevance Feedback --- p.3Chapter 1.3 --- Contributions --- p.4Chapter 1.4 --- Organization of This Work --- p.6Chapter 2 --- Background --- p.8Chapter 2.1 --- Relevance Feedback --- p.8Chapter 2.1.1 --- Heuristic Weighting Methods --- p.9Chapter 2.1.2 --- Optimization Formulations --- p.10Chapter 2.1.3 --- Various Machine Learning Techniques --- p.11Chapter 2.2 --- Support Vector Machines --- p.12Chapter 2.2.1 --- Setting of the Learning Problem --- p.12Chapter 2.2.2 --- Optimal Separating Hyperplane --- p.13Chapter 2.2.3 --- Soft-Margin Support Vector Machine --- p.15Chapter 2.2.4 --- One-Class Support Vector Machine --- p.16Chapter 3 --- Relevance Feedback with Biased SVM --- p.18Chapter 3.1 --- Introduction --- p.18Chapter 3.2 --- Biased Support Vector Machine --- p.19Chapter 3.3 --- Relevance Feedback Using Biased SVM --- p.22Chapter 3.3.1 --- Advantages of BSVM in Relevance Feedback --- p.22Chapter 3.3.2 --- Relevance Feedback Algorithm by BSVM --- p.23Chapter 3.4 --- Experiments --- p.24Chapter 3.4.1 --- Datasets --- p.24Chapter 3.4.2 --- Image Representation --- p.25Chapter 3.4.3 --- Experimental Results --- p.26Chapter 3.5 --- Discussions --- p.29Chapter 3.6 --- Summary --- p.30Chapter 4 --- Optimizing Learning with SVM Constraint --- p.31Chapter 4.1 --- Introduction --- p.31Chapter 4.2 --- Related Work and Motivation --- p.33Chapter 4.3 --- Optimizing Learning with SVM Constraint --- p.35Chapter 4.3.1 --- Problem Formulation and Notations --- p.35Chapter 4.3.2 --- Learning boundaries with SVM --- p.35Chapter 4.3.3 --- OPL for the Optimal Distance Function --- p.38Chapter 4.3.4 --- Overall Similarity Measure with OPL and SVM --- p.40Chapter 4.4 --- Experiments --- p.41Chapter 4.4.1 --- Datasets --- p.41Chapter 4.4.2 --- Image Representation --- p.42Chapter 4.4.3 --- Performance Evaluation --- p.43Chapter 4.4.4 --- Complexity and Time Cost Evaluation --- p.45Chapter 4.5 --- Discussions --- p.47Chapter 4.6 --- Summary --- p.48Chapter 5 --- Group-based Relevance Feedback --- p.49Chapter 5.1 --- Introduction --- p.49Chapter 5.2 --- SVM Ensembles --- p.50Chapter 5.3 --- Group-based Relevance Feedback Using SVM Ensembles --- p.51Chapter 5.3.1 --- (x+l)-class Assumption --- p.51Chapter 5.3.2 --- Proposed Architecture --- p.52Chapter 5.3.3 --- Strategy for SVM Combination and Group Ag- gregation --- p.52Chapter 5.4 --- Experiments --- p.54Chapter 5.4.1 --- Experimental Implementation --- p.54Chapter 5.4.2 --- Performance Evaluation --- p.55Chapter 5.5 --- Discussions --- p.56Chapter 5.6 --- Summary --- p.57Chapter 6 --- Log-based Relevance Feedback --- p.58Chapter 6.1 --- Introduction --- p.58Chapter 6.2 --- Related Work and Motivation --- p.60Chapter 6.3 --- Log-based Relevance Feedback Using SLSVM --- p.61Chapter 6.3.1 --- Problem Statement --- p.61Chapter 6.3.2 --- Soft Label Support Vector Machine --- p.62Chapter 6.3.3 --- LRF Algorithm by SLSVM --- p.64Chapter 6.4 --- Experimental Results --- p.66Chapter 6.4.1 --- Datasets --- p.66Chapter 6.4.2 --- Image Representation --- p.66Chapter 6.4.3 --- Experimental Setup --- p.67Chapter 6.4.4 --- Performance Comparison --- p.68Chapter 6.5 --- Discussions --- p.73Chapter 6.6 --- Summary --- p.75Chapter 7 --- Application: Web Image Learning --- p.76Chapter 7.1 --- Introduction --- p.76Chapter 7.2 --- A Learning Scheme for Searching Semantic Concepts --- p.77Chapter 7.2.1 --- Searching and Clustering Web Images --- p.78Chapter 7.2.2 --- Learning Semantic Concepts with Relevance Feed- back --- p.73Chapter 7.3 --- Experimental Results --- p.79Chapter 7.3.1 --- Dataset and Features --- p.79Chapter 7.3.2 --- Performance Evaluation --- p.80Chapter 7.4 --- Discussions --- p.82Chapter 7.5 --- Summary --- p.82Chapter 8 --- Conclusions and Future Work --- p.84Chapter 8.1 --- Conclusions --- p.84Chapter 8.2 --- Future Work --- p.85Chapter A --- List of Publications --- p.87Bibliography --- p.10

Semantic image retrieval using relevance feedback and transaction logs

Due to the recent improvements in digital photography and storage capacity, storing large amounts of images has been made possible, and efficient means to retrieve images matching a user’s query are needed. Content-based Image Retrieval (CBIR) systems automatically extract image contents based on image features, i.e. color, texture, and shape. Relevance feedback methods are applied to CBIR to integrate users’ perceptions and reduce the gap between high-level image semantics and low-level image features. The precision of a CBIR system in retrieving semantically rich (complex) images is improved in this dissertation work by making advancements in three areas of a CBIR system: input, process, and output. The input of the system includes a mechanism that provides the user with required tools to build and modify her query through feedbacks. Users behavioral in CBIR environments are studied, and a new feedback methodology is presented to efficiently capture users’ image perceptions. The process element includes image learning and retrieval algorithms. A Long-term image retrieval algorithm (LTL), which learns image semantics from prior search results available in the system’s transaction history, is developed using Factor Analysis. Another algorithm, a short-term learner (STL) that captures user’s image perceptions based on image features and user’s feedbacks in the on-going transaction, is developed based on Linear Discriminant Analysis. Then, a mechanism is introduced to integrate these two algorithms to one retrieval procedure. Finally, a retrieval strategy that includes learning and searching phases is defined for arranging images in the output of the system. The developed relevance feedback methodology proved to reduce the effect of human subjectivity in providing feedbacks for complex images. Retrieval algorithms were applied to images with different degrees of complexity. LTL is efficient in extracting the semantics of complex images that have a history in the system. STL is suitable for query and images that can be effectively represented by their image features. Therefore, the performance of the system in retrieving images with visual and conceptual complexities was improved when both algorithms were applied simultaneously. Finally, the strategy of retrieval phases demonstrated promising results when the query complexity increases

Biased classification for relevance feedback in content-based image retrieval.

Peng, Xiang.Thesis (M.Phil.)--Chinese University of Hong Kong, 2007.Includes bibliographical references (leaves 98-115).Abstracts in English and Chinese.Abstract --- p.iAcknowledgement --- p.ivChapter 1 --- Introduction --- p.1Chapter 1.1 --- Problem Statement --- p.3Chapter 1.2 --- Major Contributions --- p.6Chapter 1.3 --- Thesis Outline --- p.7Chapter 2 --- Background Study --- p.9Chapter 2.1 --- Content-based Image Retrieval --- p.9Chapter 2.1.1 --- Image Representation --- p.11Chapter 2.1.2 --- High Dimensional Indexing --- p.15Chapter 2.1.3 --- Image Retrieval Systems Design --- p.16Chapter 2.2 --- Relevance Feedback --- p.19Chapter 2.2.1 --- Self-Organizing Map in Relevance Feedback --- p.20Chapter 2.2.2 --- Decision Tree in Relevance Feedback --- p.22Chapter 2.2.3 --- Bayesian Classifier in Relevance Feedback --- p.24Chapter 2.2.4 --- Nearest Neighbor Search in Relevance Feedback --- p.25Chapter 2.2.5 --- Support Vector Machines in Relevance Feedback --- p.26Chapter 2.3 --- Imbalanced Classification --- p.29Chapter 2.4 --- Active Learning --- p.31Chapter 2.4.1 --- Uncertainly-based Sampling --- p.33Chapter 2.4.2 --- Error Reduction --- p.34Chapter 2.4.3 --- Batch Selection --- p.35Chapter 2.5 --- Convex Optimization --- p.35Chapter 2.5.1 --- Overview of Convex Optimization --- p.35Chapter 2.5.2 --- Linear Program --- p.37Chapter 2.5.3 --- Quadratic Program --- p.37Chapter 2.5.4 --- Quadratically Constrained Quadratic Program --- p.37Chapter 2.5.5 --- Cone Program --- p.38Chapter 2.5.6 --- Semi-definite Program --- p.39Chapter 3 --- Imbalanced Learning with BMPM for CBIR --- p.40Chapter 3.1 --- Research Motivation --- p.41Chapter 3.2 --- Background Review --- p.42Chapter 3.2.1 --- Relevance Feedback for CBIR --- p.42Chapter 3.2.2 --- Minimax Probability Machine --- p.42Chapter 3.2.3 --- Extensions of Minimax Probability Machine --- p.44Chapter 3.3 --- Relevance Feedback using BMPM --- p.45Chapter 3.3.1 --- Model Definition --- p.45Chapter 3.3.2 --- Advantages of BMPM in Relevance Feedback --- p.46Chapter 3.3.3 --- Relevance Feedback Framework by BMPM --- p.47Chapter 3.4 --- Experimental Results --- p.47Chapter 3.4.1 --- Experiment Datasets --- p.48Chapter 3.4.2 --- Performance Evaluation --- p.50Chapter 3.4.3 --- Discussions --- p.53Chapter 3.5 --- Summary --- p.53Chapter 4 --- BMPM Active Learning for CBIR --- p.55Chapter 4.1 --- Problem Statement and Motivation --- p.55Chapter 4.2 --- Background Review --- p.57Chapter 4.3 --- Relevance Feedback by BMPM Active Learning . --- p.58Chapter 4.3.1 --- Active Learning Concept --- p.58Chapter 4.3.2 --- General Approaches for Active Learning . --- p.59Chapter 4.3.3 --- Biased Minimax Probability Machine --- p.60Chapter 4.3.4 --- Proposed Framework --- p.61Chapter 4.4 --- Experimental Results --- p.63Chapter 4.4.1 --- Experiment Setup --- p.64Chapter 4.4.2 --- Performance Evaluation --- p.66Chapter 4.5 --- Summary --- p.68Chapter 5 --- Large Scale Learning with BMPM --- p.70Chapter 5.1 --- Introduction --- p.71Chapter 5.1.1 --- Motivation --- p.71Chapter 5.1.2 --- Contribution --- p.72Chapter 5.2 --- Background Review --- p.72Chapter 5.2.1 --- Second Order Cone Program --- p.72Chapter 5.2.2 --- General Methods for Large Scale Problems --- p.73Chapter 5.2.3 --- Biased Minimax Probability Machine --- p.75Chapter 5.3 --- Efficient BMPM Training --- p.78Chapter 5.3.1 --- Proposed Strategy --- p.78Chapter 5.3.2 --- Kernelized BMPM and Its Solution --- p.81Chapter 5.4 --- Experimental Results --- p.82Chapter 5.4.1 --- Experimental Testbeds --- p.83Chapter 5.4.2 --- Experimental Settings --- p.85Chapter 5.4.3 --- Performance Evaluation --- p.87Chapter 5.5 --- Summary --- p.92Chapter 6 --- Conclusion and Future Work --- p.93Chapter 6.1 --- Conclusion --- p.93Chapter 6.2 --- Future Work --- p.94Chapter A --- List of Symbols and Notations --- p.96Chapter B --- List of Publications --- p.98Bibliography --- p.10

Using biased support vector machine in image retrieval with self-organizing map.

Chan Chi Hang.Thesis submitted in: August 2004.Thesis (M.Phil.)--Chinese University of Hong Kong, 2005.Includes bibliographical references (leaves 105-114).Abstracts in English and Chinese.Abstract --- p.iAcknowledgement --- p.ivChapter 1 --- Introduction --- p.1Chapter 1.1 --- Problem Statement --- p.3Chapter 1.2 --- Major Contributions --- p.5Chapter 1.3 --- Publication List --- p.6Chapter 1.4 --- Thesis Organization --- p.7Chapter 2 --- Background Survey --- p.9Chapter 2.1 --- Relevance Feedback Framework --- p.9Chapter 2.1.1 --- Relevance Feedback Types --- p.11Chapter 2.1.2 --- Data Distribution --- p.12Chapter 2.1.3 --- Training Set Size --- p.14Chapter 2.1.4 --- Inter-Query Learning and Intra-Query Learning --- p.15Chapter 2.2 --- History of Relevance Feedback Techniques --- p.16Chapter 2.3 --- Relevance Feedback Approaches --- p.19Chapter 2.3.1 --- Vector Space Model --- p.19Chapter 2.3.2 --- Ad-hoc Re-weighting --- p.26Chapter 2.3.3 --- Distance Optimization Approach --- p.29Chapter 2.3.4 --- Probabilistic Model --- p.33Chapter 2.3.5 --- Bayesian Approach --- p.39Chapter 2.3.6 --- Density Estimation Approach --- p.42Chapter 2.3.7 --- Support Vector Machine --- p.48Chapter 2.4 --- Presentation Set Selection --- p.52Chapter 2.4.1 --- Most-probable strategy --- p.52Chapter 2.4.2 --- Most-informative strategy --- p.52Chapter 3 --- Biased Support Vector Machine for Content-Based Image Retrieval --- p.57Chapter 3.1 --- Motivation --- p.57Chapter 3.2 --- Background --- p.58Chapter 3.2.1 --- Regular Support Vector Machine --- p.59Chapter 3.2.2 --- One-class Support Vector Machine --- p.61Chapter 3.3 --- Biased Support Vector Machine --- p.63Chapter 3.4 --- Interpretation of parameters in BSVM --- p.67Chapter 3.5 --- Soft Label Biased Support Vector Machine --- p.69Chapter 3.6 --- Interpretation of parameters in Soft Label BSVM --- p.73Chapter 3.7 --- Relevance Feedback Using Biased Support Vector Machine --- p.74Chapter 3.7.1 --- Advantages of BSVM in Relevance Feedback . . --- p.74Chapter 3.7.2 --- Relevance Feedback Algorithm By BSVM --- p.75Chapter 3.8 --- Experiments --- p.78Chapter 3.8.1 --- Synthetic Dataset --- p.80Chapter 3.8.2 --- Real-World Dataset --- p.81Chapter 3.8.3 --- Experimental Results --- p.83Chapter 3.9 --- Conclusion --- p.86Chapter 4 --- Self-Organizing Map-based Inter-Query Learning --- p.88Chapter 4.1 --- Motivation --- p.88Chapter 4.2 --- Algorithm --- p.89Chapter 4.2.1 --- Initialization and Replication of SOM --- p.89Chapter 4.2.2 --- SOM Training for Inter-Query Learning --- p.90Chapter 4.2.3 --- Incorporate with Intra-Query Learning --- p.92Chapter 4.3 --- Experiments --- p.93Chapter 4.3.1 --- Synthetic Dataset --- p.95Chapter 4.3.2 --- Real-World Dataset --- p.95Chapter 4.3.3 --- Experimental Results --- p.97Chapter 4.4 --- Conclusion --- p.98Chapter 5 --- Conclusion --- p.102Bibliography --- p.10

Attribute-based Image Retrieval: Towards Bridging the Semantic and Intention Gaps

Ph.DDOCTOR OF PHILOSOPH

Content-based image retrieval-- a small sample learning approach.

Tao Dacheng.Thesis (M.Phil.)--Chinese University of Hong Kong, 2004.Includes bibliographical references (leaves 70-75).Abstracts in English and Chinese.Chapter Chapter 1 --- Introduction --- p.1Chapter 1.1 --- Content-based Image Retrieval --- p.1Chapter 1.2 --- SVM based RF in CBIR --- p.3Chapter 1.3 --- DA based RF in CBIR --- p.4Chapter 1.4 --- Existing CBIR Engines --- p.5Chapter 1.5 --- Practical Applications of CBIR --- p.10Chapter 1.6 --- Organization of this thesis --- p.11Chapter Chapter 2 --- Statistical Learning Theory and Support Vector Machine --- p.12Chapter 2.1 --- The Recognition Problem --- p.12Chapter 2.2 --- Regularization --- p.14Chapter 2.3 --- The VC Dimension --- p.14Chapter 2.4 --- Structure Risk Minimization --- p.15Chapter 2.5 --- Support Vector Machine --- p.15Chapter 2.6 --- Kernel Space --- p.17Chapter Chapter 3 --- Discriminant Analysis --- p.18Chapter 3.1 --- PCA --- p.18Chapter 3.2 --- KPCA --- p.18Chapter 3.3 --- LDA --- p.20Chapter 3.4 --- BDA --- p.20Chapter 3.5 --- KBDA --- p.21Chapter Chapter 4 --- Random Sampling Based SVM --- p.24Chapter 4.1 --- Asymmetric Bagging SVM --- p.25Chapter 4.2 --- Random Subspace Method SVM --- p.26Chapter 4.3 --- Asymmetric Bagging RSM SVM --- p.26Chapter 4.4 --- Aggregation Model --- p.30Chapter 4.5 --- Dissimilarity Measure --- p.31Chapter 4.6 --- Computational Complexity Analysis --- p.31Chapter 4.7 --- QueryGo Image Retrieval System --- p.32Chapter 4.8 --- Toy Experiments --- p.35Chapter 4.9 --- Statistical Experimental Results --- p.36Chapter Chapter 5 --- SSS Problems in KBDA RF --- p.42Chapter 5.1 --- DKBDA --- p.43Chapter 5.1.1 --- DLDA --- p.43Chapter 5.1.2 --- DKBDA --- p.43Chapter 5.2 --- NKBDA --- p.48Chapter 5.2.1 --- NLDA --- p.48Chapter 5.2.2 --- NKBDA --- p.48Chapter 5.3 --- FKBDA --- p.49Chapter 5.3.1 --- FLDA --- p.49Chapter 5.3.2 --- FKBDA --- p.49Chapter 5.4 --- Experimental Results --- p.50Chapter Chapter 6 --- NDA based RF for CBIR --- p.52Chapter 6.1 --- NDA --- p.52Chapter 6.2 --- SSS Problem in NDA --- p.53Chapter 6.2.1 --- Regularization method --- p.53Chapter 6.2.2 --- Null-space method --- p.54Chapter 6.2.3 --- Full-space method --- p.54Chapter 6.3 --- Experimental results --- p.55Chapter 6.3.1 --- K nearest neighbor evaluation for NDA --- p.55Chapter 6.3.2 --- SSS problem --- p.56Chapter 6.3.3 --- Evaluation experiments --- p.57Chapter Chapter 7 --- Medical Image Classification --- p.59Chapter 7.1 --- Introduction --- p.59Chapter 7.2 --- Region-based Co-occurrence Matrix Texture Feature --- p.60Chapter 7.3 --- Multi-level Feature Selection --- p.62Chapter 7.4 --- Experimental Results --- p.63Chapter 7.4.1 --- Data Set --- p.64Chapter 7.4.2 --- Classification Using Traditional Features --- p.65Chapter 7.4.3 --- Classification Using the New Features --- p.66Chapter Chapter 8 --- Conclusion --- p.68Bibliography --- p.7