Design, implementation, and evaluation of scalable content-based image retrieval techniques.

Wong, Yuk Man.Thesis (M.Phil.)--Chinese University of Hong Kong, 2007.Includes bibliographical references (leaves 95-100).Abstracts in English and Chinese.Abstract --- p.iiAcknowledgement --- p.vChapter 1 --- Introduction --- p.1Chapter 1.1 --- Overview --- p.1Chapter 1.2 --- Contribution --- p.3Chapter 1.3 --- Organization of This Work --- p.5Chapter 2 --- Literature Review --- p.6Chapter 2.1 --- Content-based Image Retrieval --- p.6Chapter 2.1.1 --- Query Technique --- p.6Chapter 2.1.2 --- Relevance Feedback --- p.7Chapter 2.1.3 --- Previously Proposed CBIR systems --- p.7Chapter 2.2 --- Invariant Local Feature --- p.8Chapter 2.3 --- Invariant Local Feature Detector --- p.9Chapter 2.3.1 --- Harris Corner Detector --- p.9Chapter 2.3.2 --- DOG Extrema Detector --- p.10Chapter 2.3.3 --- Harris-Laplacian Corner Detector --- p.13Chapter 2.3.4 --- Harris-Affine Covariant Detector --- p.14Chapter 2.4 --- Invariant Local Feature Descriptor --- p.15Chapter 2.4.1 --- Scale Invariant Feature Transform (SIFT) --- p.15Chapter 2.4.2 --- Shape Context --- p.17Chapter 2.4.3 --- PCA-SIFT --- p.18Chapter 2.4.4 --- Gradient Location and Orientation Histogram (GLOH) --- p.19Chapter 2.4.5 --- Geodesic-Intensity Histogram (GIH) --- p.19Chapter 2.4.6 --- Experiment --- p.21Chapter 2.5 --- Feature Matching --- p.27Chapter 2.5.1 --- Matching Criteria --- p.27Chapter 2.5.2 --- Distance Measures --- p.28Chapter 2.5.3 --- Searching Techniques --- p.29Chapter 3 --- A Distributed Scheme for Large-Scale CBIR --- p.31Chapter 3.1 --- Overview --- p.31Chapter 3.2 --- Related Work --- p.33Chapter 3.3 --- Scalable Content-Based Image Retrieval Scheme --- p.34Chapter 3.3.1 --- Overview of Our Solution --- p.34Chapter 3.3.2 --- Locality-Sensitive Hashing --- p.34Chapter 3.3.3 --- Scalable Indexing Solutions --- p.35Chapter 3.3.4 --- Disk-Based Multi-Partition Indexing --- p.36Chapter 3.3.5 --- Parallel Multi-Partition Indexing --- p.37Chapter 3.4 --- Feature Representation --- p.43Chapter 3.5 --- Empirical Evaluation --- p.44Chapter 3.5.1 --- Experimental Testbed --- p.44Chapter 3.5.2 --- Performance Evaluation Metrics --- p.44Chapter 3.5.3 --- Experimental Setup --- p.45Chapter 3.5.4 --- Experiment I: Disk-Based Multi-Partition Indexing Approach --- p.45Chapter 3.5.5 --- Experiment II: Parallel-Based Multi-Partition Indexing Approach --- p.48Chapter 3.6 --- Application to WWW Image Retrieval --- p.55Chapter 3.7 --- Summary --- p.55Chapter 4 --- Image Retrieval System for IND Detection --- p.60Chapter 4.1 --- Overview --- p.60Chapter 4.1.1 --- Motivation --- p.60Chapter 4.1.2 --- Related Work --- p.61Chapter 4.1.3 --- Objective --- p.62Chapter 4.1.4 --- Contribution --- p.63Chapter 4.2 --- Database Construction --- p.63Chapter 4.2.1 --- Image Representations --- p.63Chapter 4.2.2 --- Index Construction --- p.64Chapter 4.2.3 --- Keypoint and Image Lookup Tables --- p.67Chapter 4.3 --- Database Query --- p.67Chapter 4.3.1 --- Matching Strategies --- p.68Chapter 4.3.2 --- Verification Processes --- p.71Chapter 4.3.3 --- Image Voting --- p.75Chapter 4.4 --- Performance Evaluation --- p.76Chapter 4.4.1 --- Evaluation Metrics --- p.76Chapter 4.4.2 --- Results --- p.77Chapter 4.4.3 --- Summary --- p.81Chapter 5 --- Shape-SIFT Feature Descriptor --- p.82Chapter 5.1 --- Overview --- p.82Chapter 5.2 --- Related Work --- p.83Chapter 5.3 --- SHAPE-SIFT Descriptors --- p.84Chapter 5.3.1 --- Orientation assignment --- p.84Chapter 5.3.2 --- Canonical orientation determination --- p.84Chapter 5.3.3 --- Keypoint descriptor --- p.87Chapter 5.4 --- Performance Evaluation --- p.88Chapter 5.5 --- Summary --- p.90Chapter 6 --- Conclusions and Future Work --- p.92Chapter 6.1 --- Conclusions --- p.92Chapter 6.2 --- Future Work --- p.93Chapter A --- Publication --- p.94Bibliography --- p.9

Content based video retrieval via spatial-temporal information discovery.

Content based video retrieval (CBVR) has been strongly motivated by a variety of realworld applications. Most state-of-the-art CBVR systems are built based on Bag-of-visual- Words (BovW) framework for visual resources representation and access. The framework, however, ignores spatial and temporal information contained in videos, which plays a fundamental role in unveiling semantic meanings. The information includes not only the spatial layout of visual content on a still frame (image), but also temporal changes across the sequential frames. Specially, spatially and temporally co-occurring visual words, which are extracted under the BovW framework, often tend to collaboratively represent objects, scenes, or events in the videos. The spatial and temporal information discovery would be useful to advance the CBVR technology. In this thesis, we propose to explore and analyse the spatial and temporal information from a new perspective: i) co-occurrence of the visual words is formulated as a correlation matrix, ii) spatial proximity and temporal coherence are analytically and empirically studied to re ne this correlation. Following this, a quantitative spatial and temporal correlation (STC) model is de ned. The STC discovered from either the query example (denoted by QC) or the data collection (denoted by DC) are assumed to determine speci- city of the visual words in the retrieval model, i:e: selected Words-Of-Interest are found more important for certain topics. Based on this hypothesis, we utilized the STC matrix to establish a novel visual content similarity measurement method and a query reformulation scheme for the retrieval model. Additionally, the STC also characterizes the context of the visual words, and accordingly a STC-Based context similarity measurement is proposed to detect the synonymous visual words. The method partially solves an inherent error of visual vocabulary under the BovW framework. Systematic experimental evaluations on public TRECVID and CC WEB VIDEO video collections demonstrate that the proposed methods based on the STC can substantially improve retrieval e ectiveness of the BovW framework. The retrieval model based on STC outperforms state-of-the-art CBVR methods on the data collections without storage and computational expense. Furthermore, the rebuilt visual vocabulary in this thesis is more compact and e ective. Above methods can be incorporated together for e ective and e cient CBVR system implementation. Based on the experimental results, it is concluded that the spatial-temporal correlation e ectively approximates the semantical correlation. This discovered correlation approximation can be utilized for both visual content representation and similarity measurement, which are key issues for CBVR technology development