Bridging the semantic gap in content-based image retrieval.

To manage large image databases, Content-Based Image Retrieval (CBIR) emerged as a new research subject. CBIR involves the development of automated methods to use visual features in searching and retrieving. Unfortunately, the performance of most CBIR systems is inherently constrained by the low-level visual features because they cannot adequately express the user\u27s high-level concepts. This is known as the semantic gap problem. This dissertation introduces a new approach to CBIR that attempts to bridge the semantic gap. Our approach includes four components. The first one learns a multi-modal thesaurus that associates low-level visual profiles with high-level keywords. This is accomplished through image segmentation, feature extraction, and clustering of image regions. The second component uses the thesaurus to annotate images in an unsupervised way. This is accomplished through fuzzy membership functions to label new regions based on their proximity to the profiles in the thesaurus. The third component consists of an efficient and effective method for fusing the retrieval results from the multi-modal features. Our method is based on learning and adapting fuzzy membership functions to the distribution of the features\u27 distances and assigning a degree of worthiness to each feature. The fourth component provides the user with the option to perform hybrid querying and query expansion. This allows the enrichment of a visual query with textual data extracted from the automatically labeled images in the database. The four components are integrated into a complete CBIR system that can run in three different and complementary modes. The first mode allows the user to query using an example image. The second mode allows the user to specify positive and/or negative sample regions that should or should not be included in the retrieved images. The third mode uses a Graphical Text Interface to allow the user to browse the database interactively using a combination of low-level features and high-level concepts. The proposed system and ail of its components and modes are implemented and validated using a large data collection for accuracy, performance, and improvement over traditional CBIR techniques

Penerapan Algoritme Segmentasi Mean Shift Dan Pemilihan Fitur Dasar Warna Pada Sistem Temu Kembali Citra Berbasis ISI

Sistem Temu Kembali Citra Berbasis lsi adalah sistem yang mengorganisir data citra hasil pengenalan objek, mengenali data citra baru dan mampu melakukan pencarian data citra berdasarkan objek dari citra da/am jumlah citra besar dan variasi gambar yang beragam. Salah satu implementasi penggunaan Sistem Temu Kembali Citra Berbasis lsi adalah pencarian koleksi barang di museum. User dapat mempero/eh informasi gambar tentang barang-barang koleksi museum hanya dengan memasukkan nama objek dari barang koleksi. Pada Tugas Akhir ini akan difokuskan pertama pada pembuatan Sistem Temu Kembali Citra Berbasis lsi dengan tiga tahapan. Pada tahapan pertama pelatihan data, data citra yang te/ah diberi label/keyword berisikan nama-nama objek pada citra disegmentasi menggunakan algoritme Mean Shift untuk menghasilkan region-region. Untuk menghasilkan vektor fitur citra maka dilakukan ekstraksi fitur dasar warna pada hasil segmentasi, menge/ompokkanlclustering region ke dalam kelas-ke/aslblob-b/ob dengan menggunakan algoritme K-Means berdasarkan vektor fitur, memprediksi keterhubungan antara blob dengan word dengan menggunakan algoritme Expectation Maximization. Tahap kedua, Pengenalan Data Citra baru di/akukan dengan menggunakan algoritme Nearest-Neighbour berdasarkan kedekatan vektor fitur data citra baru dengan data citra hasil pelatihan. Pada tahap ketiga dilakukan Pencarian Data Citra berdasarkan data input user berupa teks yang mewakili nama objek yang dicari dari data citra yang telah dihasi/kan pada tahapan pelatihan data. Fokus kedua yaitu ana/isis kinerja Sistem Temu Kembali Citra Berbasis lsi hasil dari pemilihanfitur dasar warna dalam domain (L*u*v, RGB, L *a*b, L *a*b dan CrCb), penggunaan algoritme mean shift pada tahapan awallprepocessing. Berdasarkan hasil uji coba, penentuan jumlah k/aster yang lebih banyak terbukti meningkatkan kinerja Sistem Temu Kembali Citra Berbasis lsi serta pemilihan fitur dasar warna domain L *u*v /ebih memiliki nilai kinerja yang lebih tinggi daripada domain warna yang lain (L *u*v, RGB, L *a*b, L *a*b, CrCb, dan kombinasinya

USING SOCIAL ANNOTATIONS TO IMPROVE WEB SEARCH

Web-based tagging systems, which include social bookmarking systems such as Delicious, have become increasingly popular. These systems allow participants to annotate or tag web resources. This research examined the use of social annotations to improve the quality of web searches. The research involved three components. First, social annotations were used to index resources. Two annotation-based indexing methods were proposed: annotation based indexing and full text with annotation indexing. Second, social annotations were used to improve search result ranking. Six annotation based ranking methods were proposed: Popularity Count, Propagate Popularity Count, Query Weighted Popularity Count, Query Weighted Propagate Popularity Count, Match Tag Count and Normalized Match Tag Count. Third, social annotations were used to both index and rank resources. The result from the first experiment suggested that both static feature and similarity feature should be considered when using social annotations to re-rank search result. The result of the second experiment showed that using only annotation as an index of resources may not be a good idea. Since social Annotations could be viewed as a high level concept of the content, combining them to the content of resource could add some more important concepts to the resources. Last but not least, the result from the third experiment confirmed that the combination of using social annotations to rank the search result and using social annotations as resource index augmentation provided a promising rank of search results. It showed that social annotations could benefit web search

Aplicación del modelo Bag-of-Words al reconocimiento de imágenes

Object recognition on images has been more investigated in the recent years. Its principal application is the image retrieval and, therefore, image searchers would find the solution to the query based on whether the image has certain objects in its visual content or not instead of based on the adjacent textual annotations. Content based image retrieval would improve notoriously the quality of searchers. It is neccesary to have models that classify an image based on its low level features. In this project, it is used the ‘Bag of words’ model. Multimedia information retrieval entails many fields involved, and has many applications. The objective of this project is the indexing of images of a database based on content. It tries to eliminate the semantic gap finding the descriptors of each imagen, and therefore decide to which class or which semantic concept belongs.--------------------------------------------------------------------El reconocimiento de objetos en imágenes es un campo cada vez más investigado y que se aplica principalmente a la recuperación de imágenes basada en contenido, es decir, a buscadores de imágenes que encontrarán la solución a una consulta basándose en si la imagen contiene ciertos objetos o no en función de su contenido visual, y no de las anotaciones textuales colindantes. Su aplicación surge de la necesidad de sistemas de gestión automatizada de documentos multimedia que sustituyan a la gestión manual, ya que ciertas bases de datos de información multimedia tienen tamaños impracticables para realizar una anotación manual. La recuperación de imágenes basada en contenido mejoraría significativamente la calidad de las búsquedas. Para ello es necesario disponer de modelos que se enfrenten a la clasificación de una imagen a partir de sus características de bajo nivel. En este proyecto se va a utilizar el modelo Bag-of-words (BoW). La recuperación de información multimedia conlleva muchos campos involucrados: clasificadores de información, estadísticas de señales, visión artificial… Por otro lado, también tiene multitud de aplicaciones: buscadores Web, detección de rostros en fotografías, recuperación de imágenes médicas, robótica, etc. Este proyecto tiene como objetivo la indexación de las imágenes de una base de datos basándose en el contenido. Trata de eliminar la laguna semántica hallando los descriptores de cada imagen de la base de datos para luego discernir a qué clase o concepto semántico pertenecen.Ingeniería Técnica en Sonido e Image

Local selection of features and its applications to image search and annotation

In multimedia applications, direct representations of data objects typically involve hundreds or thousands of features. Given a query object, the similarity between the query object and a database object can be computed as the distance between their feature vectors. The neighborhood of the query object consists of those database objects that are close to the query object. The semantic quality of the neighborhood, which can be measured as the proportion of neighboring objects that share the same class label as the query object, is crucial for many applications, such as content-based image retrieval and automated image annotation. However, due to the existence of noisy or irrelevant features, errors introduced into similarity measurements are detrimental to the neighborhood quality of data objects. One way to alleviate the negative impact of noisy features is to use feature selection techniques in data preprocessing. From the original vector space, feature selection techniques select a subset of features, which can be used subsequently in supervised or unsupervised learning algorithms for better performance. However, their performance on improving the quality of data neighborhoods is rarely evaluated in the literature. In addition, most traditional feature selection techniques are global, in the sense that they compute a single set of features across the entire database. As a consequence, the possibility that the feature importance may vary across different data objects or classes of objects is neglected. To compute a better neighborhood structure for objects in high-dimensional feature spaces, this dissertation proposes several techniques for selecting features that are important to the local neighborhood of individual objects. These techniques are then applied to image applications such as content-based image retrieval and image label propagation. Firstly, an iterative K-NN graph construction method for image databases is proposed. A local variant of the Laplacian Score is designed for the selection of features for individual images. Noisy features are detected and sparsified iteratively from the original standardized feature vectors. This technique is incorporated into an approximate K-NN graph construction method so as to improve the semantic quality of the graph. Secondly, in a content-based image retrieval system, a generalized version of the Laplacian Score is used to compute different feature subspaces for images in the database. For online search, a query image is ranked in the feature spaces of database images. Those database images for which the query image is ranked highly are selected as the query results. Finally, a supervised method for the local selection of image features is proposed, for refining the similarity graph used in an image label propagation framework. By using only the selected features to compute the edges leading from labeled image nodes to unlabeled image nodes, better annotation accuracy can be achieved. Experimental results on several datasets are provided in this dissertation, to demonstrate the effectiveness of the proposed techniques for the local selection of features, and for the image applications under consideration

Automatic image annotation and retrieval using weighted feature selection

Abstract. The development of technology generates huge amounts of non-textual information, such as images. An efficient image annotation and retrieval system is highly desired. Clustering algorithms make it possible to represent visual features of images with finite symbols. Based on this, many statistical models, which analyze correspondence between visual features and words and discover hidden semantics, have been published. These models improve the annotation and retrieval of large image databases. However, image data usually have a large number of dimensions. Traditional clustering algorithms assign equal weights to these dimensions, and become confounded in the process of dealing with these dimensions. In this paper, we propose weighted feature selection algorithm as a solution to this problem. For a given cluster, we determine relevant features based on histogram analysis and assign greater weight to relevant features as compared to less relevant features. We have implemented various different models to link visual tokens with keywords based on the clustering results of K-means algorithm with weighted feature selection and without feature selection, and evaluated performance using precision, recall and correspondence accuracy using benchmark dataset. The results show that weighted feature selection is better than traditional ones for automatic image annotation and retrieval