Investigating the Behavior of Compact Composite Descriptors in Early Fusion, Late Fusion and Distributed Image Retrieval

In Content-Based Image Retrieval (CBIR) systems, the visual content of the images is mapped into a new space named the feature space. The features that are chosen must be discriminative and sufficient for the description of the objects. The key to attaining a successful retrieval system is to choose the right features that represent the images as unique as possible. A feature is a set of characteristics of the image, such as color, texture, and shape. In addition, a feature can be enriched with information about the spatial distribution of the characteristic that it describes. Evaluation of the performance of low-level features is usually done on homogenous benchmarking databases with a limited number of images. In real-world image retrieval systems, databases have a much larger scale and may be heterogeneous. This paper investigates the behavior of Compact Composite Descriptors (CCDs) on heterogeneous databases of a larger scale. Early and late fusion techniques are tested and their performance in distributed image retrieval is calculated. This study demonstrates that, even if it is not possible to overcome the semantic gap in image retrieval by feature similarity, it is still possible to increase the retrieval effectiveness

ImSe : Instant Interactive Image Retrieval System with Exploration/Exploitation trade-off

Imagine a journalist looking for an illustration to his article about patriotism in a database of unannotated images. The idea of a suitable image is very vague and the best way to navigate through the database is to provide feedback to the images proposed by an Image Retrieval system in order to enable the system to learn what the ideal target image of the user is. Thus, at each search iteration a set of n images is displayed and the user must indicate how relevant they are to his/her target. When considering real-life problems we must also take into account the system's time-complexity and scalability to work with Big Data. To tackle this issue we utilize hierarchical Gaussian Process Bandits with visual Self-Organizing Map as a preprocessing technique. A prototype system called ImSe was developed and tested in experiments with real users in different types of tasks. The experiments show favorable results and indicate the benefits of proposed algorithms in different types of tasks

A Practical Framework for Executing Complex Queries over Encrypted Multimedia Data

Over the last few years, data storage in cloud based services has been very popular due to easy management and monetary advantages of cloud computing. Recent developments showed that such data could be leaked due to various attacks. To address some of these attacks, encrypting sensitive data before sending to cloud emerged as an important protection mechanism. If the data is encrypted with traditional techniques, selective retrieval of encrypted data becomes challenging. To address this challenge, efficient searchable encryption schemes have been developed over the years. Almost all of the existing searchable encryption schemes are developed for keyword searches and require running some code on the cloud servers. However, many of the existing cloud storage services (e.g., Dropbox, Box, Google Drive, etc.) only allow simple data object retrieval and do not provide computational support needed to realize most of the searchable encryption schemes. In this paper, we address the problem of efficient execution of complex search queries over wide range of encrypted data types (e.g., image files) without requiring customized computational support from the cloud servers. To this end, we provide an extensible framework for supporting complex search queries over encrypted multimedia data. Before any data is uploaded to the cloud, important features are extracted to support different query types (e.g., extracting facial features to support face recognition queries) and complex queries are converted to series of object retrieval tasks for cloud service. Our results show that this framework may support wide range of image retrieval queries on encrypted data with little overhead and without any change to underlying data storage services

Recuperación de imágenes en artículos científicos usando estrategias de anotación automática

En este trabajo se explora la utilización de estrategias de anotación automática sobre información textual y visual obtenida de artículos científicos, la forma en que este contenido se relaciona y la representación de esta información, con el _n de desarrollar un sistema de recuperación de información por contenido específico para este tipo de colecciones. Para esto, un nuevo modelo de representación, recuperación y anotación automática de imágenes es propuesto. Este modelo está basado en estrategias de semántica latente para representaciones estructuradas. El sistema desarrollado durante este trabajo es llamado Litermed, el cual implementa el modelo propuesto y ofrece las funcionalidades de procesamiento necesarias para la transformación de archivos correspondientes a artículos científicos en la representación propuesta. Para esto se desarrollaron fases como: extracción de imágenes de archivos PDF, extracción de características textuales y visuales, construcción de índices de características con sus respectivas anotaciones, clasificación de modalidad de imágenes, solución y evaluación de consultas visuales. Además, Litermed permite la realización de consultas por medio de su interfaz web utilizando como consulta imágenes de ejemplo. Para la realización de una evaluación cuantitativa del sistema, se propone el uso de un versión modificada de un conjunto de datos conocido. Los resultados indican que el modelo propuesto de anotación automática mejora el desempeño obtenido por estrategias de recuperación por contenido del estado del arte.Abstract. In this work, we explore the use of automatic annotation strategies for text-visual information from research papers, as well as the relationship between the content and the representation to build a retrieval system for this specific type of documents. To achieved that, we propose a novel strategy for the representation, search and automatic annotation of images. This model, is based on strategies of latent semantic analysis for structured representations. The system that implements the proposed model is called Litermed. This system is able to process the research papers _les to achieve the proposed representation. The processing phases are decomposed as follow: image extraction from research paper _les (PDF), text-visual features extraction, index _les construction with associated annotations, modality image classification, solution and evaluation of visual queries. Additionaly, Litermed allows run visual queries over a web based interface. Finally, an exhaustive automatic evaluation is performed over a modified version of a public well know dataset. The results show that the proposed model outperforms the state-of-the-art methods of query-by-example search.Maestrí

Recuperación de imágenes en artículos científicos usando estrategias de anotación automática

En este trabajo se explora la utilización de estrategias de anotación automática sobre información textual y visual obtenida de artículos científicos, la forma en que este contenido se relaciona y la representación de esta información, con el _n de desarrollar un sistema de recuperación de información por contenido específico para este tipo de colecciones. Para esto, un nuevo modelo de representación, recuperación y anotación automática de imágenes es propuesto. Este modelo está basado en estrategias de semántica latente para representaciones estructuradas. El sistema desarrollado durante este trabajo es llamado Litermed, el cual implementa el modelo propuesto y ofrece las funcionalidades de procesamiento necesarias para la transformación de archivos correspondientes a artículos científicos en la representación propuesta. Para esto se desarrollaron fases como: extracción de imágenes de archivos PDF, extracción de características textuales y visuales, construcción de índices de características con sus respectivas anotaciones, clasificación de modalidad de imágenes, solución y evaluación de consultas visuales. Además, Litermed permite la realización de consultas por medio de su interfaz web utilizando como consulta imágenes de ejemplo. Para la realización de una evaluación cuantitativa del sistema, se propone el uso de un versión modificada de un conjunto de datos conocido. Los resultados indican que el modelo propuesto de anotación automática mejora el desempeño obtenido por estrategias de recuperación por contenido del estado del arte.Abstract. In this work, we explore the use of automatic annotation strategies for text-visual information from research papers, as well as the relationship between the content and the representation to build a retrieval system for this specific type of documents. To achieved that, we propose a novel strategy for the representation, search and automatic annotation of images. This model, is based on strategies of latent semantic analysis for structured representations. The system that implements the proposed model is called Litermed. This system is able to process the research papers _les to achieve the proposed representation. The processing phases are decomposed as follow: image extraction from research paper _les (PDF), text-visual features extraction, index _les construction with associated annotations, modality image classification, solution and evaluation of visual queries. Additionaly, Litermed allows run visual queries over a web based interface. Finally, an exhaustive automatic evaluation is performed over a modified version of a public well know dataset. The results show that the proposed model outperforms the state-of-the-art methods of query-by-example search.Maestrí

Recuperación de Imágenes en Artículos Cientitíficos usando estrategias de Anotación Automática

En este trabajo se explora la utilización de estrategias de anotación automática sobre información textual y visual obtenida de artículos cientí�ficos, la forma en que este contenido se relaciona y la representación de esta información, con el �n de desarrollar un sistema de recuperación de información por contenido especí�co para este tipo de colecciones. Para esto, un nuevo modelo de representación, recuperación y anotación automática de imágenes es propuesto. Este modelo está basado en estrategias de semántica latente para representaciónes estructuradas. El sistema desarrollado durante este trabajo es llamado Litermed, el cual implementa el modelo propuesto y ofrece las funcionalidades de procesamiento necesarias para la transformación de archivos correspondientes a artículos cientí�cos en la representación propuesta. Para esto se desarrollaron fases como: extracción de imágenes de archivos PDF, extracción de características textuales y visuales, construcción de índices de características con sus respectivas anotaciones, clasi�cación de modalidad de imágenes, solución y evaluación de consultas visuales. Además, Litermed permite la realización de consultas por medio de su interfaz web utilizando como consulta imágenes de ejemplo. Para la realización de una evaluación cuantitativa del sistema, se propone el uso de un versión modi�cada de un conjunto de datos conocido. Los resultados indican que el modelo propuesto de anotación automática mejora el desempeño obtenido por estrategias de recuperación por contenido del estado del arte.Abstract. In this work, we explore the use of automatic annotation strategies for text-visual information from research papers, as well as the relationship between the content and the representation to build a retrieval system for this specific type of documents. To achieved that, we propose a novel strategy for the representation, search and automatic annotation of images. This model, is based on strategies of latent semantic analysis for structured representations. The system that implements the proposed model is called Litermed. This system is able to process the research papers �les to achieve the proposed representation. The processing phases are decomposed as follow: image extraction from research paper files (PDF), text-visual features extraction, index files construction with associated annotations, modality image classi fication, solution and evaluation of visual queries. Additionaly, Litermed allows run visual queries over a web based interface. Finally, an exhuastive automatic evaluation is performed over a modified version of a public well know dataset. The results show that the proposed model outperforms the state-of-the-art methods of query-by-example search.Maestrí

A framework for automated landmark recognition in community contributed image corpora

Any large library of information requires efficient ways to organise it and methods that allow people to access information efficiently and collections of digital images are no exception. Automatically creating high-level semantic tags based on image content is difficult, if not impossible to achieve accurately. In this thesis a framework is presented that allows for the automatic creation of rich and accurate tags for images with landmarks as the main object. This framework uses state of the art computer vision techniques fused with the wide range of contextual information that is available with community contributed imagery. Images are organised into clusters based on image content and spatial data associated with each image. Based on these clusters different types of classifiers are* trained to recognise landmarks contained within the images in each cluster. A novel hybrid approach is proposed combining these classifiers with an hierarchical matching approach to allow near real-time classification and captioning of images containing landmarks

A tree grammar-based visual password scheme

A thesis submitted to the Faculty of Science, University of the Witwatersrand, Johannesburg, in fulfilment of the requirements for the degree of Doctor of Philosophy. Johannesburg, August 31, 2015.Visual password schemes can be considered as an alternative to alphanumeric passwords. Studies have shown that alphanumeric passwords can, amongst others, be eavesdropped, shoulder surfed, or guessed, and are susceptible to brute force automated attacks. Visual password schemes use images, in place of alphanumeric characters, for authentication. For example, users of visual password schemes either select images (Cognometric) or points on an image (Locimetric) or attempt to redraw their password image (Drawmetric), in order to gain authentication. Visual passwords are limited by the so-called password space, i.e., by the size of the alphabet from which users can draw to create a password and by susceptibility to stealing of passimages by someone looking over your shoulders, referred to as shoulder surfing in the literature. The use of automatically generated highly similar abstract images defeats shoulder surfing and means that an almost unlimited pool of images is available for use in a visual password scheme, thus also overcoming the issue of limited potential password space. This research investigated visual password schemes. In particular, this study looked at the possibility of using tree picture grammars to generate abstract graphics for use in a visual password scheme. In this work, we also took a look at how humans determine similarity of abstract computer generated images, referred to as perceptual similarity in the literature. We drew on the psychological idea of similarity and matched that as closely as possible with a mathematical measure of image similarity, using Content Based Image Retrieval (CBIR) and tree edit distance measures. To this end, an online similarity survey was conducted with respondents ordering answer images in order of similarity to question images, involving 661 respondents and 50 images. The survey images were also compared with eight, state of the art, computer based similarity measures to determine how closely they model perceptual similarity. Since all the images were generated with tree grammars, the most popular measure of tree similarity, the tree edit distance, was also used to compare the images. Eight different types of tree edit distance measures were used in order to cover the broad range of tree edit distance and tree edit distance approximation methods. All the computer based similarity methods were then correlated with the online similarity survey results, to determine which ones more closely model perceptual similarity. The results were then analysed in the light of some modern psychological theories of perceptual similarity. This work represents a novel approach to the Passfaces type of visual password schemes using dynamically generated pass-images and their highly similar distractors, instead of static pictures stored in an online database. The results of the online survey were then accurately modelled using the most suitable tree edit distance measure, in order to automate the determination of similarity of our generated distractor images. The information gathered from our various experiments was then used in the design of a prototype visual password scheme. The generated images were similar, but not identical, in order to defeat shoulder surfing. This approach overcomes the following problems with this category of visual password schemes: shoulder surfing, bias in image selection, selection of easy to guess pictures and infrastructural limitations like large picture databases, network speed and database security issues. The resulting prototype developed is highly secure, resilient to shoulder surfing and easy for humans to use, and overcomes the aforementioned limitations in this category of visual password schemes