Utilising semantic technologies for intelligent indexing and retrieval of digital images

The proliferation of digital media has led to a huge interest in classifying and indexing media objects for generic search and usage. In particular, we are witnessing colossal growth in digital image repositories that are difficult to navigate using free-text search mechanisms, which often return inaccurate matches as they in principle rely on statistical analysis of query keyword recurrence in the image annotation or surrounding text. In this paper we present a semantically-enabled image annotation and retrieval engine that is designed to satisfy the requirements of the commercial image collections market in terms of both accuracy and efficiency of the retrieval process. Our search engine relies on methodically structured ontologies for image annotation, thus allowing for more intelligent reasoning about the image content and subsequently obtaining a more accurate set of results and a richer set of alternatives matchmaking the original query. We also show how our well-analysed and designed domain ontology contributes to the implicit expansion of user queries as well as the exploitation of lexical databases for explicit semantic-based query expansion

COMPARATIVE ANALYSIS OF IMAGE RETRIEVAL TECHNIQUES IN CYBERSPACE

Purpose: With the popularity and remarkable usage of digital images in various domains, the existing image retrieval techniques need to be enhanced. The content-based image retrieval is playing a vital role to retrieve the requested data from the database available in cyberspace. CBIR from cyberspace is a popular and interesting research area nowadays for a better outcome. The searching and downloading of the requested images accurately based on meta-data from the cyberspace by using CBIR techniques is a challenging task. The purpose of this study is to explore the various image retrieval techniques for retrieving the data available in cyberspace.  Methodology: Whenever a user wishes to retrieve an image from the web, using present search engines, a bunch of images is retrieved based on a user query. But, most of the resultant images are unrelated to the user query. Here, the user puts their text-based query in the web-based search engine and compute the related images and retrieval time. Main Findings:  This study compares the accuracy and retrieval-time of the requested image. After the detailed analysis, the main finding is none of the used web-search engines viz. Flickr, Pixabay, Shutterstock, Bing, Everypixel, retrieved the accurate related images based on the entered query.   Implications: This study is discussing and performs a comparative analysis of various content-based image retrieval techniques from cyberspace. Novelty of Study: Research community has been making efforts towards efficient retrieval of useful images from the web but this problem has not been solved and it still prevails as an open research challenge. This study makes some efforts to resolve this research challenge and perform a comparative analysis of the outcome of various web-search engines

Design and Implementation of a Multimedia Information Retrieval Engine for the MSR-Bing Image Retrieval Challenge

The aim of this work is to design and implement a multimedia information retrieval engine for the MSR-Bing Retrieval Challenge provided by Microsoft. The challenge is based on the Clickture dataset, generated from click logs of Bing image search. The system has to predict the relevance of images with respect to text queries, by associating a score to a pair (image, text query) that indicates how the text query is good at describing the image content. We attempt to combine textual and visual information, by performing text-based and content-based image retrieval. The framework used to extract visual features is Caffe, an efficient implementation of deep Convolutional Neural Network(CNN). Decision is taken using a knowledge base containing triplets each consisting of a text query, an image, and the number of times that a users clicked on the image, in correspondence of the text query. Two strategies were proposed. In one case we analyse the intersection among the riplets elements retrieved respectively using the textual query and the image itself. In the other case we analyse the union. To solve efficiency issues we proposed an approach that index visual features using Apache Lucene, that is a text search engine library written entirely in Java, suitable for nearly any application requiring full-text search abilities. To this aim, we have converted image features into a textual form, to index them into an inverted index by means of Lucene. In this way we were able to set up a robust retrieval system that combines full-text search with content-based image retrieval capabilities. To prove that our search of textually and visually similar images really works, a small web-based prototype has been implemented. We evaluated different versions of our system over the development set in order to evaluate the measures of similarity to compare images, and to assess the best sorting strategy. Finally, our proposed approaches have been compared with those implemented by the winners of previous challenge editions

Físchlár-TRECVid2004: Combined text- and image-based searching of video archives

The Fischlar-TRECVid-2004 system was developed for Dublin City University's participation in the 2004 TRECVid video information retrieval benchmarking activity. The system allows search and retrieval of video shots from over 60 hours of content. The shot retrieval engine employed is based on a combination of query text matched against spoken dialogue combined with image-image matching where a still image (sourced externally), or a keyframe (from within the video archive itself), is matched against all keyframes in the video archive. Three separate text retrieval engines are employed for closed caption text, automatic speech recognition and video OCR. Visual shot matching is primarily based on MPEG-7 low-level descriptors. The system supports relevance feedback at the shot level enabling augmentation and refinement using relevant shots located by the user. Two variants of the system were developed, one that supports both text- and image-based searching and one that supports image only search. A user evaluation experiment compared the use of the two systems. Results show that while the system combining text- and image-based searching achieves greater retrieval effectiveness, users make more varied and extensive queries with the image only based searching version

RISE: A ROBUST IMAGE SEARCH ENGINE

This thesis advances RISE (Robust Image Search Engine), an image database application designed to build and search an image repository. rise is built on the foundation of a CBIR (Content Based Image Retrieval) system. The basic goal of this system is to compute content similarity of images based on their color signatures. The color signature of an image is computed by systematically dividing the image into a number of small blocks and computing the average color of each block using ideas from DCT (Discrete Cosine Transform) that forms the basis for JPEG (Joint Photographic Experts Group) compression format. The average color extracted from each block is used to construct a tree structure and finally, the tree structure is compared with similar structures already stored in the database. During the query process, an image is given to the system as a query image and the system returns a set of images that have similar content or color distribution as the given image. The query image is processed to create its signature which is then matched against similar signature of images already stored in the database. The content similarity is measured by computing normalized Euclidean distance between the query image and the images already stored in the database. RISE has a GUI (Graphic User Interface) front end and a Java servlet in the back end that searches the images stored in the database and returns the results to the web browser. RISE enhances the performance of image operations of the system using JAI (Java Advance Imaging) tools

TRECVID 2004 experiments in Dublin City University

In this paper, we describe our experiments for TRECVID 2004 for the Search task. In the interactive search task, we developed two versions of a video search/browse system based on the Físchlár Digital Video System: one with text- and image-based searching (System A); the other with only image (System B). These two systems produced eight interactive runs. In addition we submitted ten fully automatic supplemental runs and two manual runs. A.1, Submitted Runs: • DCUTREC13a_{1,3,5,7} for System A, four interactive runs based on text and image evidence. • DCUTREC13b_{2,4,6,8} for System B, also four interactive runs based on image evidence alone. • DCUTV2004_9, a manual run based on filtering faces from an underlying text search engine for certain queries. • DCUTV2004_10, a manual run based on manually generated queries processed automatically. • DCU_AUTOLM{1,2,3,4,5,6,7}, seven fully automatic runs based on language models operating over ASR text transcripts and visual features. • DCUauto_{01,02,03}, three fully automatic runs based on exploring the benefits of multiple sources of text evidence and automatic query expansion. A.2, In the interactive experiment it was confirmed that text and image based retrieval outperforms an image-only system. In the fully automatic runs, DCUauto_{01,02,03}, it was found that integrating ASR, CC and OCR text into the text ranking outperforms using ASR text alone. Furthermore, applying automatic query expansion to the initial results of ASR, CC, OCR text further increases performance (MAP), though not at high rank positions. For the language model-based fully automatic runs, DCU_AUTOLM{1,2,3,4,5,6,7}, we found that interpolated language models perform marginally better than other tested language models and that combining image and textual (ASR) evidence was found to marginally increase performance (MAP) over textual models alone. For our two manual runs we found that employing a face filter disimproved MAP when compared to employing textual evidence alone and that manually generated textual queries improved MAP over fully automatic runs, though the improvement was marginal. A.3, Our conclusions from our fully automatic text based runs suggest that integrating ASR, CC and OCR text into the retrieval mechanism boost retrieval performance over ASR alone. In addition, a text-only Language Modelling approach such as DCU_AUTOLM1 will outperform our best conventional text search system. From our interactive runs we conclude that textual evidence is an important lever for locating relevant content quickly, but that image evidence, if used by experienced users can aid retrieval performance. A.4, We learned that incorporating multiple text sources improves over ASR alone and that an LM approach which integrates shot text, neighbouring shots and entire video contents provides even better retrieval performance. These findings will influence how we integrate textual evidence into future Video IR systems. It was also found that a system based on image evidence alone can perform reasonably and given good query images can aid retrieval performance

Concept-based Multimedia Information Retrieval System using Ontology Search in Cultural Heritage

The richness of Cultural Heritage and Natural History is abundant. Many of the cultural heritage collection in Library, National Archive, and Museum in the form physical object or digital format in a different type of media (text, image, audio and video). One cultural heritage object can have the relationship with other objects in different media format and do not mention query term explicitly. Using the various media format causes problems in search. A monolithic search engine like Google, Bing, Google Image, Youtube, or Findsounds only retrieve one media format. Besides, the search result of the existing search engine is less relevant and incomplete in searching cultural heritage. Several multimedia information retrieval techniques used in building the relationship using ontology like ontology based search, content-based search with ontology and hybrid search with ontology. This paper proposes Concept-based Multimedia Information Retrieval System (MIRS) with ontology using Indonesia’s cultural heritage dataset to increase relevance and completeness of the system. Concept-based MIRS using manually built thesauri or by extracting latent word relationship and concept from the Ontology that provides definition and formal structure for describing the implicit and explicit concepts and its relationship in cultural heritage documentation. Ontology-based Semantic similarity measure is defined which measure the semantic relationship between document based on the likeness of their meaning. The search results indicate that the document being retrieved becomes highly relevant, more complete, enrich the keyword and in varying media formats when is compared to existing search engine results such google, bing, google image, youtube and findsounds in specific domain

CAMEL: Concept Annotated iMagE Libraries

Copyright 2001 Society of Photo-Optical Instrumentation Engineers. One print or electronic copy may be made for personal use only. Systematic electronic or print reproduction and distribution, duplication of any material in this paper for a fee or for commercial purposes, or modification of the content of the paper are prohibited. http://dx.doi.org/10.1117/12.410975The problem of content-based image searching has received considerable attention in the last few years. Thousands of images are now available on the internet, andmany important applications require searching of images in domains such as E-commerce, medical imaging, weather prediction, satellite imagery, and so on. Yet, content-based image querying is still largely unestablished as a mainstream field, nor is it widely used by search engines. We believe that two of the major hurdles for this poor acceptance are poor retrieval quality and usability. In this paper, we introduce the CAMEL system—an acronym for Concept Annotated iMagE Libraries—as an effort to address both of the above problems. The CAMEL system provides and easy-to-use, and yet powerful, text-only query interface, which allows users to search for images based on visual concepts, identified by specifying relevant keywords. Conceptually, CAMEL annotates images with the visual concepts that are relevant to them. In practice, CAMEL defines visual concepts by looking at sample images off-line and extracting their relevant visual features. Once defined, such visual concepts can be used to search for relevant images on the fly, using content-based search methods. The visual concepts are stored in a Concept Library and are represented by an associated set of wavelet features, which in our implementation were extracted by the WALRUS image querying system. Even though the CAMEL framework applies independently of the underlying query engine, for our prototype we have chosenWALRUS as a back-end, due to its ability to extract and query with image region features. CAMEL improves retrieval quality because it allows experts to build very accurate representations of visual concepts that can be used even by novice users. At the same time, CAMEL improves usability by supporting the familiar text-only interface currently used by most search engines on the web. Both improvements represent a departure from traditional approaches to improving image query systems—instead of focusing on query execution, we emphasize query specification by allowing simpler and yet more precise query specification

RISE: A Robust Image Search Engine

In this article we address the problem of organizing images for effective and efficient retrieval in large image database systems. Specifically, we describe the design and architecture of RISE, a Robust Image Search Engine. RISE is designed to build and search an image repository, with an interface that allows for the query and maintenance of the database over the Internet using any browser. RISE is built on the foundation of a CBIR (Content Based Image Retrieval) system and computes the similarity of images using their color signatures. The signature of an image in the database is computed by systematically dividing the image into a set of small blocks of pixels and then computing the average color of each block. This is based on the Discrete Cosine Transform (DCT) that forms the basis for popular JPEG image file format. The average color in each pixel block forms the characters of our image description. Organizing these pixel blocks into a tree structure allows us to create the words or tokens for the image. Thus the tokens represent the spatial distribution of the color in the image. The tokens for each image in the database are first computed and stored in a relational database as their signatures. Using a commercial relational database system (RDBMS) to store and query signatures of images improves the efficiency of the system. A query image provided by a user is first parsed to build the tokens which are then compared with the tokens for images in the database. During the query process, tokenization improves the efficiency by quantifying the degree of match between the query image and images in the database. The content similarity is measured by computing normalized Euclidean distance between corresponding tokens in query and stored images where correspondence is defined by the relative location of those tokens. The location of pixel blocks is maintained by using a quad tree structure that also improves performance by early pruning of search space. The distance is computed in perceptual color space, specifically L * a * b * and at different levels of detail. The perceptual color space allows RISE to ignore small variations in color while different levels of detail allow it to select a set of images for further exploration, or discard a set altogether. RISE only compares the precomputed color signature images that are stored in an RDBMS. It is very efficient since there is no need to extract complete information for every image. RISE is implemented using object-oriented design techniques and is deployed as a web browser-based search engine. RISE has a GUI (Graphical User Interface) front-end and a Java servlet in the back-end that searches the images stored in the database and returns the results to the web browser. RISE enhances the performance of image operations of the system by using JAI (Java Advance Imaging) tools, which obviates the dependence on a single image file format. In addition, the use of RDBMS and Java also facilitates the portability of 1 2 Goswami, Bhatia, Samal the system