Shape-based Image Retrieval Using Spatio-temporal Relation Computations

[[abstract]]Content-based retrieval of multimedia information is one of the most difficult research topics in multimedia computing and information retrieval. In this paper, we present a visual system which allows content-based retrieval of still image. The recognition algorithms we used are based on spatio-temporal relations. Two approaches and algorithms were developed based on the similarity between polygons. The system is incorporated with a visual interface which allows the user to specify polygons as the shape specification of pictures. The preliminary experience shows that, within an image database containing about 300 bitmapped images, the system is able to retrieve correct information of a high satisfaction.[[notice]]補正完

Using contour information and segmentation for object registration, modeling and retrieval

This thesis considers different aspects of the utilization of contour information and syntactic and semantic image segmentation for object registration, modeling and retrieval in the context of content-based indexing and retrieval in large collections of images. Target applications include retrieval in collections of closed silhouettes, holistic w ord recognition in handwritten historical manuscripts and shape registration. Also, the thesis explores the feasibility of contour-based syntactic features for improving the correspondence of the output of bottom-up segmentation to semantic objects present in the scene and discusses the feasibility of different strategies for image analysis utilizing contour information, e.g. segmentation driven by visual features versus segmentation driven by shape models or semi-automatic in selected application scenarios. There are three contributions in this thesis. The first contribution considers structure analysis based on the shape and spatial configuration of image regions (socalled syntactic visual features) and their utilization for automatic image segmentation. The second contribution is the study of novel shape features, matching algorithms and similarity measures. Various applications of the proposed solutions are presented throughout the thesis providing the basis for the third contribution which is a discussion of the feasibility of different recognition strategies utilizing contour information. In each case, the performance and generality of the proposed approach has been analyzed based on extensive rigorous experimentation using as large as possible test collections

New Approach for Symbol Recognition Combining Shape Context of Interest Points with Sparse Representation

International audienceIn this paper, we propose a new approach for symbol description. Our method is built based on the combination of shape context of interest points descriptor and sparse representation. More specifically, we first learn a dictionary describing shape context of interest point descriptors. Then, based on information retrieval techniques, we build a vector model for each symbol based on its sparse representation in a visual vocabulary whose visual words are columns in the learned dictionary. The retrieval task is performed by ranking symbols based on similarity between vector models. The evaluation of our method, using benchmark datasets, demonstrates the validity of our approach and shows that it outperforms related state-of-theart methods

Trade mark similarity assessment support system

Trade marks are valuable intangible intellectual property (IP) assets with potentially high reputational value that can be protected. Similarity between trade marks may potentially lead to infringement. That similarity is normally assessed based on the visual, conceptual and phonetic aspects of the trade marks in question. Hence, this thesis addresses this issue by proposing a trade mark similarity assessment support system that uses the three main aspects of trade mark similarity as a mechanism to avoid future infringement. A conceptual model of the proposed trade mark similarity assessment support system is first proposed and developed based on the similarity assessment criteria outlined in a trade mark manual. The proposed model is the first contribution of this study, and it consists of visual, conceptual, phonetic and inference engine modules. The second contribution of this work is an algorithm that compares trade marks based on their visual similarity. The algorithm performs a similarity assessment using content-based image retrieval (CBIR) technology and an integrated visual descriptor derived using the low-level image feature, i.e. the shape feature. The performance of the algorithm is then assessed using information retrieval based measures. The obtained result demonstrates better retrieval performance in comparison to the state of the art algorithm. The conceptual aspect of trade mark similarity is then examined and analysed using a proposed algorithm that employs semantic technology in the conceptual module. This contribution enables the computation of the conceptual similarity between trade marks, with the utilisation of an external knowledge source in the form of a lexical ontology, together with natural language processing and set similarity theory. The proposed algorithm is evaluated using both information VI retrieval and human collective opinion measures. The retrieval result produced by the proposed algorithm outperforms the traditional string similarity comparison algorithm in both measures. The phonetic module examines the phonetic similarity of trade marks using another proposed algorithm that utilises phoneme analysis. This algorithm employs phonological features, which are extracted based on human speech articulation. In addition, the algorithm also provides a mechanism to compare the phonetic aspect of trade marks with typographic characters. The proposed algorithm is the fourth contribution of this study. It is evaluated using an information retrieval based measure. The result shows better retrieval performance in comparison to the traditional string similarity algorithm. The final contribution of this study is a methodology to aggregate the overall similarity score between trade marks. It is motivated by the understanding that trade mark similarity should be assessed holistically; that is, the visual, conceptual and phonetic aspects should be considered together. The proposed method is developed in the inference engine module; it utilises fuzzy logic for the inference process. A set of fuzzy rules, which consists of several membership functions, is also derived in this study based on the trade mark manual and a collection of trade mark disputed cases is analysed. The method is then evaluated using both information retrieval and human collective opinion. The proposed method improves the retrieval accuracy and the experiment also proves that the aggregated similarity score correlates well with the score produced from human collective opinion. The evaluations performed in the course of this study employ the following datasets: the MPEG-7 shape dataset, the MPEG-7 trade marks dataset, a collection of 1400 trade marks from real trade mark dispute cases, and a collection of 378,943 company names

Presenting Visual Information to the User: Combining Computer Vision and Interface Design

In this work, we suggest better ways to present visual information (image databases) for browsing and retrieval. Thumbnails obtained from an image set give a good overview of its contents. Instead of simply downsampling images to obtain thumbnails, we first find salient regions (saliency map) using local statistical features of the image. We crop and downsample the images based on these saliency maps, and obtain better thumbnails. The suggested methods of finding salient regions are faster than existing methods while giving comparable results. Secondly, we have developed a Content Based Image Retrieval (CBIR) system to provide empirical evidence (by user study) that similarity based grouped and hierarchical placement of images is better than random placement. Using an effective shape based similarity measure we conclude that visual search is very useful in image retrieval systems. We conducted a field test to check the robustness of the system in varying photography conditions

Investigating the Behavior of Compact Composite Descriptors in Early Fusion, Late Fusion and Distributed Image Retrieval

In Content-Based Image Retrieval (CBIR) systems, the visual content of the images is mapped into a new space named the feature space. The features that are chosen must be discriminative and sufficient for the description of the objects. The key to attaining a successful retrieval system is to choose the right features that represent the images as unique as possible. A feature is a set of characteristics of the image, such as color, texture, and shape. In addition, a feature can be enriched with information about the spatial distribution of the characteristic that it describes. Evaluation of the performance of low-level features is usually done on homogenous benchmarking databases with a limited number of images. In real-world image retrieval systems, databases have a much larger scale and may be heterogeneous. This paper investigates the behavior of Compact Composite Descriptors (CCDs) on heterogeneous databases of a larger scale. Early and late fusion techniques are tested and their performance in distributed image retrieval is calculated. This study demonstrates that, even if it is not possible to overcome the semantic gap in image retrieval by feature similarity, it is still possible to increase the retrieval effectiveness

Content-based indexing of low resolution documents

In any multimedia presentation, the trend for attendees taking pictures of slides that interest them during the presentation using capturing devices is gaining popularity. To enhance the image usefulness, the images captured could be linked to image or video database. The database can be used for the purpose of file archiving, teaching and learning, research and knowledge management, which concern image search. However, the above-mentioned devices include cameras or mobiles phones have low resolution resulted from poor lighting and noise. Content-Based Image Retrieval (CBIR) is considered among the most interesting and promising fields as far as image search is concerned. Image search is related with finding images that are similar for the known query image found in a given image database. This thesis concerns with the methods used for the purpose of identifying documents that are captured using image capturing devices. In addition, the thesis also concerns with a technique that can be used to retrieve images from an indexed image database. Both concerns above apply digital image processing technique. To build an indexed structure for fast and high quality content-based retrieval of an image, some existing representative signatures and the key indexes used have been revised. The retrieval performance is very much relying on how the indexing is done. The retrieval approaches that are currently in existence including making use of shape, colour and texture features. Putting into consideration these features relative to individual databases, the majority of retrievals approaches have poor results on low resolution documents, consuming a lot of time and in the some cases, for the given query image, irrelevant images are obtained. The proposed identification and indexing method in the thesis uses a Visual Signature (VS). VS consists of the captures slides textual layout’s graphical information, shape’s moment and spatial distribution of colour. This approach, which is signature-based are considered for fast and efficient matching to fulfil the needs of real-time applications. The approach also has the capability to overcome the problem low resolution document such as noisy image, the environment’s varying lighting conditions and complex backgrounds. We present hierarchy indexing techniques, whose foundation are tree and clustering. K-means clustering are used for visual features like colour since their spatial distribution give a good image’s global information. Tree indexing for extracted layout and shape features are structured hierarchically and Euclidean distance is used to get similarity image for CBIR. The assessment of the proposed indexing scheme is conducted based on recall and precision, a standard CBIR retrieval performance evaluation. We develop CBIR system and conduct various retrieval experiments with the fundamental aim of comparing the accuracy during image retrieval. A new algorithm that can be used with integrated visual signatures, especially in late fusion query was introduced. The algorithm has the capability of reducing any shortcoming associated with normalisation in initial fusion technique. Slides from conferences, lectures and meetings presentation are used for comparing the proposed technique’s performances with that of the existing approaches with the help of real data. This finding of the thesis presents exciting possibilities as the CBIR systems is able to produce high quality result even for a query, which uses low resolution documents. In the future, the utilization of multimodal signatures, relevance feedback and artificial intelligence technique are recommended to be used in CBIR system to further enhance the performance

Image-Based Query by Example Using MPEG-7 Visual Descriptors

This project presents the design and implementation of a Content-Based Image Retrieval (CBIR) system where queries are formulated by visual examples through a graphical interface. Visual descriptors and similarity measures implemented in this work followed mainly those defined in the MPEG-7 standard although, when necessary, extensions are proposed. Despite the fact that this is an image-based system, all the proposed descriptors have been implemented for both image and region queries, allowing the future system upgrade to support region-based queries. This way, even a contour shape descriptor has been developed, which has no sense for the whole image. The system has been assessed on different benchmark databases; namely, MPEG-7 Common Color Dataset, and Corel Dataset. The evaluation has been performed for isolated descriptors as well as for combinations of them. The strategy studied in this work to gather the information obtained from the whole set of computed descriptors is weighting the rank list for each isolated descriptor

Shape-based invariant features extraction for object recognition

International audienceThe emergence of new technologies enables generating large quantity of digital information including images; this leads to an increasing number of generated digital images. Therefore it appears a necessity for automatic systems for image retrieval. These systems consist of techniques used for query specification and re-trieval of images from an image collection. The most frequent and the most com-mon means for image retrieval is the indexing using textual keywords. But for some special application domains and face to the huge quantity of images, key-words are no more sufficient or unpractical. Moreover, images are rich in content; so in order to overcome these mentioned difficulties, some approaches are pro-posed based on visual features derived directly from the content of the image: these are the content-based image retrieval (CBIR) approaches. They allow users to search the desired image by specifying image queries: a query can be an exam-ple, a sketch or visual features (e.g., colour, texture and shape). Once the features have been defined and extracted, the retrieval becomes a task of measuring simi-larity between image features. An important property of these features is to be in-variant under various deformations that the observed image could undergo. In this chapter, we will present a number of existing methods for CBIR applica-tions. We will also describe some measures that are usually used for similarity measurement. At the end, and as an application example, we present a specific ap-proach, that we are developing, to illustrate the topic by providing experimental results