Content-based image retrieval using colour and shape fused features

Multi-feature methods are able to contribute to a more effective method compared to single-feature methods since feature fusion methods will be able to close the gap that exists in the single-feature methods. This paper presents a feature fusion method, which focuses on extracting colour and shape features for content-based image retrieval (CBIR). The colour feature is extracted based on the proposed Multi-resolution Joint Auto Correlograms (MJAC), while the shape information is obtained through the proposed Extended Generalised Ridgelet-Fourier (EGRF). These features are fused together through a proposed integrated scheme. The feature fusion method has been tested on the SIMPLIcity image database, where several retrieval measurements are utilised to compare the effectiveness of the proposed method with few other comparable methods. The retrieval results show that the proposed Integrated Colour-shape (ICS) descriptor has successfully obtained the best overall retrieval performance in all the retrieval measurements as compared to the benchmark methods, which include precision (53.50%), precision at 11 standard recall levels (52.48%), and rank (17.40)

Current challenges in content based image retrieval by means of low-level feature combining

The aim of this paper is to discuss a fusion of the two most popular low-level image features - colour and shape - in the aspect of content-based image retrieval. By combining them we can achieve much higher accuracy in various areas, e.g. pattern recognition, object representation, image retrieval. To achieve such a goal two general strategies (sequential and parallel) for joining elementary queries were proposed. Usually they are employed to construct a processing structure, where each image is being decomposed into regions, based on shapes with some characteristic properties - colour and its distribution. In the paper we provide an analysis of this proposition as well as the exemplary results of application in the Content Based Image Retrieval problem. The original contribution of the presented work is related to different fusions of several shape and colour descriptors (standard and non-standard ones) and joining them into parallel or sequential structures giving considerable improvements in content-based image retrieval. The novelty is based on the fact that many existing methods (even complex ones) work in single domain (shape or colour), while the proposed approach joins features from different areas

Automatic Image Annotation for Semantic Image Retrieval

This paper addresses the challenge of automatic annotation of images for semantic image retrieval. In this research, we aim to identify visual features that are suitable for semantic annotation tasks. We propose an image classification system that combines MPEG-7 visual descriptors and support vector machines. The system is applied to annotate cityscape and landscape images. For this task, our analysis shows that the colour structure and edge histogram descriptors perform best, compared to a wide range of MPEG-7 visual descriptors. On a dataset of 7200 landscape and cityscape images representing real-life varied quality and resolution, the MPEG-7 colour structure descriptor and edge histogram descriptor achieve a classification rate of 82.8% and 84.6%, respectively. By combining these two features, we are able to achieve a classification rate of 89.7%. Our results demonstrate that combining salient features can significantly improve classification of images

Multi modal multi-semantic image retrieval

PhDThe rapid growth in the volume of visual information, e.g. image, and video can overwhelm users’ ability to find and access the specific visual information of interest to them. In recent years, ontology knowledge-based (KB) image information retrieval techniques have been adopted into in order to attempt to extract knowledge from these images, enhancing the retrieval performance. A KB framework is presented to promote semi-automatic annotation and semantic image retrieval using multimodal cues (visual features and text captions). In addition, a hierarchical structure for the KB allows metadata to be shared that supports multi-semantics (polysemy) for concepts. The framework builds up an effective knowledge base pertaining to a domain specific image collection, e.g. sports, and is able to disambiguate and assign high level semantics to ‘unannotated’ images. Local feature analysis of visual content, namely using Scale Invariant Feature Transform (SIFT) descriptors, have been deployed in the ‘Bag of Visual Words’ model (BVW) as an effective method to represent visual content information and to enhance its classification and retrieval. Local features are more useful than global features, e.g. colour, shape or texture, as they are invariant to image scale, orientation and camera angle. An innovative approach is proposed for the representation, annotation and retrieval of visual content using a hybrid technique based upon the use of an unstructured visual word and upon a (structured) hierarchical ontology KB model. The structural model facilitates the disambiguation of unstructured visual words and a more effective classification of visual content, compared to a vector space model, through exploiting local conceptual structures and their relationships. The key contributions of this framework in using local features for image representation include: first, a method to generate visual words using the semantic local adaptive clustering (SLAC) algorithm which takes term weight and spatial locations of keypoints into account. Consequently, the semantic information is preserved. Second a technique is used to detect the domain specific ‘non-informative visual words’ which are ineffective at representing the content of visual data and degrade its categorisation ability. Third, a method to combine an ontology model with xi a visual word model to resolve synonym (visual heterogeneity) and polysemy problems, is proposed. The experimental results show that this approach can discover semantically meaningful visual content descriptions and recognise specific events, e.g., sports events, depicted in images efficiently. Since discovering the semantics of an image is an extremely challenging problem, one promising approach to enhance visual content interpretation is to use any associated textual information that accompanies an image, as a cue to predict the meaning of an image, by transforming this textual information into a structured annotation for an image e.g. using XML, RDF, OWL or MPEG-7. Although, text and image are distinct types of information representation and modality, there are some strong, invariant, implicit, connections between images and any accompanying text information. Semantic analysis of image captions can be used by image retrieval systems to retrieve selected images more precisely. To do this, a Natural Language Processing (NLP) is exploited firstly in order to extract concepts from image captions. Next, an ontology-based knowledge model is deployed in order to resolve natural language ambiguities. To deal with the accompanying text information, two methods to extract knowledge from textual information have been proposed. First, metadata can be extracted automatically from text captions and restructured with respect to a semantic model. Second, the use of LSI in relation to a domain-specific ontology-based knowledge model enables the combined framework to tolerate ambiguities and variations (incompleteness) of metadata. The use of the ontology-based knowledge model allows the system to find indirectly relevant concepts in image captions and thus leverage these to represent the semantics of images at a higher level. Experimental results show that the proposed framework significantly enhances image retrieval and leads to narrowing of the semantic gap between lower level machinederived and higher level human-understandable conceptualisation

An improved image segmentation algorithm for salient object detection

Semantic object detection is one of the most important and challenging problems in image analysis. Segmentation is an optimal approach to detect salient objects, but often fails to generate meaningful regions due to over-segmentation. This paper presents an improved semantic segmentation approach which is based on JSEG algorithm and utilizes multiple region merging criteria. The experimental results demonstrate that the proposed algorithm is encouraging and effective in salient object detection

A Smart Content-Based Image Retrieval Approach Based on Texture Feature and Slantlet Transform

With the advancement of digital storing and capturing technologies in recent years, an image retrieval system has been widely known for Internet usage. Several image retrieval methods have been proposed to find similar images from a collection of digital images to a specified query image. Content-based image retrieval (CBIR) is a subfield of image retrieval techniques that extracts features and descriptions content such as color, texture, and shapes from a huge database of images. This paper proposes a two-tier image retrieval approach, a coarse matching phase, and a fine-matching phase. The first phase is used to extract spatial features, and the second phase extracts texture features based on the Slantlet transform. The findings of this study revealed that texture features are reliable and capable of producing excellent results and unsusceptible to low resolution and proved that the SLT-based texture feature is the perfect mate. The proposed method\u27s experimental results have outperformed the benchmark results with precision gaps of 28.0 % for the Caltech 101 dataset. The results demonstrate that the two-tier strategy performed well with the successive phase (fine-matching) and the preceding phase (coarse matching) working hand in hand harmoniously