The main concern dealing with content-based image retrieval (CBIR) is to bridge the semantic gap. The high level query posed by the user and low level features extracted by the machine illustrates the problem of semantic gap. To solve the problem of semantic gap, this paper presents a hybrid technique using an emergence index and fuzzy logic for efficient retrieval of images based on the colour feature. Emergence index (EI) is proposed to understand the hidden meaning of the image. Fuzzy similarity measure is developed to calculate the similarity between the target image and the images in the database. The images were ranked based on their similarity along with the fuzzy similarity distance measure. The preliminary experiments conducted on small set of images and promising results were obtained

Deb, Sagarmay

Kulkarni, Siddhivinayak

Federation ResearchOnline

Human Perception based Image Retrieval using Emergence Index and Fuzzy Similarity Measure  Sagarmay Deb1 and Siddhivinayak Kulkarni2        1School of Commerce and Management Southern Cross University Military Road East Lismore NSW 2480, Australia Sagarmay.deb@scu.edu.au  2School of Information Technology and Mathematical Science University of Ballarat Mt Helen Campus University Drive PO Box 663, Ballarat Victoria, 3353, Australia S.Kulkarni@ballarat.edu.au  ABSTRACT  The main concern dealing with Content-based Image Retrieval (CBIR) is to bridge the semantic gap. The high level query posed by the user and low level features extracted by the machine illustrates the problem of semantic gap. To solve the problem of semantic gap, this paper presents a hybrid technique using an emergence index and fuzzy logic for efficient retrieval of images based on the colour feature. Emergence Index (EI) is proposed to understand the hidden meaning of the image. Fuzzy Similarity Measure is developed to calculate the similarity between the target image and the images in the database. The images were ranked based on their similarity along with the fuzzy similarity distance measure. The preliminary experiments conducted on small set of images and promising results were obtained.  1.  INTRODUCTION Content-based image retrieval (CBIR) is a bottleneck of multimedia database access system. In the last decade, many image retrieval techniques have been developed based on colour, shape, texture and spatial locations where retrievals were done in an automated way with least user interference. Fuzzy logic based technique has been applied for solving the problems in CBIR. Fuzzy logic is used for measuring the similarity between the two images as fuzzy hamming distance, region groupings, colour histogram as feature extraction techniques, and posing the queries etc. Fuzzy colour histogram is proposed by considering the colour similarity of each pixel’s colour associated to all the histogram bins through fuzzy-set membership function [1].  Fuzzy logic based colour histogram and their corresponding fuzzy distances are proposed for the retrieval of colour images for image database [2]. Fuzzy logic is applied to the traditional colour histogram for solving the problem of inaccuracy in typical colour feature. The similarity is defined through a balanced combination between global and regional similarity measures incorporating all the features [3].  The Fuzzy Hamming Distance (FHD) is an extension of Hamming Distance for real valued vectors. Because the feature space of each image is a real-valued, the fuzzy Hamming Distance can be successfully used as image similarity measure. FHD is applied for colour histograms of the two images. The authors claim that FHD not only considers the number of different colours but also the magnitude of this difference [4] [5]. In [6] supports concept-based image retrieval as well as the inexact match with a fuzzy triple matching performed when evaluating queries.  In [7] an image is represented by a set of segmented regions each of which is characterised by a fuzzy feature reflecting colour, texture and shape properties. A CBIR system which automatically clusters images using features of those images which are fuzzy in nature. The resultant clusters must be described by linguistic variables which are more meaningful to humans than traditional approaches, also fuzzy features result in better clustering than traditional approach. Fuzzy image labelling method that assigns multiple semantic labels together with confidence measures to each region in an image [8].   1-4244-1502-0/07/$25.00 © 2007 IEEE ISSNIP 2007359  Figure 1: (a) Original shape, (b) and (c) emergent shapes derived from (a) [21]  Thorough and meaningful image segmentation is essential for accurate image retrieval, is still a problem. Also finding the semantic meanings out of an image from low-level features like colour, shape, texture and spatial locations and connect it to high-level features like chair, table, car, house etc. is another unresolved problem. This is because of the inherent problem of computer perception, as computer cannot replicate a human mind. Image retrieval can be achieved based on low-level visual features like colour [9], texture [10], shape [11] or high level semantics [12] or both [13].   As quite a few commercial and non-commercial models are developed, none of them made attempts to study the hidden or implicit meanings of the images. We achieve more accurate and different search outcomes when hidden meanings are also taken into account for both precision and recall. For example, we can consider a square with one diagonal. This is the explicit or outer meaning of the image. But when we consider implicit or hidden meanings, we get two triangles in it. This is what we call emergence. Calculation of similarity between the target image and the images in database is another important issue for retrieving the images from the database. Various distance measures such as city block distance [14], Euclidean distance [15], histogram intersection distance [16], Mahalanobis distance have been used in image retrieval. We propose the calculation of similarity based on fuzzy logic. Fuzzy logic has been already developed for natural language queries for CBIR [17]. The performances of some of these measures have been evaluated on image databases and results are discussed in this paper.   The paper provides definitions and application of emergence index in image query processing. The images are segmented based on major object in an image. Similarity distance is calculated using fuzzy similarity measure for various colours. Section 2 details the concept of emergence index, Section 3 describes fuzzy logic based similarity distance measure, Section 4 explains the experimental results and Section 5 deals with analysis and comparison and conclusion and future research are described in Section 6.  2. CONCEPT OF EMERGENCE INDEX  Shape emergence is defined as emergence of single or multiple shapes from a particular shape.  Figure 1 shows examples of shape emergence. The symbolic representation of shapes could be defined using infinite maximal lines which are straight lines with no limit. I = {N; constraints}, where N is the number of infinite maximal lines, which effectively define an image I and constraints are limitations, which define behaviours or properties that come out from the infinite maximal lines [18]. In order to calculate the emergence index, we developed the following equation for the emergence index, EI  = f(D,F,V,C,E), where D for domain where the image belongs, F for features, V for variables which defines the features’ constraints under which the features are defined, C for constraints and E for emergence characteristics of images. Any image, static or in motion, could be expressed semantically in terms of the above-mentioned five parameters [19]. An application of emergence index in practical problems is in geographic location. We consider the image of a map of a township where there is a park, a lake, roads and residential area. The roads surrounding the park and the lake form the shape of a bowl. This is the example of an embedded shape emergence where emergence is a set of the whole image. We know images are generated in huge numbers. So if we want to locate this particular image from a table containing large volume of data, then we can have an input of a bowl. Then this input of the bowl will find a match with the emergent shape of the bowl in this image of the map and select it for display. This is the advantage of using emergence index [20]. Figure 2 shows the original images and along with segmented images considering the emergence shape.   Figure 2: Segmentation of Images based on Emergent Shapes Identification 360 3. FUZZY LOGIC BASED SIMILARITY MEASURE  All the images in database are segmented based on the colour regions using emergence index technique. The colour feature is extracted from each image using colour histogram techniques and stored in a database. Np represents the total number of pixels in an image. For each colour value (red, green, blue etc.) the number of the pixels that belong to the value are recorded and denoted by Nf where f ∈ Fu. The feature representation set Fu is the set of colours useful for retrieval. Due to the size of the image collection, it is necessary to use a database for managing the image content information. The chief priority is to store this data in such way as to facilitate the fastest possible retrieval time in order to make rapid browsing feasible. The numbers of the pixels are calculated for each colour. The red colour feature component FR is calculated for each colour in an image by the following formula: FR = NR/NP.  NR represents the number of red pixels in an image. Similarly the feature component is calculated for all the colours. For image retrieval based on similarity measure, the fuzzy distance between the two images should be calculated. This distance is used to retrieve and index images based on the similarity. Let us consider the extracted features for each image are in the form of: I1 = [I1C1, I1C2... I1Ci …I1Cn], where I1 is the first image in database and C indicates the colour extracted from each image. Similarly, these colour features are extracted from each image and stored in feature database.  The images are represented by an n-dimensional colour feature vectors. The similarity is calculated for each component of the feature vectors. Hence for each pair of images, we have n similarity measures. The global similarity is given by: FuzzyAND = Min [(Q1, T1), (Q2, T2)…(Qn, Tn)]  FuzzyOR = Max [(Q1, T1), (Q2, T2)…(Qn, Tn)], where Q1 represents the first colour vector for the query image, similarly T1 represents first colour vector for the target image.  The global similarity measure is calculated by: ∑∑= =−=kjni FuzzyORFuzzyANDFuzzyORDF0 0..   An F.D. stand for Fuzzy Distance, n is the colour feature vectors and k are the number of images in database. The similarity measure for each image is calculated in accordance with the query image and indexed based upon selection sort algorithm.  4. EXPERIMENTAL RESULTS  In the preliminary experiments, the prototype was kept simple. Only 250 images were downloaded from World Wide Web (WWW) in different categories like flowers, animals, natural scene, people etc.   Figure 3: Top five query results for three different images using emergence index and fuzzy similarity distance  Colour Feature extraction for all the images have been performed offline. To check the performance of the developed fuzzy based similarity measure, number of experiments was conducted. Some of the experimental results are discussed in this paper. The Figure 3 shows the top five results returned for a query. The image in the first row represents a query image and below is the retrieved images with emergence index along with their fuzzy similarity distance. Similar image has the least fuzzy similarity distance and it increases from top to bottom.   5. ANALYSIS AND COMPARISON  The performance of the proposed emergence index and fuzzy similarity distance was compared with other developed similarity measures such as Euclidean distance, Chi-square distance, histogram intersection and city block distance. The results were compared using precision and recall method  361    Figure 4: Comparison of retrieval performance for various similarity measures   using the consistent criteria and image database for each similarity measure. Recall signifies the relevant images in the database that are retrieved in response to a query. Precision is the proportion of the retrieved images that are relevant to the query.  Figure 4 shows the retrieval performance based on other similarity measures. After comparing and analysing all the similarity measure algorithms, it was observed that each algorithm except histogram intersection performed relatively well.   6. CONCLUSION AND FUTURE RESEARCH  Accessing the multimedia databases using emergence index and fuzzy similarity distance is presented in this paper. The emergence gives rise to altogether different meaning and could be used in accessing large databases in more efficient way. In the second stage, similarity measure has formed very important step for retrieving the images based on their features. There are many similarity function exists such as Euclidean distance, histogram intersection etc. Along with these existing measures, Fuzzy logic based image similarity distance has been proposed and used for retrieving the images based on their colour feature. The proposed similarity learning is compared with existing similarity measures. Combining the emergence index and fuzzy logic based similarity measure has improved the overall retrieval results in CBIR. The future research will lead to the extension of fusing emergence index and fuzzy logic for other features of the image.   REFERENCES  [1] J. Han and K. Ma, Fuzzy Colour Histogram and Its Use in Colour Image Retrieval, IEEE Transactions on Image Processing, Vol. 11, No. 8, August 2002, pp. 944-952. [2]  C. Vertan and N. Boujemaa, Using Fuzzy Histograms and Distances for Colour Image retrieval, Challenge of Image Retrieval, Brighton, page numbers not available, 2000. [3] R. Zhang and Z. Zhang, A Clustering based Approach to Efficient Image Retrieval, 14th IEEE Conference on Tools with Artificial Intelligence, pp. 339-346, 2002. [4] M. Ionescu, and A. Ralescu, Fuzzy Hamming Distance in a Content based Image Retrieval Systems, Proceedings of FUZZ-IEEE 2004, Budapest, Page numbers not available, 2004. [5] M. Ionescu, and A. Ralescu, Image Clustering for a Fuzzy Hamming Distance based CBIR Systems, Proceedings of the Sixteen Midwest Artificial Intelligence and Cognitive Science Conference, MAICS-2005, pp. 102-108, 2005. [6]  J. Yang, An Image Retrieval Model based on Fuzzy Triples, Journal on Fuzzy Sets and Systems, Vol. 121, No. 3, pp. 459 – 470, 2001. [7]  Y. Chen and J. Wang, A Region based Fuzzy Feature Matching Approach to Content based Image Retrieval, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, No. 9, pp. 1252-1267, 2002. 362[8]  M. Paterno, F. Lim, and W. Leow, Fuzzy Semantic Labelling for Image Retrieval, International Conference on Multimedia and Expo, Vol. 2, pp. 767-770, 2004. [9]  G. Pass, R. Zabih, J. Miller, Comparing Images using Colour Coherence Vector, ACM      Multimedia,   Boston, MA, pp. 65-73, 1996. [10] B. Manjunath, and W. Ma, Texture Features for Browsing and Retrieval of Image Data”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 18, No. 8, August 1996. [11] M. Safar, C. Shahabi, C., and X. Sun, Image Retrieval by Shape: A Comparative Study, Proceedings of IEEE International Conference on Multimedia and Exposition (ICME), USA, 2000. [12] D. Forsyth, Finding Pictures of Objects in Large Collections of Images”, Report of the NSF/ARPA Workshop on 3D Object Representation for Computer Vision, pp. 335, 1996. [13] R. Zhao, and W. Grosky, Bridging the Semantic Gap in Image Retrieval, Distributed Multimedia Databases: Techniques and Applications, Idea Group Publishing, Hershey, PA 17033-1240, USA,  pp. 14-36, 2002. [14] M. Stricker and M. Orengo, Similarity of Colour Images. In the Proceedings of SPIE: Storage and Retrieval for Image and Video Databases, Vol. 2420, pp. 381-392, 1995. [15] H. Voorhees and T. Poggio, Computing Texture Boundries from Images, Nature, pp. 333-367, 1988. [16] J. Smith, Integrated Spatial and Feature Image System: Retrieval, Analysis and Compression, PhD Thesis, Columbia University, 1997. [17] B. Verma and S. Kulkarni, Fuzzy Logic Based Interpretation and Fusion of Colour Queries, Journal of Fuzzy Sets and Systems, Volume 147, No. 1, pp. 99-118, 2004. [18] J. Gero, Shape Emergence and Symbolic Reasoning using Maximal Lines Unpublished Notes, Design Computing Unit, Department of Architectural and Design Science, University of Sydney, Sydney, 1992. [19] S. Deb, and Y. Zhang, Concepts of Emergence Index in Image Databases, Distributed Multimedia Databases: Techniques and Applications, Idea Group Publishing, Hershey, PA17033-1240, USA, pp. 73-88, 2001. [20] S. Deb, and Y. Zhang, Retrieval of Geographic Location with Emergence Index in Multimedia, Second International Workshop on Intelligent Multimedia Computing and Networking, Durham, North Carolina, USA,  pp. 976-979, March 8-13, 2002.  [21] J. Gero, Visual Emergence in Design Collaboration, Key Center of Design Computing, International Journal of CADCAM and Computer Graphics, Vol. 8, No. 3, 349-357, 1993.    363

Human perception based image retrieval using emergence index and fuzzy similarity measure

https://researchonline.federation.edu.au/vital/access/services/Download/vital:3590/SOURCE1

Human perception based image retrieval using emergence index and fuzzy similarity measure

Abstract

Similar works

Full text

Available Versions

Federation ResearchOnline