[[alternative]]An Efficient Shape-Representation Method for Content Based Image Retrieval

計畫編號：NSC93-2213-E032-006研究期間：200408~200507研究經費：593,000[[abstract]]以內容為基礎之影像查詢(CBIR)的研究可分為特徵選取、物件表示以及結果比 對。假如以物件的外形輪廓表示物件的特徵，那麼邊緣點偵測就是抽取這類特徵的第 一個步驟。當找完了邊緣點後，一個好的物件表示法必須能夠克服物件在影像中的移 位、旋轉、以及放大或縮小等問題。甚至對於物件外形在一定程度內的損毀下也必須 能夠有好的比對結果。這些問題都是在利用物件外形特徵來表示物件時以及比對過程 中相當重要的議題。 因此本計畫將提出一個有效率及強健的以物件外形特徵為基礎的影像查詢系統。 我們使用一快速的邊緣點偵測演算法來偵測出影像中所有可能的邊緣點，並提出一新 的物件表示法—爬山式序列表示法(Mountain Climbing Sequence (MCS))。此表示法 對於前面所提之影像中的移位、旋轉、以及放大或縮小等問題都可以達到不變的效果。 另外，由於邊緣點的偵測就目前的研究經驗上並無法保証能夠找一物件的完整外形， 因此我們也將嘗試在現有的外形特徵表示法下，克服物件外形不完整抽取的情況，甚 至於在物件少部份被遮蔽的狀況也能得到好的比對結果。[[sponsorship]]行政院國家科學委員

Current challenges in content based image retrieval by means of low-level feature combining

The aim of this paper is to discuss a fusion of the two most popular low-level image features - colour and shape - in the aspect of content-based image retrieval. By combining them we can achieve much higher accuracy in various areas, e.g. pattern recognition, object representation, image retrieval. To achieve such a goal two general strategies (sequential and parallel) for joining elementary queries were proposed. Usually they are employed to construct a processing structure, where each image is being decomposed into regions, based on shapes with some characteristic properties - colour and its distribution. In the paper we provide an analysis of this proposition as well as the exemplary results of application in the Content Based Image Retrieval problem. The original contribution of the presented work is related to different fusions of several shape and colour descriptors (standard and non-standard ones) and joining them into parallel or sequential structures giving considerable improvements in content-based image retrieval. The novelty is based on the fact that many existing methods (even complex ones) work in single domain (shape or colour), while the proposed approach joins features from different areas

Neural network-based shape retrieval using moment invariants and Zernike moments.

Shape is one of the fundamental image features for use in Content-Based Image Retrieval (CBIR). Compared with other visual features such as color and texture, it is extremely powerful and provides capability for object recognition and similarity-based image retrieval. In this thesis, we propose a Neural Network-Based Shape Retrieval System using Moment Invariants and Zernike Moments. Moment Invariants and Zernike Moments are two region-based shape representation schemes and are derived from the shape in an image and serve as image features. k means clustering is used to group similar images in an image collection into k clusters whereas Neural Network is used to facilitate retrieval against a given query image. Neural Network is trained by the clustering result on all of the images in the collection using back-propagation algorithm. In this scheme, Neural Network serves as a classifier such that moments are inputs to the Neural Network and the output is one of the k classes that have the largest similarities to the query image. Paper copy at Leddy Library: Theses & Major Papers - Basement, West Bldg. / Call Number: Thesis2005 .C444. Source: Masters Abstracts International, Volume: 44-03, page: 1396. Thesis (M.Sc.)--University of Windsor (Canada), 2005

A review on content-based image retrieval representation and description for fish

There is an increasing interest in the description and representation of fish species images. For that purpose, Content-based Image Retrieval (CBIR) is applied. Due to the uncontrolled deep sea underwater environment, it is very hard to accurately estimate the similarities between the fishes and retrieves them according to its species due to ineffective visual features extraction for fish image representation. In this paper, CBIR for representation and description of fish is reviewed. Shape is one of the most important features to describe fish. This paper considers the combination of global and local shape features. Existing combination is carefully studied and the importances of global and local shape features are presented. The focus of possible future works is also suggested

Multi modal multi-semantic image retrieval

PhDThe rapid growth in the volume of visual information, e.g. image, and video can overwhelm users’ ability to find and access the specific visual information of interest to them. In recent years, ontology knowledge-based (KB) image information retrieval techniques have been adopted into in order to attempt to extract knowledge from these images, enhancing the retrieval performance. A KB framework is presented to promote semi-automatic annotation and semantic image retrieval using multimodal cues (visual features and text captions). In addition, a hierarchical structure for the KB allows metadata to be shared that supports multi-semantics (polysemy) for concepts. The framework builds up an effective knowledge base pertaining to a domain specific image collection, e.g. sports, and is able to disambiguate and assign high level semantics to ‘unannotated’ images. Local feature analysis of visual content, namely using Scale Invariant Feature Transform (SIFT) descriptors, have been deployed in the ‘Bag of Visual Words’ model (BVW) as an effective method to represent visual content information and to enhance its classification and retrieval. Local features are more useful than global features, e.g. colour, shape or texture, as they are invariant to image scale, orientation and camera angle. An innovative approach is proposed for the representation, annotation and retrieval of visual content using a hybrid technique based upon the use of an unstructured visual word and upon a (structured) hierarchical ontology KB model. The structural model facilitates the disambiguation of unstructured visual words and a more effective classification of visual content, compared to a vector space model, through exploiting local conceptual structures and their relationships. The key contributions of this framework in using local features for image representation include: first, a method to generate visual words using the semantic local adaptive clustering (SLAC) algorithm which takes term weight and spatial locations of keypoints into account. Consequently, the semantic information is preserved. Second a technique is used to detect the domain specific ‘non-informative visual words’ which are ineffective at representing the content of visual data and degrade its categorisation ability. Third, a method to combine an ontology model with xi a visual word model to resolve synonym (visual heterogeneity) and polysemy problems, is proposed. The experimental results show that this approach can discover semantically meaningful visual content descriptions and recognise specific events, e.g., sports events, depicted in images efficiently. Since discovering the semantics of an image is an extremely challenging problem, one promising approach to enhance visual content interpretation is to use any associated textual information that accompanies an image, as a cue to predict the meaning of an image, by transforming this textual information into a structured annotation for an image e.g. using XML, RDF, OWL or MPEG-7. Although, text and image are distinct types of information representation and modality, there are some strong, invariant, implicit, connections between images and any accompanying text information. Semantic analysis of image captions can be used by image retrieval systems to retrieve selected images more precisely. To do this, a Natural Language Processing (NLP) is exploited firstly in order to extract concepts from image captions. Next, an ontology-based knowledge model is deployed in order to resolve natural language ambiguities. To deal with the accompanying text information, two methods to extract knowledge from textual information have been proposed. First, metadata can be extracted automatically from text captions and restructured with respect to a semantic model. Second, the use of LSI in relation to a domain-specific ontology-based knowledge model enables the combined framework to tolerate ambiguities and variations (incompleteness) of metadata. The use of the ontology-based knowledge model allows the system to find indirectly relevant concepts in image captions and thus leverage these to represent the semantics of images at a higher level. Experimental results show that the proposed framework significantly enhances image retrieval and leads to narrowing of the semantic gap between lower level machinederived and higher level human-understandable conceptualisation

Color image quality measures and retrieval

The focus of this dissertation is mainly on color image, especially on the images with lossy compression. Issues related to color quantization, color correction, color image retrieval and color image quality evaluation are addressed. A no-reference color image quality index is proposed. A novel color correction method applied to low bit-rate JPEG image is developed. A novel method for content-based image retrieval based upon combined feature vectors of shape, texture, and color similarities has been suggested. In addition, an image specific color reduction method has been introduced, which allows a 24-bit JPEG image to be shown in the 8-bit color monitor with 256-color display. The reduction in download and decode time mainly comes from the smart encoder incorporating with the proposed color reduction method after color space conversion stage. To summarize, the methods that have been developed can be divided into two categories: one is visual representation, and the other is image quality measure. Three algorithms are designed for visual representation: (1) An image-based visual representation for color correction on low bit-rate JPEG images. Previous studies on color correction are mainly on color image calibration among devices. Little attention was paid to the compressed image whose color distortion is evident in low bit-rate JPEG images. In this dissertation, a lookup table algorithm is designed based on the loss of PSNR in different compression ratio. (2) A feature-based representation for content-based image retrieval. It is a concatenated vector of color, shape, and texture features from region of interest (ROI). (3) An image-specific 256 colors (8 bits) reproduction for color reduction from 16 millions colors (24 bits). By inserting the proposed color reduction method into a JPEG encoder, the image size could be further reduced and the transmission time is also reduced. This smart encoder enables its decoder using less time in decoding. Three algorithms are designed for image quality measure (IQM): (1) A referenced IQM based upon image representation in very low-dimension. Previous studies on IQMs are based on high-dimensional domain including spatial and frequency domains. In this dissertation, a low-dimensional domain IQM based on random projection is designed, with preservation of the IQM accuracy in high-dimensional domain. (2) A no-reference image blurring metric. Based on the edge gradient, the degree of image blur can be measured. (3) A no-reference color IQM based upon colorfulness, contrast and sharpness