Trademark image retrieval by local features

The challenge of abstract trademark image retrieval as a test of machine vision algorithms has attracted considerable research interest in the past decade. Current operational trademark retrieval systems involve manual annotation of the images (the current ‘gold standard’). Accordingly, current systems require a substantial amount of time and labour to access, and are therefore expensive to operate. This thesis focuses on the development of algorithms that mimic aspects of human visual perception in order to retrieve similar abstract trademark images automatically. A significant category of trademark images are typically highly stylised, comprising a collection of distinctive graphical elements that often include geometric shapes. Therefore, in order to compare the similarity of such images the principal aim of this research has been to develop a method for solving the partial matching and shape perception problem. There are few useful techniques for partial shape matching in the context of trademark retrieval, because those existing techniques tend not to support multicomponent retrieval. When this work was initiated most trademark image retrieval systems represented images by means of global features, which are not suited to solving the partial matching problem. Instead, the author has investigated the use of local image features as a means to finding similarities between trademark images that only partially match in terms of their subcomponents. During the course of this work, it has been established that the Harris and Chabat detectors could potentially perform sufficiently well to serve as the basis for local feature extraction in trademark image retrieval. Early findings in this investigation indicated that the well established SIFT (Scale Invariant Feature Transform) local features, based on the Harris detector, could potentially serve as an adequate underlying local representation for matching trademark images. There are few researchers who have used mechanisms based on human perception for trademark image retrieval, implying that the shape representations utilised in the past to solve this problem do not necessarily reflect the shapes contained in these image, as characterised by human perception. In response, a ii practical approach to trademark image retrieval by perceptual grouping has been developed based on defining meta-features that are calculated from the spatial configurations of SIFT local image features. This new technique measures certain visual properties of the appearance of images containing multiple graphical elements and supports perceptual grouping by exploiting the non-accidental properties of their configuration. Our validation experiments indicated that we were indeed able to capture and quantify the differences in the global arrangement of sub-components evident when comparing stylised images in terms of their visual appearance properties. Such visual appearance properties, measured using 17 of the proposed metafeatures, include relative sub-component proximity, similarity, rotation and symmetry. Similar work on meta-features, based on the above Gestalt proximity, similarity, and simplicity groupings of local features, had not been reported in the current computer vision literature at the time of undertaking this work. We decided to adopted relevance feedback to allow the visual appearance properties of relevant and non-relevant images returned in response to a query to be determined by example. Since limited training data is available when constructing a relevance classifier by means of user supplied relevance feedback, the intrinsically non-parametric machine learning algorithm ID3 (Iterative Dichotomiser 3) was selected to construct decision trees by means of dynamic rule induction. We believe that the above approach to capturing high-level visual concepts, encoded by means of meta-features specified by example through relevance feedback and decision tree classification, to support flexible trademark image retrieval and to be wholly novel. The retrieval performance the above system was compared with two other state-of-the-art image trademark retrieval systems: Artisan developed by Eakins (Eakins et al., 1998) and a system developed by Jiang (Jiang et al., 2006). Using relevance feedback, our system achieves higher average normalised precision than either of the systems developed by Eakins’ or Jiang. However, while our trademark image query and database set is based on an image dataset used by Eakins, we employed different numbers of images. It was not possible to access to the same query set and image database used in the evaluation of Jiang’s trademark iii image retrieval system evaluation. Despite these differences in evaluation methodology, our approach would appear to have the potential to improve retrieval effectiveness

Design and evaluation of a shape retrieval system

PhD ThesisWhile automated storage and retrieval systems for textual and numeric data are now commonplace, the development of analogous systems for pictorial data has lagged behind - not through the lack of need for such systems, but because their development involves a number of significant problems. The aim of this project is to investigate these problems by designing and evaluating an information retrieval system for a specific class of picture, 2-dimensional engineering drawings. This involves consideration of the retrieval capabilities needed by such· a system, what storage structures it would require, how the salient features of each drawing should be represented, how query and stored shapes should be matched, what features were of greatest importance in retrieval, and the interfaces necessary to formulate queries and display results. A form of hierarchical boundary representation has been devised for stored shapes, in which each boundary can be viewed as a series of levels of steadily increasing complexity. A set of rules for boundary and segment ordering ensures that as far as possible, each shape has a unique representation. For each level at which each boundary can be viewed, a set of invariant shape features characterizing that level is extracted and added to the shape representation stored in the database. Two classes of boundary feature have been defmed; global features, characteristic of the boundary as a whole, and local features, corresponding to individual fragments of the boundary. To complete the shape description, position features are also computed and stored, to specify the pattern of inner boundaries within the overall shape. Six different tYPes of shape retrieval have been distinguished, although the prototype system can offer only three of these - exact shape matching, partial shape matching and similarity matching. Complete or incomplete query shapes can be built up at a terminal, and subjected to a feature extraction process similar to that for stored drawings, yielding a query fIle that can be matched against the shape database. A variety of matching techniques is provided, including similarity estimation using global or local features, tests for the existence of specified local features in stored drawings, and cumulative angle vs distance matching between query and stored shape boundaries. Results can be displayed in text or graphical form. The retrieval performance of the system in similarity matching mode has been evaluated by comparing its rankings of shapes retrieved in response to test queries with those obtained by a group of human subjects faced with the same task. Results, expressed as normalized recall and precision, are encouraging, particularly for similarity estimation using either global or local boundary features. While the detailed results are of limited significance until confrrmed with larger test collections, they appear sufficiently promising to warrant the development of a more advanced prototype capable of handling 3-D geometric models. Some design aspects of the system would appear to be applicable to a wider range of pictorial information systems

Geometric Approaches for 3D Shape Denoising and Retrieval

A key issue in developing an accurate 3D shape recognition system is to design an efficient shape descriptor for which an index can be built, and similarity queries can be answered efficiently. While the overwhelming majority of prior work on 3D shape analysis has concentrated primarily on rigid shape retrieval, many real objects such as articulated motions of humans are nonrigid and hence can exhibit a variety of poses and deformations. Motivated by the recent surge of interest in content-based analysis of 3D objects in computeraided design and multimedia computing, we develop in this thesis a unified theoretical and computational framework for 3D shape denoising and retrieval by incorporating insights gained from algebraic graph theory and spectral geometry. We first present a regularized kernel diffusion for 3D shape denoising by solving partial differential equations in the weighted graph-theoretic framework. Then, we introduce a computationally fast approach for surface denoising using the vertexcentered finite volume method coupled with the mesh covariance fractional anisotropy. Additionally, we propose a spectral-geometric shape skeleton for 3D object recognition based on the second eigenfunction of the Laplace-Beltrami operator in a bid to capture the global and local geometry of 3D shapes. To further enhance the 3D shape retrieval accuracy, we introduce a graph matching approach by assigning geometric features to each endpoint of the shape skeleton. Extensive experiments are carried out on two 3D shape benchmarks to assess the performance of the proposed shape retrieval framework in comparison with state-of-the-art methods. The experimental results show that the proposed shape descriptor delivers best-in-class shape retrieval performance

Cross-Paced Representation Learning with Partial Curricula for Sketch-based Image Retrieval

In this paper we address the problem of learning robust cross-domain representations for sketch-based image retrieval (SBIR). While most SBIR approaches focus on extracting low- and mid-level descriptors for direct feature matching, recent works have shown the benefit of learning coupled feature representations to describe data from two related sources. However, cross-domain representation learning methods are typically cast into non-convex minimization problems that are difficult to optimize, leading to unsatisfactory performance. Inspired by self-paced learning, a learning methodology designed to overcome convergence issues related to local optima by exploiting the samples in a meaningful order (i.e. easy to hard), we introduce the cross-paced partial curriculum learning (CPPCL) framework. Compared with existing self-paced learning methods which only consider a single modality and cannot deal with prior knowledge, CPPCL is specifically designed to assess the learning pace by jointly handling data from dual sources and modality-specific prior information provided in the form of partial curricula. Additionally, thanks to the learned dictionaries, we demonstrate that the proposed CPPCL embeds robust coupled representations for SBIR. Our approach is extensively evaluated on four publicly available datasets (i.e. CUFS, Flickr15K, QueenMary SBIR and TU-Berlin Extension datasets), showing superior performance over competing SBIR methods