The Optimisation of Elementary and Integrative Content-Based Image Retrieval Techniques

Image retrieval plays a major role in many image processing applications. However, a number of factors (e.g. rotation, non-uniform illumination, noise and lack of spatial information) can disrupt the outputs of image retrieval systems such that they cannot produce the desired results. In recent years, many researchers have introduced different approaches to overcome this problem. Colour-based CBIR (content-based image retrieval) and shape-based CBIR were the most commonly used techniques for obtaining image signatures. Although the colour histogram and shape descriptor have produced satisfactory results for certain applications, they still suffer many theoretical and practical problems. A prominent one among them is the well-known “curse of dimensionality “. In this research, a new Fuzzy Fusion-based Colour and Shape Signature (FFCSS) approach for integrating colour-only and shape-only features has been investigated to produce an effective image feature vector for database retrieval. The proposed technique is based on an optimised fuzzy colour scheme and robust shape descriptors. Experimental tests were carried out to check the behaviour of the FFCSS-based system, including sensitivity and robustness of the proposed signature of the sampled images, especially under varied conditions of, rotation, scaling, noise and light intensity. To further improve retrieval efficiency of the devised signature model, the target image repositories were clustered into several groups using the k-means clustering algorithm at system runtime, where the search begins at the centres of each cluster. The FFCSS-based approach has proven superior to other benchmarked classic CBIR methods, hence this research makes a substantial contribution towards corresponding theoretical and practical fronts

Vision-Based Object Recognition and 3-D Pose Estimation Using Conic Features

This thesis deals with monocular vision-based object recognition and 3-D pose estimation based on conic features. Conic features including circles and ellipses are frequently observed in many man-made objects in real word as well as have the merit of robustness potentially in feature extraction in vision-based applications. Although the 3-D pose estimation problem of conic features in 3-D space has been studied well since 1990, the previous work has not provided a unique solution completely for full 3-D pose parameters (i.e., 3-orientations and 3-positions) due to complexity from high nonlinearity of a general conic. This thesis, therefore, renews conic features in a new perspective on geometric invariants in both 3-D space and 2-D projective space, incorporating other geometric features with conics. First, as the most essential step in dealing with conics, this thesis shows that the pose parameters of a circular feature in 3-D space can be derived analytically from incorporating a coplanar point. A procedure of pose parameter recovery is described in detail, and its performance is evaluated and discussed in view of pose estimation errors and sensitivity. Second, it is also revealed that the pose of an elliptic feature can be resolved when two coplanar points are incorporated on the basis of the polarity of two points for a conic in 2-D projective space. This thesis proposes a series of algorithms to determine the 3-D pose parameters uniquely, and evaluates the proposed method through a measure of estimation performance and sensitivity depending on point locations. Third, a pair of two conics is dealt with, which is regarded as an extension of the idea of the incorporation scheme to another conic feature from point features. Under the polarity concept, this thesis proves that the problem involving a pair of two conics can be formulated with the problem of one ellipse with two points so that its solution is derived in the same form as in the ellipse case. In order to treat two or more conic objects as well as to deal with an object recognition problem, the rest of thesis concentrates on the theoretical foundation of multiple object recognition. First, some effective modeling approaches are described. A general object model is specially designed to model multiple objects for object recognition and pose recovery in view of spatial geometry. In particular, this thesis defines a pairwise conic model that can describes the geometrical relation between two conics invariantly in 2-D projective space, which consists of a pairwise conic (PC), a pairwise conic invariant (PCI), and a pairwise conic pole (PCP). Based on the two kinds of models, an object learning and recognition system is proposed as a general framework for multiple object recognition. Considering simplicity and flexibility in object learning stage, this thesis introduces a semi-automatic learning scheme to construct the multiple object model from a model image at once. To utilize geometric relations among multiple objects effectively in object recognition, this thesis specifies some feature functions based on the pairwise conic model, and then describes an object recognition method in a fashion of linear-chain conditional random field (CRF). In particular, as a post refinement step of the recognition, a geometric alignment procedure is also proposed in algorithmic details to improve recognition performance against noisy conditions. Last, the multiple object recognition method is evaluated intensively through two practical applications that deal with a place recognition and an elevator button recognition problem for service robots. A series of experiment results supports the effectiveness of the proposed method, maintaining reliable performance against noisy conditions in the presence of perspective distortion and partial object occlusions.Contents Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix 1 Introduction 1 1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 Research objective and expected contribution . . . . . . . . . . . . . . . . . . 6 1.4 Organization of thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2 3-D Pose Estimation of a Circular Feature 10 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.1.2 Problem formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.1.3 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.1.4 Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.2 Preliminaries: an elliptic cone in 3-D space and its homogeneous representation in 2-D projective space . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.2.1 Homogeneous representation . . . . . . . . . . . . . . . . . . . . . . . 15 2.2.2 Principal planes of a cone versus diagonalization of a conic matrix Q . 16 2.3 3-D interpretation of a circular feature for 3-D pose estimation . . . . . . . . 19 2.3.1 3-D orientation estimation . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.3.2 3-D position estimation . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.3.3 Composition of homogeneous transformation and discrimination for the unique solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.4 Experiment results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.4.1 A numerical example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.4.2 Evaluation of pose estimation performance . . . . . . . . . . . . . . . 29 2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 3 3-D Pose Estimation of an Elliptic Feature 35 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 3.1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 3.1.2 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 3.1.3 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.2 Interpretation of an elliptic feature with coplanar points in 2-D projective space 38 3.2.1 The minimal number of points for pose estimation . . . . . . . . . . . 39 3.2.2 Analysis of possible constraints for relative positions of two points to an ellipse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 3.2.3 Feature selection scheme for stable homography estimation . . . . . . 43 3.3 3-D pose estimation algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 3.3.1 Extraction of triangular features from an elliptic object . . . . . . . . 47 3.3.2 Homography decomposition . . . . . . . . . . . . . . . . . . . . . . . . 50 3.3.3 Composition of homogeneous transformation matrix with unique solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 3.4 Experiment results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 3.4.1 Experimental setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 3.4.2 Evaluation of the proposed method . . . . . . . . . . . . . . . . . . . . 54 3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4 3-D Pose Estimation of a Pair of Conic Features 61 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 4.2 3-D pose estimation of a conic feature incorporated with line features . . . . 61 4.3 3-D pose estimation of a conic feature incorporated with another conic feature 63 4.3.1 Some examples of self-polar triangle and invariants . . . . . . . . . . . 65 4.3.2 3-D pose estimation of a pair of coplanar conics . . . . . . . . . . . . . 67 4.3.3 Examples of 3-D pose estimation of a conic feature incorporated with another conic feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 5 Multiple Object Recognition Based on Pairwise Conic Model 77 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 5.2 Learning of geometric relation of multiple objects . . . . . . . . . . . . . . . . 78 5.3 Pairwise conic model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 5.3.1 De_nitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 5.4 Multiple object recognition based on pairwise conic model and conditional random _elds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 5.4.1 Graphical model for multiple object recognition . . . . . . . . . . . . . 86 5.4.2 Linear-chain conditional random _eld . . . . . . . . . . . . . . . . . . 87 5.4.3 Determination of low-level feature functions for multiple object recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 5.4.4 Range selection trick for e_ciently computing the costs of low-level feature functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 5.4.5 Evaluation of observation sequence . . . . . . . . . . . . . . . . . . . . 93 5.4.6 Object recognition based on hierarchical CRF . . . . . . . . . . . . . . 95 5.5 Geometric alignment algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 97 5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 6 Application to Place Recognition for Service Robots 105 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 6.1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 6.1.2 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 6.2 Feature extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 6.2.1 Detection of 2-D geometric shapes . . . . . . . . . . . . . . . . . . . . 107 6.2.2 Examples of shape feature extraction . . . . . . . . . . . . . . . . . . . 109 6.3 Object modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 6.3.1 A place model that describes multiple landmark objects . . . . . . . . 112 6.3.2 Pairwise conic model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 6.3.3 Incorporation of non-conic features with a pairwise conic model . . . . 114 6.4 Place learning and recognition system . . . . . . . . . . . . . . . . . . . . . . 121 6.4.1 HCRF-based recognition . . . . . . . . . . . . . . . . . . . . . . . . . . 122 6.5 Experiment results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 6.5.1 Experimental setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 6.5.2 Performance evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . 127 6.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 7 Application to Elevator Button Recognition 136 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 7.1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 7.1.2 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 7.1.3 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 7.2 Object modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 7.2.1 Geometric model for multiple button objects . . . . . . . . . . . . . . 140 7.2.2 Pairwise conic model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 7.3 Learning and recognition system . . . . . . . . . . . . . . . . . . . . . . . . . 141 7.3.1 Button object learning . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 7.3.2 CRF-based recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 7.4 Experiment results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 7.4.1 Experimental setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 7.4.2 Performance evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . 151 7.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 8 Concluding remarks 159 8.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 8.2 Further work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 References 161 Summary (in Korean) 16