Semi-Supervised Image Classification based on a Multi-Feature Image Query Language

Abstract

The area of Content-Based Image Retrieval (CBIR) deals with a wide range of research disciplines. Being closely related to text retrieval and pattern recognition, the probably most serious issue to be solved is the so-called \semantic gap". Except for very restricted use-cases, machines are not able to recognize the semantic content of digital images as well as humans. This thesis identifies the requirements for a crucial part of CBIR user interfaces, a multimedia-enabled query language. Such a language must be able to capture the user's intentions and translate them into a machine-understandable format. An approach to tackle this translation problem is to express high-level semantics by merging low-level image features. Two related methods are improved for either fast (retrieval) or accurate(categorization) merging. A query language has previously been developed by the author of this thesis. It allows the formation of nested Boolean queries. Each query term may be text- or content-based and the system merges them into a single result set. The language is extensible by arbitrary new feature vector plug-ins and thus use-case independent. This query language should be capable of mapping semantics to features by applying machine learning techniques; this capability is explored. A supervised learning algorithm based on decision trees is used to build category descriptors from a training set. Each resulting \query descriptor" is a feature-based description of a concept which is comprehensible and modifiable. These descriptors could be used as a normal query and return a result set with a high CBIR based precision/recall of the desired category. Additionally, a method for normalizing the similarity profiles of feature vectors has been developed which is essential to perform categorization tasks. To prove the capabilities of such queries, the outcome of a semi-supervised training session with \leave-one-object-out" cross validation is compared to a reference system. Recent work indicates that the discriminative power of the query-based descriptors is similar and is likely to be improved further by implementing more recent feature vectors