DOI 10.1007/s11042-007-0106-y Evaluation of content-based image descriptors by statistical methods

Abstract Evaluation of visual information retrieval systems is usually performed by executing test queries and computing recall- and precision-like measures based on predefined media collections and ground truth information. This process is complex and time consuming. For the evaluation of feature transformations (transformation of visual media objects to feature vectors) it would be desirable to have simpler methods available as well. In this paper we introduce a supplementary evaluation procedure for features that is founded on statistical data analysis. A second novelty is that we make use of the existing visual MPEG-7 descriptors to judge the characteristics of feature transformations. The proposed procedure is divided into four steps: (1) feature extraction, (2) merging with MPEG-7 data and normalisation, (3) statistical data analysis and (4) visualisation and interpretation. Three types of statistical methods are used for evaluation: (1) univariate description (moments, etc.), (2) identification of similarities between feature elements (e.g. cluster analysis) and (3) identification of dependencies between variables (e.g. factor analysis). Statistical analysis provides beneficial insights into the structure of features that can be exploited for feature redesign. Application and advantages of the proposed approach are shown in a number of toy examples

Semantics in Content-based Multimedia Retrieval

This contribution investigates the content-based feature extraction methods used in visual information retrieval, focusing on concepts that are employed for the semantic representation of media content. The background part describes the building blocks of feature extraction functions. Since numerous methods have been proposed we concentrate on the metaconcepts. The building blocks lead to a discussion of starting points for semantic enrichment of low-level features. The second part reviews features from the perspective of data quality. A case study on content-based MPEG-7 features illustrates the relativity of terms like “low-level, ” “highlevel” and “semantics”. For example, often more semantics mean just more redundancy. The final part sketches the application of features in retrieval scenarios. The results of a case study suggest that – from the retrieval perspective, too – “semantic enrichment of low-level features ” is a partially questionable concept. The performance of classification-based retrieval, it seems, does hardly depend on the context of features.