Scalable metadata for search, sonification and display

Abstract

Presented at the 8th International Conference on Auditory Display (ICAD), Kyoto, Japan, July 2-5, 2002.This paper argues for the need - and usefulness - of scalable content-based metadata. Scalability is here defined as the conjunction of two properties: arbitrary resolution, and convertibility between resolutions. The need follows directly from the projected exponential trend of media data size, that equally affects metadata. In addition to addressing this need, scalable metadata are useful because they are hierarchical in nature, and incorporate statistics effective for search (in automatic media handling systems) or sonification and display (in interactive media handling systems). Scalable metadata are built upon a small number of statistical operations that offer the right scalability properties: extrema (min, max), mean, variance, covariance, histogram, etc. These statistics are used alone or in combination to produce summary descriptions with a resolution tailored to the needs and constraints of the application. They can also be understood as parametrizations of the distributions of full-resolution descriptor values that they summarize. As such, they support inference mechanisms upon to build search and matching algorithms. For interactive applications, scalable content-based descriptors can be used to produce visual displays that support zooming and navigation within multimedia collections of arbitrary size, under the assistance of visual and auditory feedback