Video indexing based on mosaic representations

Abstract

Abstract | Videois a rich source of information. It provides visual information about scenes. However, this information is implicitly buried inside the raw video data, and is provided with the cost of very high temporal redundancy. While the standard sequential form of video storage is adequate for viewing in a "movie mode", it fails to support rapid access to information of interest that is required in many of the emerging applications of video. This paper presents an approach for e cient access, use and manipulation of video data. The video data is rst transformed from its sequential and redundant frame-based representation in which the information about the scene is distributed over many frames, to an explicit and compact scene-based representation, to which each frame can be directly related. This compact reorganization of the video data supports non-linear browsing and e cient indexing to provide rapid access directly to information of interest. The paper describes a new set of methods for indexing into the video sequence based on the scene-based representation. These indexing methods are based on geometric and dynamic information contained in the video. These methods complement the more traditional "content-based indexing " methods which utilizes image appearance information (namely color and texture properties), but are considerably simpler to achieve and are highly computationally e cient. Keywords|video indexing, video browsing, compact video representations, mosaics, video manipulation, video annotation, video compression, video databases. I