We present a novel scheme for object-based video sequence representation using appearance spaces. Our scheme enables fully automatic extraction of semantic video objects for a class of sequences, and their supervised organization in an object-class hierarchy. The hierarchy can be used for generic classification of query video objects, and transcoding using semantics of video objects

Gaurav Garg