Object-based video representations: shape compression and object segmentation

Object-based video representations are considered to be useful for easing the process of multimedia content production and enhancing user interactivity in multimedia productions. Object-based video presents several new technical challenges, however. Firstly, as with conventional video representations, compression of the video data is a requirement. For object-based representations, it is necessary to compress the shape of each video object as it moves in time. This amounts to the compression of moving binary images. This is achieved by the use of a technique called context-based arithmetic encoding. The technique is utilised by applying it to rectangular pixel blocks and as such it is consistent with the standard tools of video compression. The blockbased application also facilitates well the exploitation of temporal redundancy in the sequence of binary shapes. For the first time, context-based arithmetic encoding is used in conjunction with motion compensation to provide inter-frame compression. The method, described in this thesis, has been thoroughly tested throughout the MPEG-4 core experiment process and due to favourable results, it has been adopted as part of the MPEG-4 video standard. The second challenge lies in the acquisition of the video objects. Under normal conditions, a video sequence is captured as a sequence of frames and there is no inherent information about what objects are in the sequence, not to mention information relating to the shape of each object. Some means for segmenting semantic objects from general video sequences is required. For this purpose, several image analysis tools may be of help and in particular, it is believed that video object tracking algorithms will be important. A new tracking algorithm is developed based on piecewise polynomial motion representations and statistical estimation tools, e.g. the expectationmaximisation method and the minimum description length principle

Progressive contour coding in the wavelet domain

This paper presents a new wavelet-based image contour coding technique, suitable for representing either shapes or generic contour maps. Starting from a contour map (e.g. a segmentation map or the result of a contour extraction operator), this is first converted in a one-dimensional signal. Coordinate jumps among different contour extremities are converted, if under a suitable threshold, into signal discontinuities which can be compactly represented in the wavelet domain. Otherwise, the exceeding discontinuities are coded as side information. This side information is minimized by an optimized contour segment sequencing. The obtained 1D signal is decomposed and coded in the wavelet domain by using a 1D version of an improved implementation of the SPIHT algorithm. This technique can efficiently code every kind of 2D contour map, from one to many unconnected contour segments. It guarantees a fully embedded progressive coding, state-of-art coding performance, good approximation capabilities for both open and closed contours, and visually graceful degradation at low bit-rates

Shape representation and coding of visual objets in multimedia applications — An overview

Emerging multimedia applications have created the need for new functionalities in digital communications. Whereas existing compression standards only deal with the audio-visual scene at a frame level, it is now necessary to handle individual objects separately, thus allowing scalable transmission as well as interactive scene recomposition by the receiver. The future MPEG-4 standard aims at providing compression tools addressing these functionalities. Unlike existing frame-based standards, the corresponding coding schemes need to encode shape information explicitly. This paper reviews existing solutions to the problem of shape representation and coding. Region and contour coding techniques are presented and their performance is discussed, considering coding efficiency and rate-distortion control capability, as well as flexibility to application requirements such as progressive transmission, low-delay coding, and error robustnes

Progressive contour coding in the wavelet domain

This paper presents a new wavelet-based image contour coding technique, suitable for representing either shapes or generic contour maps. Starting from a contour map (e.g. a segmentation map or the result of an edge detector process), a unique one-dimensional signal is generated from the set of contour points. Coordinate jumps between contour extremities when under a tolerance threshold represent signal discontinuities but they can still be compactly coded in the wavelet domain. Exceeding threshold discontinuities are coded as side information. This side information and the amount of remaining discontinuity are minimized by an optimized contour segment sequencing. The obtained 1D signal is decomposed and coded in the wavelet domain by using a 1D extension of the SPIHT algorithm. The described technique can efficiently code any kind of 2D contour map, from one to many unconnected contour segments. It guarantees a fully embedded progressive coding, state-of-art coding performance, good approximation capabilities for both open and closed contours, and graceful visual degradation at low bit-rates

Vehicle make and model recognition using bag of expressions

This article belongs to the Section Intelligent SensorsVehicle make and model recognition (VMMR) is a key task for automated vehicular surveillance (AVS) and various intelligent transport system (ITS) applications. In this paper, we propose and study the suitability of the bag of expressions (BoE) approach for VMMR-based applications. The method includes neighborhood information in addition to visual words. BoE improves the existing power of a bag of words (BOW) approach, including occlusion handling, scale invariance and view independence. The proposed approach extracts features using a combination of different keypoint detectors and a Histogram of Oriented Gradients (HOG) descriptor. An optimized dictionary of expressions is formed using visual words acquired through k-means clustering. The histogram of expressions is created by computing the occurrences of each expression in the image. For classification, multiclass linear support vector machines (SVM) are trained over the BoE-based features representation. The approach has been evaluated by applying cross-validation tests on the publicly available National Taiwan Ocean University-Make and Model Recognition (NTOU-MMR) dataset, and experimental results show that it outperforms recent approaches for VMMR. With multiclass linear SVM classification, promising average accuracy and processing speed are obtained using a combination of keypoint detectors with HOG-based BoE description, making it applicable to real-time VMMR systems.Muhammad Haroon Yousaf received funding from the Higher Education Commission, Pakistan for Swarm Robotics Lab under the National Centre for Robotics and Automation (NCRA). The authors also acknowledge support from the Directorate of ASR& TD, University of Engineering and Technology Taxila, Pakistan