Scalable Multiresolution Image Segmentation and Its Application in Video Object Extraction Algorithm

This paper presents a novel multiresolution image segmentation method based on the discrete wavelet transform and Markov Random Field (MRF) modelling. A major contribution of this work is to add spatial scalability to the segmentation algorithm producing the same segmentation pattern at different resolutions. This property makes it suitable for the scalable object-based wavelet coding. The correlation between different resolutions of pyramid is considered by a multiresolution analysis which is incorporated into the objective function of the MRF segmentation algorithm. Allowing for smoothness terms in the objective function at different resolutions improves border smoothness and creates visually more pleasing objects/regions, particularly at lower resolutions where downsampling distortions are more visible. Application of the spatial segmentation in video segmentation, compared to traditional image/video object extraction algorithms, produces more visually pleasing shape masks at different resolutions which is applicable for object-based video wavelet coding. Moreover it allows for larger motion, better noise tolerance and less computational complexity. In addition to spatial scalability, the proposed algorithm outperforms the standard image/video segmentation algorithms, in both objective and subjective tests

A survey on video segmentation for real-time applications

Video object segmentation is to extract moving and static objects from consecutive video frames. It is a prerequisite for visual content retrieval (e.g., MPEG-7 related schemes), objectbased compression and coding (e.g., MPEG-4 codecs), object recognition, object tracking, security video surveillance, traffic monitoring for law enforcement, and many other application

Low Bit Rate Video Coding

Variable length bit rate (VLBR) broadly encompasses video coding which mandates a temporal frequency of 10 frames per second (fps) or less. Object-based video coding represents a very promising option for VLBR coding, though the problems of object identification and segmentation need to be addressed by further research. Pattern-based coding is a simplified object segmentation process that is computationally much less expensive, though a real-time, content-dependent pattern generation approach will certainly improve its acceptance for VLBR coding. In this paper pattern based coding technique is used. In this paper, a very low bit-rate video coding algorithm that focuses on moving regions is performed. The aim is to improve the coding performance, which gives better subjective and objective quality than that of the conventional coding methods at the same bit rate. Eight patterns are pre-defined to approximate the moving regions in a macroblock. The patterns are then used for motion estimation and compensation to reduce the prediction errors. Furthermore, in order to increase the compression performance, the residual errors of a macroblock are rearranged into a block with no significant increase of high-order DCT coefficients. As a result, both the prediction efficiency and the compression efficiency are improved. This paper shows that using pattern based coding the compression ratio is better

Semi-automatic video object segmentation for multimedia applications

A semi-automatic video object segmentation tool is presented for segmenting both still pictures and image sequences. The approach comprises both automatic segmentation algorithms and manual user interaction. The still image segmentation component is comprised of a conventional spatial segmentation algorithm (Recursive Shortest Spanning Tree (RSST)), a hierarchical segmentation representation method (Binary Partition Tree (BPT)), and user interaction. An initial segmentation partition of homogeneous regions is created using RSST. The BPT technique is then used to merge these regions and hierarchically represent the segmentation in a binary tree. The semantic objects are then manually built by selectively clicking on image regions. A video object-tracking component enables image sequence segmentation, and this subsystem is based on motion estimation, spatial segmentation, object projection, region classification, and user interaction. The motion between the previous frame and the current frame is estimated, and the previous object is then projected onto the current partition. A region classification technique is used to determine which regions in the current partition belong to the projected object. User interaction is allowed for object re-initialisation when the segmentation results become inaccurate. The combination of all these components enables offline video sequence segmentation. The results presented on standard test sequences illustrate the potential use of this system for object-based coding and representation of multimedia

Allowing content-based functionalities in segmentation-based coding schemes

This paper deals with the use of the segmentation tools and principles presented in [10] and [13] for allowing content-based functionalities. In this framework, means for supervised selection of objects in the scene are proposed. In addition, a technique for object tracking in the context of segmentation-based video coding is presented. The technique is independent of the type of segmentation approach used in the coding scheme. The algorithm relies on a double partition of the image that yields spatially homogeneous regions. This double partition permits to obtain the position and shape of the previous object in the current image while computing the projected partition. In order to demonstrate the potentialities of this algorithm, it is applied in a specific coding scheme so that content-based functionalities, such as selective coding, are allowed.Peer ReviewedPostprint (published version

Pengenalan Objek Manusia pada Video Menggunakan Metode Unsupervised Segmentation dengan Adaptive Neural Network Classifier

ABSTRAKSI: Saat ini, video digital dimanfaatkan secara luas untuk berbagai aplikasi. Teknologi video telah menjadi salah satu teknologi terpenting dalam komunikasi multimedia, antara lain adalah untuk videophone dan video converence. Penggunaan video tidak terbatas untuk keperluan komunikasi, namun juga dimanfaatkan dalam bidang pendidikan, kesehatan, hiburan, bisnis, dan lain-lain. Object based video coding telah memberikan kontirbusi yang cukup besar untuk perkembangan teknologi video digital. Salah satunya adalah pengenalan objek manusia pada video yang dapat menambah kemudahan penggunaan, pengolahan dan analisis video digital.Dalam Tugas Akhir ini, metode unsepervised segmentation dengan adaptive neural network classifier digunakan untuk pengenalan objek manusia pada video, yaitu dengan memisahkan objek manusia tersebut dari backgroundnya. Adaptive neural netwrok classifier digunakan untuk menangani klasifikasi elemen objek yang bergerak pada video.Kata Kunci : pengenalan objek, segmentasi, klasifikasi, video digital, neural networkABSTRACT: Nowadays, digital video have been used widely for many application. Video technology have being one of the most important element in multimedia telecommunication, such as videophone and videoconference. The usability of video technology isn’t just for communication but also use for entertainment, education, business, health care, etc. Object based video coding give a big contribution for development of video technology. One example of object based video coding is human object recognition which can increase the usability of digital video technology.In this Final Assignment, unsepervised segmentation with adaptive neural network classifier used for human object recognition in video, that is extract human object from its background. Adaptive neural network classifier is used to handle classification problem of the element of the moving object.Keyword: object recognition, segmentation, clssification, digital video, neural networ

Stand-Alone Objective Segmentation Quality Evaluation

The identification of objects in video sequences, that is, video segmentation, plays a major role in emerging interactive multimedia services, such as those enabled by the ISO MPEG-4 and MPEG-7 standards. In this context, assessing the adequacy of the identified objects to the application targets, that is, evaluating the segmentation quality, assumes a crucial importance. Video segmentation technology has received considerable attention in the literature, with algorithms being proposed to address various types of applications. However, the segmentation quality performance evaluation of those algorithms is often ad hoc, and a well-established solution is not available. In fact, the field of objective segmentation quality evaluation is still maturing; recently, some more efforts have been made, mainly following the emergence of the MPEG object-based coding and description standards. This paper discusses the problem of objective segmentation quality evaluation in its most difficult scenario: standalone evaluation, that is, when a reference segmentation is not available for comparative evaluation. In particular, objective metrics are proposed for the evaluation of standalone segmentation quality for both individual objects and overall segmentation partitions

On Evaluating Video Object Segmentation Quality: A Perceptually Driven Objective Metric

The task of extracting objects in video sequences emerges in many applications such as object-based video coding (e.g., MPEG-4) and content-based video indexing and retrieval (e.g., MPEG-7). The MPEG-4 standard provides specifications for the coding of video objects, but does not address the problem of how to extract foreground objects in image sequences. Therefore, for specific applications, evaluating the quality of foreground/background segmentation results is necessary to allow for an appropriate selection of segmentation algorithms and for tuning their parameters for optimal performance. Many segmentation algorithms have been proposed along with a number of evaluation criteria. Nevertheless, formal psychophysical experiments evaluating the quality of different video foreground object segmentation results have not yet been conducted. In this paper, a generic framework for both subjective and objective segmentation quality evaluation is presented. An objective quality assessment method for segmentation evaluation is derived on the basis of perceptual factors through subjective experiments. The performance of the proposed method is shown on different state-of-the-art foreground/background segmentation algorithms and our method is compared to other objective methods which do not include perceptual factors. Moreover, on the basis of subjective results, weighting strategies are introduced into the proposed metric to meet the specificity of different segmentation applications e.g., video compression, video surveillance and mixed reality. Experimental results confirm the efficiency of the proposed approach

Estimation of depth fields suitable for video compression using 3-D structures and motion of objects

Cataloged from PDF version of article.Intensity prediction along motion trajectories removes temporal redundancy considerably in video compression algorithms. In threedimensional (3-D) object-based video coding, both 3-D motion and depth values are required for temporal prediction. The required 3-D motion parameters for each object are found by the correspondence-based Ematrix method. The estimation of the correspondences—two-dimensional (2-D) motion field—between the frames and segmentation of the scene into objects are achieved simultaneously by minimizing a Gibbs energy. The depth field is estimated by jointly minimizing a defined distortion and bitrate criterion using the 3-D motion parameters. The resulting depth field is efficient in the rate-distortion sense. Bit-rate values corresponding to the lossless encoding of the resultant depth fields are obtained using predictive coding; prediction errors are encoded by a Lempel–Ziv algorithm. The results are satisfactory for real-life video scenes

Video coding for compression and content-based functionality

The lifetime of this research project has seen two dramatic developments in the area of digital video coding. The first has been the progress of compression research leading to a factor of two improvement over existing standards, much wider deployment possibilities and the development of the new international ITU-T Recommendation H.263. The second has been a radical change in the approach to video content production with the introduction of the content-based coding concept and the addition of scene composition information to the encoded bit-stream. Content-based coding is central to the latest international standards efforts from the ISO/IEC MPEG working group. This thesis reports on extensions to existing compression techniques exploiting a priori knowledge about scene content. Existing, standardised, block-based compression coding techniques were extended with work on arithmetic entropy coding and intra-block prediction. These both form part of the H.263 and MPEG-4 specifications respectively. Object-based coding techniques were developed within a collaborative simulation model, known as SIMOC, then extended with ideas on grid motion vector modelling and vector accuracy confidence estimation. An improved confidence measure for encouraging motion smoothness is proposed. Object-based coding ideas, with those from other model and layer-based coding approaches, influenced the development of content-based coding within MPEG-4. This standard made considerable progress in this newly adopted content based video coding field defining normative techniques for arbitrary shape and texture coding. The means to generate this information, the analysis problem, for the content to be coded was intentionally not specified. Further research work in this area concentrated on video segmentation and analysis techniques to exploit the benefits of content based coding for generic frame based video. The work reported here introduces the use of a clustering algorithm on raw data features for providing initial segmentation of video data and subsequent tracking of those image regions through video sequences. Collaborative video analysis frameworks from COST 21 l qual and MPEG-4, combining results from many other segmentation schemes, are also introduced