Object-based video representations: shape compression and object segmentation

Object-based video representations are considered to be useful for easing the process of multimedia content production and enhancing user interactivity in multimedia productions. Object-based video presents several new technical challenges, however. Firstly, as with conventional video representations, compression of the video data is a requirement. For object-based representations, it is necessary to compress the shape of each video object as it moves in time. This amounts to the compression of moving binary images. This is achieved by the use of a technique called context-based arithmetic encoding. The technique is utilised by applying it to rectangular pixel blocks and as such it is consistent with the standard tools of video compression. The blockbased application also facilitates well the exploitation of temporal redundancy in the sequence of binary shapes. For the first time, context-based arithmetic encoding is used in conjunction with motion compensation to provide inter-frame compression. The method, described in this thesis, has been thoroughly tested throughout the MPEG-4 core experiment process and due to favourable results, it has been adopted as part of the MPEG-4 video standard. The second challenge lies in the acquisition of the video objects. Under normal conditions, a video sequence is captured as a sequence of frames and there is no inherent information about what objects are in the sequence, not to mention information relating to the shape of each object. Some means for segmenting semantic objects from general video sequences is required. For this purpose, several image analysis tools may be of help and in particular, it is believed that video object tracking algorithms will be important. A new tracking algorithm is developed based on piecewise polynomial motion representations and statistical estimation tools, e.g. the expectationmaximisation method and the minimum description length principle

Segmentation-based mesh design for motion estimation

Dans la plupart des codec vidéo standard, l'estimation des mouvements entre deux images se fait généralement par l'algorithme de concordance des blocs ou encore BMA pour « Block Matching Algorithm ». BMA permet de représenter l'évolution du contenu des images en décomposant normalement une image par blocs 2D en mouvement translationnel. Cette technique de prédiction conduit habituellement à de sévères distorsions de 1'artefact de bloc lorsque Ie mouvement est important. De plus, la décomposition systématique en blocs réguliers ne dent pas compte nullement du contenu de l'image. Certains paramètres associes aux blocs, mais inutiles, doivent être transmis; ce qui résulte d'une augmentation de débit de transmission. Pour paillier a ces défauts de BMA, on considère les deux objectifs importants dans Ie codage vidéo, qui sont de recevoir une bonne qualité d'une part et de réduire la transmission a très bas débit d'autre part. Dans Ie but de combiner les deux exigences quasi contradictoires, il est nécessaire d'utiliser une technique de compensation de mouvement qui donne, comme transformation, de bonnes caractéristiques subjectives et requiert uniquement, pour la transmission, l'information de mouvement. Ce mémoire propose une technique de compensation de mouvement en concevant des mailles 2D triangulaires a partir d'une segmentation de l'image. La décomposition des mailles est construite a partir des nœuds repartis irrégulièrement Ie long des contours dans l'image. La décomposition résultant est ainsi basée sur Ie contenu de l'image. De plus, étant donné la même méthode de sélection des nœuds appliquée à l'encodage et au décodage, la seule information requise est leurs vecteurs de mouvement et un très bas débit de transmission peut ainsi être réalise. Notre approche, comparée avec BMA, améliore à la fois la qualité subjective et objective avec beaucoup moins d'informations de mouvement. Dans la premier chapitre, une introduction au projet sera présentée. Dans Ie deuxième chapitre, on analysera quelques techniques de compression dans les codec standard et, surtout, la populaire BMA et ses défauts. Dans Ie troisième chapitre, notre algorithme propose et appelé la conception active des mailles a base de segmentation, sera discute en détail. Ensuite, les estimation et compensation de mouvement seront décrites dans Ie chapitre 4. Finalement, au chapitre 5, les résultats de simulation et la conclusion seront présentés.Abstract: In most video compression standards today, the generally accepted method for temporal prediction is motion compensation using block matching algorithm (BMA). BMA represents the scene content evolution with 2-D rigid translational moving blocks. This kind of predictive scheme usually leads to distortions such as block artefacts especially when the motion is important. The two most important aims in video coding are to receive a good quality on one hand and a low bit-rate on the other. This thesis proposes a motion compensation scheme using segmentation-based 2-D triangular mesh design method. The mesh is constructed by irregularly spread nodal points selected along image contour. Based on this, the generated mesh is, to a great extent, image content based. Moreover, the nodes are selected with the same method on the encoder and decoder sides, so that the only information that has to be transmitted are their motion vectors, and thus very low bit-rate can be achieved. Compared with BMA, our approach could improve subjective and objective quality with much less motion information."--Résumé abrégé par UM

An overview of video super-resolution algorithms

We investigate some excellent algorithms in the field of video space super-resolution based on artificial intelligence, structurally analyze the network structure of the algorithm and the commonly used loss functions. We also analyze the characteristics of algorithms in the new field of video space-time super-resolution. This work helps researchers to deeply understand the video super-resolution technology based on artificial intelligence

A Review Paper on Video De-Interlacing Multiple Techniques

In this paper present video interlacing de-interlacing and various techniques. Focus on the different techniques of video De- Interlacing that are Intra Field, Inter Field, Motion Adaptive, Motion Compensated De- interlacing and Spatio-Temporal Interpolation. De- Interlaced video use the full resolution of each scan so produced high quality image and remove flicker problem. Techniques are work on the scan line of object Intra Field techniques use pixels of the moving object, Inter Field works on stationary regions of object, Motion Adaptive works on the edge of the Object and Motion Compensation focus video sequence and brightness variation. Advantage of using De-interlacing technique is: Better Moving object image, no flickers and high vertical resolution

Visual motion : algorithms for analysis and application

Thesis (M.S.)--Massachusetts Institute of Technology, Dept. of Architecture, 1990.Includes bibliographical references (leaves 71-73).by Michael Adam Sokolov.M.S

Motion estimation and correction for simultaneous PET/MR using SIRF and CIL

SIRF is a powerful PET/MR image reconstruction research tool for processing data and developing new algorithms. In this research, new developments to SIRF are presented, with focus on motion estimation and correction. SIRF's recent inclusion of the adjoint of the resampling operator allows gradient propagation through resampling, enabling the MCIR technique. Another enhancement enabled registering and resampling of complex images, suitable for MRI. Furthermore, SIRF's integration with the optimization library CIL enables the use of novel algorithms. Finally, SPM is now supported, in addition to NiftyReg, for registration. Results of MR and PET MCIR reconstructions are presented, using FISTA and PDHG, respectively. These demonstrate the advantages of incorporating motion correction and variational and structural priors. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 2'

Motion estimation and correction for simultaneous PET/MR using SIRF and CIL

SIRF is a powerful PET/MR image reconstruction research tool for processing data and developing new algorithms. In this research, new developments to SIRF are presented, with focus on motion estimation and correction. SIRF's recent inclusion of the adjoint of the resampling operator allows gradient propagation through resampling, enabling the MCIR technique. Another enhancement enabled registering and resampling of complex images, suitable for MRI. Furthermore, SIRF's integration with the optimization library CIL enables the use of novel algorithms. Finally, SPM is now supported, in addition to NiftyReg, for registration. Results of MR and PET MCIR reconstructions are presented, using FISTA and PDHG, respectively. These demonstrate the advantages of incorporating motion correction and variational and structural priors. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 2'

Object-based 3-d motion and structure analysis for video coding applications

Ankara : Department of Electrical and Electronics Engineering and the Institute of Engineering and Sciences of Bilkent University, 1997.Thesis (Ph.D.) -- -Bilkent University, 1997.Includes bibliographical references leaves 102-115Novel 3-D motion analysis tools, which can be used in object-based video codecs, are proposed. In these tools, the movements of the objects, which are observed through 2-D video frames, are modeled in 3-D space. Segmentation of 2-D frames into objects and 2-D dense motion vectors for each object are necessary as inputs for the proposed 3-D analysis. 2-D motion-based object segmentation is obtained by Gibbs formulation; the initialization is achieved by using a fast graph-theory based region segmentation algorithm which is further improved to utilize the motion information. Moreover, the same Gibbs formulation gives the needed dense 2-D motion vector field. The formulations for the 3-D motion models are given for both rigid and non- rigid moving objects. Deformable motion is modeled by a Markov random field which permits elastic relations between neighbors, whereas, rigid 3-D motion parameters are estimated using the E-matrix method. Some improvements on the E-matrix method are proposed to make this algorithm more robust to gross errors like the consequence of incorrect segmentation of 2-D correspondences between frames. Two algorithms are proposed to obtain dense depth estimates, which are robust to input errors and suitable for encoding, respectively. While the former of these two algorithms gives simply a MAP estimate, the latter uses rate-distortion theory. Finally, 3-D motion models are further utilized for occlusion detection and motion compensated temporal interpolation, and it is observed that for both applications 3-D motion models have superiority over their 2-D counterparts. Simulation results on artificial and real data show the advantages of the 3-D motion models in object-based video coding algorithms.Alatan, A AydinPh.D