Motion-based Segmentation and Classification of Video Objects

In this thesis novel algorithms for the segmentation and classification of video objects are developed. The segmentation procedure is based on motion and is able to extract moving objects acquired by either a static or a moving camera. The classification of those objects is performed by matching their outlines gathered from a number of consecutive frames of the video with preprocessed views of prototypical objects stored in a database. This thesis contributes to four areas of image processing and computer vision: motion analysis, implicit active contour models, motion-based segmentation, and object classification. In detail, in the field of motion analysis, the tensor-based motion estimation approach is extended by a non-maximum suppression scheme, which improves the identification of relevant image structures significantly. In order to analyze videos that contain large image displacements, a feature-based motion estimation method is developed. In addition, to include camera operations into the segmentation process, a robust camera motion estimator based on least trimmed squares regression is presented. In the area of implicit active contour models, a model that unifies geometric and geodesic active contours is developed. For this model an efficient numerical implementation based on a new narrow-band method and a semi-implicit discretization is provided. Compared to standard algorithms these optimizations reduce the computational complexity significantly. Integrating the results of the motion analysis into the fast active contour implementation, novel algorithms for motion-based segmentation are developed. In the field of object classification, a shape-based classification approach is extended and adapted to image sequence processing. Finally, a system for video object classification is derived by combining the proposed motion-based segmentation algorithms with the shape-based classification approach

A video object generation tool allowing friendly user interaction

In this paper we describe an interactive video object segmentation tool developed in the framework of the ACTS-AC098 MOMUSYS project. The Video Object Generator with User Environment (VOGUE) combines three different sets of automatic and semi-automatic-tool (spatial segmentation, object tracking and temporal segmentation) with general purpose tools for user interaction. The result is an integrated environment allowing the user-assisted segmentation of any sort of video sequences in a friendly and efficient manner.Peer ReviewedPostprint (published version

Interaction between high-level and low-level image analysis for semantic video object extraction

Authors of articles published in EURASIP Journal on Advances in Signal Processing are the copyright holders of their articles and have granted to any third party, in advance and in perpetuity, the right to use, reproduce or disseminate the article, according to the SpringerOpen copyright and license agreement (http://www.springeropen.com/authors/license)

The aceToolbox: low-level audiovisual feature extraction for retrieval and classification

In this paper we present an overview of a software platform that has been developed within the aceMedia project, termed the aceToolbox, that provides global and local lowlevel feature extraction from audio-visual content. The toolbox is based on the MPEG-7 eXperimental Model (XM), with extensions to provide descriptor extraction from arbitrarily shaped image segments, thereby supporting local descriptors reflecting real image content. We describe the architecture of the toolbox as well as providing an overview of the descriptors supported to date. We also briefly describe the segmentation algorithm provided. We then demonstrate the usefulness of the toolbox in the context of two different content processing scenarios: similarity-based retrieval in large collections and scene-level classification of still images

Multi-Cue Structure Preserving MRF for Unconstrained Video Segmentation

Video segmentation is a stepping stone to understanding video context. Video segmentation enables one to represent a video by decomposing it into coherent regions which comprise whole or parts of objects. However, the challenge originates from the fact that most of the video segmentation algorithms are based on unsupervised learning due to expensive cost of pixelwise video annotation and intra-class variability within similar unconstrained video classes. We propose a Markov Random Field model for unconstrained video segmentation that relies on tight integration of multiple cues: vertices are defined from contour based superpixels, unary potentials from temporal smooth label likelihood and pairwise potentials from global structure of a video. Multi-cue structure is a breakthrough to extracting coherent object regions for unconstrained videos in absence of supervision. Our experiments on VSB100 dataset show that the proposed model significantly outperforms competing state-of-the-art algorithms. Qualitative analysis illustrates that video segmentation result of the proposed model is consistent with human perception of objects