Augmenting training sets with still images for video concept detection

Accessing the visual information of video content is a challenging task. Automatic annotation techniques have made significant progress, however they still suffer from the lack of appropriate training data. To overcome this problem we propose the use of still images taken from a photo sharing website as an additional resource for training. However, a mere extension of the training set with still images does not yield a large gain in classification accuracy. We show that using a combination of techniques for bridging the differences between still images and video keyframes improves classification performance compared to simply augmenting the training set

A new generic texture synthesis approach for enhanced H.264/MPEG4-AVC video coding

A new generic texture synthesis approach, which is inspired by the work of Kwatra et al., is presented in this paper. The approach is hierarchical, non-parametric, patch-based and for that applicable to a large class of spatio-temporal textures with and without local motion activity. In this work, it is shown, how the new texture synthesizer can be integrated into a content-based video coding framework. That is, the decoder reconstructs textures like water, grass, etc. that are usually very costly to encode. For that, the new texture synthesizer in conjunction with side information that is generated by the encoder is required. The reconstruction of above-mentioned textures at the decoder side basically corresponds to stuffing holes in a video sequence. Spurious edges are thereby avoided by using graph cuts to generate irregular contours at transitions between natural and synthetic textures and preferably place them (the contours) in high-frequency regions, where they are less visible

Texture synthesis method for generic video sequences

An effective texture synthesis method is presented that is inspired by the work of Kwatra et al. [1]. Their algorithm is non-parametric and patch-based. Blending between overlapping patches is optimized using graph cut techniques. We generalize the initial approach [ 1 ] to achieve a new synthesis algorithm that yields improved results for a much larger class of natural video sequences. For that, two major extensions have been provided: (1) the ability to handle constrained texture synthesis applications and (2) robustness against global camera motion. Constrained synthesis thereby refers to integrating synthetic textures into natural video sequences, as opposed to unconstrained texture synthesis, where (infinite) spatio-temporal extensions of single textures are generated. Camera motion compensation enables applicability of the synthesis algorithm to video sequences with a moving camera. The results presented in this paper show that the proposed improvements yield significant subjective gains compared to the initial algorithm

Merging MPEG-7 descriptors for image content analysis

A segmentation algorithm for image content analysis is presented. We assume that the textures in a video scene can be labeled subjectively relevant or irrelevant. Relevant textures are defined as containing subjectively meaningful details, while irrelevant textures can be seen as image content with less important subjective details. We apply this idea to video coding using a texture analyzer and a texture synthesizer. The texture analyzer (encoder side) identifies the texture regions with unimportant subjective details and generates side information for the texture synthesizer (decoder side), which in turn inserts synthetic textures at the specified locations. The focus of this paper is the texture analyzer, which uses multiple MPEG-7 descriptors simultaneously for similarity estimation. The texture analyzer is based on a split and merge segmentation approach. Its current implementation yields an identification rate of up to 96% and an average gain of up to 10% compared to single descriptor usage

Modelling image completion distortions in texture analysis-synthesis coding

Perception based coding with Texture Analysis and Synthesis (TAS) is a promising way to increase the compression efficiency of modern coding schemes, such as High Efficiency Video Coding (HEVC). TAS approaches typically employ an analysis step which specifies which blocks could be reconstructed by a texture synthesizer. Even though synthesized blocks are perceptually similar to their original versions, they may produce relatively high Mean Squared Error with regard to the original signal. Hence, the contribution of this paper will be a novel image quality assessment method for image completion based on visual attention variations. A subjective experiment has been designed to provide insight into the perceived distortions which are induced by TAS techniques and it is shown that saliency changes are a promising predictor for perceptual distortion. Free access to the image database is provided to encourage further research on this topic

Temporally consistent soccer field registration

In this paper, a system for temporally consistent registration of a soccer field is proposed. It is implemented using 2D field line and arc detection as well as motion estimation for frames where too few field lines are detected. We show that a combination of these technologies yields more consistent results than common state-of-the-art approaches. As an important additional contribution, a Kalman filter is applied to suppress remaining misdetections. The system is able to register the soccer field with a mean error of 2.53 m regardless of the visible field region

Adaptive 2D-AR framework for texture completion

Texture extrapolation techniques enable to fill large holes of missing information. Many applications can be targeted such as image and video coding, channel block losses, object removal, filling of 3D disocclusions etc. For more than two decades, many approaches have been developed, even though each contains pros and cons which force to choose the best compromise for the targeted application. In this paper, we propose to continue exploring and improving a popular parametric completion method using the autoregressive (AR) model. In this framework, the training area is automatically optimized. A consistency criterion also enables to assess and regularize the model. Moreover, a post-processing step enables to remove the remaining seam artefacts. A comparison with the state-of-the-art is provided for both subjective quality and complexity which remains a major constraint for texture completion

On the efficiency of image completion methods for intra prediction in video coding with large block structures

Intra prediction is a fundamental tool in video coding with hybrid block-based architecture. Recent investigations have shown that one of the most beneficial elements for a higher compression performance in high-resolution videos is the incorporation of larger block structures. Thus in this work, we investigate the performance of novel intra prediction modes based on different image completion techniques in a new video coding scheme with large block structures. Image completion methods exploit the fact that high frequency image regions yield high coding costs when using classical H.264/AVC prediction modes. This problem is tackled by investigating the incorporation of several intra predictors using the concept of Laplace partial differential equation (PDE), Least Square (LS) based linear prediction and the Auto Regressive model. A major aspect of this article is the evaluation of the coding performance in a qualitative (i.e. coding efficiency) manner. Experimental resul ts show significant improvements in compression (up to 7.41 %) by integrating the LS-based linear intra prediction