On Evaluating Video Object Segmentation Quality: A Perceptually driven Objective Metric

Segmentation of moving objects in image sequences plays an important role in video processing and analysis. Evaluating the quality of segmentation results is necessary to allow the appropriate selection of segmentation algorithms and to tune their parameters for optimal performance. Many segmentation algorithms have been proposed along with a number of evaluation criteria. Nevertheless, no formal psychophysical experiments evaluating the quality of different video object segmentation results have been conducted. In this paper, a generic framework for segmentation quality evaluation is presented. A perceptually driven automatic method for segmentation evaluation is proposed and compared against state-of-the-art. Moreover, on the basis of subjective results, weighting strategies are introduced into the proposed objective metric to meet the specificity of different segmentation applications such as video compression and mixed reality. Experimental results confirm the efficiency of the proposed approach

On Evaluating Video Object Segmentation Quality: A Perceptually Driven Objective Metric

The task of extracting objects in video sequences emerges in many applications such as object-based video coding (e.g., MPEG-4) and content-based video indexing and retrieval (e.g., MPEG-7). The MPEG-4 standard provides specifications for the coding of video objects, but does not address the problem of how to extract foreground objects in image sequences. Therefore, for specific applications, evaluating the quality of foreground/background segmentation results is necessary to allow for an appropriate selection of segmentation algorithms and for tuning their parameters for optimal performance. Many segmentation algorithms have been proposed along with a number of evaluation criteria. Nevertheless, formal psychophysical experiments evaluating the quality of different video foreground object segmentation results have not yet been conducted. In this paper, a generic framework for both subjective and objective segmentation quality evaluation is presented. An objective quality assessment method for segmentation evaluation is derived on the basis of perceptual factors through subjective experiments. The performance of the proposed method is shown on different state-of-the-art foreground/background segmentation algorithms and our method is compared to other objective methods which do not include perceptual factors. Moreover, on the basis of subjective results, weighting strategies are introduced into the proposed metric to meet the specificity of different segmentation applications e.g., video compression, video surveillance and mixed reality. Experimental results confirm the efficiency of the proposed approach

Image synthesis based on a model of human vision

Modern computer graphics systems are able to construct renderings of such high quality that viewers are deceived into regarding the images as coming from a photographic source. Large amounts of computing resources are expended in this rendering process, using complex mathematical models of lighting and shading. However, psychophysical experiments have revealed that viewers only regard certain informative regions within a presented image. Furthermore, it has been shown that these visually important regions contain low-level visual feature differences that attract the attention of the viewer. This thesis will present a new approach to image synthesis that exploits these experimental findings by modulating the spatial quality of image regions by their visual importance. Efficiency gains are therefore reaped, without sacrificing much of the perceived quality of the image. Two tasks must be undertaken to achieve this goal. Firstly, the design of an appropriate region-based model of visual importance, and secondly, the modification of progressive rendering techniques to effect an importance-based rendering approach. A rule-based fuzzy logic model is presented that computes, using spatial feature differences, the relative visual importance of regions in an image. This model improves upon previous work by incorporating threshold effects induced by global feature difference distributions and by using texture concentration measures. A modified approach to progressive ray-tracing is also presented. This new approach uses the visual importance model to guide the progressive refinement of an image. In addition, this concept of visual importance has been incorporated into supersampling, texture mapping and computer animation techniques. Experimental results are presented, illustrating the efficiency gains reaped from using this method of progressive rendering. This visual importance-based rendering approach is expected to have applications in the entertainment industry, where image fidelity may be sacrificed for efficiency purposes, as long as the overall visual impression of the scene is maintained. Different aspects of the approach should find many other applications in image compression, image retrieval, progressive data transmission and active robotic vision