Full-reference stereoscopic video quality assessment using a motion sensitive HVS model

Stereoscopic video quality assessment has become a major research topic in recent years. Existing stereoscopic video quality metrics are predominantly based on stereoscopic image quality metrics extended to the time domain via for example temporal pooling. These approaches do not explicitly consider the motion sensitivity of the Human Visual System (HVS). To address this limitation, this paper introduces a novel HVS model inspired by physiological findings characterising the motion sensitive response of complex cells in the primary visual cortex (V1 area). The proposed HVS model generalises previous HVS models, which characterised the behaviour of simple and complex cells but ignored motion sensitivity, by estimating optical flow to measure scene velocity at different scales and orientations. The local motion characteristics (direction and amplitude) are used to modulate the output of complex cells. The model is applied to develop a new type of full-reference stereoscopic video quality metrics which uniquely combine non-motion sensitive and motion sensitive energy terms to mimic the response of the HVS. A tailored two-stage multi-variate stepwise regression algorithm is introduced to determine the optimal contribution of each energy term. The two proposed stereoscopic video quality metrics are evaluated on three stereoscopic video datasets. Results indicate that they achieve average correlations with subjective scores of 0.9257 (PLCC), 0.9338 and 0.9120 (SRCC), 0.8622 and 0.8306 (KRCC), and outperform previous stereoscopic video quality metrics including other recent HVS-based metrics

Impact of disparity error on user experience of interacting with stereoscopic 3D video content

The stereoscopic three-dimensional (3D) displays can offer immersive experience to the audience by artificially stimulating binocular stereopsis in the human visual system. The binocular disparity between the left and right view is the key factor in creating the impression of depth, distinguishing the stereo 3D video from other types of video paradigms. Taking into consideration of the imperfections of current disparity estimation algorithms, this paper focus on the impact of disparity error on the user experience of pointing and selecting stereo 3D content. The conducted user study into perception tolerance suggests that users can tolerate disparity errors to a certain degree, where the level of tolerance varies with perceived distance from the screen. In addition, the study demonstrates that for a typical interaction task, reduction of accuracy is proportional to the disparity level of targeted 3D objects

Analysing Animal Behaviour in Wildlife Videos Using Face Detection and Tracking

This paper presents an algorithm that categorises animal locomotive behaviour by combining detection and tracking of animal faces in wildlife videos. As an example, the algorithm is applied to lion faces

A Multiresolution Technique for Video Indexing and Retrieval

This paper presents a novel approach to the multiresolution analysis and scalability in video indexing and retrieval. A scalable algorithm for video parsing and key-frame extraction is introduced. The technique is based on real-time analysis of MPEG motion variables and scalable metrics simplification by discrete contour evolution. Furthermore, a hierarchical key-frame retrieval method using scalable colour histogram analysis is presented. It offers customisable levels of detail in the descriptor space, where the relevance order is determined by degradation of the image, and not by degradation of the image histogram. To assess the performance of the approach several experiments have been conducted. Selected results are reported in this paper. 1

Efficient Key-Frame Extraction and Video Analysis

Content based video indexing and retrieval has its foundations in the analyses of the prime video temporal structures. Consequently, technologies for video segmentation and key-frame extraction have become crucial for the development of advanced digital video systems. Conventional algorithms for video partitioning and keyframe extraction are mainly implemented autonomously. By focusing the analysis on the compressed video features, this paper introduces a real-time algorithm for scene change detection and key-frame extraction that generates the frame difference metrics by analysing statistics of the macro-block features extracted from the MPEG compressed stream. The key-frame extraction method is implemented using difference metrics curve simplification by discrete contour evolution algorithm. This approach resulted in a fast and robust algorithm. Results of computer simulations are reported. 1

Temporal Segmentation of MPEG Video Streams

Many algorithms for temporal video partitioning rely on the analysis of uncompressed video features. Since the information relevant to the partitioning process can be extracted directly from the MPEG compressed stream, higher efficiency can be achieved utilizing information from the MPEG compressed domain. This paper introduces a real-time algorithm for scene change detection that analyses the statistics of the macroblock features extracted directly from the MPEG stream. A method for extraction of the continuous frame difference that transforms the 3D video stream into a 1D curve is presented. This transform is then further employed to extract temporal units within the analysed video sequence. Results of computer simulations are reported

Real-time Face Detection and Tracking of Animals

This paper presents a real-time method for extracting information about the locomotive activity of animals in wildlife videos by detecting and tracking the animals' faces. As an example application, the system is trained on lions. The underlying detection strategy is based on the concepts used in the Viola-Jones detector [1], an algorithm that was originally used for human face detection utilising Haar-like features and AdaBoost classifiers. Smooth and accurate tracking is achieved by integrating the detection algorithm with a low-level feature tracker. A specific coherence model that dynamically estimates the likelihood of the actual presence of an animal based on temporal confidence accumulation is employed to ensure a reliable and temporally continuous detection/tracking capability. The information generated by the tracker can be used to automatically classify and annotate basic locomotive behaviours in wildlife video repositories. © 2006 IEE

Towards real-time shot detection in the mpeg-compressed domain

As content based video indexing and retrieval has its foundations in the prime video structures, such as a shot or a scene, the algorithms for video partitioning have become crucial in contemporary development of digital video technology. Conventional algorithms for video partitioning mainly focus on the analysis of compressed video features, since the information relevant to the partitioning process can be extracted directly from the MPEG compressed stream and used for the detection of shot boundaries. However, most of the proposed algorithms do not show real time capabilities that are essential for video applications. This paper introduces a real time algorithm for cut detection. It analyses the statistics of the features extracted from the MPEG compressed stream, such as the macroblock type, and extends the same metrics to algorithms for gradual change detection. Our analysis led to a fast and robust algorithm for cut detection. Future research will be directed towards the use of the same concept for improving the real-time gradual change detection algorithms. Results of computer simulations are reported. 1