Multi-view video coding via virtual view generation

In this paper, a multi-view video coding method via generation of virtual picture sequences is proposed. Pictures are synthesized for the sake of better exploitation of the redundancies between neighbouring views in a multi-view sequence. Pictures are synthesized through a 3D warping method to estimate certain views in a multi-view set. Depth map and associated colour video sequences are used for view generation and tests. H. 264/AVC coding standard based MVC draft software is used for coding colour videos and depth maps as well as certain views which are predicted from the virtually generated views. Results for coding these views with the proposed method are compared against the reference H. 264/AVC simulcast method under some low delay coding scenarios. The rate-distortion performance of the proposed method outperforms that of the reference method at all bit-rates

3D fatigue from stereoscopic 3D video displays: Comparing objective and subjective tests using electroencephalography

The use of stereoscopic display has increased in recent times, with a growing range of applications using 3D videos for visual entertainment, data visualization, and medical applications. However, stereoscopic 3D video can lead to adverse reactions amongst some viewers, including visual fatigue, headache and nausea; such reactions can further lead to Visually Induced Motion Sickness (VIMS). Whilst motion sickness symptoms can occur from other types of visual displays, this paper investigates the rapid adjustment triggered by human pupils as a potential cause of 3D fatigue due to VIMS from stereoscopic 3D displays. Using Electroencephalogram (EEG) biosignals and eye blink tools to measure the 3D fatigue, a series of objective and subjective experiments were conducted to investigate the effect of stereoscopic 3D across a series of video sequences

Fast multi-view video plus depth coding with hierarchical bi-prediction

This research work is partially funded by STEPS-Malta and partially by the EU–ESF 1.25.The Multi-view Video Coding (MVC) standard was developed for efficient encoding of multi-view videos. Part of it requires the calculation of both disparity and motion estimations using a bi-prediction structure. These estimations involve an exhaustive search for the optimal compensation vectors from multiple forward and backward reference frames which, while being very efficient in terms of compression, results in high computational costs. This paper proposes a solution that utilizes the multi-view geometry along with the available depth data, to calculate more accurate predictors for both motion and disparity estimations, and for both directions of the prediction structure. Simulation results demonstrate that this technique is reliable enough to allow a substantial reduction in the search areas in all the reference frames. This in turn results in a significant speed-up gain of 3.2 times with a negligible influence on the coding efficiency, while encoding both the color and the depth MVVs.peer-reviewe

Exploiting depth information for fast motion and disparity estimation in multi-view video coding

This research work is partially funded by the Strategic Educational Pathways Scholarship Scheme (STEPS-Malta). This scholarship is partly financed by the European Union – European Social Fund (ESF 1.25).Multi-view Video Coding (MVC) employs both motion and disparity estimation within the encoding process. These provide a significant increase in coding efficiency at the expense of a substantial increase in computational requirements. This paper presents a fast motion and disparity estimation technique that utilizes the multi-view geometry together with the depth information and the corresponding encoded motion vectors from the reference view, to produce more reliable motion and disparity vector predictors for the current view. This allows for a smaller search area which reduces the computational cost of the multi-view encoding system. Experimental results confirm that the proposed techniques can provide a speed-up gain of up to 4.2 times, with a negligible loss in the rate-distortion performance for both the color and the depth MVC.peer-reviewe

Exploiting depth information for fast multi-view video coding

This research work is partially funded by the Strategic Educational Pathways Scholarship Scheme (STEPS-Malta). This scholarship is partly financed by the European Union – European Social Fund (ESF 1.25).Multi-view video coding exploits inter-view redundancies to compress the video streams and their associated depth information. These techniques utilize disparity estimation techniques to obtain disparity vectors (DVs) across different views. However, these methods contribute to the majority of the computational power needed for multi-view video encoding. This paper proposes a solution for fast disparity estimation based on multi-view geometry and depth information. A DV predictor is first calculated followed by an iterative or a fast search estimation process which finds the optimal DV in the search area dictated by the predictor. Simulation results demonstrate that this predictor is reliable enough to determine the area of the optimal DVs to allow a smaller search range. Furthermore, results show that the proposed approach achieves a speedup of 2.5 while still preserving the original rate-distortion performance.peer-reviewe

Fast inter-mode decision in multi-view video plus depth coding

Motion and disparity estimations are employed in Multi-view Video Coding (MVC) to remove redundancies present between temporal and different viewpoint frames, respectively, in both the color and the depth multi-view videos. These constitute the major computational expensive tasks of the video encoder, as iterative search for the optimal mode and its appropriate compensation vectors is employed to reduce the Rate-Distortion Optimization (RDO) cost function. This paper proposes a solution to limit the number of modes that are tested for RDO to encode the inter-view predicted views. The decision is based on the encoded information obtained from the corresponding Macroblock in the Base view, identified accurately by using the multi-view geometry together with the depth data. Results show that this geometric technique manages to reduce about 70% of the estimation's computational time and can also be used with fast geometric estimations to reduce up to 95% of the original encoding time. These gains are obtained with little degradation on the multi-view video quality for both color and depth MVC.peer-reviewe

Improved depth maps coding efficiency of 3D videos

The research work disclosed in this publication is partially funded by the Strategic Educational Pathways Scholarship Scheme (Malta). The scholarship is part-financed by the European Union – European Social Fund.Immersive 3D video services demand the transmission of the viewpoints' depth map together with the texture multiview video to allow arbitrary reconstruction of intermediate viewpoints required for free-view navigation and 3D depth perception. The Multi-view Video Coding (MVC) standard is generally used to encode these auxiliary depth maps and since their estimation process is highly computational intensive, the coding time increases. This paper proposes a technique that exploits the multi-view geometry together with the depth map itself to calculate more accurate initial compensation vectors for the Macro-blocks' estimation. Starting from a more accurate position allows for a smaller search area, reducing the computations required during depth map MVC. Furthermore, the SKIP mode is extended to predict also the disparity vectors from the neighborhood encoded vectors, to omit some of them from transmission. Results demonstrate that these modifications provide an average computational reduction of up-to 87% with a bitrate saving of about 8.3% while encoding an inter-view predicted viewpoint from a depth map multi-view video.peer-reviewe

A novel view-level target bit rate distribution estimation technique for real-time multi-view video plus depth

This paper presents a novel view-level target bit rate distribution estimation technique for real-time Multi-view video plus depth using a statistical model that is based on the prediction mode distribution. Experiments using various standard test sequences show the efficacy of the technique, as the model manages to estimate online the view-level target bit rate distribution with an absolute mean estimation error of 2% and a standard deviation of 0.9%. Moreover, this technique provides adaptation of the view-level bit rate distribution providing scene change handling capability.peer-reviewe