Reducing 3D video coding complexity through more efficient disparity estimation

3D video coding for transmission exploits the Disparity Estimation (DE) to remove the inter-view redundancies present within both the texture and the depth map multi-view videos. Good estimation accuracy can be achieved by partitioning the macro-block into smaller subblocks partitions. However, the DE process must be performed on each individual sub-block to determine the optimal mode and their disparity vectors, in terms of ratedistortion efficiency. This vector estimation process is heavy on computational resources, thus, the coding computational cost becomes proportional to the number of search points and the inter-view modes tested during the rate-distortion optimization. In this paper, a solution that exploits the available depth map data, together with the multi-view geometry, is proposed to identify a better DE search area; such that it allows a reduction in its search points. It also exploits the number of different depth levels present within the current macro-block to determine which modes can be used for DE to further reduce its computations. Simulation results demonstrate that this can save up to 95% of the encoding time, with little influence on the coding efficiency of the texture and the depth map multi-view video coding. This makes 3D video coding more practical for any consumer devices, which tend to have limited computational power.peer-reviewe

Fast inter-mode decision in multi-view video plus depth coding

Motion and disparity estimations are employed in Multi-view Video Coding (MVC) to remove redundancies present between temporal and different viewpoint frames, respectively, in both the color and the depth multi-view videos. These constitute the major computational expensive tasks of the video encoder, as iterative search for the optimal mode and its appropriate compensation vectors is employed to reduce the Rate-Distortion Optimization (RDO) cost function. This paper proposes a solution to limit the number of modes that are tested for RDO to encode the inter-view predicted views. The decision is based on the encoded information obtained from the corresponding Macroblock in the Base view, identified accurately by using the multi-view geometry together with the depth data. Results show that this geometric technique manages to reduce about 70% of the estimation's computational time and can also be used with fast geometric estimations to reduce up to 95% of the original encoding time. These gains are obtained with little degradation on the multi-view video quality for both color and depth MVC.peer-reviewe

Exploiting depth information for fast multi-view video coding

This research work is partially funded by the Strategic Educational Pathways Scholarship Scheme (STEPS-Malta). This scholarship is partly financed by the European Union – European Social Fund (ESF 1.25).Multi-view video coding exploits inter-view redundancies to compress the video streams and their associated depth information. These techniques utilize disparity estimation techniques to obtain disparity vectors (DVs) across different views. However, these methods contribute to the majority of the computational power needed for multi-view video encoding. This paper proposes a solution for fast disparity estimation based on multi-view geometry and depth information. A DV predictor is first calculated followed by an iterative or a fast search estimation process which finds the optimal DV in the search area dictated by the predictor. Simulation results demonstrate that this predictor is reliable enough to determine the area of the optimal DVs to allow a smaller search range. Furthermore, results show that the proposed approach achieves a speedup of 2.5 while still preserving the original rate-distortion performance.peer-reviewe

Improved depth maps coding efficiency of 3D videos

The research work disclosed in this publication is partially funded by the Strategic Educational Pathways Scholarship Scheme (Malta). The scholarship is part-financed by the European Union – European Social Fund.Immersive 3D video services demand the transmission of the viewpoints' depth map together with the texture multiview video to allow arbitrary reconstruction of intermediate viewpoints required for free-view navigation and 3D depth perception. The Multi-view Video Coding (MVC) standard is generally used to encode these auxiliary depth maps and since their estimation process is highly computational intensive, the coding time increases. This paper proposes a technique that exploits the multi-view geometry together with the depth map itself to calculate more accurate initial compensation vectors for the Macro-blocks' estimation. Starting from a more accurate position allows for a smaller search area, reducing the computations required during depth map MVC. Furthermore, the SKIP mode is extended to predict also the disparity vectors from the neighborhood encoded vectors, to omit some of them from transmission. Results demonstrate that these modifications provide an average computational reduction of up-to 87% with a bitrate saving of about 8.3% while encoding an inter-view predicted viewpoint from a depth map multi-view video.peer-reviewe

Error concealment techniques for H.264/MVC encoded sequences

This work is partially funded by the Strategic Educational Pathways Scholarship Scheme (STEPS-Malta). This scholarship is partly financed by the European Union–European Social Fund (ESF 1.25).The H.264/MVC standard offers good compression ratios for multi-view sequences by exploiting spatial, temporal and interview image dependencies. This works well in error-free channels, however in the event of transmission errors, it leads to the propagation of the distorted macro-blocks, degrading the quality of experience of the user. This paper reviews the state-of-the-art error concealment solutions and proposes a low complexity concealment method that can be used with multi-view video coding. The error resilience techniques used to aid error concealment are also identified. Results obtained demonstrate that good multi-view video reconstruction can be obtained with this approach.peer-reviewe

Performance of enhanced error concealment techniques in multi-view video coding systems

This research work is partially funded by the Strategic Educational Pathways Scholarship Scheme (STEPS-Malta). This scholarship is partly financed by the European Union - European Social Fund (ESF 1.25).Transmission of multi-view video encoded bit-streams over error-prone channels demands robust error concealment techniques. This paper studies the performance of solutions that exploit the neighbourhood spatial, temporal and inter-view information for this scope. Furthermore, different boundary distortion measurements, motion compensation refinement and temporal error concealment of Anchor frames were exploited to improve the results obtained by the basic error concealment techniques. Results show that a gain in performance is obtained with the implementation of each independent concealment technique. Furthermore, Peak Signal-to-Noise Ratio (PSNR) gains of about 4dB relative to the standard were achieved when adopting a hybrid error concealment approach.peer-reviewe

Fast multi-view video plus depth coding with hierarchical bi-prediction

This research work is partially funded by STEPS-Malta and partially by the EU–ESF 1.25.The Multi-view Video Coding (MVC) standard was developed for efficient encoding of multi-view videos. Part of it requires the calculation of both disparity and motion estimations using a bi-prediction structure. These estimations involve an exhaustive search for the optimal compensation vectors from multiple forward and backward reference frames which, while being very efficient in terms of compression, results in high computational costs. This paper proposes a solution that utilizes the multi-view geometry along with the available depth data, to calculate more accurate predictors for both motion and disparity estimations, and for both directions of the prediction structure. Simulation results demonstrate that this technique is reliable enough to allow a substantial reduction in the search areas in all the reference frames. This in turn results in a significant speed-up gain of 3.2 times with a negligible influence on the coding efficiency, while encoding both the color and the depth MVVs.peer-reviewe

Exploiting depth information for fast motion and disparity estimation in multi-view video coding

This research work is partially funded by the Strategic Educational Pathways Scholarship Scheme (STEPS-Malta). This scholarship is partly financed by the European Union – European Social Fund (ESF 1.25).Multi-view Video Coding (MVC) employs both motion and disparity estimation within the encoding process. These provide a significant increase in coding efficiency at the expense of a substantial increase in computational requirements. This paper presents a fast motion and disparity estimation technique that utilizes the multi-view geometry together with the depth information and the corresponding encoded motion vectors from the reference view, to produce more reliable motion and disparity vector predictors for the current view. This allows for a smaller search area which reduces the computational cost of the multi-view encoding system. Experimental results confirm that the proposed techniques can provide a speed-up gain of up to 4.2 times, with a negligible loss in the rate-distortion performance for both the color and the depth MVC.peer-reviewe