3D high definition video coding on a GPU-based heterogeneous system

H.264/MVC is a standard for supporting the sensation of 3D, based on coding from 2 (stereo) to N views. H.264/MVC adopts many coding options inherited from single view H.264/AVC, and thus its complexity is even higher, mainly because the number of processing views is higher. In this manuscript, we aim at an efficient parallelization of the most computationally intensive video encoding module for stereo sequences. In particular, inter prediction and its collaborative execution on a heterogeneous platform. The proposal is based on an efficient dynamic load balancing algorithm and on breaking encoding dependencies. Experimental results demonstrate the proposed algorithm's ability to reduce the encoding time for different stereo high definition sequences. Speed-up values of up to 90× were obtained when compared with the reference encoder on the same platform. Moreover, the proposed algorithm also provides a more energy-efficient approach and hence requires less energy than the sequential reference algorith

Fast Motion Estimation Algorithms for Block-Based Video Coding Encoders

The objective of my research is reducing the complexity of video coding standards in real-time scalable and multi-view applications

Overview of 3D Video: Coding Algorithms, Implementations and Standardization

Projecte final de carrera fet en col.laboració amb Linköping Institute of TechnologyEnglish: 3D technologies have aroused a great interest over the world in the last years. Television, cinema and videogames are introducing, little by little, 3D technologies into the mass market. This comes as a result of the research done in the 3D field, solving many of its limitations such as quality, contents creation or 3D displays. This thesis focus on 3D video, considering concepts that concerns the coding issues and the video formats. The aim is to provide an overview of the current state of 3D video, including the standardization and some interesting implementations and alternatives that exist. In the report necessary background information is presented in order to understand the concepts developed: compression techniques, the different video formats, their standardization and some advances or alternatives to the processes previously explained. Finally, a comparison between the different concepts is presented to complete the overview, ending with some conclusions and proposed ideas for future works.Castellano: Las tecnologías 3D han despertado un gran interés en todo el mundo en los últimos años. Televisión, cine y videojuegos están introduciendo, poco a poco, ésta tecnología en el mercado. Esto es resultado de la investigación realizada en el campo de las 3D, solucionando muchas de sus limitaciones, como la calidad, la creación de contenidos o las pantallas 3D. Este proyecto se centra en el video 3D, considerando los conceptos relacionados con la codificación y los formatos de vídeo. El objetivo es proporcionar una visión del estado actual del vídeo 3D, incluyendo los estándares y algunas de las implementaciones más interesantes que existen. En la memoria, se presenta información adicional para facilitar el seguimiento de los conceptos desarrollados: técnicas de compresión, formatos de vídeo, su estandarización y algunos avances o alternativas a los procesos explicados. Finalmente, se presentan diferentes comparaciones entre los conceptos tratados, acabando el documento con las conclusiones obtenidas e ideas propuestas para futuros trabajos.Català: Les tecnologies 3D han despertat un gran interès a tot el món en els últims anys. Televisió, cinema i videojocs estan introduint, lentament, aquesta tecnologia en el mercat. Això és resultat de la investigació portada a terme en el camp de les 3D, solucionant moltes de les seves limitacions, com la qualitat, la creació de continguts o les pantalles 3D. Aquest proyecte es centra en el video 3D, considerant els conceptes relacionats amb la codificació i els formats de video. L'objectiu és proporcionar una visió de l'estat actual del video 3D, incloent-hi els estandàrds i algunes de les implementacions més interessants que existeixen. A la memòria, es presenta informació adicional per facilitar el seguiment dels conceptes desenvolupats: tècniques de compressió, formats de video, la seva estandardització i alguns avenços o alternatives als procesos explicats. Finalment, es presenten diferents comparacions entre els conceptes tractats i les conclusions obtingudes, juntament amb propostes per a futurs treballs

High Performance Multiview Video Coding

Following the standardization of the latest video coding standard High Efficiency Video Coding in 2013, in 2014, multiview extension of HEVC (MV-HEVC) was published and brought significantly better compression performance of around 50% for multiview and 3D videos compared to multiple independent single-view HEVC coding. However, the extremely high computational complexity of MV-HEVC demands significant optimization of the encoder. To tackle this problem, this work investigates the possibilities of using modern parallel computing platforms and tools such as single-instruction-multiple-data (SIMD) instructions, multi-core CPU, massively parallel GPU, and computer cluster to significantly enhance the MVC encoder performance. The aforementioned computing tools have very different computing characteristics and misuse of the tools may result in poor performance improvement and sometimes even reduction. To achieve the best possible encoding performance from modern computing tools, different levels of parallelism inside a typical MVC encoder are identified and analyzed. Novel optimization techniques at various levels of abstraction are proposed, non-aggregation massively parallel motion estimation (ME) and disparity estimation (DE) in prediction unit (PU), fractional and bi-directional ME/DE acceleration through SIMD, quantization parameter (QP)-based early termination for coding tree unit (CTU), optimized resource-scheduled wave-front parallel processing for CTU, and workload balanced, cluster-based multiple-view parallel are proposed. The result shows proposed parallel optimization techniques, with insignificant loss to coding efficiency, significantly improves the execution time performance. This , in turn, proves modern parallel computing platforms, with appropriate platform-specific algorithm design, are valuable tools for improving the performance of computationally intensive applications

Low Complexity Multiview Video Coding

3D video is a technology that has seen a tremendous attention in the recent years. Multiview Video Coding (MVC) is an extension of the popular H.264 video coding standard and is commonly used to compress 3D videos. It offers an improvement of 20% to 50% in compression efficiency over simulcast encoding of multiview videos using the conventional H.264 video coding standard. However, there are two important problems associated with it: (i) its superior compression performance comes at the cost of significantly higher computational complexity which hampers the real-world realization of MVC encoder in applications such as 3D live broadcasting and interactive Free Viewpoint Television (FTV), and (ii) compressed 3D videos can suffer from packet loss during transmission, which can degrade the viewing quality of the 3D video at the decoder. This thesis aims to solve these problems by presenting techniques to reduce the computational complexity of the MVC encoder and by proposing a consistent error concealment technique for frame losses in 3D video transmission. The thesis first analyses the complexity of the MVC encoder. It then proposes two novel techniques to reduce the complexity of motion and disparity estimation. The first method achieves complexity reduction in the disparity estimation process by exploiting the relationship between temporal levels, type of macroblocks and search ranges while the second method achieves it by exploiting the geometrical relation- ship between motion and disparity vectors in stereo frames. These two methods are then combined with other state-of-the-art methods in a unique framework where gains add up. Experimental results show that the proposed low-complexity framework can reduce the encoding time of the standard MVC encoder by over 93% while maintaining similar compression efficiency performance. The addition of new View Synthesis Prediction (VSP) modes to the MVC encoding framework improves the compression efficiency of MVC. However, testing additional modes comes at the cost of increased encoding complexity. In order to reduce the encoding complexity, the thesis, next, proposes a bayesian early mode decision technique for a VSP enhanced MVC coder. It exploits the statistical similarities between the RD costs of the VSP SKIP mode in neighbouring views to terminate the mode decision process early. Results indicate that the proposed technique can reduce the encoding time of the enhanced MVC coder by over 33% at similar compression efficiency levels. Finally, compressed 3D videos are usually required to be broadcast to a large number of users where transmission errors can lead to frame losses which can degrade the video quality at the decoder. A simple reconstruction of the lost frames can lead to inconsistent reconstruction of the 3D scene which may negatively affect the viewing experience of a user. In order to solve this problem, the thesis proposes, at the end, a consistency model for recovering frames lost during transmission. The proposed consistency model is used to evaluate inter-view and temporal consistencies while selecting candidate blocks for concealment. Experimental results show that the proposed technique is able to recover the lost frames with high consistency and better quality than two standard error concealment methods and a baseline technique based on the boundary matching algorithm

Fast mode decision for Multiview Video Coding based on scene geometry

A new fast mode decision (FMD) algorithm for multi-view video coding (MVC) is presented. The codification of the views is based on the analysis of the homogeneity of the depth map and corrected with the motion analysis of a reference view, which is encoded based on traditional methods and on the use of the disparity differences between the views. This approach reduces the burden of the rate-distortion motion analysis using the availability of a depth map and the presence of the disparity vectors, which are assumed to be provided by the acquisition proces

Fast motion estimation algorithms for block-based video coding encoders

The objective of my research is reducing the complexity of video coding standards in real-time scalable and multi-view applications.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Low-Complexity Multiview Video Coding

We consider the problem of complexity reduction in Multiview Video Coding (MVC). We provide a unique comprehensive study that integrates and compares the different low complexity encoding techniques that have been proposed at different levels of the MVC system. In addition, we propose a novel complexity reduction method that takes advantage of the relationship between disparity vectors along time. The relationship is exploited with respect to the motion activity in the frame, as well as with the position of the frame in the Group of Pictures. We integrate this technique into our unique comprehensive framework and evaluate the performance of the resulting system in different setups. We show that the effective combination of complexity reduction techniques results in saving up to 93% in encoding time at the cost of only 0.08 dB in peak signal-to-noise ratio (PSNR) and 1.64% increase in bitrate compared to the standard MVC implementation (JMVM 6.0)

Pengurangan Kompleksitas Komputasi Pada Multiview HEVC Berbasis Perangkat FPGA

Dengan meningkatnya kualitas dan resolusi konten video, terutama video 3D, kompleksitas komputasi dalam pemrosesannya juga meningkat secara signifikan. Salah satu standar video yang populer, HEVC memiliki ekstensi yang dinamakan Multiview HEVC (MV-HEVC) dan 3D-HEVC dengan jumlah data dan resolusi yang tinggi, mengakibatkan adanya peningkatan kompleksitas komputasi. Penelitian ini bertujuan mengurangi kompleksitas komputasi dari video MV-HEVC dengan menerapkan mode decision berupa ECU, CFM, ESD, dan deblocking filter yang diujicobakan pada platform PC berbasis Linux dan platform Xilinx All Programmable SoC. Dari hasil eksperimen didapatkan pengurangan kompleksitas komputasi yang dilihat dari perbandingan dari waktu encoding. Platform Xilinx All Programmable SoC mampu memperoleh waktu encoding yang lebih cepat 35,85% daripada PC berbasis Linux. Selanjutnya untuk kualitas video yang dihasilkan antara kedua platform tersebut hampir sama dilihat dari nilai bitrate dan PSNR. =========================================================================== Due to the increasing quality and resolution of videocontent, especially 3D video, the computational complexity forits processing also significantly increases. One of the popularformat, HEVC has extensions called Multiview HEVC (MV-HEVC) and 3D-HEVC with high amounts of data and highresolution that resulting in increased computational complexity.This study aims to reduce the computational complexity of MVHEVC videos by implementing mode decision such as ECU,CFM, ESD, and deblocking filterswhich are tested on Linuxbased PC platforms and the Xilinx All Programmable SoCplatform. From the experimental results obtained the reductionin computational complexity can be seen from the comparisonofencoding time, the Xilinx All Programmable SoC platform isable to obtain encoding times that are faster than Linux-basedPCs. For the quality of the video produced between the two theplatform is not significant from the bitrate and PSNR values