Adaptive motion estimation algorithm and hardware designs for H.264 multiview video coding

Multiview Video Coding (MVC) is the process of efficiently compressing stereo (2 views) or multiview video signals. The improved compression efficiency achieved by H.264 MVC comes with a significant increase in computational complexity. Therefore, in this thesis, we propose novel techniques for significantly reducing the amount of computations performed by full search motion estimation algorithm for H.264 MVC, and therefore significantly reducing the energy consumption of full search motion estimation hardware for H.264 MVC with very small PSNR loss and bitrate increase. We also propose an adaptive fast motion estimation algorithm for reducing the amount of computations performed by H.264 MVC motion estimation, and therefore reducing the energy consumption of H.264 MVC motion estimation hardware even more with additional very small PSNR loss and bitrate increase. We also propose an adaptive H.264 MVC motion estimation hardware for implementing the proposed adaptive fast motion estimation algorithm. The proposed motion estimation hardware is implemented in Verilog HDL and mapped to a Xilinx Virtex-6 FPGA. The proposed motion estimation hardware has less energy consumption than the full search motion estimation hardware for H.264 MVC and the full search motion estimation hardware for H.264 MVC including the proposed computation reduction techniques

High-Level Synthesis Based VLSI Architectures for Video Coding

High Efficiency Video Coding (HEVC) is state-of-the-art video coding standard. Emerging applications like free-viewpoint video, 360degree video, augmented reality, 3D movies etc. require standardized extensions of HEVC. The standardized extensions of HEVC include HEVC Scalable Video Coding (SHVC), HEVC Multiview Video Coding (MV-HEVC), MV-HEVC+ Depth (3D-HEVC) and HEVC Screen Content Coding. 3D-HEVC is used for applications like view synthesis generation, free-viewpoint video. Coding and transmission of depth maps in 3D-HEVC is used for the virtual view synthesis by the algorithms like Depth Image Based Rendering (DIBR). As first step, we performed the profiling of the 3D-HEVC standard. Computational intensive parts of the standard are identified for the efficient hardware implementation. One of the computational intensive part of the 3D-HEVC, HEVC and H.264/AVC is the Interpolation Filtering used for Fractional Motion Estimation (FME). The hardware implementation of the interpolation filtering is carried out using High-Level Synthesis (HLS) tools. Xilinx Vivado Design Suite is used for the HLS implementation of the interpolation filters of HEVC and H.264/AVC. The complexity of the digital systems is greatly increased. High-Level Synthesis is the methodology which offers great benefits such as late architectural or functional changes without time consuming in rewriting of RTL-code, algorithms can be tested and evaluated early in the design cycle and development of accurate models against which the final hardware can be verified

Pengurangan Kompleksitas Komputasi Pada Multiview HEVC Berbasis Perangkat FPGA

Dengan meningkatnya kualitas dan resolusi konten video, terutama video 3D, kompleksitas komputasi dalam pemrosesannya juga meningkat secara signifikan. Salah satu standar video yang populer, HEVC memiliki ekstensi yang dinamakan Multiview HEVC (MV-HEVC) dan 3D-HEVC dengan jumlah data dan resolusi yang tinggi, mengakibatkan adanya peningkatan kompleksitas komputasi. Penelitian ini bertujuan mengurangi kompleksitas komputasi dari video MV-HEVC dengan menerapkan mode decision berupa ECU, CFM, ESD, dan deblocking filter yang diujicobakan pada platform PC berbasis Linux dan platform Xilinx All Programmable SoC. Dari hasil eksperimen didapatkan pengurangan kompleksitas komputasi yang dilihat dari perbandingan dari waktu encoding. Platform Xilinx All Programmable SoC mampu memperoleh waktu encoding yang lebih cepat 35,85% daripada PC berbasis Linux. Selanjutnya untuk kualitas video yang dihasilkan antara kedua platform tersebut hampir sama dilihat dari nilai bitrate dan PSNR. =========================================================================== Due to the increasing quality and resolution of videocontent, especially 3D video, the computational complexity forits processing also significantly increases. One of the popularformat, HEVC has extensions called Multiview HEVC (MV-HEVC) and 3D-HEVC with high amounts of data and highresolution that resulting in increased computational complexity.This study aims to reduce the computational complexity of MVHEVC videos by implementing mode decision such as ECU,CFM, ESD, and deblocking filterswhich are tested on Linuxbased PC platforms and the Xilinx All Programmable SoCplatform. From the experimental results obtained the reductionin computational complexity can be seen from the comparisonofencoding time, the Xilinx All Programmable SoC platform isable to obtain encoding times that are faster than Linux-basedPCs. For the quality of the video produced between the two theplatform is not significant from the bitrate and PSNR values

Algorithm/Architecture Co-Exploration of Visual Computing: Overview and Future Perspectives

Concurrently exploring both algorithmic and architectural optimizations is a new design paradigm. This survey paper addresses the latest research and future perspectives on the simultaneous development of video coding, processing, and computing algorithms with emerging platforms that have multiple cores and reconfigurable architecture. As the algorithms in forthcoming visual systems become increasingly complex, many applications must have different profiles with different levels of performance. Hence, with expectations that the visual experience in the future will become continuously better, it is critical that advanced platforms provide higher performance, better flexibility, and lower power consumption. To achieve these goals, algorithm and architecture co-design is significant for characterizing the algorithmic complexity used to optimize targeted architecture. This paper shows that seamless weaving of the development of previously autonomous visual computing algorithms and multicore or reconfigurable architectures will unavoidably become the leading trend in the future of video technology

Homogeneous and heterogeneous MPSoC architectures with network-on-chip connectivity for low-power and real-time multimedia signal processing

Two multiprocessor system-on-chip (MPSoC) architectures are proposed and compared in the paper with reference to audio and video processing applications. One architecture exploits a homogeneous topology; it consists of 8 identical tiles, each made of a 32-bit RISC core enhanced by a 64-bit DSP coprocessor with local memory. The other MPSoC architecture exploits a heterogeneous-tile topology with on-chip distributed memory resources; the tiles act as application specific processors supporting a different class of algorithms. In both architectures, the multiple tiles are interconnected by a network-on-chip (NoC) infrastructure, through network interfaces and routers, which allows parallel operations of the multiple tiles. The functional performances and the implementation complexity of the NoC-based MPSoC architectures are assessed by synthesis results in submicron CMOS technology. Among the large set of supported algorithms, two case studies are considered: the real-time implementation of an H.264/MPEG AVC video codec and of a low-distortion digital audio amplifier. The heterogeneous architecture ensures a higher power efficiency and a smaller area occupation and is more suited for low-power multimedia processing, such as in mobile devices. The homogeneous scheme allows for a higher flexibility and easier system scalability and is more suited for general-purpose DSP tasks in power-supplied devices

High Performance Multiview Video Coding

Following the standardization of the latest video coding standard High Efficiency Video Coding in 2013, in 2014, multiview extension of HEVC (MV-HEVC) was published and brought significantly better compression performance of around 50% for multiview and 3D videos compared to multiple independent single-view HEVC coding. However, the extremely high computational complexity of MV-HEVC demands significant optimization of the encoder. To tackle this problem, this work investigates the possibilities of using modern parallel computing platforms and tools such as single-instruction-multiple-data (SIMD) instructions, multi-core CPU, massively parallel GPU, and computer cluster to significantly enhance the MVC encoder performance. The aforementioned computing tools have very different computing characteristics and misuse of the tools may result in poor performance improvement and sometimes even reduction. To achieve the best possible encoding performance from modern computing tools, different levels of parallelism inside a typical MVC encoder are identified and analyzed. Novel optimization techniques at various levels of abstraction are proposed, non-aggregation massively parallel motion estimation (ME) and disparity estimation (DE) in prediction unit (PU), fractional and bi-directional ME/DE acceleration through SIMD, quantization parameter (QP)-based early termination for coding tree unit (CTU), optimized resource-scheduled wave-front parallel processing for CTU, and workload balanced, cluster-based multiple-view parallel are proposed. The result shows proposed parallel optimization techniques, with insignificant loss to coding efficiency, significantly improves the execution time performance. This , in turn, proves modern parallel computing platforms, with appropriate platform-specific algorithm design, are valuable tools for improving the performance of computationally intensive applications

Error resilient H.264 coded video transmission over wireless channels

The H.264/AVC recommendation was first published in 2003 and builds on the concepts of earlier standards such as MPEG-2 and MPEG-4. The H.264 recommendation represents an evolution of the existing video coding standards and was developed in response to the growing need for higher compression. Even though H.264 provides for greater compression, H.264 compressed video streams are very prone to channel errors in mobile wireless fading channels such as 3G due to high error rates experienced. Common video compression techniques include motion compensation, prediction methods, transformation, quantization and entropy coding, which are the common elements of a hybrid video codecs. The ITU-T recommendation H.264 introduces several new error resilience tools, as well as several new features such as Intra Prediction and Deblocking Filter. The channel model used for the testing was the Rayleigh Fading channel with the noise component simulated as Additive White Gaussian Noise (AWGN) using QPSK as the modulation technique. The channel was used over several Eb/N0 values to provide similar bit error rates as those found in the literature. Though further research needs to be conducted, results have shown that when using the H.264 error resilience tools in protecting encoded bitstreams to minor channel errors improvement in the decoded video quality can be observed. The tools did not perform as well with mild and severe channel errors significant as the resultant bitstream was too corrupted. From this, further research in channel coding techniques is needed to determine if the bitstream can be protected from these sorts of error rate