Frame Interpolation for Cloud-Based Mobile Video Streaming

© 2016 IEEE. Cloud-based High Definition (HD) video streaming is becoming popular day by day. On one hand, it is important for both end users and large storage servers to store their huge amount of data at different locations and servers. On the other hand, it is becoming a big challenge for network service providers to provide reliable connectivity to the network users. There have been many studies over cloud-based video streaming for Quality of Experience (QoE) for services like YouTube. Packet losses and bit errors are very common in transmission networks, which affect the user feedback over cloud-based media services. To cover up packet losses and bit errors, Error Concealment (EC) techniques are usually applied at the decoder/receiver side to estimate the lost information. This paper proposes a time-efficient and quality-oriented EC method. The proposed method considers H.265/HEVC based intra-encoded videos for the estimation of whole intra-frame loss. The main emphasis in the proposed approach is the recovery of Motion Vectors (MVs) of a lost frame in real-time. To boost-up the search process for the lost MVs, a bigger block size and searching in parallel are both considered. The simulation results clearly show that our proposed method outperforms the traditional Block Matching Algorithm (BMA) by approximately 2.5 dB and Frame Copy (FC) by up to 12 dB at a packet loss rate of 1%, 3%, and 5% with different Quantization Parameters (QPs). The computational time of the proposed approach outperforms the BMA by approximately 1788 seconds

Cost and Coding Efficient Motion Estimation Design Considerations for High Efficiency Video Coding (HEVC) Standard

This paper focuses on motion estimation engine design in future high-efficiency video coding (HEVC) encoders. First, a methodology is explained to analyze hardware implementation cost in terms of hardware area, memory size and memory bandwidth for various possible motion estimation engine designs. For 11 different configurations, hardware cost as well as the coding efficiency are quantified and are compared through a graphical analysis to make design decisions. It has been shown that using smaller block sizes (e.g. 4 × 4) imposes significantly larger hardware requirements at the expense of modest improvements in coding efficiency. Secondly, based on the analysis on various configurations, one configuration is chosen and algorithm improvements are presented to further reduce hardware implementation cost of the selected configuration. Overall, the proposed changes provide 56 × on-chip bandwidth, 151 × off-chip bandwidth, 4.3 × core area and 4.5 × on-chip memory area savings when compared to the hardware implementation of the HM-3.0 design.Texas Instruments Incorporate

Implementation of a motion estimation algorithm for Intel FPGAs using OpenCL

Producción CientíficaMotion Estimation is one of the main tasks behind any video encoder. It is a compu- tationally costly task; therefore, it is usually delegated to specific or reconfigurable hardware, such as FPGAs. Over the years, multiple FPGA implementations have been developed, mainly using hardware description languages such as Verilog or VHDL. Since programming using hardware description languages is a complex task, it is desirable to use higher-level languages to develop FPGA applications.The aim of this work is to evaluate OpenCL, in terms of expressiveness, as a tool for devel- oping this kind of FPGA applications. To do so, we present and evaluate a parallel implementation of the Block Matching Motion Estimation process using OpenCL for Intel FPGAs, usable and tested on an Intel Stratix 10 FPGA. The implementa- tion efficiently processes Full HD frames completely inside the FPGA. In this work, we show the resource utilization when synthesizing the code on an Intel Stratix 10 FPGA, as well as a performance comparison with multiple CPU implementations with varying levels of optimization and vectorization capabilities. We also compare the proposed OpenCL implementation, in terms of resource utilization and perfor- mance, with estimations obtained from an equivalent VHDL implementation.Junta de Castilla y León - Consejería de Educación de la Proyecto PROPHET-2 (VA226P20)Ministerio de Economía, Industria y Competitividad: (PID2019- 104834 GB-I00) and European Regional Development Fund (ERDF) program: Project PCAS (TIN2017-88614-R)Ministerio de Ciencia e Innovación (PID2019-104184RB-I00 / AEI / 10.13039/501100011033)Xunta de Galicia y fondos FEDER de la UE (Centro de Investigación de Galicia acreditación 2019-2022, ref. ED431G 2019/01; Consolidation Program of Competitive Reference Groups, ref. ED431C 2021/30Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación y “European Union NextGenerationEU/PRTR” : (MCIN/ AEI/10.13039/501100011033) - grant TED2021-130367B-I00Publicación en abierto financiada por el Consorcio de Bibliotecas Universitarias de Castilla y León (BUCLE), con cargo al Programa Operativo 2014ES16RFOP009 FEDER 2014-2020 DE CASTILLA Y LEÓN, Actuación:20007-CL - Apoyo Consorcio BUCL

An FPGA Implementation of HW/SW Codesign Architecture for H.263 Video Coding

Chapitre 12 http://www.intechopen.com/download/pdf/pdfs_id/1574

A Survey on Block Matching Algorithms for Video Coding

Block matching algorithm (BMA) for motion estimation (ME) is the heart to many motion-compensated video-coding techniques/standards, such as ISO MPEG-1/2/4 and ITU-T H.261/262/263/264/265, to reduce the temporal redundancy between different frames. During the last three decades, hundreds of fast block matching algorithms have been proposed. The shape and size of search patterns in motion estimation will influence more on the searching speed and quality of performance. This article provides an overview of the famous block matching algorithms and compares their computational complexity and motion prediction quality

Parallel H.264/AVC Fast Rate-Distortion Optimized Motion Estimation using Graphics Processing Unit and Dedicated Hardware

Heterogeneous systems on a single chip composed of CPU, Graphical Processing Unit (GPU), and Field Programmable Gate Array (FPGA) are expected to emerge in near future. In this context, the System on Chip (SoC) can be dynamically adapted to employ different architectures for execution of data-intensive applications. Motion estimation is one such task that can be accelerated using FPGA and GPU for high performance H.264/AVC encoder implementation. In most of works on parallel implementation of motion estimation, the bit rate cost of motion vectors is generally ignored. On the contrary, this paper presents a fast rate-distortion optimized parallel motion estimation algorithm implemented on GPU using OpenCL and FPGA/ASIC using VHDL. The predicted motion vectors are estimated from temporally preceding motion vectors and used for evaluating the bit rate cost of the motion vectors simultaneously. The experimental results show that the proposed scheme achieves significant speedup on GPU and FPGA, and has comparable ratedistortion performance with respect to sequential fast motion estimation algorithm

An Investigation of Block Searching Algorithms for Video Frame Codecs

Block matching is the most computationally demanding aspect of the video encoding process. In many applications real-time video encoding is desired and therefore it is important that the encoding is fast. Also where handheld devices such as a PDA or mobile phone are concerned a less computationally intensive algorithm means a simpler processor can be used which saves on hardware costs and also extends battery life. An optimised algorithm also allows these devices to be used in low bandwidth wireless networks. The challenge is to decrease the computational load on the system without compromising the quality of the video stream too much, thus enabling easier and less expensive implementations of real-time encoding. This thesis appraises some of the principal Block Search Algorithms used in Video compression today. This work follows on from the work of Aroh Barjatya who implemented 7 common Block Search Algorithms to predict P-frames in MATLAB. Three further hybrid DS algorithms are implemented in MATLAB. Additional code is added to produce plots of the main metrics and to calculate some statistics such as Average Searching Points, Average PSNR and the Speed Improvement Ratio with respect to the Diamond Search and the Exhaustive Search. For a comparative analysis with previous studies 3 standard industry test sequences are used. The first sequence, Miss America is a typical videoconferencing scene with limited object motion and a stationary background. The second sequence, Flower Garden consists mainly of stationary objects, but with a fast camera panning motion. The third sequence, Football contains large local object motion. The performance of the 3 implemented algorithms were assessed by the aforementioned statistics. Simulation results showed that the NCDS was the fastest algorithm amongst the 3 hybrid DS algorithms simulated. A speedup ranging from 10% for the complex motion sequence Flower Garden to nearly 54% for the low motion video conferencing sequence Miss America was recorded. All 3 algorithms performed very competitively in terms of PSNR compared to the DS even though they use a lower number of search points on average. It was shown that the NCDS has marginally worse PSNR performance than the DS compared to the other 2 algorithms – the highest being a drop in PSNR of 0.680dB for the Flower Garden sequence. However, the speed improvements for NCDS are quite substantial and thus would justify its use over the DS. The results from the implementation concurred with the literature therefore validating the implementation. The implementation was used as a guide in nominating a ‘robust’ Block Search Algorithm. When the DS, CDS, SCDS and the NCDS were compared with ARPS it was shown that ARPS generally gave both higher PSNR and higher search speed for all 3 sequences. The reason for the good performance of ARPS is that it quickly directs the search into the local region of the global minimum by calculating the Predicted Motion Vector. The minimum error from a rood pattern of nodes is found and then a final refined search calculates the motion vector. Simulation results showed that ARPS was the best algorithm amongst the 10 algorithms simulated from the point of view of speed (lowest number of search points used per macroblock) and video quality (PSNR). For real-time encoding of video the best fast block motion algorithm to advise is ARPS

A Three-Point Directional Search Block Matching Algorithm

This paper proposes compact directional asymmetric search patterns, which we have named as three-point directional search (TDS). In most fast search motion estimation algorithms, a symmetric search pattern is usually set at the minimum block distortion point at each step of the search. The design of the symmetrical pattern in these algorithms relies primarily on the assumption that the direction of convergence is equally alike in each direction with respect to the search center. Therefore, the monotonic property of real-world video sequences is not properly used by these algorithms. The strategy of TDS is to keep searching for the minimum block distortion point in the most probable directions, unlike the previous fast search motion estimation algorithms where all the directions are checked. Therefore, the proposed method significantly reduces the number of search points for locating a motion vector. Compared to conventional fast algorithms, the proposed method has the fastest search speed and most satisfactory PSNR values for all test sequences