A reconfigurable frame interpolation hardware architecture for high definition video

Since Frame Rate Up-Conversion (FRC) is started to be used in recent consumer electronics products like High Definition TV, real-time and low cost implementation of FRC algorithms has become very important. Therefore, in this paper, we propose a low cost hardware architecture for realtime implementation of frame interpolation algorithms. The proposed hardware architecture is reconfigurable and it allows adaptive selection of frame interpolation algorithms for each Macroblock. The proposed hardware architecture is implemented in VHDL and mapped to a low cost Xilinx XC3SD1800A-4 FPGA device. The implementation results show that the proposed hardware can run at 101 MHz on this FPGA and consumes 32 BRAMs and 15384 slices

Perception-oriented methodology for robust motion estimation design

Optimizing a motion estimator (ME) for picture rate conversion is challenging. This is because there are many types of MEs and, within each type, many parameters, which makes subjective assessment of all the alternatives impractical. To solve this problem, we propose an automatic design methodology that provides `well-performing MEs' from the multitude of options. Moreover, we prove that applying this methodology results in subjectively pleasing quality of the upconverted video, even while our objective performance metrics are necessarily suboptimal. This proof involved a user rating of 93 MEs in 3 video sequences. The 93 MEs were systematically selected from a total of 7000 ME alternatives. The proposed methodology may provide an inspiration for similar tough multi-dimensional optimization tasks with unreliable metrics

Improved Side Information Generation for Distributed Video Coding by Exploiting Spatial and Temporal Correlations

Distributed Video Coding (DVC) is a new paradigm in video coding, which is receiving a lot of interests nowadays. Side Information (SI) generation is a key function in the DVC decoder, and plays a key-role in determining the performance of the codec. This paper proposes an improved motion compensated frame interpolation for SI generation in DVC, which exploits both spatial and temporal correlations in the sequences. Partially decoded Wyner-Ziv (WZ) frames, based on initial SI by motion compensated temporal interpolation, are exploited to improve the performance of the whole SI generation. More specifically, an enhanced temporal frame interpolation is proposed, including motion vector refinement and smoothing, optimal compensation mode selection, and a new matching criterion for motion estimation. The improved SI technique is also applied to a new hybrid spatial and temporal error concealment scheme to conceal errors in WZ frames, where the error-concealed results from spatial concealment are used to improve the performance of temporal concealment. Simulation results show that the proposed scheme can achieve up to 1.0 dB improvement in rate distortion performance in WZ frames for video with high motion, when compared to state-of-the-art DVC. In addition, both the objective and perceptual quality of the corrupted sequences are significantly improved by the proposed hybrid error concealment scheme, outperforming both spatial and temporal concealments alone

An improved algorithm for deinterlacing video streams

The MPEG-4 standard for computerized video incorporates the concept of a video object pLane While in the simplest case this can be the full rectangular frame, the standard supports a hierarchical set of arbitrary shaped planes, one for each content sensitive video object. Herein is proposed a method for extracting arbitrary planes from video that does not already contain video object plane information; Deinterlacing is the process of taking two video fields, each at half the height of the finalized image frame, and combining them into that finalized frame. As the fields are not captured simultaneously, temporal artifacts may result. Herein is proposed a method to use the above mentioned video object planes to calculate the intra-field motion of objects in the video stream and correct for such motion leading to a higher quality deinterlaced output.*; *This dissertation is a compound document (contains both a paper copy and a CD as part of the dissertation)

HDTV transmission format conversion and migration path

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1997.Includes bibliographical references (leaves 77-79).by Lon E. Sunshine.Ph.D

A review of digital video tampering: from simple editing to full synthesis.

Video tampering methods have witnessed considerable progress in recent years. This is partly due to the rapid development of advanced deep learning methods, and also due to the large volume of video footage that is now in the public domain. Historically, convincing video tampering has been too labour intensive to achieve on a large scale. However, recent developments in deep learning-based methods have made it possible not only to produce convincing forged video but also to fully synthesize video content. Such advancements provide new means to improve visual content itself, but at the same time, they raise new challenges for state-of-the-art tampering detection methods. Video tampering detection has been an active field of research for some time, with periodic reviews of the subject. However, little attention has been paid to video tampering techniques themselves. This paper provides an objective and in-depth examination of current techniques related to digital video manipulation. We thoroughly examine their development, and show how current evaluation techniques provide opportunities for the advancement of video tampering detection. A critical and extensive review of photo-realistic video synthesis is provided with emphasis on deep learning-based methods. Existing tampered video datasets are also qualitatively reviewed and critically discussed. Finally, conclusions are drawn upon an exhaustive and thorough review of tampering methods with discussions of future research directions aimed at improving detection methods

Low power motion estimation based frame rate up-conversion hardware designs

Recently flat panel high definition television (HDTV) displays with 100 Hz, 120 Hz and 240 Hz picture rates are introduced. However, video materials are captured and broadcast in different temporal resolutions ranging from 24 Hz to 60 Hz. In order to display these video formats correctly on high picture rate displays, new frames should be generated and inserted into the original video sequence to increase its frame rate. Therefore, frame rate upconversion (FRUC) has become a necessity. Motion compensated FRUC (MC-FRUC) algorithms provide better quality results than non-motion compensated FRUC algorithms. These MC-FRUC algorithms consist of two main stages, motion estimation (ME) and motion compensated interpolation (MCI). In ME, motion vectors (MV) are calculated between successive frames, and in MCI this MV data is used to generate a new frame that is inserted between two successive frames, thus doubling the frame rate. In addition to these two main steps, intermediate steps such as refinement of the MV field by various algorithms like motion vector smoothing and bilateral ME refinement may be used to improve the quality of the interpolated video. In this thesis, a perfect absolute difference technique for block matching ME hardware is proposed. The proposed technique reduces the power consumption of a full search ME hardware by 2.2% on a XC2VP30-7 FPGA without any PSNR loss. In addition, a global motion estimation (GME) algorithm and its hardware implementation are proposed. The proposed GME algorithm increases PSNR of 3D recursive search ME algorithm by 2.5% and its hardware implementation is capable of processing 341 720p frames per second. An adaptive technique for GME, which reduces the energy consumption of the GME hardware by 14.37% on a XC6VLX75T FPGA with a 0.17% PSNR loss, is also proposed. Furthermore, an early termination technique for the adaptive bilateral motion estimation (ABIME) algorithm is proposed. The proposed technique reduces the energy consumption of the ABIME hardware by 29% with a 0.04% PSNR loss on a XC6VLX75T FPGA. In addition, an efficient weighted coefficient overlapped block motion compensation (WC-OBMC) hardware which reduces the dynamic power consumption of the reference WC-OBMC hardware by 22% is proposed. The proposed hardware is capable of processing 57 720p frames per second on a XC6VLX75T FPGA. Finally, the ABIME hardware is implemented on a Xilinx ML605 FPGA board