Implementing video compression algorithms on reconfigurable devices

The increasing density offered by Field Programmable Gate Arrays(FPGA), coupled with their short design cycle, has made them a popular choice for implementing a wide range of algorithms and complete systems. In this thesis the implementation of video compression algorithms on FPGAs is studied. Two areas are specifically focused on; the integration of a video encoder into a complete system and the power consumption of FPGA based video encoders. Two FPGA based video compression systems are described, one which targets surveillance applications and one which targets video conferencing applications. The FPGA video surveillance system makes use of a novel memory format to improve the efficiency with which input video sequences can be loaded over the system bus. The power consumption of a FPGA video encoder is analyzed. The results indicating that the motion estimation encoder stage requires the most power consumption. An algorithm, which reuses the intra prediction results generated during the encoding process, is then proposed to reduce the power consumed on an FPGA video encoder’s external memory bus. Finally, the power reduction algorithm is implemented within an FPGA video encoder. Results are given showing that, in addition to reducing power on the external memory bus, the algorithm also reduces power in the motion estimation stage of a FPGA based video encoder

HEVC와 JPEG 하드웨어 부호화기를 위한 DCT의 Approximate Calculation

학위논문 (석사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2015. 8. 이혁재.Discrete Cosine Transform (DCT) is widely used for various image and video compression applications because of its excellent energy compaction property. DCT is computationally intensive and the calculations are parallelizable. Therefore it is often implemented in hardware for speeding up the calculation. However due to large size of DCT or multiple modules of DCT required for some applications, the hardware area taken up by DCT in image or video encoders become significant. The DCT required in most applications doesnt need to be exact. Taking advantage of this fact, here a novel approach is provided to reduce the hardware area cost of the DCT module. The DCT hardware module consists of combinational logic and memory. Both the components are reduced and the complete implementation is described. The application being aimed at is for HEVC and JPEG, however the idea is applicable to any DCT hardware implementation. Finally the degradation caused to encoded image and video in terms of BDBR is discussed and the gate count results from the synthesis is provided.Chapter 1 Introduction 1 1.1 2D DCT Hardware Module . . . . . . . . . . . . . . . . . . . . . 2 1.1.1 Pipelining the process . . . . . . . . . . . . . . . . . . . . 5 1.2 Approximate DCT . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Chapter 2 Related Works 9 Chapter 3 The Moving Window Idea for Bit-Width Reduction 12 3.1 ML Recovery for Moving Window . . . . . . . . . . . . . . . . . 16 Chapter 4 Approximate DCT for HEVC 19 4.1 HEVC Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 4.2 HEVC Encoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 4.3 DCT in HEVC Encoder . . . . . . . . . . . . . . . . . . . . . . . 21 4.4 Approximate DCT in HEVC . . . . . . . . . . . . . . . . . . . . 23 4.4.1 The three components of the DCT module . . . . . . . . 27 4.4.2 Optimizing Partial Butterfly Adder/Subtractors . . . . . 29 4.4.3 Optimizing the multiplication module . . . . . . . . . . . 30 4.4.3.1 Multiple Constant Multiplication (MCM) . . . . 32 4.4.3.2 Approximate MCM . . . . . . . . . . . . . . . . 32 4.4.4 Optimizing the transpose memory . . . . . . . . . . . . . 36 Chapter 5 Approximate DCT for JPEG 39 5.1 JPEG Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 5.2 Approximate DCT . . . . . . . . . . . . . . . . . . . . . . . . . . 41 5.3 Application of Moving Window to DCT transpose memory . . . 42 5.3.1 Ideal implementation . . . . . . . . . . . . . . . . . . . . . 43 5.3.2 Window position based on first row . . . . . . . . . . . . . 43 5.3.2.1 Cases of failure . . . . . . . . . . . . . . . . . . . 46 5.3.3 Position based on first column . . . . . . . . . . . . . . . 48 5.3.3.1 Cases of failure . . . . . . . . . . . . . . . . . . . 49 5.4 Hybrid implementation . . . . . . . . . . . . . . . . . . . . . . . . 50 Chapter 6 Experimental Results 54 6.1 HEVC Experiments and Results . . . . . . . . . . . . . . . . . . 55 6.2 JPEG Experiments and Results . . . . . . . . . . . . . . . . . . . 55 Chapter 7 Conclusion 64Maste

Survey of FPGA applications in the period 2000 – 2015 (Technical Report)

Romoth J, Porrmann M, Rückert U. Survey of FPGA applications in the period 2000 – 2015 (Technical Report).; 2017.Since their introduction, FPGAs can be seen in more and more different fields of applications. The key advantage is the combination of software-like flexibility with the performance otherwise common to hardware. Nevertheless, every application field introduces special requirements to the used computational architecture. This paper provides an overview of the different topics FPGAs have been used for in the last 15 years of research and why they have been chosen over other processing units like e.g. CPUs

Visual Content Characterization Based on Encoding Rate-Distortion Analysis

Visual content characterization is a fundamentally important but under exploited step in dataset construction, which is essential in solving many image processing and computer vision problems. In the era of machine learning, this has become ever more important, because with the explosion of image and video content nowadays, scrutinizing all potential content is impossible and source content selection has become increasingly difficult. In particular, in the area of image/video coding and quality assessment, it is highly desirable to characterize/select source content and subsequently construct image/video datasets that demonstrate strong representativeness and diversity of the visual world, such that the visual coding and quality assessment methods developed from and validated using such datasets exhibit strong generalizability. Encoding Rate-Distortion (RD) analysis is essential for many multimedia applications. Examples of applications that explicitly use RD analysis include image encoder RD optimization, video quality assessment (VQA), and Quality of Experience (QoE) optimization of streaming videos etc. However, encoding RD analysis has not been well investigated in the context of visual content characterization. This thesis focuses on applying encoding RD analysis as a visual source content characterization method with image/video coding and quality assessment applications in mind. We first conduct a video quality subjective evaluation experiment for state-of-the-art video encoder performance analysis and comparison, where our observations reveal severe problems that motivate the needs of better source content characterization and selection methods. Then the effectiveness of RD analysis in visual source content characterization is demonstrated through a proposed quality control mechanism for video coding by eigen analysis in the space of General Quality Parameter (GQP) functions. Finally, by combining encoding RD analysis with submodular set function optimization, we propose a novel method for automating the process of representative source content selection, which helps boost the RD performance of visual encoders trained with the selected visual contents

Depth-Map Image Compression Based on Region and Contour Modeling

In this thesis, the problem of depth-map image compression is treated. The compilation of articles included in the thesis provides methodological contributions in the fields of lossless and lossy compression of depth-map images.The first group of methods addresses the lossless compression problem. The introduced methods are using the approach of representing the depth-map image in terms of regions and contours. In the depth-map image, a segmentation defines the regions, by grouping pixels having similar properties, and separates them using (region) contours. The depth-map image is encoded by the contours and the auxiliary information needed to reconstruct the depth values in each region.One way of encoding the contours is to describe them using two matrices of horizontal and vertical contour edges. The matrices are encoded using template context coding where each context tree is optimally pruned. In certain contexts, the contour edges are found deterministically using only the currently available information. Another way of encoding the contours is to describe them as a sequence of contour segments. Each such segment is defined by an anchor (starting) point and a string of contour edges, equivalent to a string of chain-code symbols. Here we propose efficient ways to select and encode the anchor points and to generate contour segments by using a contour crossing point analysis and by imposing rules that help in minimizing the number of anchor points.The regions are reconstructed at the decoder using predictive coding or the piecewise constant model representation. In the first approach, the large constant regions are found and one depth value is encoded for each such region. For the rest of the image, suitable regions are generated by constraining the local variation of the depth level from one pixel to another. The nonlinear predictors selected specifically for each region are combining the results of several linear predictors, each fitting optimally a subset of pixels belonging to the local neighborhood. In the second approach, the depth value of a given region is encoded using the depth values of the neighboring regions already encoded. The natural smoothness of the depth variation and the mutual exclusiveness of the values in neighboring regions are exploited to efficiently predict and encode the current region's depth value.The second group of methods is studying the lossy compression problem. In a first contribution, different segmentations are generated by varying the threshold for the depth local variability. A lossy depth-map image is obtained for each segmentation and is encoded based on predictive coding, quantization and context tree coding. In another contribution, the lossy versions of one image are created either by successively merging the constant regions of the original image, or by iteratively splitting the regions of a template image using horizontal or vertical line segments. Merging and splitting decisions are greedily taken, according to the best slope towards the next point in the rate-distortion curve. An entropy coding algorithm is used to encode each image.We propose also a progressive coding method for coding the sequence of lossy versions of a depth-map image. The bitstream is encoded so that any lossy version of the original image is generated, starting from a very low resolution up to lossless reconstruction. The partitions of the lossy versions into regions are assumed to be nested so that a higher resolution image is obtained by splitting some regions of a lower resolution image. A current image in the sequence is encoded using the a priori information from a previously encoded image: the anchor points are encoded relative to the already encoded contour points; the depth information of the newly resulting regions is recovered using the depth value of the parent region.As a final contribution, the dissertation includes a study of the parameterization of planar models. The quantized heights at three-pixel locations are used to compute the optimal plane for each region. The three-pixel locations are selected so that the distortion due to the approximation of the plane over the region is minimized. The planar model and the piecewise constant model are competing in the merging process, where the two regions to be merged are those ensuring the optimal slope in the rate-distortion curve

Advances in Hydrogels

Hydrogels are a class of soft materials with crosslinked network structures. They show good biocompatibility, biodegradability, hydrophilicity, and mechanical properties similar to those of tissue, so they have a wide range of applications. In recent years, a variety of multifunctional hydrogels with excellent performance have been developed, greatly expanding the depth and breadth of their applications. This book is the reprint of the Special Issue “Advances in Hydrogels”, which focused on the recent advances regarding hydrogels, aiming to provide reference for researchers in related fields. This book included one editorial, thirteen original research articles, and three valuable reviews from thirteen different countries including Canada, China, Thailand, Mexico, India, Saudi Arabia, Chile, Germany, the Czech Republic, Colombia, Romania, Israel, and the USA

Fusing spatial and temporal components for real-time depth data enhancement of dynamic scenes

The depth images from consumer depth cameras (e.g., structured-light/ToF devices) exhibit a substantial amount of artifacts (e.g., holes, flickering, ghosting) that needs to be removed for real-world applications. Existing methods cannot entirely remove them and perform slow. This thesis proposes a new real-time spatio-temporal depth image enhancement filter that completely removes flickering and ghosting, and significantly reduces holes. This thesis also presents a novel depth-data capture setup and two data reduction methods to optimize the performance of the proposed enhancement method