Real-time complexity constrained encoding

Complex software appliances can be deployed on hardware with limited available computational resources. This computational boundary puts an additional constraint on software applications. This can be an issue for real-time applications with a fixed time constraint such as low delay video encoding. In the context of High Efficiency Video Coding (HEVC), a limited number of publications have focused on controlling the complexity of an HEVC video encoder. In this paper, a technique is proposed to control complexity by deciding between 2Nx2N merge mode and full encoding, at different Coding Unit (CU) depths. The technique is demonstrated in two encoders. The results demonstrate fast convergence to a given complexity threshold, and a limited loss in rate-distortion performance (on average 2.84% Bjontegaard delta rate for 40% complexity reduction)

LOW-DELAY WINDOW-BASED RATE CONTROL SCHEME FOR VIDEO QUALITY OPTIMIZATION IN VIDEO ENCODER

ABSTRACT The consistent video quality and encoding latency due to buffering are two important aspects in designing rate control scheme for the application of real-time video coding system. To well balance these two contrary objectives, we firstly analyze the constraint of buffer latency and the definition of a "consistent" video quality. Then a window-based rate control scheme is proposed with one window for controlling the rate and latency, while the other window for optimizing video quality. By applying low complexity frame level ratedistortion model in the testing sequences, our proposed method shows excellent performance in balancing the encoder buffer latency and optimized video quality. Besides, this one-pass rate control scheme is highly practical for the real-time video coding application. Rate control, buffer latency, video quality, window-based, video encode

Lowpass Filtering of Rate-Distortion Functions for Quality Smoothing in Real-Time Video Communication

Digital Object Identifier 10.1109/TCSVT.2005.852417In variable-bit-rate (VBR) video coding, the video is pre-processed to collect sequence-level statistics, which are used for global bit allocation in the actual encoding stage to obtain a smoothed video presentation quality. However, in real-time video recording and network streaming, this type of two-pass encoding scheme is not allowed because the access to future frames and global statistics is not available. To address this issue, we introduce the concept of low-pass filtering of rate-distortion (R-D) functions and develop a smoothed rate control (SRC) framework for real-time video recording and streaming. Theoretically, we prove that, using a geometric averaging filter, the SRC algorithm is able to maintain a smoothed video presentation quality while achieving the target bit rate automatically. We also analyze the buffer requirement of the SRC algorithm in real-time video streaming, and propose a scheme to seamlessly integrate robust buffer control into the SRC framework. The proposed SRC algorithm has very low computational complexity and implementation cost. Our extensive experimental results demonstrate that the SRC algorithm significantly reduces the picture quality variation in the encoded video clips

Complexity management of H.264/AVC video compression.

The H. 264/AVC video coding standard offers significantly improved compression efficiency and flexibility compared to previous standards. However, the high computational complexity of H. 264/AVC is a problem for codecs running on low-power hand held devices and general purpose computers. This thesis presents new techniques to reduce, control and manage the computational complexity of an H. 264/AVC codec. A new complexity reduction algorithm for H. 264/AVC is developed. This algorithm predicts "skipped" macroblocks prior to motion estimation by estimating a Lagrange ratedistortion cost function. Complexity savings are achieved by not processing the macroblocks that are predicted as "skipped". The Lagrange multiplier is adaptively modelled as a function of the quantisation parameter and video sequence statistics. Simulation results show that this algorithm achieves significant complexity savings with a negligible loss in rate-distortion performance. The complexity reduction algorithm is further developed to achieve complexity-scalable control of the encoding process. The Lagrangian cost estimation is extended to incorporate computational complexity. A target level of complexity is maintained by using a feedback algorithm to update the Lagrange multiplier associated with complexity. Results indicate that scalable complexity control of the encoding process can be achieved whilst maintaining near optimal complexity-rate-distortion performance. A complexity management framework is proposed for maximising the perceptual quality of coded video in a real-time processing-power constrained environment. A real-time frame-level control algorithm and a per-frame complexity control algorithm are combined in order to manage the encoding process such that a high frame rate is maintained without significantly losing frame quality. Subjective evaluations show that the managed complexity approach results in higher perceptual quality compared to a reference encoder that drops frames in computationally constrained situations. These novel algorithms are likely to be useful in implementing real-time H. 264/AVC standard encoders in computationally constrained environments such as low-power mobile devices and general purpose computers

An Energy-efficient Live Video Coding and Communication over Unreliable Channels

In the field of multimedia communications there exist many important applications where live or real-time video data is captured by a camera, compressed and transmitted over the channel which can be very unreliable and, at the same time, computational resources or battery capacity of the transmission device are very limited. For example, such scenario holds for video transmission for space missions, vehicle-to-infrastructure video delivery, multimedia wireless sensor networks, wireless endoscopy, video coding on mobile phones, high definition wireless video surveillance and so on. Taking into account such restrictions, a development of efficient video coding techniques for these applications is a challenging problem. The most popular video compression standards, such as H.264/AVC, are based on the hybrid video coding concept, which is very efficient when video encoding is performed off-line or non real-time and the pre-encoded video is played back. However, the high computational complexity of the encoding and the high sensitivity of the hybrid video bit stream to losses in the communication channel constitute a significant barrier of using these standards for the applications mentioned above. In this thesis, as an alternative to the standards, a video coding based on three-dimensional discrete wavelet transform (3-D DWT) is considered as a candidate to provide a good trade-off between encoding efficiency, computational complexity and robustness to channel losses. Efficient tools are proposed to reduce the computational complexity of the 3-D DWT codec. These tools cover all levels of the codec’s development such as adaptive binary arithmetic coding, bit-plane entropy coding, wavelet transform, packet loss protection based on error-correction codes and bit rate control. These tools can be implemented as end-to-end solution and directly used in real-life scenarios. The thesis provides theoretical, simulation and real-world results which show that the proposed 3-D DWT codec can be more preferable than the standards for live video coding and communication over highly unreliable channels and or in systems where the video encoding computational complexity or power consumption plays a critical role

Distributed Coding/Decoding Complexity in Video Sensor Networks

Video Sensor Networks (VSNs) are recent communication infrastructures used to capture and transmit dense visual information from an application context. In such large scale environments which include video coding, transmission and display/storage, there are several open problems to overcome in practical implementations. This paper addresses the most relevant challenges posed by VSNs, namely stringent bandwidth usage and processing time/power constraints. In particular, the paper proposes a novel VSN architecture where large sets of visual sensors with embedded processors are used for compression and transmission of coded streams to gateways, which in turn transrate the incoming streams and adapt them to the variable complexity requirements of both the sensor encoders and end-user decoder terminals. Such gateways provide real-time transcoding functionalities for bandwidth adaptation and coding/decoding complexity distribution by transferring the most complex video encoding/decoding tasks to the transcoding gateway at the expense of a limited increase in bit rate. Then, a method to reduce the decoding complexity, suitable for system-on-chip implementation, is proposed to operate at the transcoding gateway whenever decoders with constrained resources are targeted. The results show that the proposed method achieves good performance and its inclusion into the VSN infrastructure provides an additional level of complexity control functionality

Research and developments of Dirac video codec

This thesis was submitted for the degree of Doctor of Philosophy and was awarded by Brunel University.In digital video compression, apart from storage, successful transmission of the compressed video data over the bandwidth limited erroneous channels is another important issue. To enable a video codec for broadcasting application, it is required to implement the corresponding coding tools (e.g. error-resilient coding, rate control etc.). They are normally non-normative parts of a video codec and hence their specifications are not defined in the standard. In Dirac as well, the original codec is optimized for storage purpose only and so, several non-normative part of the encoding tools are still required in order to be able to use in other types of application. Being the "Research and Developments of the Dirac Video Codec" as the research title, phase I of the project is mainly focused on the error-resilient transmission over a noisy channel. The error-resilient coding method used here is a simple and low complex coding scheme which provides the error-resilient transmission of the compressed video bitstream of Dirac video encoder over the packet erasure wired network. The scheme combines source and channel coding approach where error-resilient source coding is achieved by data partitioning in the wavelet transformed domain and channel coding is achieved through the application of either Rate-Compatible Punctured Convolutional (RCPC) Code or Turbo Code (TC) using un-equal error protection between header plus MV and data. The scheme is designed mainly for the packet-erasure channel, i.e. targeted for the Internet broadcasting application. But, for a bandwidth limited channel, it is still required to limit the amount of bits generated from the encoder depending on the available bandwidth in addition to the error-resilient coding. So, in the 2nd phase of the project, a rate control algorithm is presented. The algorithm is based upon the Quality Factor (QF) optimization method where QF of the encoded video is adaptively changing in order to achieve average bitrate which is constant over each Group of Picture (GOP). A relation between the bitrate, R and the QF, which is called Rate-QF (R-QF) model is derived in order to estimate the optimum QF of the current encoding frame for a given target bitrate, R. In some applications like video conferencing, real-time encoding and decoding with minimum delay is crucial, but, the ability to do real-time encoding/decoding is largely determined by the complexity of the encoder/decoder. As we all know that motion estimation process inside the encoder is the most time consuming stage. So, reducing the complexity of the motion estimation stage will certainly give one step closer to the real-time application. So, as a partial contribution toward realtime application, in the final phase of the research, a fast Motion Estimation (ME) strategy is designed and implemented. It is the combination of modified adaptive search plus semi-hierarchical way of motion estimation. The same strategy was implemented in both Dirac and H.264 in order to investigate its performance on different codecs. Together with this fast ME strategy, a method which is called partial cost function calculation in order to further reduce down the computational load of the cost function calculation was presented. The calculation is based upon the pre-defined set of patterns which were chosen in such a way that they have as much maximum coverage as possible over the whole block. In summary, this research work has contributed to the error-resilient transmission of compressed bitstreams of Dirac video encoder over a bandwidth limited error prone channel. In addition to this, the final phase of the research has partially contributed toward the real-time application of the Dirac video codec by implementing a fast motion estimation strategy together with partial cost function calculation idea.BBC R&D and Brunel University

A real time generic variable pattern selection algorithm for very low bit-rate video coding

The selection of an optimal regular-shaped pattern set for very low bit-rate video coding, focusing on moving regions has been the objective of much recent research in order to try and improve bit-rate efficiency. Selecting the optimal pattern set however, is an NP hard problem. This paper presents a generic variable pattern selection (GVPS) algorithm, which introduces a pattern selection parameter that is able to control the performance in terms of computational complexity as well as bit-rate and picture quality. While using a sub-optimal variable pattern set, GVPS obtains a coding performance comparable to near-optimal algorithms, such as the k-change neighbourhood solution, while being much less computationally intensive, so that it is able to process all types of video sequences in real-time, with minimal pre-processing overheads

Complexity management for video encoders.

Software implementation of block-based video coding standards has been used in a wide range of applications. In many cases, such as real-time multimedia systems or power-constrained systems, the coding performance of software-only video encoders and decoders is limited by computational complexity. This thesis presents research work to develop techniques to manage computational complexity of video encoders. These techniques aim to provide significant complexity saving as well as adaptively controlling the computational complexity. This thesis first investigates experimentally the most computationally intensive functions in a video encoder. Based on the results of profile tests, several functions are selected as candidates, on which complexity reduction algorithms will be performed. These functions include discrete cosine transform and related functions as well as motion estimation. Adaptive complexity-reduction algorithms are proposed for computationally expensive functions: discrete cosine transform and motion estimation functions respectively. It is shown that these algorithms can flexibly control the computational complexity of each function with negligible loss of video quality. The inherent characteristics of coded macroblocks are investigated through experimental tests and they are categorized into "skipped" and" unskipped" macroblocks based on two parameters. An innovative algorithm is developed to reduce the computational complexity by predicting "skipped" macroblock prior to encoding and not carrying out the coding process on these macroblocks. The approaches described in this thesis can not only achieve adaptive control of the computational complexity of a video encoder, but also can manage the trade-off between complexity and distortion. These proposed algorithms are evaluated in terms of complexity reduction performance, rate-distortion performance and subjective and objective visual quality by experimental testing.The advantages and disadvantages of each algorithm are discussed

Register-transfer level design of sum of absolute transformed difference for high efficiency video coding

High Efficiency Video Coding (HEVC) is the state-of-the-art video coding standard which offers 50% improvement in coding efficiency over its predecessor Advanced Video Coding (AVC). Compared to AVC, HEVC supports up to 33 angular modes, DC mode and planar mode. The significant rise in the number of intra prediction mode however has increased the computational complexity. Sum of Absolute Transformed Difference (SATD), a fast Rate Distortion Optimization (RDO) intra prediction algorithm in the HEVC standard, is one of the most complex and compute-intensive part of the encoding process. SATD alone can takes up to 40% of the total encoding time; hence off-loading it to dedicated hardware accelerators is necessary to address the increasing need for real-time video coding in accordance with the push for coding efficiency. This work proposes a Verilog-described N × N SATD hardware architecture which is based on Hadamard Transform. The architecture would support a variable block size from 4 × 4 to 32 × 32 with 1-D horizontal and 1-D vertical Hadamard Transform. At the same time, it is designed to achieve throughput optimization by pipelining and feedthrough control. The performance of the implemented SATD is then evaluated in terms of utilization, timing and power