Quality of Experience (QoE)-Aware Fast Coding Unit Size Selection for HEVC Intra-prediction

The exorbitant increase in the computational complexity of modern video coding standards, such as High Efficiency Video Coding (HEVC), is a compelling challenge for resource-constrained consumer electronic devices. For instance, the brute force evaluation of all possible combinations of available coding modes and quadtree-based coding structure in HEVC to determine the optimum set of coding parameters for a given content demand a substantial amount of computational and energy resources. Thus, the resource requirements for real time operation of HEVC has become a contributing factor towards the Quality of Experience (QoE) of the end users of emerging multimedia and future internet applications. In this context, this paper proposes a content-adaptive Coding Unit (CU) size selection algorithm for HEVC intra-prediction. The proposed algorithm builds content-specific weighted Support Vector Machine (SVM) models in real time during the encoding process, to provide an early estimate of CU size for a given content, avoiding the brute force evaluation of all possible coding mode combinations in HEVC. The experimental results demonstrate an average encoding time reduction of 52.38%, with an average Bjøntegaard Delta Bit Rate (BDBR) increase of 1.19% compared to the HM16.1 reference encoder. Furthermore, the perceptual visual quality assessments conducted through Video Quality Metric (VQM) show minimal visual quality impact on the reconstructed videos of the proposed algorithm compared to state-of-the-art approaches

Virtual Frames as Long-Term Reference Frames for HEVC Inter-Prediction

High Efficiency Video Coding(HEVC) employs both past or future frames when encoding the current frame in a video sequence. This paper proposes a framework for using virtual reference frames, to achieve increased coding gains in the long-term for repetitive scenes in static camera scenarios

A Decoding-Complexity and Rate-Controlled Video-Coding Algorithm for HEVC

Video playback on mobile consumer electronic (CE) devices is plagued by fluctuations in the network bandwidth and by limitations in processing and energy availability at the individual devices. Seen as a potential solution, the state-of-the-art adaptive streaming mechanisms address the first aspect, yet the efficient control of the decoding-complexity and the energy use when decoding the video remain unaddressed. The quality of experience (QoE) of the end-users’ experiences, however, depends on the capability to adapt the bit streams to both these constraints (i.e., network bandwidth and device’s energy availability). As a solution, this paper proposes an encoding framework that is capable of generating video bit streams with arbitrary bit rates and decoding-complexity levels using a decoding-complexity–rate–distortion model. The proposed algorithm allocates rate and decoding-complexity levels across frames and coding tree units (CTUs) and adaptively derives the CTU-level coding parameters to achieve their imposed targets with minimal distortion. The experimental results reveal that the proposed algorithm can achieve the target bit rate and the decoding-complexity with 0.4% and 1.78% average errors, respectively, for multiple bit rate and decoding-complexity levels. The proposed algorithm also demonstrates a stable frame-wise rate and decoding-complexity control capability when achieving a decoding-complexity reduction of 10.11 (%/dB). The resultant decoding-complexity reduction translates into an overall energy-consumption reduction of up to 10.52 (%/dB) for a 1 dB peak signal-to-noise ratio (PSNR) quality loss compared to the HM 16.0 encoded bit streams

Improving HEVC Coding Efficiency Using Virtual Long-Term Reference Pictures

Inter-frame prediction in HEVC uses two types of reference pictures: short-term and long-term. Out of these long-term reference (LTR) pictures enable exploiting correlation among frames with extended temporal distances. In addition, LTR pictures improve the inter-frame prediction where video scenes are repeated such as in TV-series episodes, news broad-casts and movies. In this context, this paper proposes an algorithm to calculate LTR pictures using artificially generated virtual reference frames for static-camera scenes. The experimental results demonstrate an average coding improvement of2.34%in terms of Bjøntegaard Delta Bit Rate(BDBR), when compared with the HEVC reference encoder HM16.8

iCUS: Intelligent CU Size Selection for HEVC Inter Prediction

The hierarchical quadtree partitioning of Coding Tree Units (CTU) is one of the striking features in HEVC that contributes towards its superior coding performance over its predecessors. However, the brute force evaluation of the quadtree hierarchy using the Rate-Distortion (RD) optimisation, to determine the best partitioning structure for a given content, makes it one of the most time-consuming operations in HEVC encoding. In this context, this paper proposes an intelligent fast Coding Unit (CU) size selection algorithm to expedite the encoding process of HEVC inter-prediction. The proposed algorithm introduces (i) two CU split likelihood modelling and classification approaches using Support Vector Machines (SVM) and Bayesian probabilistic models, and (ii) a fast CU selection algorithm that makes use of both offline trained SVMs and online trained Bayesian probabilistic models. Finally, (iii) a computational complexity to coding efficiency trade-off mechanism is introduced to flexibly control the algorithm to suit different encoding requirements. The experimental results of the proposed algorithm demonstrate an average encoding time reduction performance of 53.46%, 61.15%, and 58.15% for Low Delay B , Random Access , and Low Delay P configurations, respectively, with Bjøntegaard Delta-Bit Rate (BD-BR) losses of 2.35%, 2.9%, and 2.35%, respectively, when evaluated across a wide range of content types and quality level

HEVC encoder optimization and decoding complexity-aware video encoding.

The increased demand for high quality video evidently elevates the bandwidth requirements of the communication channels being used, which in return demands for more efficient video coding algorithms within the media distribution tool chain. As such, High Efficiency Video Coding (HEVC) video coding standard is a potential solution that demonstrates a significant coding efficiency improvement over its predecessors. HEVC constitutes an assortment of novel coding tools and features that contribute towards its superior coding performance, yet at the same time demand more computational, processing and energy resources; a crucial bottleneck, especially in the case of resource constrained Consumer Electronic (CE) devices. In this context, the first contribution in this thesis presents a novel content adaptive Coding Unit (CU) size prediction algorithm for HEVC-based low-delay video encoding. In this case, two independent content adaptive CU size selection models are introduced while adopting a moving window-based feature selection process to ensure that the framework remains robust and dynamically adapts to any varying video content. The experimental results demonstrate a consistent average encoding time reduction ranging from 55% - 58% and 57% - 61% with average Bjøntegaard Delta Bit Rate (BDBR) increases of 1.93% - 2.26% and 2.14% - 2.33% compared to the HEVC 16.0 reference software for the low delay P and low delay B configurations, respectively, across a wide range of content types and bit rates. The video decoding complexity and the associated energy consumption are tightly coupled with the complexity of the codec as well as the content being decoded. Hence, video content adaptation is extensively considered as an application layer solution to reduce the decoding complexity and thereby the associated energy consumption. In this context, the second contribution in this thesis introduces a decoding complexity-aware video encoding algorithm for HEVC using a novel decoding complexity-rate-distortion model. The proposed algorithm demonstrates on average a 29.43% and 13.22% decoding complexity reductions for the same quality with only a 6.47% BDBR increase when using the HM 16.0 and openHEVC decoders, respectively. Moreover, decoder energy consumption analysis reveals an overall energy reduction of up to 20% for the same video quality. Adaptive video streaming is considered as a potential solution in the state-of-the-art to cope with the uncertain fluctuations in the network bandwidth. Yet, the simultaneous consideration of both bit rate and decoding complexity for content adaptation with minimal quality impact is extremely challenging due to the dynamics of the video content. In response, the final contribution in this thesis introduces a content adaptive decoding complexity and rate controlled encoding framework for HEVC. The experimental results reveal that the proposed algorithm achieves a stable rate and decoding complexity controlling performance with an average error of only 0.4% and 1.78%, respectively. Moreover, the proposed algorithm is capable of generating HEVC bit streams that exhibit up to 20.03 %/dB decoding complexity reduction which result in up to 7.02 %/dB decoder energy reduction per 1dB Peak Signal-to-Noise Ratio (PSNR) quality loss

A Decoding-Complexity and Rate-Controlled Video-Coding Algorithm for HEVC

Video playback on mobile consumer electronic (CE) devices is plagued by fluctuations in the network bandwidth and by limitations in processing and energy availability at the individual devices. Seen as a potential solution, the state-of-the-art adaptive streaming mechanisms address the first aspect, yet the efficient control of the decoding-complexity and the energy use when decoding the video remain unaddressed. The quality of experience (QoE) of the end-users’ experiences, however, depends on the capability to adapt the bit streams to both these constraints (i.e., network bandwidth and device’s energy availability). As a solution, this paper proposes an encoding framework that is capable of generating video bit streams with arbitrary bit rates and decoding-complexity levels using a decoding-complexity–rate–distortion model. The proposed algorithm allocates rate and decoding-complexity levels across frames and coding tree units (CTUs) and adaptively derives the CTU-level coding parameters to achieve their imposed targets with minimal distortion. The experimental results reveal that the proposed algorithm can achieve the target bit rate and the decoding-complexity with 0.4% and 1.78% average errors, respectively, for multiple bit rate and decoding-complexity levels. The proposed algorithm also demonstrates a stable frame-wise rate and decoding-complexity control capability when achieving a decoding-complexity reduction of 10.11 (%/dB). The resultant decoding-complexity reduction translates into an overall energy-consumption reduction of up to 10.52 (%/dB) for a 1 dB peak signal-to-noise ratio (PSNR) quality loss compared to the HM 16.0 encoded bit streams

An adaptive video streaming framework for Scalable HEVC (SHVC) standard

This paper presents an implementation of a Media Aware Network Element (MANE) for dynamic video content adaptation in Scalable HEVC (SHVC) video streaming. The experimental results discuss the varying quality-to-playback time ratio and decoding power consumption with random access period in SHVC encoding under fluctuating and persistent network bandwidth conditions