Efficient HEVC-based video adaptation using transcoding

In a video transmission system, it is important to take into account the great diversity of the network/end-user constraints. On the one hand, video content is typically streamed over a network that is characterized by different bandwidth capacities. In many cases, the bandwidth is insufficient to transfer the video at its original quality. On the other hand, a single video is often played by multiple devices like PCs, laptops, and cell phones. Obviously, a single video would not satisfy their different constraints. These diversities of the network and devices capacity lead to the need for video adaptation techniques, e.g., a reduction of the bit rate or spatial resolution. Video transcoding, which modifies a property of the video without the change of the coding format, has been well-known as an efficient adaptation solution. However, this approach comes along with a high computational complexity, resulting in huge energy consumption in the network and possibly network latency. This presentation provides several optimization strategies for the transcoding process of HEVC (the latest High Efficiency Video Coding standard) video streams. First, the computational complexity of a bit rate transcoder (transrater) is reduced. We proposed several techniques to speed-up the encoder of a transrater, notably a machine-learning-based approach and a novel coding-mode evaluation strategy have been proposed. Moreover, the motion estimation process of the encoder has been optimized with the use of decision theory and the proposed fast search patterns. Second, the issues and challenges of a spatial transcoder have been solved by using machine-learning algorithms. Thanks to their great performance, the proposed techniques are expected to significantly help HEVC gain popularity in a wide range of modern multimedia applications

Receiver-Driven Video Adaptation

In the span of a single generation, video technology has made an incredible impact on daily life. Modern use cases for video are wildly diverse, including teleconferencing, live streaming, virtual reality, home entertainment, social networking, surveillance, body cameras, cloud gaming, and autonomous driving. As these applications continue to grow more sophisticated and heterogeneous, a single representation of video data can no longer satisfy all receivers. Instead, the initial encoding must be adapted to each receiver's unique needs. Existing adaptation strategies are fundamentally flawed, however, because they discard the video's initial representation and force the content to be re-encoded from scratch. This process is computationally expensive, does not scale well with the number of videos produced, and throws away important information embedded in the initial encoding. Therefore, a compelling need exists for the development of new strategies that can adapt video content without fully re-encoding it. To better support the unique needs of smart receivers, diverse displays, and advanced applications, general-use video systems should produce and offer receivers a more flexible compressed representation that supports top-down adaptation strategies from an original, compressed-domain ground truth. This dissertation proposes an alternate model for video adaptation that addresses these challenges. The key idea is to treat the initial compressed representation of a video as the ground truth, and allow receivers to drive adaptation by dynamically selecting which subsets of the captured data to receive. In support of this model, three strategies for top-down, receiver-driven adaptation are proposed. First, a novel, content-agnostic entropy coding technique is implemented in which symbols are selectively dropped from an input abstract symbol stream based on their estimated probability distributions to hit a target bit rate. Receivers are able to guide the symbol dropping process by supplying the encoder with an appropriate rate controller algorithm that fits their application needs and available bandwidths. Next, a domain-specific adaptation strategy is implemented for H.265/HEVC coded video in which the prediction data from the original source is reused directly in the adapted stream, but the residual data is recomputed as directed by the receiver. By tracking the changes made to the residual, the encoder can compensate for decoder drift to achieve near-optimal rate-distortion performance. Finally, a fully receiver-driven strategy is proposed in which the syntax elements of a pre-coded video are cataloged and exposed directly to clients through an HTTP API. Instead of requesting the entire stream at once, clients identify the exact syntax elements they wish to receive using a carefully designed query language. Although an implementation of this concept is not provided, an initial analysis shows that such a system could save bandwidth and computation when used by certain targeted applications.Doctor of Philosoph

Toward a General Parametric Model for Assessing the Impact of Video Transcoding on Objective Video Quality

Video transcoding can cause degradation to an original video. Currently, there is no general model that assesses the impact of video transcoding on video quality. Such a model could play a critical role in evaluating the quality of the transcoded video, and thereby optimizing delivery of video to end-users while meeting their expectations. The main contribution of this research is the development and substantiation of a general parametric model, called the Video Transcoding Objective-quality Model (VTOM), that provides an extensible video transcoding service selection mechanism, which takes into account both the format and characteristics of the original video and the desired output, i.e., viewing format with preferred quality of service. VTOM represents a mathematical function that uses a set of media-related parameters for the original video and desired output, including codec, bit rate, frame rate, and frame size to predict the quality of the transcoded video generated from a specific transcoding. VTOM includes four quality sub-models, each describing the impact of each of these parameters on objective video quality, as well as a weighted-product aggregation function that combines these quality sub-models with four additional error sub-models in a single function for assessing the overall video quality. I compared the predicted quality results generated from the VTOM with quality values generated from an existing objective-quality metric. These comparisons yielded results that showed good correlations, with low error values. VTOM helps the researchers and developers of video delivery systems and applications to calculate the degradation that video transcoding can cause on the fly, rather than evaluate it statistically using statistical methods that only consider the desired output. Because VTOM takes into account the quality of the input video, i.e., video format and characteristics, and the desired quality of the output video, it can be used for dynamic video transcoding service selection and composition. A number of quality metrics were examined and used in development of VTOM and its assessment. However, this research discovered that, to date, there are no suitable metrics in the literature for comparing two videos with different frame rates. Therefore, this dissertation defines a new metric, called Frame Rate Metric (FRM) as part of its contributions. FRM can use any frame-based quality metric for comparing frames from both videos. Finally, this research presents and adapts four Quality of Service (QoS)-aware video transcoding service selection algorithms. The experimental results showed that these four algorithms achieved good results in terms of time complexity, success ratio, and user satisfaction rate

Architectural support for ubiquitous access to multimedia content

Tese de doutoramento. Engenharia Electrotécnica e de Computadores (Telecomunicações). Faculdade de Engenharia. Universidade do Porto. 200