A taxonomy of the parameters used by decision methods for adaptive video transmission

International audienceNowadays, video data transfers account for much of the Internet traffic and a huge number of users use this service on a daily base. Even if videos are usually stored in several bitrates on servers, the video sending rate does not take into account network conditions which are changing dynamically during transmission. Therefore, the best bitrate is not used which causes sub-optimal video quality when the video bitrate is under the available bandwidth or packet loss when it is over it. One solution is to deploy adaptive video, which adapts video parameters such as bitrate or frame resolution to network conditions. Many ideas are proposed in the literature, yet no paper provides a global view on adaptation methods in order to classify them. This article fills this gap by discussing several adaptation methods through a taxonomy of the parameters used for adaptation. We show that, in the research community, the sender generally takes the decision of adaptation whereas in the solutions supported by major current companies the receiver takes this decision. We notably suggest, without evaluation, a valuable and realistic adaptation method, gathering the advantages of the presented methods

Video Traffic Characteristics of Modern Encoding Standards: H.264/AVC with SVC and MVC Extensions and H.265/HEVC

abstract: Video encoding for multimedia services over communication networks has significantly advanced in recent years with the development of the highly efficient and flexible H.264/AVC video coding standard and its SVC extension. The emerging H.265/HEVC video coding standard as well as 3D video coding further advance video coding for multimedia communications. This paper first gives an overview of these new video coding standards and then examines their implications for multimedia communications by studying the traffic characteristics of long videos encoded with the new coding standards. We review video coding advances from MPEG-2 and MPEG-4 Part 2 to H.264/AVC and its SVC and MVC extensions as well as H.265/HEVC. For single-layer (nonscalable) video, we compare H.265/HEVC and H.264/AVC in terms of video traffic and statistical multiplexing characteristics. Our study is the first to examine the H.265/HEVC traffic variability for long videos. We also illustrate the video traffic characteristics and statistical multiplexing of scalable video encoded with the SVC extension of H.264/AVC as well as 3D video encoded with the MVC extension of H.264/AVC.View the article as published at https://www.hindawi.com/journals/tswj/2014/189481

Intra Coding Strategy for Video Error Resiliency: Behavioral Analysis

One challenge in video transmission is to deal with packet loss. Since the compressed video streams are sensitive to data loss, the error resiliency of the encoded video becomes important. When video data is lost and retransmission is not possible, the missed data should be concealed. But loss concealment causes distortion in the lossy frame which also propagates into the next frames even if their data are received correctly. One promising solution to mitigate this error propagation is intra coding. There are three approaches for intra coding: intra coding of a number of blocks selected randomly or regularly, intra coding of some specific blocks selected by an appropriate cost function, or intra coding of a whole frame. But Intra coding reduces the compression ratio; therefore, there exists a trade-off between bitrate and error resiliency achieved by intra coding. In this paper, we study and show the best strategy for getting the best rate-distortion performance. Considering the error propagation, an objective function is formulated, and with some approximations, this objective function is simplified and solved. The solution demonstrates that periodical I-frame coding is preferred over coding only a number of blocks as intra mode in P-frames. Through examination of various test sequences, it is shown that the best intra frame period depends on the coding bitrate as well as the packet loss rate. We then propose a scheme to estimate this period from curve fitting of the experimental results, and show that our proposed scheme outperforms other methods of intra coding especially for higher loss rates and coding bitrates

Non-MPM Mode Coding for Intra Prediction in Video Coding

The High Efficiency Video Coding standard introduced thirty-five intra prediction modes. It employed a method based on three most probable modes (MPM) to improve intra mode coding. This method significantly improved the performance by extracting three MPMs out of the thirty-five intra modes. The Joint Video Exploration Team (JVET) defines sixty-seven intra prediction modes for a possible future video coding standard. In the latest JVET development, six MPMs are chosen, and the remaining sixty-one modes are divided into sixteen “selected” and forty-five “non-selected” modes. These non-MPM modes are coded using fixed length coding. This research focusses on finding more efficient ways to code these intra prediction modes, including MPM modes and non-MPM modes. A method is proposed to select and order the sixty-one non-MPM modes based on probability statistics. The modes that fall into selected category are coded using shorter codes and non-selected modes are coded using larger codes, which is in line with the principle of entropy coding. Experimental results prove performance improvement when compared to JEM7.0 software as a reference

A Collaborative Adaptive Wiener Filter for Image Restoration Using a Spatial-Domain Multi-patch Correlation Model

We present a new patch-based image restoration algorithm using an adaptive Wiener filter (AWF) with a novel spatial-domain multi-patch correlation model. The new filter structure is referred to as a collaborative adaptive Wiener filter (CAWF). The CAWF employs a finite size moving window. At each position, the current observation window represents the reference patch. We identify the most similar patches in the image within a given search window about the reference patch. A single-stage weighted sum of all of the pixels in the similar patches is used to estimate the center pixel in the reference patch. The weights are based on a new multi-patch correlation model that takes into account each pixel’s spatial distance to the center of its corresponding patch, as well as the intensity vector distances among the similar patches. One key advantage of the CAWF approach, compared with many other patch-based algorithms, is that it can jointly handle blur and noise. Furthermore, it can also readily treat spatially varying signal and noise statistics. To the best of our knowledge, this is the first multi-patch algorithm to use a single spatial-domain weighted sum of all pixels within multiple similar patches to form its estimate and the first to use a spatial-domain multi-patch correlation model to determine the weights. The experimental results presented show that the proposed method delivers high performance in image restoration in a variety of scenarios

Design Of Computer Vision Systems For Optimizing The Threat Detection Accuracy

This dissertation considers computer vision (CV) systems in which a central monitoring station receives and analyzes the video streams captured and delivered wirelessly by multiple cameras. It addresses how the bandwidth can be allocated to various cameras by presenting a cross-layer solution that optimizes the overall detection or recognition accuracy. The dissertation presents and develops a real CV system and subsequently provides a detailed experimental analysis of cross-layer optimization. Other unique features of the developed solution include employing the popular HTTP streaming approach, utilizing homogeneous cameras as well as heterogeneous ones with varying capabilities and limitations, and including a new algorithm for estimating the effective medium airtime. The results show that the proposed solution significantly improves the CV accuracy. Additionally, the dissertation features an improved neural network system for object detection. The proposed system considers inherent video characteristics and employs different motion detection and clustering algorithms to focus on the areas of importance in consecutive frames, allowing the system to dynamically and efficiently distribute the detection task among multiple deployments of object detection neural networks. Our experimental results indicate that our proposed method can enhance the mAP (mean average precision), execution time, and required data transmissions to object detection networks. Finally, as recognizing an activity provides significant automation prospects in CV systems, the dissertation presents an efficient activity-detection recurrent neural network that utilizes fast pose/limbs estimation approaches. By combining object detection with pose estimation, the domain of activity detection is shifted from a volume of RGB (Red, Green, and Blue) pixel values to a time-series of relatively small one-dimensional arrays, thereby allowing the activity detection system to take advantage of highly capable neural networks that have been trained on large GPU clusters for thousands of hours. Consequently, capable activity detection systems with considerably fewer training sets and processing hours can be built