Machine Learning for Multimedia Communications

Machine learning is revolutionizing the way multimedia information is processed and transmitted to users. After intensive and powerful training, some impressive efficiency/accuracy improvements have been made all over the transmission pipeline. For example, the high model capacity of the learning-based architectures enables us to accurately model the image and video behavior such that tremendous compression gains can be achieved. Similarly, error concealment, streaming strategy or even user perception modeling have widely benefited from the recent learningoriented developments. However, learning-based algorithms often imply drastic changes to the way data are represented or consumed, meaning that the overall pipeline can be affected even though a subpart of it is optimized. In this paper, we review the recent major advances that have been proposed all across the transmission chain, and we discuss their potential impact and the research challenges that they raise

Joint Exploitation of Residual Source Information and MAC Layer CRC Redundancy for Robust Video Decoding

International audienceThis paper presents a MAP estimation method allowing the robust decoding of compressed video streams by exploiting the bitstream structure (i.e., information about the source, related to variable-length codes and source characteristics) together with the knowledge of the MAC layer CRC (here considered as additional redundancy on the MAC packet). This method is implemented via a sequential decoding algorithm in which the branch selection metric in the decoding trellis incorporates a CRC-dependent factor, and the paths which are not compatible with the source constraints are pruned. A first implementation of the proposed algorithm performs exact computations of the metrics, and is thus computationally expensive. Therefore, we also introduce a suboptimal (with tunable complexity) version of the proposed metric computation. This technique is then applied to the robust decoding of sequences encoded using the H.264/AVC standard based on CAVLC, and transmitted using aWiFi-like packet structure. Significant link budget improvement results are demonstrated for BPSK modulated signals sent over AWGN channels, even in the presence of channel coding

Optimized cross-layer forward error correction coding for H.264 AVC video transmission over wireless channels

Forward error correction (FEC) codes that can provide unequal error protection (UEP) have been used recently for video transmission over wireless channels. These video transmission schemes may also benefit from the use of FEC codes both at the application layer (AL) and the physical layer (PL). However, the interaction and optimal setup of UEP FEC codes at the AL and the PL have not been previously investigated. In this paper, we study the cross-layer design of FEC codes at both layers for H.264 video transmission over wireless channels. In our scheme, UEP Luby transform codes are employed at the AL and rate-compatible punctured convolutional codes at the PL. In the proposed scheme, video slices are first prioritized based on their contribution to video quality. Next, we investigate the four combinations of cross-layer FEC schemes at both layers and concurrently optimize their parameters to minimize the video distortion and maximize the peak signal-to-noise ratio. We evaluate the performance of these schemes on four test H.264 video streams and show the superiority of optimized cross-layer FEC design.Peer reviewedElectrical and Computer Engineerin

Hybrid video quality prediction: reviewing video quality measurement for widening application scope

A tremendous number of objective video quality measurement algorithms have been developed during the last two decades. Most of them either measure a very limited aspect of the perceived video quality or they measure broad ranges of quality with limited prediction accuracy. This paper lists several perceptual artifacts that may be computationally measured in an isolated algorithm and some of the modeling approaches that have been proposed to predict the resulting quality from those algorithms. These algorithms usually have a very limited application scope but have been verified carefully. The paper continues with a review of some standardized and well-known video quality measurement algorithms that are meant for a wide range of applications, thus have a larger scope. Their individual artifacts prediction accuracy is usually lower but some of them were validated to perform sufficiently well for standardization. Several difficulties and shortcomings in developing a general purpose model with high prediction performance are identified such as a common objective quality scale or the behavior of individual indicators when confronted with stimuli that are out of their prediction scope. The paper concludes with a systematic framework approach to tackle the development of a hybrid video quality measurement in a joint research collaboration.Polish National Centre for Research and Development (NCRD) SP/I/1/77065/10, Swedish Governmental Agency for Innovation Systems (Vinnova

A New Compressive Video Sensing Framework for Mobile Broadcast

A new video coding method based on compressive sampling is proposed. In this method, a video is coded using compressive measurements on video cubes. Video reconstruction is performed by minimization of total variation (TV) of the pixelwise discrete cosine transform coefficients along the temporal direction. A new reconstruction algorithm is developed from TVAL3, an efficient TV minimization algorithm based on the alternating minimization and augmented Lagrangian methods. Video coding with this method is inherently scalable, and has applications in mobile broadcast