No-reference video quality estimation based on machine learning for passive gaming video streaming applications

Recent years have seen increasing growth and popularity of gaming services, both interactive and passive. While interactive gaming video streaming applications have received much attention, passive gaming video streaming, in-spite of its huge success and growth in recent years, has seen much less interest from the research community. For the continued growth of such services in the future, it is imperative that the end user gaming quality of experience (QoE) is estimated so that it can be controlled and maximized to ensure user acceptance. Previous quality assessment studies have shown not so satisfactory performance of existing No-reference (NR) video quality assessment (VQA) metrics. Also, due to the inherent nature and different requirements of gaming video streaming applications, as well as the fact that gaming videos are perceived differently from non-gaming content (as they are usually computer generated and contain artificial/synthetic content), there is a need for application specific light-weight, no-reference gaming video quality prediction models. In this paper, we present two NR machine learning based quality estimation models for gaming video streaming, NR-GVSQI and NR-GVSQE, using NR features such as bitrate, resolution, blockiness, etc. We evaluate their performance on different gaming video datasets and show that the proposed models outperform the current state-of-the-art no-reference metrics, while also reaching a prediction accuracy comparable to the best known full reference metric

An innovative machine learning-based scheduling solution for improving live UHD video streaming quality in highly dynamic network environments

The latest advances in terms of network technologies open up new opportunities for high-end applications, including using the next generation video streaming technologies. As mobile devices become more affordable and powerful, an increasing range of rich media applications could offer a highly realistic and immersive experience to mobile users. However, this comes at the cost of very stringent Quality of Service (QoS) requirements, putting significant pressure on the underlying networks. In order to accommodate these new rich media applications and overcome their associated challenges, this paper proposes an innovative Machine Learning-based scheduling solution which supports increased quality for live omnidirectional (360◦) video streaming. The proposed solution is deployed in a highly dy-namic Unmanned Aerial Vehicle (UAV)-based environment to support immersive live omnidirectional video streaming to mobile users. The effectiveness of the proposed method is demonstrated through simulations and compared against three state-of-the-art scheduling solutions, such as: Static Prioritization (SP), Required Activity Detection Scheduler (RADS) and Frame Level Scheduler (FLS). The results show that the proposed solution outperforms the other schemes involved in terms of PSNR, throughput and packet loss rate

Toward Generalized Psychovisual Preprocessing For Video Encoding

Deep perceptual preprocessing has recently emerged as a new way to enable further bitrate savings across several generations of video encoders without breaking standards or requiring any changes in client devices. In this article, we lay the foundation for a generalized psychovisual preprocessing framework for video encoding and describe one of its promising instantiations that is practically deployable for video-on-demand, live, gaming, and user-generated content (UGC). Results using state-of-the-art advanced video coding (AVC), high efficiency video coding (HEVC), and versatile video coding (VVC) encoders show that average bitrate [Bjontegaard delta-rate (BD-rate)] gains of 11%-17% are obtained over three state-of-the-art reference-based quality metrics [Netflix video multi-method assessment fusion (VMAF), structural similarity index (SSIM), and Apple advanced video quality tool (AVQT)], as well as the recently proposed nonreference International Telecommunication Union-Telecommunication?(ITU-T) P.1204 metric. The proposed framework on CPU is shown to be twice faster than × 264 medium-preset encoding. On GPU hardware, our approach achieves 714 frames/sec for 1080p video (below 2 ms/frame), thereby enabling its use in very-low-latency live video or game streaming applications

Feature-aware uniform tessellations on video manifold for content-sensitive supervoxels

Over-segmenting a video into supervoxels has strong potential to reduce the complexity of computer vision applications. Content-sensitive supervoxels (CSS) are typically smaller in content-dense regionsand larger in content-sparse regions. In this paper, we propose to compute feature-aware CSS (FCSS) that are regularly shaped 3D primitive volumes well aligned with local object/region/motion boundaries in video.To compute FCSS, we map a video to a 3-dimensional manifold, in which the volume elements of video manifold give a good measure of the video content density. Then any uniform tessellation on manifold can induce CSS. Our idea is that among all possible uniform tessellations, FCSS find one whose cell boundaries well align with local video boundaries. To achieve this goal, we propose a novel tessellation method that simultaneously minimizes the tessellation energy and maximizes the average boundary distance.Theoretically our method has an optimal competitive ratio O(1). We also present a simple extension of FCSS to streaming FCSS for processing long videos that cannot be loaded into main memory at once. We evaluate FCSS, streaming FCSS and ten representative supervoxel methods on four video datasets and two novel video applications. The results show that our method simultaneously achieves state-of-the-art performance with respect to various evaluation criteria