Streaming Video QoE Modeling and Prediction: A Long Short-Term Memory Approach

HTTP based adaptive video streaming has become a popular choice of streaming due to the reliable transmission and the flexibility offered to adapt to varying network conditions. However, due to rate adaptation in adaptive streaming, the quality of the videos at the client keeps varying with time depending on the end-to-end network conditions. Further, varying network conditions can lead to the video client running out of playback content resulting in rebuffering events. These factors affect the user satisfaction and cause degradation of the user quality of experience (QoE). It is important to quantify the perceptual QoE of the streaming video users and monitor the same in a continuous manner so that the QoE degradation can be minimized. However, the continuous evaluation of QoE is challenging as it is determined by complex dynamic interactions among the QoE influencing factors. Towards this end, we present LSTM-QoE, a recurrent neural network based QoE prediction model using a Long Short-Term Memory (LSTM) network. The LSTM-QoE is a network of cascaded LSTM blocks to capture the nonlinearities and the complex temporal dependencies involved in the time varying QoE. Based on an evaluation over several publicly available continuous QoE databases, we demonstrate that the LSTM-QoE has the capability to model the QoE dynamics effectively. We compare the proposed model with the state-of-the-art QoE prediction models and show that it provides superior performance across these databases. Further, we discuss the state space perspective for the LSTM-QoE and show the efficacy of the state space modeling approaches for QoE prediction

How to Perform AMP? Cubic Adjustments for Improving the QoE

[EN] Adaptive Media Playout (AMP) consists of smoothly and dynamically adjusting the media playout rate to recover from undesired (e.g., buffer overflow/underflow or out-of-sync) situations. The existing AMP solutions are mainly characterized by two main aspects. The first one is their goal (e.g., keeping the buffers¿ occupancy into safe ranges or enabling media synchronization). The second one is the criteria that determine the need for triggering the playout adjustments (e.g., buffer fullness or asynchrony levels). This paper instead focuses on a third key aspect, which has not been sufficiently investigated yet: the specific adjustment strategy to be performed. In particular, we propose a novel AMP strategy, called Cubic AMP, which is based on employing a cubic interpolation method to adjust a deviated playout point to a given reference. On the one hand, mathematical analysis and graphical examples show that our proposal provides superior performance than other existing linear and quadratic AMP strategies in terms of the smoothness of the playout curve, while significantly outperforming the quadratic AMP strategy regarding the duration of the adjustment period and without increasing the computational complexity. It has also been proved and discussed that higher-order polynomial interpolation methods are less convenient than cubic ones. On the other hand, the results of subjective tests confirm that our proposal provides better Quality of Experience (QoE) than the other existing AMP strategies.This work has been funded, partially, by the “Fondo Europeo de Desarrollo Regional (FEDER)” and the Spanish Ministry of Economy and Competitiveness, under its R&D&I Support Program, in project with Ref. TEC2013-45492-R.Montagud, M.; Boronat, F.; Roig, B.; Sapena Piera, A. (2017). How to Perform AMP? Cubic Adjustments for Improving the QoE. Computer Communications. 103:61-73. https://doi.org/10.1016/j.comcom.2017.01.017S617310

Model and performance of a no-reference quality assessment metric for video streaming

Video streaming via Transmission Control Protocol (TCP) networks has become a popular and highly demanded service, but its quality assessment in both objective and subjective terms has not been properly addressed. In this paper, based on statistical analysis a full analytic model of a no-reference objective metric, namely pause intensity (PI), for video quality assessment is presented. The model characterizes the video playout buffer behavior in connection with the network performance (throughput) and the video playout rate. This allows for instant quality measurement and control without requiring a reference video. PI specifically addresses the need for assessing the quality issue in terms of the continuity in the playout of TCP streaming videos, which cannot be properly measured by other objective metrics such as peak signal-to-noise-ratio, structural similarity, and buffer underrun or pause frequency. The performance of the analytical model is rigidly verified by simulation results and subjective tests using a range of video clips. It is demonstrated that PI is closely correlated with viewers' opinion scores regardless of the vastly different composition of individual elements, such as pause duration and pause frequency which jointly constitute this new quality metric. It is also shown that the correlation performance of PI is consistent and content independent

Enhancing the broadcasted TV consumption experience with broadband omnidirectional video content

[EN] The current wide range of heterogeneous consumption devices and delivery technologies, offers the opportunity to provide related contents in order to enhance and enrich the TV consumption experience. This paper describes a solution to handle the delivery and synchronous consumption of traditional broadcast TV content and related broadband omnidirectional video content. The solution is intended to support both hybrid (broadcast/broadband) delivery technologies and has been designed to be compatible with the Hybrid Broadcast Broadband TV (HbbTV) standard. In particular, some specifications of HbbTV, such as the use of global timestamps or discovery mechanisms, have been adopted. However, additional functionalities have been designed to achieve accurate synchronization and to support the playout of omnidirectional video content in current consumption devices. In order to prove that commercial hybrid environments could be immediately enhanced with this type of content, the proposed solution has been included in a testbed, and objectively and subjectively evaluated. Regarding the omnidirectional video content, the two most common types of projections are supported: equirectangular and cube map. The results of the objective assessment show that the playout of broadband delivered omnidirectional video content in companion devices can be accurately synchronized with the playout on TV of traditional broadcast 2D content. The results of the subjective assessment show the high interest of users in this type of new enriched and immersive experience that contributes to enhance their Quality of Experience (QoE) and engagement.This work was supported by the Generalitat Valenciana, Investigacion Competitiva Proyectos, through the Research and Development Program Grants for Research Groups to be Consolidated, under Grant AICO/2017/059 and Grant AICO/2017Marfil-Reguero, D.; Boronat, F.; López, J.; Vidal Meló, A. (2019). Enhancing the broadcasted TV consumption experience with broadband omnidirectional video content. IEEE Access. 7:171864-171883. https://doi.org/10.1109/ACCESS.2019.2956084S171864171883

Streaming Video over HTTP with Consistent Quality

In conventional HTTP-based adaptive streaming (HAS), a video source is encoded at multiple levels of constant bitrate representations, and a client makes its representation selections according to the measured network bandwidth. While greatly simplifying adaptation to the varying network conditions, this strategy is not the best for optimizing the video quality experienced by end users. Quality fluctuation can be reduced if the natural variability of video content is taken into consideration. In this work, we study the design of a client rate adaptation algorithm to yield consistent video quality. We assume that clients have visibility into incoming video within a finite horizon. We also take advantage of the client-side video buffer, by using it as a breathing room for not only network bandwidth variability, but also video bitrate variability. The challenge, however, lies in how to balance these two variabilities to yield consistent video quality without risking a buffer underrun. We propose an optimization solution that uses an online algorithm to adapt the video bitrate step-by-step, while applying dynamic programming at each step. We incorporate our solution into PANDA -- a practical rate adaptation algorithm designed for HAS deployment at scale.Comment: Refined version submitted to ACM Multimedia Systems Conference (MMSys), 201

An automated model for the assessment of QoE of adaptive video streaming over wireless networks

[EN] Nowadays, heterogeneous devices are widely utilizing Hypertext Transfer Protocol (HTTP) to transfer the data. Furthermore, HTTP adaptive video streaming (HAS) technology transmits the video data over wired and wireless networks. In adaptive technology services, a client's application receives a streaming video through the adaptation of its quality to the network condition. However, such a technology has increased the demand for Quality of Experience (QoE) in terms of prediction and assessment. It can also cause a challenging behavior regarding subjective and objective QoE evaluations of HTTP adaptive video over time since each Quality of Service (QoS) parameter affects the QoE of end-users separately. This paper introduces a methodology design for the evaluation of subjective QoE in adaptive video streaming over wireless networks. Besides, some parameters are considered such as video characteristics, segment length, initial delay, switch strategy, stalls, as well as QoS parameters. The experiment's evaluation demonstrated that objective metrics can be mapped to the most significant subjective parameters for user's experience. The automated model could function to demonstrate the importance of correlation for network behaviors' parameters. Consequently, it directly influences the satisfaction of the end-user's perceptual quality. In comparison with other recent related works, the model provided a positive Pearson Correlation value. Simulated results give a better performance between objective Structural Similarity (SSIM) and subjective Mean Opinion Score (MOS) evaluation metrics for all video test samples.This work has been partially supported by the "Ministerio de Economia y Competitividad" in the "Programa Estatal de Fomento de la Investigacion Cientifica y Tecnica de Excelencia, Subprograma Estatal de Generacion de Conocimiento" within the Project under Grant TIN2017-84802-C2-1-P. This study has been partially done in the computer science departments at the (University of Sulaimani and Halabja).Taha, M.; Ali, A.; Lloret, J.; Gondim, PRL.; Canovas, A. (2021). An automated model for the assessment of QoE of adaptive video streaming over wireless networks. Multimedia Tools and Applications. 80(17):26833-26854. https://doi.org/10.1007/s11042-021-10934-92683326854801

Quality of experience for adaptive streaming using HTTP Dynamic Streaming

This thesis investigates how to establish the relationship between OSI layer 7 parameters of video streaming and the QoE of the user, and to evaluate which methods are most tting for the estimation of QoE. The project is made in cooperation with LTH and Acreo, and is a part of the Next generation over-the-top multimedia services (NOTTS) 7and the Eco system for Future Media Distribution (EFRAIM) project. The underlying techniques, which form the environment of our research of estimating the QoE, is adaptive bitrate streaming over TCP. The purpose is to investigate how a service, that provides a user with the means to benchmark the received quality of the Over the top (OTT) streaming service, can be built and distributed. Today there exists no such service that takes the viewers subjective opinion into consideration. There have been extensive research on some connected elds and issues but none with a unied solution to streaming adaptive bitrate video over TCP with its particular behavior and eect caused on the streamed video. In this report we evaluated two dierent methods of prediction of QoE, Pause Intensity based on the number of pauses and their length during playback, and a Linear bitrate model based on the average bitrate quality and its standard deviation. We also made a small user test with our streaming client software to evaluate the two methods to decide which one is the most benecial to use. The test showed that although one of the most irritating playback deficiencies is when pauses occur, the linear bitrate model delivered the most accurate predictions