Look ahead to improve QoE in DASH streaming

[EN] When a video is encoded with constant quality, the resulting bitstream will have variable bitrate due to the inherent nature of the video encoding process. This paper proposes a video Adaptive Bitrate Streaming (ABR) algorithm, called Look Ahead, which takes into account this bitrate variability in order to calculate, in real time, the appropriate quality level that minimizes the number of interruptions during the playback. The algorithm is based on the Dynamic Adaptive Streaming over HTTP (DASH) standard for on-demand video services. In fact, it has been implemented and integrated into ExoPlayer v2, the latest version of the library developed by Google to play DASH contents. The proposed algorithm is compared to the Müller and Segment Aware Rate Adaptation (SARA) algorithms as well as to the default ABR algorithm integrated into ExoPlayer. The comparison is carried out by using the most relevant parameters that affect the Quality of Experience (QoE) in video playback services, that is, number and duration of stalls, average quality of the video playback and number of representation switches. These parameters can be combined to define a QoE model. In this sense, this paper also proposes two new QoE models for the evaluation of ABR algorithms. One of them considers the bitrate of every segment of each representation, and the second is based on VMAF (Video Multimethod Assessment Fusion), a Video Quality Assessment (VQA) method developed by Netflix. The evaluations presented in the paper reflect: first, that Look Ahead outperforms the Müller, SARA and the ExoPlayer ABR algorithms in terms of number and duration of video playback stalls, with hardly decreasing the average video quality; and second, that the two QoE models proposed are more accurate than other similar models existing in the literature.This work is supported by the PAID-10-18 Program of the Universitat Politecnica de Valencia (Ayudas para contratos de acceso al sistema espanol de Ciencia, Tecnologia e Innovacion, en estructuras de investigacion de la Universitat Politecnica de Valencia) and by the Project 20180810 from the Universitat Politecnica de Valencia ("Tecnologias de distribucion y procesado de informacion multimedia y QoE").Belda Ortega, R.; De Fez Lava, I.; Arce Vila, P.; Guerri Cebollada, JC. (2020). Look ahead to improve QoE in DASH streaming. Multimedia Tools and Applications. 79(33-34):25143-25170. https://doi.org/10.1007/s11042-020-09214-9S25143251707933-34Akhshabi S, Narayanaswamy S, Begen AC, Dovrolis C (2012) An experimental evaluation of rate-adaptive video players over HTTP. Signal process. Image Commun 27(4):271–287. https://doi.org/10.1016/j.image.2011.10.003Android Developers webpage, ExoPlayer. Available online at: https://developer.android.com/guide/topics/media/exoplayer.html . Accessed: Jun. (2019)Bampis CG, Li Z, Bovik AC (2018) SpatioTemporal feature integration and model fusion for full reference video quality assessment. IEEE Trans on Circuits and Syst for Video Tech 29:2256–2270. https://doi.org/10.1109/TCSVT.2018.2868262Barman N, Martini MG (2019) QoE modeling for HTTP adaptive video streaming - a survey and open challenges. IEEE Access 7:30831–30859. https://doi.org/10.1109/ACCESS.2019.2901778Belda R (2013) Algoritmo de adaptación DASH: Look Ahead. Master Thesis. Universitat Politècnica de València. http://hdl.handle.net/10251/33359 .Belda R, de Fez I, Arce P, Guerri J C (2018) Look ahead: a DASH adaptation algorithm. Proc. of the IEEE Int. Symp. On broadband multimed. Syst. And broadcast., Valencia, Spain: article no. 158. https://doi.org/10.1109/BMSB.2018.8436718 .Blender Foundation webpage. Available online at: https://www.blender.org/foundation . Accessed: Jun. (2019).Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20-3:273–297. https://doi.org/10.1023/A:1022627411411DASH Industry forum webpage. Available online at: http://dashif.org . Accessed: Jun. (2019)Ghadiyaram D, Pan J, Bovik AC (2019) A subjective and objective study of stalling events in mobile streaming videos. IEEE Trans on Circuits and Syst for Video Technol 29(1):183–197. https://doi.org/10.1109/TCSVT.2017.2768542Ghent University. 4G/LTE bandwidth logs. Available online at: http://users.ugent.be/~jvdrhoof/dataset-4g . Accessed: Jun. (2019).Github webpage. A DASH segment size aware rate adaptation model for DASH. Available online at: https://github.com/pari685/AStream . Accessed: Jun. (2019)GitHub website. Dashgen, Multimedia Communications Group. Available online at: https://github.com/comm-iteam/dashgen . Accessed: Jun. (2019).van der Hooft J, Petrangeli S, Wauters T, Huysegems R, Alface PR, Bostoen T, De Turck F (2016) HTTP/2-based adaptive streaming of HEVC video over 4G/LTE networks. IEEE Commun Lett 20(1):2177–2180. https://doi.org/10.1109/LCOMM.2016.2601087Huang TY, Johari R, McKeown N, Trunnell M, Watson M (2014) A buffer-based approach to rate adaptation: evidence from a large video streaming service. Proc. of the 2014 ACM Conf. On SIGCOMM, Chicago, IL, USA: 187-198. https://doi.org/10.1145/2619239.2626296Institute of Telecommunications and Multimedia Applications website. Look Ahead Demo. Available online at: https://lookahead.iteam.upv.es . Accessed: Jun. (2019)ISO/IEC 23009–1:2014 (2014) Dynamic adaptive streaming over HTTP (DASH) - Part 1: media presentation description and segment formats.Juluri P, Tamarapalli V, Medhi D (2015) SARA: segment aware rate adaptation algorithm for dynamic adaptive streaming over HTTP. Proc. of the IEEE Int. Conf. On Commun. Workshop (ICCW), London, UK: 1765-1770. https://doi.org/10.1109/ICCW.2015.7247436 .Juluri P, Tamarapalli V, Medhi D (2016) QoE management in DASH systems using the segment aware rate adaptation algorithm. Proc. of the IEEE/IFIP Netw. Oper. And Manag. Symp. (NOMS), Istanbul, Turkey: 129-136. https://doi.org/10.1109/NOMS.2016.7502805 .Kua J, Armitage G, Branch P (2017) A survey of rate adaptation techniques for dynamic adaptive streaming over HTTP. IEEE Commun Surv & Tutor 19(3):1842–1866. https://doi.org/10.1109/COMST.2017.2685630Lee S, Youn K, Chung K (2015) Adaptive video quality control scheme to improve QoE of MPEG DASH. Proc. of IEEE Int. Conf. On Consum. Electron. (ICCE), Las Vegas, NV, USA: 126-127. https://doi.org/10.1109/ICCE.2015.7066348 .Li S, Zhang F, Ma L, Ngan K (2011) Image quality assessment by separately evaluating detail losses and additive impairments. IEEE Trans. on Multimed. 13-5:935–949. https://doi.org/10.1109/TMM.2011.2152382Liu C, Bouazizi I, Gabbouj M (2011) Rate adaptation for adaptive HTTP streaming. Proc. of the second annual ACM Conf. On multimed. Syst. (MMSys), San Jose, CA, USA: 169-174. https://doi.org/10.1145/1943552.1943575 .Medium webpage (2016) Toward a practical perceptual video quality metric. Available online at: https://medium.com/netflix-techblog/toward-a-practical-perceptual-video-quality-metric-653f208b9652 . Accessed: Jun. 2019.Mobile Video Service Performance Study (2015) HUAWEI white paper. Available online at: http://www.ctiforum.com/uploadfile/2015/0701/20150701091255294.pdf .Mok RKP, Luo X, Chan EWW, Chang RKC (2012) QDASH: a QoE-aware DASH system. Proc. of multim. Syst. Conf. (MMSys), Chapel Hill, NC, USA: 11-22. https://doi.org/10.1145/2155555.2155558Moldovan C, Hagn K, Sieber C, Kellerer W, Hoßfeld T (2017) Keep calm and don’t switch: about the relationship between switches and quality in HAS. Proc. of the Int. Teletraffic Congr. (ITC), Genoa, Italy: pp. 1-6. https://doi.org/10.23919/ITC.2017.8065802Müller C, Lederer S, Timmerer C (2012) An evaluation of dynamic adaptive streaming over HTTP in vehicular environments. Proc. of the 4th workshop on mob. Video (MoVid), Chapel Hill, NC, USA: 37-42. https://doi.org/10.1145/2151677.2151686Nguyen T, Vu T, Nguyen DV, Ngoc NP, and Thang TC (2015) QoE optimization for adaptive streaming with multiple VBR videos. Proc. of the Int. Conf. On comp., Manag. And Telecommun. (ComManTel), DaNang, Vietnam: 189-193. https://doi.org/10.1109/ComManTel.2015.7394285 .Qin Y, H. Shuai, Pattipati K R, Qian F, Sen S, Wang B, Yue C (2018) ABR Streaming of VBR-encoded videos: characterization, challenges, and solutions. Proc. of ACM CoNext 2018, Heraklion, Greece: 366–378. https://doi.org/10.1145/3281411.3281439 .Samain J, Carofiglio G, Muscariello L, Papalini M, Sardara M, Tortelli M, Rossi D (2017) Dynamic adaptive video streaming: towards a systematic comparison of ICN and TCP/IP. IEEE Trans on Multimed 19(10):2166–2181. https://doi.org/10.1109/TMM.2017.2733340Sheikh H, Bovik A (2006) Image information and visual quality. IEEE Trans on Image Process 15(2):430–444. https://doi.org/10.1109/TIP.2005.859378Shuai Y, Herfet T (2016). A buffer dynamic stabilizer for low-latency adaptive video streaming. Proc. of the Int. Conf. on Consum. Electron., Berlin: 1–5. https://doi.org/10.1109/ICCE-Berlin.2016.7684742 .Tavakoli S, Egger S, Seufert M, Schatz R, Brunnström K, García N (2016) Perceptual quality of HTTP adaptive streaming strategies: cross-experimental analysis of multi-laboratory and crowdsourced subjective studies. IEEE Journal on Select Areas in Commun 34-8:2141–2153. https://doi.org/10.1109/JSAC.2016.2577361Yarnagula H K, Juluri P, Mehr S K, Tamarapalli V, Medhi D (2019) QoE for Mobile clients with segment-aware rate adaptation algorithm (SARA) for DASH video streaming. ACM trans. On multimed. Comput., Commun., and Appl. (TOMM) 15(2):article no. 36 https://doi.org/10.1145/3311749 .Yin X, Sekar V, Sinopoli B (2014) Toward a principled framework to design dynamic adaptive streaming algorithms over HTTP. Proc. of the 13th ACM workshop on hot topics in Netw. (HotNets), Los Angeles, CA, USA: 1-7. https://doi.org/10.1145/2670518.2673877 .YouTube webpage (2019) Youtube press. Available online at: https://www.youtube.com/yt/about/press . Accessed: Jun. 2019.Youtube webpage, Google I/O ‘18: Building feature-rich media apps with ExoPlayer. Available online at: https://youtu.be/svdq1BWl4r8?t=2m . Published: May (2018)Yu L, Tillo T, Xiao J (2017) QoE-driven dynamic adaptive video streaming strategy with future information. IEEE Trans on Broadcast 63-3:523–534. https://doi.org/10.1109/TBC.2017.2687698Zhao S, Li Z, Medhi D, Lai P, Liu S (2017) Study of user QoE improvement for dynamic adaptive streaming over HTTP (MPEG-DASH). Proc. of the Int. Conf. On Comput., network. And Commun. (ICNC): multimed. Comput. And Commun., Santa Clara, CA, USA: 566-570. https://doi.org/10.1109/ICCNC.2017.7876191 .Zhou Y, Duan Y, Sun J, Guo Z (2014) Towards a simple and smooth rate adaption for VBR video in DASH. Proc. of the IEEE Vis. Commun. and Image Process. Conf, Valletta, pp 9–12. https://doi.org/10.1109/VCIP.2014.7051491Zhou C, Lin C-W, Guo Z (2016) mDASH: a Markov decision-based rate adaptation approach for dynamic HTTP streaming. IEEE Trans. on Multimed 18(4):738–751. https://doi.org/10.1109/TMM.2016.252265

QoE-Based Low-Delay Live Streaming Using Throughput Predictions

Recently, HTTP-based adaptive streaming has become the de facto standard for video streaming over the Internet. It allows clients to dynamically adapt media characteristics to network conditions in order to ensure a high quality of experience, that is, minimize playback interruptions, while maximizing video quality at a reasonable level of quality changes. In the case of live streaming, this task becomes particularly challenging due to the latency constraints. The challenge further increases if a client uses a wireless network, where the throughput is subject to considerable fluctuations. Consequently, live streams often exhibit latencies of up to 30 seconds. In the present work, we introduce an adaptation algorithm for HTTP-based live streaming called LOLYPOP (Low-Latency Prediction-Based Adaptation) that is designed to operate with a transport latency of few seconds. To reach this goal, LOLYPOP leverages TCP throughput predictions on multiple time scales, from 1 to 10 seconds, along with an estimate of the prediction error distribution. In addition to satisfying the latency constraint, the algorithm heuristically maximizes the quality of experience by maximizing the average video quality as a function of the number of skipped segments and quality transitions. In order to select an efficient prediction method, we studied the performance of several time series prediction methods in IEEE 802.11 wireless access networks. We evaluated LOLYPOP under a large set of experimental conditions limiting the transport latency to 3 seconds, against a state-of-the-art adaptation algorithm from the literature, called FESTIVE. We observed that the average video quality is by up to a factor of 3 higher than with FESTIVE. We also observed that LOLYPOP is able to reach a broader region in the quality of experience space, and thus it is better adjustable to the user profile or service provider requirements.Comment: Technical Report TKN-16-001, Telecommunication Networks Group, Technische Universitaet Berlin. This TR updated TR TKN-15-00

QoE-centric management of advanced multimedia services

Over the last years, multimedia content has become more prominent than ever. Particularly, video streaming is responsible for more than a half of the total global bandwidth consumption on the Internet. As the original Internet was not designed to deliver such real-time, bandwidth-consuming applications, a serious challenge is posed on how to efficiently provide the best service to the users. This requires a shift in the classical approach used to deliver multimedia content, from a pure Quality of Service (QoS) to a full Quality of Experience (QoE) perspective. While QoS parameters are mainly related to low-level network aspects, the QoE reflects how the end-users perceive a particular multimedia service. As the relationship between QoS parameters and QoE is far from linear, a classical QoS-centric delivery is not able to fully optimize the quality as perceived by the users. This paper provides an overview of the main challenges this PhD aims to tackle in the field of end-to-end QoE optimization of video streaming services and, more precisely, of HTTP Adaptive Streaming (HAS) solutions, which are quickly becoming the de facto standard for video delivery over the Internet

Streaming Video QoE Modeling and Prediction: A Long Short-Term Memory Approach

HTTP based adaptive video streaming has become a popular choice of streaming due to the reliable transmission and the flexibility offered to adapt to varying network conditions. However, due to rate adaptation in adaptive streaming, the quality of the videos at the client keeps varying with time depending on the end-to-end network conditions. Further, varying network conditions can lead to the video client running out of playback content resulting in rebuffering events. These factors affect the user satisfaction and cause degradation of the user quality of experience (QoE). It is important to quantify the perceptual QoE of the streaming video users and monitor the same in a continuous manner so that the QoE degradation can be minimized. However, the continuous evaluation of QoE is challenging as it is determined by complex dynamic interactions among the QoE influencing factors. Towards this end, we present LSTM-QoE, a recurrent neural network based QoE prediction model using a Long Short-Term Memory (LSTM) network. The LSTM-QoE is a network of cascaded LSTM blocks to capture the nonlinearities and the complex temporal dependencies involved in the time varying QoE. Based on an evaluation over several publicly available continuous QoE databases, we demonstrate that the LSTM-QoE has the capability to model the QoE dynamics effectively. We compare the proposed model with the state-of-the-art QoE prediction models and show that it provides superior performance across these databases. Further, we discuss the state space perspective for the LSTM-QoE and show the efficacy of the state space modeling approaches for QoE prediction