Quality of experience and access network traffic management of HTTP adaptive video streaming

The thesis focuses on Quality of Experience (QoE) of HTTP adaptive video streaming (HAS) and traffic management in access networks to improve the QoE of HAS. First, the QoE impact of adaptation parameters and time on layer was investigated with subjective crowdsourcing studies. The results were used to compute a QoE-optimal adaptation strategy for given video and network conditions. This allows video service providers to develop and benchmark improved adaptation logics for HAS. Furthermore, the thesis investigated concepts to monitor video QoE on application and network layer, which can be used by network providers in the QoE-aware traffic management cycle. Moreover, an analytic and simulative performance evaluation of QoE-aware traffic management on a bottleneck link was conducted. Finally, the thesis investigated socially-aware traffic management for HAS via Wi-Fi offloading of mobile HAS flows. A model for the distribution of public Wi-Fi hotspots and a platform for socially-aware traffic management on private home routers was presented. A simulative performance evaluation investigated the impact of Wi-Fi offloading on the QoE and energy consumption of mobile HAS.Die Doktorarbeit beschäftigt sich mit Quality of Experience (QoE) – der subjektiv empfundenen Dienstgüte – von adaptivem HTTP Videostreaming (HAS) und mit Verkehrsmanagement, das in Zugangsnetzwerken eingesetzt werden kann, um die QoE des adaptiven Videostreamings zu verbessern. Zuerst wurde der Einfluss von Adaptionsparameters und der Zeit pro Qualitätsstufe auf die QoE von adaptivem Videostreaming mittels subjektiver Crowdsourcingstudien untersucht. Die Ergebnisse wurden benutzt, um die QoE-optimale Adaptionsstrategie für gegebene Videos und Netzwerkbedingungen zu berechnen. Dies ermöglicht Dienstanbietern von Videostreaming verbesserte Adaptionsstrategien für adaptives Videostreaming zu entwerfen und zu benchmarken. Weiterhin untersuchte die Arbeit Konzepte zum Überwachen von QoE von Videostreaming in der Applikation und im Netzwerk, die von Netzwerkbetreibern im Kreislauf des QoE-bewussten Verkehrsmanagements eingesetzt werden können. Außerdem wurde eine analytische und simulative Leistungsbewertung von QoE-bewusstem Verkehrsmanagement auf einer Engpassverbindung durchgeführt. Schließlich untersuchte diese Arbeit sozialbewusstes Verkehrsmanagement für adaptives Videostreaming mittels WLAN Offloading, also dem Auslagern von mobilen Videoflüssen über WLAN Netzwerke. Es wurde ein Modell für die Verteilung von öffentlichen WLAN Zugangspunkte und eine Plattform für sozialbewusstes Verkehrsmanagement auf privaten, häuslichen WLAN Routern vorgestellt. Abschließend untersuchte eine simulative Leistungsbewertung den Einfluss von WLAN Offloading auf die QoE und den Energieverbrauch von mobilem adaptivem Videostreaming

Unveiling the end-user viewport resolution from encrypted video traces

International audienceVideo streaming is without doubt the most requested Internet service, and main source of pressure on the Internet infrastructure. At the same time, users are no longer satisfied by the Internet's best effort service, instead, they expect a seamless service of high quality from the side of the network. As result, Internet Service Providers (ISP) engineer their traffic so as to improve their end-users' experience and avoid economic losses. Content providers from their side, and to enforce customers privacy, have shifted towards end-to-end encryption (e.g., TLS/SSL). Video streaming relies on the dynamic adaptive streaming over HTTP protocol (DASH) which takes into consideration the underlying network conditions (e.g., delay, loss rate, and throughput) and the viewport capacity (e.g., screen resolution) to improve the experience of the end user in the limit of the available network resources. In this work, we propose an experimental framework able to infer fine-grained video flow information such as chunk sizes from encrypted YouTube video traces. We also present a novel technique to separate video and audio chunks from encrypted traces based on Gaussian Mixture Models (GMM). Then, we leverage our dataset to train models able to predict the class of viewport (either SD or HD) per video session with an average 92% accuracy and 85% F1-score. The prediction of the exact viewport resolution is also possible but shows a lower accuracy than the viewport class

Análisis comparativo del servicio de streaming de video de youtube entre las operadoras de datos móviles 4G en el Ecuador, para proponer parámetros mínimos de calidad.

Se realizó un análisis comparativo del servicio de streaming de video de YouTube, sobre las redes de datos móviles 4G en el Ecuador. Para la comparación se usó herramientas estadísticas sobre datos recolectados en campo mediante mediciones realizadas en la ciudad de Cuenca - Ecuador, en un ambiente outdoor fijo. Para las mediciones se usó la herramienta tecnológica “Tems Pocket”, de la empresa “InfoVista”; equipos con los que cuenta la Agencia de Regulación y Control de las Telecomunicaciones (ARCOTEL), institución con la que se trabajó para esta investigación. También se realizó una propuesta del uso de la herramienta “Tems Pocket” para futuras mediciones. El análisis comparativo se lo hace en base a los Indicadores Clave de Desempeño (KPI), tanto de Calidad de Servicio (QoS) como de Calidad de Experiencia (QoE). La Calidad de Experiencia (QoE) se midió a través del KPI “Average Video Quality” con el uso del estándar PEVQ-s, sobre una escala MOS. En la investigación se determinó con pruebas estadísticas que el KPI “Average Video Quality”, en las tres operadoras móviles, provienen de poblaciones de idéntico comportamiento, por lo tanto con distribuciones iguales. Finalmente con los resultados obtenidos se proponen parámetros mínimos u objetivos de calidad del servicio para los KPIs analizados, los cuales son: “Video Play Start Time” ≤ 1,3 segundos, “Service Access Time” ≤ 0,3 segundos, “Reproduction Start Delay” ≤ 1 segundos y “Average Video Quality” con MOS ≥ 4.It was conducted a comparative analysis of the service of streaming of video of You Tube on the networks of 4G mobile data in Ecuador. In order to make the comparison, statistical tools were used on data collected in the field by means of measurements carried out in Cuenca city, Ecuador, into a fixed outdoor environment. For the measurements, it was used the technological tool "Tems pocket" belonging to "Info Vista" company, equipment that the Agency of Regulation and Control of Telecommunications (ARCOTEL) owns. This institution is the place where this research was carried out. In addition to this, a proposal for the use of the "Pocket Tems" tool for future measurements was developed. The comparative analysis was done according to the Key Performance Indicators (KPI), both for Quality of Service (QoS) and quality of experience (QoE). The quality of experience (QoE) was measured by means of the KPI "Average Video Quality" with the use of the PVQ-s standard, on a MOS scale. In the investigation it was determined with statistical tests that the KPI "Average Video Quality", in the three mobile operators, came from populations of identical behaviour, therefore with equal distributions. Finally, with the collected results, minimum parameters, or service quality objectives are proposed for the analysed KPIs, which are: "Video Play Start Time" ≤1.3 seconds, "Service Access Time" ≤ 0.3, seconds "Reproduction Start Delay " ≤ 1 second, and “Average Video Quality" with MOS ≥ 4

Improving the transfer of machine learning-based video QoE estimation across diverse networks

With video streaming traffic generally being encrypted end-to-end, there is a lot of interest from network operators to find novel ways to evaluate streaming performance at the application layer. Machine learning (ML) has been extensively used to develop solutions that infer application-level Key Performance Indicators (KPI) and/or Quality of Experience (QoE) from the patterns in encrypted traffic. Having such insights provides the means for more user-centric traffic management and enables the mitigation of QoE degradations, thus potentially preventing customer churn. The ML–based QoE/KPI estimation solutions proposed in literature are typically trained on a limited set of network scenarios and it is often unclear how the obtained models perform if applied in a previously unseen setting (e.g., if the model is applied at the premises of a different network operator). In this paper, we address this gap by cross-evaluating the performance of QoE/KPI estimation models trained on 4 separate datasets generated from streaming 48000 video streaming sessions. The paper evaluates a set of methods for improving the performance of models when applied in a different network. Analyzed methods require no or considerably less application-level ground-truth data collected in the new setting, thus significantly reducing the extensiveness of required data collection

From Network Traffic Measurements to QoE for Internet Video

International audienceVideo streaming is a dominant contributor to the global Internet traffic. Consequently, monitoring video streaming Quality of Experience (QoE) is of paramount importance to network providers. Monitoring QoE of video is a challenge as most of the video traffic of today is encrypted. In this paper, we consider this challenge and present an approach based on controlled experimentation and machine learning to estimate QoE from encrypted video traces using network level measurements only. We consider a case of YouTube and play out a wide range of videos under realistic network conditions to build ML models (classification and regression) that predict the subjective MOS (Mean Opinion Score) based on the ITU P.1203 model along with the QoE metrics of startup delay, quality (spatial resolution) of playout and quality variations, and this is using only the underlying network Quality of Service (QoS) features. We comprehensively evaluate our approach with different sets of input network features and output QoE metrics. Overall, our classification models predict the QoE metrics and the ITU MOS with an accuracy of 63-90% while the regression models show low error; the ITU MOS (1-5) and the startup delay (in seconds) are predicted with a root mean square error of 0.33 and 2.66 respectively

Uma abordagem preditiva de DASH QoE baseada em aprendizado de máquina em multi-access edge computing

Orientador: Christian Rodolfo Esteve RothenbergDissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de Engenharia Elétrica e de ComputaçãoResumo: O tráfego de serviços de vídeo multimídia está crescendo rapidamente nas redes móveis nos últimos anos. Os serviços de vídeo que usam técnicas de Dynamic Adaptive Streaming sobre HTTP (DASH) dominaram o tráfego total da Internet para transportar o tráfego de vídeo. Espera-se que as operadoras de rede móvel (Mobile Network Operators - MNOs) continuem atendendo a essa demanda crescente por tráfego de vídeo suportado por DASH, ao mesmo tempo em que fornecem uma alta qualidade de experiência (Quality of Experience - QoE) aos usuários finais. Além disso, as operadoras precisam ter um conhecimento claro acerca da qualidade de vídeo percebida pelos usuários finais e relacioná-la com o monitoramento em nível de rede, ou com informações de telemetria para identificação de problemas, análise da causa raiz e predição de padrões. Para garantir um gerenciamento de tráfego de rede com reconhecimento de QoE, um pré-requisito é que os MNOs monitorem o tráfego de rede passivamente e realizem medições efetivas de indicadores-chave de desempenho (Key Performance Indicators - KPIs) de QoE, como resoluções, eventos de paralisação, entre outros, que influenciam diretamente a percepção do usuário final. Muitas abordagens da literatura foram propostas para medir os KPIs com o objetivo de fornecer uma qualidade de serviço de vídeo aceitável. A maioria das soluções exige consciência de contexto do usuário final, o que não é viável do ponto de vista do MNO. No entanto, Deep Packet Inspection (DPI), outra solução mais amplamente usada para estimar os KPIs diretamente do tráfego de rede, não é mais uma solução conveniente para as operadoras devido à adoção de criptografia de streaming de vídeo fim-a-fim sobre TCP (HTTPs) e QUIC. Portanto, o aprendizado de máquina (Machine Learning - ML) passou a ser recentemente aceito como uma solução bem reconhecida para estimar KPIs de QoE, analisando os padrões de tráfego criptografados bem como estatísticas como qualidade de serviço (Quality of Service - QoS). Este trabalho apresenta uma abordagem mais refinada e leve, baseada em aprendizado de máquina, denominada Edge QoE Probe, para estimar QoE do usuário final para o serviço de vídeo DASH, monitorando passivamente o tráfego de rede criptografado na borda da rede. Nossa abordagem pode avaliar vários KPIs de QoE, como por exemplo resolução, taxa de bits, proporção de paralisação, entre outros, tanto em tempo real quanto por sessão. Além disso, neste trabalho investigamos o desempenho do vídeo DASH sobre o protocolo de transporte tradicional TCP (HTTPs) e QUIC. Para este propósito, avaliamos experimentalmente diferentes traces de rede celular em um ambiente emulado de alta fidelidade e comparamos o desempenho comportamental de algoritmos Adaptive Bitrate Streaming (ABS) considerando KPIs de QoE sobre TCP (HTTPs) e QUIC. Nossos resultados empíricos mostram que os algoritmos tradicionais de ABS usando QUIC como transporte precisariam alterações específicas para melhorar o desempenho em termos de QoE de vídeo baseados em DASHAbstract: Multimedia video services traffic is rapidly growing in mobile networks in recent years. Video services using Dynamic Adaptive Streaming over HTTP (DASH) techniques have dominated the total internet traffic to carry video traffic. Mobile Network Operators (MNOs) are expected to run on with this growing demand for DASH-supported video traffic while providing a high Quality of Experience (QoE) to the end-users. Besides, operators need to have a crystal notion of video quality perceived by the end-users and correlate them with network-level monitoring or telemetry information for problem identification, root cause analysis, and pattern prediction. To ensure QoE–aware network traffic management, a prerequisite for the MNOs is to monitor the network traffic passively and measure objective QoE Key Performance Indicators (KPIs) (such as resolutions and stalling events) effectively that directly influence end-user subjective feedback. Many literature approaches have been proposed to measure the KPIs aimed to deliver acceptable video service quality. Most of the solutions require end-user awareness, which is not viable from the MNOs' perspective. However, Deep Packet Inspection (DPI), another most widely used solution to estimate the KPIs directly from network traffic, is not a convenient solution anymore for the operators due to the adoption of end-to-end video streaming encryption over TCP (HTTPs) and QUIC transport protocol. Hence, in recent, Machine Learning (ML) has been accepted as a well-recognized solution for estimating QoE KPIs by analyzing the encrypted traffic patterns and statistics as Quality of Service (QoS). This work presents an ML-based lightweight and fine-grained Edge QoE Probe approach to estimate the end-user QoE for DASH video service by passively monitoring the encrypted network traffic on the edge of the network. Our approach can assess numerous QoE KPIs (such as resolution, bit-rate, quality switches, startup delay, and stall ratio) both in a real-time and per-session manner. Moreover, we investigate the DASH video service performance over the traditional TCP (HTTPs) and QUIC transport protocol in this work. For this purpose, we experimentally evaluate different cellular network traces in a high-fidelity emulated testbed and compare the behavioral performance of Adaptive Bitrate Streaming (ABS) algorithms considering QoE KPIs over TCP (HTTPs) and QUIC. Our empirical results show that QUIC suffers from traditional state-of-the-art ABS algorithms' ineffectiveness to improve video streaming performance without specific changesMestradoEngenharia de ComputaçãoMestre em Engenharia ElétricaFuncam

A Methodology for Performance Benchmarking of Mobile Networks for Internet Video Streaming

International audienceVideo streaming is a dominant contributor to the global Internet traffic. Consequently, gauging network performance w.r.t. the video Quality of Experience (QoE) is of paramount importance to both telecom operators and regulators. Modern video streaming systems, e.g. YouTube, have huge catalogs of billions of different videos that vary significantly in content type. Owing to this difference, the QoE of different videos as perceived by end users can vary for the same network Quality of Service (QoS). In this paper, we present a methodology for benchmarking performance of mobile operators w.r.t Internet video that considers this variation in QoE. We take a data-driven approach to build a predictive model using supervised machine learning (ML) that takes into account a wide range of videos and network conditions. To that end, we first build and analyze a large catalog of YouTube videos. We then propose and demonstrate a framework of controlled experimentation based on active learning to build the training data for the targeted ML model. Using this model, we then devise YouScore, an estimate of the percentage of YouTube videos that may play out smoothly under a given network condition. Finally, to demonstrate the benchmarking utility of YouScore, we apply it on an open dataset of real user mobile network measurements to compare performance of mobile operators for video streaming

A quality of experience approach in smartphone video selection framework for energy efficiency

Online video streaming is getting more common in the smartphone device nowadays. Since the Corona Virus (COVID-19) pandemic hit all human across the globe in 2020, the usage of online streaming among smartphone user are getting more vital. Nevertheless, video streaming can cause the smartphone energy to drain quickly without user to realize it. Also, saving energy alone is not the most significant issues especially if with the lack of attention on the user Quality of Experience (QoE). A smartphones energy management is crucial to overcome both of these issues. Thus, a QoE Mobile Video Selection (QMVS) framework is proposed. The QMVS framework will govern the tradeoff between energy efficiency and user QoE in the smartphone device. In QMVS, video streaming will be using Dynamic Video Attribute Pre-Scheduling (DVAP) algorithm to determine the energy efficiency in smartphone devices. This process manages the video attribute such as brightness, resolution, and frame rate by turning to Video Content Selection (VCS). DVAP is handling a set of rule in the Rule Post-Pruning (RPP) method to remove an unused node in list tree of VCS. Next, QoE subjective method is used to obtain the Mean Opinion Score (MOS) of users from a survey experiment on QoE. After both experiment results (MOS and energy) are established, the linear regression technique is used to find the relationship between energy consumption and user QoE (MOS). The last process is to analyze the relationship of VCS results by comparing the DVAP to other recent video streaming applications available. Summary of experimental results demonstrate the significant reduction of 10% to 20% energy consumption along with considerable acceptance of user QoE. The VCS outcomes are essential to help users and developer deciding which suitable video streaming format that can satisfy energy consumption and user QoE

Quadri-dimensional approach for data analytics in mobile networks

The telecommunication market is growing at a very fast pace with the evolution of new technologies to support high speed throughput and the availability of a wide range of services and applications in the mobile networks. This has led to a need for communication service providers (CSPs) to shift their focus from network elements monitoring towards services monitoring and subscribers’ satisfaction by introducing the service quality management (SQM) and the customer experience management (CEM) that require fast responses to reduce the time to find and solve network problems, to ensure efficiency and proactive maintenance, to improve the quality of service (QoS) and the quality of experience (QoE) of the subscribers. While both the SQM and the CEM demand multiple information from different interfaces, managing multiple data sources adds an extra layer of complexity with the collection of data. While several studies and researches have been conducted for data analytics in mobile networks, most of them did not consider analytics based on the four dimensions involved in the mobile networks environment which are the subscriber, the handset, the service and the network element with multiple interface correlation. The main objective of this research was to develop mobile network analytics models applied to the 3G packet-switched domain by analysing data from the radio network with the Iub interface and the core network with the Gn interface to provide a fast root cause analysis (RCA) approach considering the four dimensions involved in the mobile networks. This was achieved by using the latest computer engineering advancements which are Big Data platforms and data mining techniques through machine learning algorithms.Electrical and Mining EngineeringM. Tech. (Electrical Engineering