On the analysis of youTube QoE in cellular networks through in-smartphone measurements

International audienceCellular-network operators are becoming increasingly interested in knowing the Quality of Experience (QoE) of their customers. QoE measurements represent today a main source of information to monitor, analyze, and subsequently manage operational networks. In this paper, we focus on the analysis of YouTube QoE in cellular networks, using QoE and distributed network measurements collected in real users' smart-phones. Relying on YoMoApp, a well-known tool for collecting YouTube smartphone measurements and QoE feedback in a crowdsourcing fashion, we have built a dataset covering about 360 different cellular users around the globe, throughout the past five years. Using this dataset, we study the characteristics of different QoE-relevant features for YouTube in smartphones. Measurements reveal a constant improvement of YouTube QoE in cellular networks over time, as well as an enhancement of the YouTube video streaming functioning in smartphones. Using the gathered measurements, we additionally investigate two case studies for YouTube QoE monitoring and analysis in cellular networks: the machine-learning-based prediction of QoE-relevant metrics from network-level measurements, and the modeling and assessment of YouTube QoE and user engagement in cellular networks and smartphone devices. Last but not least, we introduce the YoMoApp cloud dashboard to openly share smartphone YouTube QoE measurements, which allows anyone using the YoMoApp smartphone app to get immediate access to all the raw measurements collected at her devices

Systems and Methods for Measuring and Improving End-User Application Performance on Mobile Devices

In today's rapidly growing smartphone society, the time users are spending on their smartphones is continuing to grow and mobile applications are becoming the primary medium for providing services and content to users. With such fast paced growth in smart-phone usage, cellular carriers and internet service providers continuously upgrade their infrastructure to the latest technologies and expand their capacities to improve the performance and reliability of their network and to satisfy exploding user demand for mobile data. On the other side of the spectrum, content providers and e-commerce companies adopt the latest protocols and techniques to provide smooth and feature-rich user experiences on their applications. To ensure a good quality of experience, monitoring how applications perform on users' devices is necessary. Often, network and content providers lack such visibility into the end-user application performance. In this dissertation, we demonstrate that having visibility into the end-user perceived performance, through system design for efficient and coordinated active and passive measurements of end-user application and network performance, is crucial for detecting, diagnosing, and addressing performance problems on mobile devices. My dissertation consists of three projects to support this statement. First, to provide such continuous monitoring on smartphones with constrained resources that operate in such a highly dynamic mobile environment, we devise efficient, adaptive, and coordinated systems, as a platform, for active and passive measurements of end-user performance. Second, using this platform and other passive data collection techniques, we conduct an in-depth user trial of mobile multipath to understand how Multipath TCP (MPTCP) performs in practice. Our measurement study reveals several limitations of MPTCP. Based on the insights gained from our measurement study, we propose two different schemes to address the identified limitations of MPTCP. Last, we show how to provide visibility into the end- user application performance for internet providers and in particular home WiFi routers by passively monitoring users' traffic and utilizing per-app models mapping various network quality of service (QoS) metrics to the application performance.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/146014/1/ashnik_1.pd

Quality assessment and usage behavior of a mobile voice-over-IP service

Voice-over-IP (VoIP) services offer users a cheap alternative to the traditional mobile operators to make voice calls. Due to the increased capabilities and connectivity of mobile devices, these VoIP services are becoming increasingly popular on the mobile platform. Understanding the user's usage behavior and quality assessment of the VoIP service plays a key role in optimizing the Quality of Experience (QoE) and making the service to succeed or to fail. By analyzing the usage and quality assessments of a commercial VoIP service, this paper identifies device characteristics, context parameters, and user aspects that influence the usage behavior and experience during VoIP calls. Whereas multimedia services are traditionally evaluated by monitoring usage and quality for a limited number of test subjects and during a limited evaluation period, this study analyzes the service usage and quality assessments of more than thousand users over a period of 120 days. This allows to analyze evolutions in the usage behavior and perceived quality over time, which has not been done up to now for a widely-used, mobile, multimedia service. The results show a significant evolution over time of the number of calls, the call duration, and the quality assessment. The time of the call, the used network, and handovers during the call showed to have a significant influence on the users' quality assessments

Mobile web and app QoE monitoring for ISPs - from encrypted traffic to speed index through machine learning

International audienceWeb browsing is one of the key applications of the Internet. In this paper, we address the problem of mobile Web and App QoE monitoring from the Internet Service Provider (ISP) perspective, relying on in-network, passive measurements. Our study targets the analysis of Web and App QoE in mobile devices, including mobile browsing in smartphones and tablets, as well as mobile apps. As a proxy to Web QoE, we focus on the analysis of the well-known Speed Index (SI) metric. Given the wide adoption of end-to-end encryption, we resort to machine-learning models to infer the SI of individual web page and app loading sessions, using as input only packet level data. Empirical evaluations on a large, multi mobile-device corpus of Web and App QoE measurements for top popular websites and selected apps demonstrate that the proposed solution can properly infer the SI from in-network, encrypted-traffic measurements, relying on learning-based models. Our study also reveals relevant network and web page content characteristics impacting Web QoE in mobile devices, providing a complete overview on the mobile Web and App QoE assessment problem

Beauty is in the Eye of the Smartphone Holder - A Data Driven Analysis of YouTube Mobile QoE

International audienceMeasuring the Quality of Experience (QoE) undergone by cellular network users has become paramount for cellular ISPs. Given its overwhelming dominance and ever-growing popularity, this paper focuses on the analysis of QoE for YouTube in mobile networks. Using a large-scale dataset of crowdsourced YouTube QoE measurements collected in smartphones with YoMoApp, we analyze the evolution of multiple relevant QoE-related metrics over time for YouTube mobile users. The dataset includes measurements from more than 360 users worldwide, spanning over the last five years. Our data-driven analysis shows a systematic performance and QoE improvement of YouTube in mobile devices over time, accompanied by an improvement of cellular network performance and by an optimization of the YouTube streaming behavior for smartphones

Machine Learning Models for YouTube QoE and User Engagement Prediction in Smartphones

International audienceMeasuring and monitoring YouTube Quality of Experience is a challenging task, especially when dealing with cellular networks and smartphone users. Using a large-scale database of crowdsourced YouTube-QoE measurements in smartphones, we conceive multiple machine-learning models to infer different YouTube-QoE-relevant metrics and user-behavior-related metrics from network-level measurements, without requiring root access to the smartphone, video-player embedding, or any other reverse-engineering-like approaches. The dataset includes measurements from more than 360 users worldwide, spanning over the last five years. Our preliminary results suggest that QoE-based monitoring of YouTube mobile can be realized through machine learning models with high accuracy, relying only on network-related features and without accessing any higher-layer metric to perform the estimations