Mobile web and app QoE monitoring for ISPs - from encrypted traffic to speed index through machine learning

International audienceWeb browsing is one of the key applications of the Internet. In this paper, we address the problem of mobile Web and App QoE monitoring from the Internet Service Provider (ISP) perspective, relying on in-network, passive measurements. Our study targets the analysis of Web and App QoE in mobile devices, including mobile browsing in smartphones and tablets, as well as mobile apps. As a proxy to Web QoE, we focus on the analysis of the well-known Speed Index (SI) metric. Given the wide adoption of end-to-end encryption, we resort to machine-learning models to infer the SI of individual web page and app loading sessions, using as input only packet level data. Empirical evaluations on a large, multi mobile-device corpus of Web and App QoE measurements for top popular websites and selected apps demonstrate that the proposed solution can properly infer the SI from in-network, encrypted-traffic measurements, relying on learning-based models. Our study also reveals relevant network and web page content characteristics impacting Web QoE in mobile devices, providing a complete overview on the mobile Web and App QoE assessment problem

Do you agree? Contrasting Google's core web vitals and the impact of cookie consent banners with actual web QoE

Providing sophisticated web Quality of Experience (QoE) has become paramount for web service providers and network operators alike. Due to advances in web technologies (HTML5, responsive design, etc.), traditional web QoE models focusing mainly on loading times have to be refined and improved. In this work, we relate Google’s Core Web Vitals, a set of metrics for improving user experience, to the loading time aspects of web QoE, and investigate whether the Core Web Vitals and web QoE agree on the perceived experience. To this end, we first perform objective measurements in the web using Google’s Lighthouse. To close the gap between metrics and experience, we complement these objective measurements with subjective assessment by performing multiple crowdsourcing QoE studies. For this purpose, we developed CWeQS, a customized framework to emulate the entire web page loading process, and ask users for their experience while controlling the Core Web Vitals, which is available to the public. To properly configure CWeQS for the planned QoE study and the crowdsourcing setup, we conduct pre-studies, in which we evaluate the importance of the loading strategy of a web page and the importance of the user task. The obtained insights allow us to conduct the desired QoE studies for each of the Core Web Vitals. Furthermore, we assess the impact of cookie consent banners, which have become ubiquitous due to regulatory demands, on the Core Web Vitals and investigate their influence on web QoE. Our results suggest that the Core Web Vitals are much less predictive for web QoE than expected and that page loading times remain the main metric and influence factor in this context. We further observe that unobtrusive and acentric cookie consent banners are preferred by end-users and that additional delays caused by interacting with consent banners in order to agree to or reject cookies should be accounted along with the actual page load time to reduce waiting times and thus to improve web QoE

Measuring Roaming in Europe: Infrastructure and Implications on Users QoE

"Roam like Home" is the initiative of the European Commission to end the levy of extra charges when roaming within the European region. As a result, people can use data services more freely across Europe. However, the implications of roaming solutions on network performance have not been carefully examined yet. This paper provides an in-depth characterization of the implications of international data roaming within Europe. We build a unique roaming measurement platform using 16 different mobile networks deployed in 6 countries across Europe. Using this platform, we measure different aspects of international roaming in 4G networks in Europe, including mobile network configuration, performance characteristics, and quality of experience. We find that operators adopt a common approach to implement roaming called Home-routed roaming. This results in additional latency penalties of 60 ms or more, depending on geographical distance. This leads to worse browsing performance, with an increase in the metrics related to Quality of Experience (QoE) of users (Page Load time and Speed Index) in the order of 15-20%. We further analyze the impact of latency on QoE metrics in isolation and find that the penalty imposed by Home Routing leads to degradation on QoE metrics up to 150% in case of intercontinental roaming. We make our dataset public to allow reproducing the results

Systems and Methods for Measuring and Improving End-User Application Performance on Mobile Devices

In today's rapidly growing smartphone society, the time users are spending on their smartphones is continuing to grow and mobile applications are becoming the primary medium for providing services and content to users. With such fast paced growth in smart-phone usage, cellular carriers and internet service providers continuously upgrade their infrastructure to the latest technologies and expand their capacities to improve the performance and reliability of their network and to satisfy exploding user demand for mobile data. On the other side of the spectrum, content providers and e-commerce companies adopt the latest protocols and techniques to provide smooth and feature-rich user experiences on their applications. To ensure a good quality of experience, monitoring how applications perform on users' devices is necessary. Often, network and content providers lack such visibility into the end-user application performance. In this dissertation, we demonstrate that having visibility into the end-user perceived performance, through system design for efficient and coordinated active and passive measurements of end-user application and network performance, is crucial for detecting, diagnosing, and addressing performance problems on mobile devices. My dissertation consists of three projects to support this statement. First, to provide such continuous monitoring on smartphones with constrained resources that operate in such a highly dynamic mobile environment, we devise efficient, adaptive, and coordinated systems, as a platform, for active and passive measurements of end-user performance. Second, using this platform and other passive data collection techniques, we conduct an in-depth user trial of mobile multipath to understand how Multipath TCP (MPTCP) performs in practice. Our measurement study reveals several limitations of MPTCP. Based on the insights gained from our measurement study, we propose two different schemes to address the identified limitations of MPTCP. Last, we show how to provide visibility into the end- user application performance for internet providers and in particular home WiFi routers by passively monitoring users' traffic and utilizing per-app models mapping various network quality of service (QoS) metrics to the application performance.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/146014/1/ashnik_1.pd

Inferring Streaming Video Quality from Encrypted Traffic: Practical Models and Deployment Experience

Inferring the quality of streaming video applications is important for Internet service providers, but the fact that most video streams are encrypted makes it difficult to do so. We develop models that infer quality metrics (\ie, startup delay and resolution) for encrypted streaming video services. Our paper builds on previous work, but extends it in several ways. First, the model works in deployment settings where the video sessions and segments must be identified from a mix of traffic and the time precision of the collected traffic statistics is more coarse (\eg, due to aggregation). Second, we develop a single composite model that works for a range of different services (i.e., Netflix, YouTube, Amazon, and Twitch), as opposed to just a single service. Third, unlike many previous models, the model performs predictions at finer granularity (\eg, the precise startup delay instead of just detecting short versus long delays) allowing to draw better conclusions on the ongoing streaming quality. Fourth, we demonstrate the model is practical through a 16-month deployment in 66 homes and provide new insights about the relationships between Internet "speed" and the quality of the corresponding video streams, for a variety of services; we find that higher speeds provide only minimal improvements to startup delay and resolution

Exploiting Web Scraping for Education News Analysis Using Depth-First Search Algorithm

Online news is one source of data that is always up to date and provides information or factual data. The search engine is one of the features for users to be able to enter keywords based on the expected category quickly. The development of education in Indonesia makes it essential to discuss, in this study using unstructured data in online news with the keyword Education included as a parameter, and adding search methods in the field of Artificial Intelligence so that the data becomes more accurate. Data that used here was from online news, namely CNN Indonesia, Detikcom, and Liputan6. Using Python Programming with depth-first search method (DFS), when compared with the results data for relevant news. Web erosion using DFS will be very helpful in searching because this method can check the date data was sent and then track the destination URL. Of the three online media sites, Detikcom produces the highest monthly data yielding an average of 885 news about education. At the same time, Liputan6 has the least amount of data on average, 28 news per month, but the data obtained are very relevant compared to Detikcom and CNN Indonesia

Network Performance Management Using Application-centric Key Performance Indicators

The Internet and intranets are viewed as capable of supplying Anything, Anywhere, Anytime and e-commerce, e-government, e-community, and military C4I are now deploying many and varied applications to serve their needs. Network management is currently centralized in operations centers. To assure customer satisfaction with the network performance they typically plan, configure and monitor the network devices to insure an excess of bandwidth, that is overprovision. If this proves uneconomical or if complex and poorly understood interactions of equipment, protocols and application traffic degrade performance creating customer dissatisfaction, another more application-centric, way of managing the network will be needed. This research investigates a new qualitative class of network performance measures derived from the current quantitative metrics known as quality of service (QOS) parameters. The proposed class of qualitative indicators focuses on utilizing current network performance measures (QOS values) to derive abstract quality of experience (QOE) indicators by application class. These measures may provide a more user or application-centric means of assessing network performance even when some individual QOS parameters approach or exceed specified levels. The mathematics of functional analysis suggests treating QOS performance values as a vector, and, by mapping the degradation of the application performance to a characteristic lp-norm curve, a qualitative QOE value (good/poor) can be calculated for each application class. A similar procedure could calculate a QOE node value (satisfactory/unsatisfactory) to represent the service level of the switch or router for the current mix of application traffic. To demonstrate the utility of this approach a discrete event simulation (DES) test-bed, in the OPNET telecommunications simulation environment, was created modeling the topology and traffic of three semi-autonomous networks connected by a backbone. Scenarios, designed to degrade performance by under-provisioning links or nodes, are run to evaluate QOE for an access network. The application classes and traffic load are held constant. Future research would include refinement of the mathematics, many additional simulations and scenarios varying other independent variables. Finally collaboration with researchers in areas as diverse as human computer interaction (HCI), software engineering, teletraffic engineering, and network management will enhance the concepts modeled

An Intelligent Sampling Framework for Controlled Experimentation and QoE Modeling

International audienceFor internet applications, measuring, modeling and predicting the quality experienced by end users as a function of network conditions is challenging. A common approach for building application specific Quality of Experience (QoE) models is to rely on controlled experimentation. For accurate QoE modeling, this approach can result in a large number of experiments to carry out because of the multiplicity of the network features, their large span (e.g., bandwidth, delay) and the time needed to setup the experiments themselves. However, most often, the space of network features in which experimentations are carried out shows a high degree of similarity in the training labels of QoE. This similarity, difficult to predict beforehand, amplifies the training cost with little or no improvement in QoE modeling accuracy. So, in this paper, we aim to exploit this similarity, and propose a methodology based on active learning, to sample the experimental space intelligently, so that the training cost of experimentation is reduced. We validate our approach for the case of YouTube video streaming QoE modeling from out-of-band network performance measurements, and perform a rigorous analysis of our approach to quantify the gain of active sampling over uniform sampling