White paper on crowdsourced network and QoE measurements – definitions, use cases and challenges

The goal of the white paper at hand is as follows. The definitions of the terms build a framework for discussions around the hype topic ‘crowdsourcing’. This serves as a basis for differentiation and a consistent view from different perspectives on crowdsourced network measurements, with the goal to provide a commonly accepted definition in the community. The focus is on the context of mobile and fixed network operators, but also on measurements of different layers (network, application, user layer). In addition, the white paper shows the value of crowdsourcing for selected use cases, e.g., to improve QoE or regulatory issues. Finally, the major challenges and issues for researchers and practitioners are highlighted. This white paper is the outcome of the Würzburg seminar on “Crowdsourced Network and QoE Measurements” which took place from 25-26 September 2019 in Würzburg, Germany. International experts were invited from industry and academia. They are well known in their communities, having different backgrounds in crowdsourcing, mobile networks, network measurements, network performance, Quality of Service (QoS), and Quality of Experience (QoE). The discussions in the seminar focused on how crowdsourcing will support vendors, operators, and regulators to determine the Quality of Experience in new 5G networks that enable various new applications and network architectures. As a result of the discussions, the need for a white paper manifested, with the goal of providing a scientific discussion of the terms “crowdsourced network measurements” and “crowdsourced QoE measurements”, describing relevant use cases for such crowdsourced data, and its underlying challenges. During the seminar, those main topics were identified, intensively discussed in break-out groups, and brought back into the plenum several times. The outcome of the seminar is this white paper at hand which is – to our knowledge – the first one covering the topic of crowdsourced network and QoE measurements

ASSOCIATING USER’S PSYCHOLOGY INTO QUALITY OF SERVICE: AN EXAMPLE OF WEB ADAPTATION SERVICES

Content adaptation is a potential solution for tailoring multimedia web content according to the users’ preferences and heterogeneous devices’ constraints. Content adaptation can be done as third party service over the Internet. Users may pay for the service thus demand quality. The quality should include the human psychological factors. One of these factors is the maximum time a user can wait for the output to be displayed. Thus, response time is one of the qualities of service (QoS) to be considered in assessing the deliverability of content adaptation services. However, the advertised response time may not be deliverable accordingly during the actual service execution due to heavy load. Practically, the service provider should able to determine a current deliverable response time before the service level agreement (SLA) is settled with the users. In this paper, we propose a strategy for service providers to evaluate incoming requests and capable of offering the new response time. The proposed strategy takes into account the current server load and enables a mechanism for the user to evaluate whether the new response time can be accepted or not. We analyzed the performance of the proposed strategy in terms of SLA settlement under various conditions. The results indicate that the proposed strategy performs well

Considering User Behavior in the Quality of Experience Cycle: Towards Proactive QoE-aware Traffic Management

International audienceThe concept of Quality of Experience (QoE) of Internet services is widely recognized by service providers and network operators. They strive to deliver the best experience to their customers in order to increase revenues and avoid churn. Therefore, QoE is increasingly considered as an integral part of the reactive traffic management cycle of network operators. Additionally, QoE also constitutes a cycle of its own, which includes the user behavior and the service requirements. This work describes this QoE cycle, which is not widely taken into account yet, discusses the interactions of the two cycles, and derives implications towards an improved and proactive QoE-aware traffic management. A showcase on how network operators can obtain hints on the change of network requirements from detecting user behavior in encrypted video traffic is also presented in this paper

A proposed framework for mobile Internet QoS customer satisfaction using big data analytics techniques

In the past few years, the Nigeria telecommunication industry has experienced tremendous growth and changes to the extent that customers find it much easier to access the internet through their mobile phones.However, the growth in mobile telecoms subscribers comes with challenges of quality of service, which lead to fluctuations in customer satisfaction.Therefore, the present study proposed a customer satisfaction prediction model through the Key performance indicators obtained from the objective measurement of the network traffic using extended and exhaustive study of the literature.The proposed framework would guide mobile network operators on strategies to embark on in order to retain their customers within the network

Predicting the effect of home Wi-Fi quality on Web QoE

International audienceWi-Fi is the preferred way of accessing the Internet for many devices at home, but it is vulnerable to performance problems. The analysis of Wi-Fi quality metrics such as RSSI or PHY rate may indicate a number of problems, but users may not notice many of these problems if they don't degrade the performance of the applications they are using. In this work, we study the effects of the home Wi-Fi quality on Web browsing experience. We instrument a commodity access point (AP) to passively monitor Wi-Fi metrics and study the relationship between Wi-Fi metrics and Web QoE through controlled experiments in a Wi-Fi testbed. We use support vector regression to build a predictor of Web QoE when given Wi-Fi quality metrics available in most commercial APs. Our validation shows root-mean square errors on MOS predictions of 0.6432 in a controlled environment and of 0.9283 in our lab. We apply our predictor on Wi-Fi metrics collected in the wild from 4,880 APs to shed light on how Wi-Fi quality affects Web QoE in real homes

Quality of service (QoS) analysis frameworkn for text to speech (TTS) services

Quality of service (QoS) evaluations is significant and necessary for text to speech web service applications. Text to speech media conversion quality measurements has general and specific mechanisms for its functional and nonfunctional requirements. The main objective of this thesis is to introduce QoS framework which is able to evaluate and analyze the perceived quality of services (QoS) for text to speech (TTS) web services. To achieve this goal, the framework combines two main mechanisms for measuring the speech quality. General quality attributes measure the response time of TTS services, specific quality attributes measure intelligibility and naturalness through subjective quality measurements, which are mapped onto mean opinion score (MOS). Twenty individuals participated the experiment to test the speech quality by comparing three services fromtexttospeech.com, Natural Reader and Yakitome. Aggregate scores has been used to calculate the combination of general and specific nonfunctional QoS on TTS Web services. The result shown better scale for quality estimation, service1 (Fromtexttospeech) 47.84% is suitable TTS service provider where service2 and service3 (NaturalReader and Yakitome) are close 31.62 and 21.53% respectively and less preferred for listening tests to assess synthesized speech. It is essential to consider the user’s perspective when evaluating the quality of services for media conversion services such as text to speech (TTS) to enhance the user experience

Do you agree? Contrasting Google's core web vitals and the impact of cookie consent banners with actual web QoE

Providing sophisticated web Quality of Experience (QoE) has become paramount for web service providers and network operators alike. Due to advances in web technologies (HTML5, responsive design, etc.), traditional web QoE models focusing mainly on loading times have to be refined and improved. In this work, we relate Google’s Core Web Vitals, a set of metrics for improving user experience, to the loading time aspects of web QoE, and investigate whether the Core Web Vitals and web QoE agree on the perceived experience. To this end, we first perform objective measurements in the web using Google’s Lighthouse. To close the gap between metrics and experience, we complement these objective measurements with subjective assessment by performing multiple crowdsourcing QoE studies. For this purpose, we developed CWeQS, a customized framework to emulate the entire web page loading process, and ask users for their experience while controlling the Core Web Vitals, which is available to the public. To properly configure CWeQS for the planned QoE study and the crowdsourcing setup, we conduct pre-studies, in which we evaluate the importance of the loading strategy of a web page and the importance of the user task. The obtained insights allow us to conduct the desired QoE studies for each of the Core Web Vitals. Furthermore, we assess the impact of cookie consent banners, which have become ubiquitous due to regulatory demands, on the Core Web Vitals and investigate their influence on web QoE. Our results suggest that the Core Web Vitals are much less predictive for web QoE than expected and that page loading times remain the main metric and influence factor in this context. We further observe that unobtrusive and acentric cookie consent banners are preferred by end-users and that additional delays caused by interacting with consent banners in order to agree to or reject cookies should be accounted along with the actual page load time to reduce waiting times and thus to improve web QoE