Survey of End-to-End Mobile Network Measurement Testbeds, Tools, and Services

Mobile (cellular) networks enable innovation, but can also stifle it and lead to user frustration when network performance falls below expectations. As mobile networks become the predominant method of Internet access, developer, research, network operator, and regulatory communities have taken an increased interest in measuring end-to-end mobile network performance to, among other goals, minimize negative impact on application responsiveness. In this survey we examine current approaches to end-to-end mobile network performance measurement, diagnosis, and application prototyping. We compare available tools and their shortcomings with respect to the needs of researchers, developers, regulators, and the public. We intend for this survey to provide a comprehensive view of currently active efforts and some auspicious directions for future work in mobile network measurement and mobile application performance evaluation.Comment: Submitted to IEEE Communications Surveys and Tutorials. arXiv does not format the URL references correctly. For a correctly formatted version of this paper go to http://www.cs.montana.edu/mwittie/publications/Goel14Survey.pd

Quality of experience aware adaptive hypermedia system

The research reported in this thesis proposes, designs and tests a novel Quality of Experience Layer (QoE-layer) for the classic Adaptive Hypermedia Systems (AHS) architecture. Its goal is to improve the end-user perceived Quality of Service in different operational environments suitable for residential users. While the AHS’ main role of delivering personalised content is not altered, its functionality and performance is improved and thus the user satisfaction with the service provided. The QoE Layer takes into account multiple factors that affect Quality of Experience (QoE), such as Web components and network connection. It uses a novel Perceived Performance Model that takes into consideration a variety of performance metrics, in order to learn about the Web user operational environment characteristics, about changes in network connection and the consequences of these changes on the user’s quality of experience. This model also considers the user’s subjective opinion about his/her QoE, increasing its effectiveness and suggests strategies for tailoring Web content in order to improve QoE. The user related information is modelled using a stereotype-based technique that makes use of probability and distribution theory. The QoE-Layer has been assessed through both simulations and qualitative evaluation in the educational area (mainly distance learning), when users interact with the system in a low bit rate operational environment. The simulations have assessed “learning” and “adaptability” behaviour of the proposed layer in different and variable home connections when a learning task is performed. The correctness of Perceived Performance Model (PPM) suggestions, access time of the learning process and quantity of transmitted data were analysed. The results show that the QoE layer significantly improves the performance in terms of the access time of the learning process with a reduction in the quantity of data sent by using image compression and/or elimination. A visual quality assessment confirmed that this image quality reduction does not significantly affect the viewers’ perceived quality that was close to “good” perceptual level. For qualitative evaluation the QoE layer has been deployed on the open-source AHA! system. The goal of this evaluation was to compare the learning outcome, system usability and user satisfaction when AHA! and QoE-ware AHA systems were used. The assessment was performed in terms of learner achievement, learning performance and usability assessment. The results indicate that QoE-aware AHA system did not affect the learning outcome (the students have similar-learning achievements) but the learning performance was improved in terms of study time. Most significantly, QoE-aware AHA provides an important improvement in system usability as indicated by users’ opinion about their satisfaction related to QoE

Avaliação da qualidade de experiência de vídeo em várias tecnologias

Mestrado em Engenharia Eletrónica e TelecomunicaçõesNowadays the internet is associated with many services. Combined with this fact, there is a marked increase of the users joining this service. In this perspective, it is required that the service providers guarantee a minimum quality to the network services. The Quality of Experience of services is quite crucial in the development of services in networks. Also noteworthy, the tra c increase in multimedia services, including video streaming, increases the probability of congesting the networks. In the perspective of the service provider, the monitoring is a solution to avoid saturation in network. This way, this dissertation proposes to develop a platform that allows a multimedia tra c monitoring in the Meo Go service provided by the operator Portugal Telecom Communications. The architecture of the adaptive streaming over HTTP has been studied and tested to obtain the quality of experience metrics. This adaptive streaming technique presents the smooth streaming, an architecture made by Microsoft company, and it is used in the Meo Go service. Then, it is monitored the metrics obtained with the video player. This analysis is done objectively and subjectively. In this phase, the objective implementation of the method allows to obtain the prediction value of the Quality of Experience by consumers. The selected metrics were derived from the state / performance of network and terminal device. The obtained metrics aim to simulate human action in video score quality. Otherwise, subjectively, it is conducted a survey based in a questionnaire to compare methods. In this phase it was created an on-line platform to allow the obtain a greater number of rankings and data processing. In the obtained results, rstly in the smooth streaming player, it is shown the adaptive streaming implementation technique. On the next phase, test scenarios were created to demonstrate the functioning of the method in many cases, with greater relevance for those ones with higher dynamic complexity. From the perspective of subjective and objective methods, these have values that con rm the architecture of the implemented module. Over time, the performance of the scoring the quality of video streaming services approaches the one in a human mental action.Nos dias de hoje a Internet é um dos meios com mais serviços associados. Conjugado a este facto, existe um acentuado aumento de utilizadores a aderir a este serviço. Nesta perspectiva existe a necessidade de garantir uma qualidade mínima por parte dos prestadores de serviços. A Qualidade de Experiência que os consumidores têm dos serviços é bastante crucial no desenvolvimento e optimização dos serviços nas redes. É ainda de salientar que o aumento do tráfego multimédia, nomeadamente os streamings de vídeo, apresenta incrementos na probabilidade de as redes se congestionarem. Na perspectiva do prestador de serviços a monitorização é a solução para evitar a saturação total. Neste sentido, esta dissertação pretende desenvolver uma plataforma que permite a monitorização do tráfego de multimédia do serviço do Meo Go, fornecido pela operadora Portugal Telecom Comunicações. Neste trabalho foi necessário investigar e testar a arquitectura do streaming adaptativo sobre HTTP para ser possível obter métricas de qualidade de experiência. Este streaming adaptativo apresenta a técnica de smooth streaming, sendo esta arquitectura projectada pela empresa Microsoft e utilizada no serviço Meo Go. Posteriormente foram monitorizadas as métricas que se obtiveram no player de vídeo. Esta análise foi realizada de forma objectiva e subjectiva. Nesta fase da implementação objectiva do método em que se pretende obter uma predição do valor de Qualidade de Experiência por parte do consumidor, foram seleccionadas as métricas oriundas do estado/desempenho da rede e do dispositivo terminal. As métricas obtidas entram num processo de tratamento que pretende simular a ação humana nas classificações da qualidade dos vídeos. De outra forma, subjectivamente, foi realizada uma pesquisa, com base num questionário, de modo a comparar os métodos. Nesta etapa foi gerada uma plataforma online que possibilitou obter um maior número de classificações dos vídeos para posteriormente se proceder ao tratamento de dados. Nos resultados obtidos, primeiramente ao nível do player de smooth streaming, estes permitem analisar a técnica de implementação de streaming adaptativo. Numa fase seguinte foram criados cenários de teste para comprovar o funcionamento do método em diversas situações, tendo com maior relevância aqueles que contêm dinâmicas mais complexas. Na perspectiva dos métodos subjectivo e objectivo, estes apresentam valores que confirmam a arquitectura do módulo implementado. Adicionalmente, o desempenho do método em classificar a qualidade de serviço de vídeo streaming, ao longo do tempo, apresentou valores que se aproximam da dinâmica esperada numa ação mental humana

How to accelerate your internet : a practical guide to bandwidth management and optimisation using open source software

xiii, 298 p. : ill. ; 24 cm.Libro ElectrónicoAccess to sufficient Internet bandwidth enables worldwide electronic collaboration, access to informational resources, rapid and effective communication, and grants membership to a global community. Therefore, bandwidth is probably the single most critical resource at the disposal of a modern organisation. The goal of this book is to provide practical information on how to gain the largest possible benefit from your connection to the Internet. By applying the monitoring and optimisation techniques discussed here, the effectiveness of your network can be significantly improved

A network paradigm for very high capacity mobile and fixed telecommunications ecosystem sustainable evolution

For very high capacity networks (VHC), the main objective is to improve the quality of the end-user experience. This implies compliance with key performance indicators (KPIs) required by applications. Key performance indicators at the application level are throughput, download time, round trip time, and video delay. They depend on the end-to-end connection between the server and the end-user device. For VHC networks, Telco operators must provide the required application quality. Moreover, they must meet the objectives of economic sustainability. Today, Telco operators rarely achieve the above objectives, mainly due to the push to increase the bit-rate of access networks without considering the end-to-end KPIs of the applications. The main contribution of this paper concerns the definition of a deployment framework to address performance and cost issues for VHC networks. We show three actions on which it is necessary to focus. First, limiting bit-rate through video compression. Second, contain the rate of packet loss through artificial intelligence algorithms for line stabilization. Third, reduce latency (i.e., round-trip time) with edge-cloud computing. The concerted and gradual application of these measures can allow a Telco to get out of the ultra-broadband "trap" of the access network, as defined in the paper. We propose to work on end-to-end optimization of the bandwidth utilization ratio. This leads to a better performance experienced by the end-user. It also allows a Telco operator to create new business models and obtain new revenue streams at a sustainable cost. To give a clear example, we describe how to realize mobile virtual and augmented reality, which is one of the most challenging future services.Comment: 42 pages, 4 tables, 6 figures. v2: Revised Englis

Systems and Methods for Measuring and Improving End-User Application Performance on Mobile Devices

In today's rapidly growing smartphone society, the time users are spending on their smartphones is continuing to grow and mobile applications are becoming the primary medium for providing services and content to users. With such fast paced growth in smart-phone usage, cellular carriers and internet service providers continuously upgrade their infrastructure to the latest technologies and expand their capacities to improve the performance and reliability of their network and to satisfy exploding user demand for mobile data. On the other side of the spectrum, content providers and e-commerce companies adopt the latest protocols and techniques to provide smooth and feature-rich user experiences on their applications. To ensure a good quality of experience, monitoring how applications perform on users' devices is necessary. Often, network and content providers lack such visibility into the end-user application performance. In this dissertation, we demonstrate that having visibility into the end-user perceived performance, through system design for efficient and coordinated active and passive measurements of end-user application and network performance, is crucial for detecting, diagnosing, and addressing performance problems on mobile devices. My dissertation consists of three projects to support this statement. First, to provide such continuous monitoring on smartphones with constrained resources that operate in such a highly dynamic mobile environment, we devise efficient, adaptive, and coordinated systems, as a platform, for active and passive measurements of end-user performance. Second, using this platform and other passive data collection techniques, we conduct an in-depth user trial of mobile multipath to understand how Multipath TCP (MPTCP) performs in practice. Our measurement study reveals several limitations of MPTCP. Based on the insights gained from our measurement study, we propose two different schemes to address the identified limitations of MPTCP. Last, we show how to provide visibility into the end- user application performance for internet providers and in particular home WiFi routers by passively monitoring users' traffic and utilizing per-app models mapping various network quality of service (QoS) metrics to the application performance.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/146014/1/ashnik_1.pd

An HTTP/2 push-based approach for low-latency live streaming with super-short segments

Over the last years, streaming of multimedia content has become more prominent than ever. To meet increasing user requirements, the concept of HTTP Adaptive Streaming (HAS) has recently been introduced. In HAS, video content is temporally divided into multiple segments, each encoded at several quality levels. A rate adaptation heuristic selects the quality level for every segment, allowing the client to take into account the observed available bandwidth and the buffer filling level when deciding the most appropriate quality level for every new video segment. Despite the ability of HAS to deal with changing network conditions, a low average quality and a large camera-to-display delay are often observed in live streaming scenarios. In the meantime, the HTTP/2 protocol was standardized in February 2015, providing new features which target a reduction of the page loading time in web browsing. In this paper, we propose a novel push-based approach for HAS, in which HTTP/2's push feature is used to actively push segments from server to client. Using this approach with video segments with a sub-second duration, referred to as super-short segments, it is possible to reduce the startup time and end-to-end delay in HAS live streaming. Evaluation of the proposed approach, through emulation of a multi-client scenario with highly variable bandwidth and latency, shows that the startup time can be reduced with 31.2% compared to traditional solutions over HTTP/1.1 in mobile, high-latency networks. Furthermore, the end-to-end delay in live streaming scenarios can be reduced with 4 s, while providing the content at similar video quality

User-activity aware strategies for mobile information access

Information access suffers tremendously in wireless networks because of the low correlation between content transferred across low-bandwidth wireless links and actual data used to serve user requests. As a result, conventional content access mechanisms face such problems as unnecessary bandwidth consumption and large response times, and users experience significant performance degradation. In this dissertation, we analyze the cause of those problems and find that the major reason for inefficient information access in wireless networks is the absence of any user-activity awareness in current mechanisms. To solve these problems, we propose three user-activity aware strategies for mobile information access. Through simulations and implementations, we show that our strategies can outperform conventional information access schemes in terms of bandwidth consumption and user-perceived response times.Ph.D.Committee Chair: Raghupathy Sivakumar; Committee Member: Chuanyi Ji; Committee Member: George Riley; Committee Member: Magnus Egerstedt; Committee Member: Umakishore Ramachandra

Towards enabling cross-layer information sharing to improve today's content delivery systems

Content is omnipresent and without content the Internet would not be what it is today. End users consume content throughout the day, from checking the latest news on Twitter in the morning, to streaming music in the background (while working), to streaming movies or playing online games in the evening, and to using apps (e.g., sleep trackers) even while we sleep in the night. All of these different kinds of content have very specific and different requirements on a transport—on one end, online gaming often requires a low latency connection but needs little throughput, and, on the other, streaming a video requires high throughput, but it performs quite poorly under packet loss. Yet, all content is transferred opaquely over the same transport, adhering to a strict separation of network layers. Even a modern transport protocol such as Multi-Path TCP, which is capable of utilizing multiple paths, cannot take the (above) requirements or needs of that content into account for its path selection. In this work we challenge the layer separation and show that sharing information across the layers is beneficial for consuming web and video content. To this end, we created an event-based simulator for evaluating how applications can make informed decisions about which interfaces to use delivering different content based on a set of pre-defined policies that encode the (performance) requirements or needs of that content. Our policies achieve speedups of a factor of two in 20% of our cases, have benefits in more than 50%, and create no overhead in any of the cases. For video content we created a full streaming system that allows an even finer grained information sharing between the transport and the application. Our streaming system, called VOXEL, enables applications to select dynamically and on a frame granularity which video data to transfer based on the current network conditions. VOXEL drastically reduces video stalls in the 90th-percentile by up to 97% while not sacrificing the stream's visual fidelity. We confirmed our performance improvements in a real-user study where 84% of the participants clearly preferred watching videos streamed with VOXEL over the state-of-the-art.Inhalte sind allgegenwärtig und ohne Inhalte wäre das Internet nicht das, was es heute ist. Endbenutzer konsumieren Inhalte von früh bis spät - es beginnt am Morgen mit dem Lesen der neusten Nachrichten auf Twitter, dem online hören von Musik während der Arbeit, wird fortgeführt mit dem Schauen von Filmen über Online-Streaming Dienste oder dem spielen von Mehrspieler Online Spielen am Abend, und sogar dem, mit dem Internet synchronisierten, Überwachens des eigenen Schlafes in der Nacht. All diese verschiedenen Arten von Inhalten haben sehr spezifische und unterschiedliche Ansprüche an den Transport über das Internet - auf der einen Seite sind es Online Spiele, die eine sehr geringe Latenz, aber kaum Durchsatz benötigen, auf der Anderen gibt es Video-Streaming Dienste, die einen sehr hohen Datendurchsatz benötigen, aber, sehr nur schlecht mit Paketverlust umgehen können. Jedoch werden all diese Inhalte über den selben, undurchsichtigen, Transportweg übertragen, weil an eine strikte Unterteilung der Netzwerk- und Transportschicht festgehalten wird. Sogar ein modernes Übertragungsprotokoll wie MPTCP, welches es ermöglicht mehrere Netzwerkpfade zu nutzen, kann die (oben genannten) Anforderungen oder Bedürfnisse des Inhaltes, nicht für die Pfadselektierung, in Betracht ziehen. In dieser Arbeit fordern wir die Trennung der Schichten heraus und zeigen, dass ein Informationsaustausch zwischen den Netzwerkschichten von großem Vorteil für das Konsumieren von Webseiten und Video Inhalten sein kann. Hierzu haben wir einen Ereignisorientierten Simulator entwickelt, mit dem wir untersuchten wie Applikationen eine informierte Entscheidung darüber treffen können, welche Netzwerkschnittstellen für verschiedene Inhalte, basierend auf vordefinierten Regeln, welche die Leistungsvorgaben oder Bedürfnisse eines Inhalts kodieren, benutzt werden sollen. Unsere Regeln erreichen eine Verbesserung um einen Faktor von Zwei in 20% unserer Testfälle, haben einen Vorteil in mehr als 50% der Fälle und erzeugen in keinem Fall einen Mehraufwand. Für Video Inhalte haben wir ein komplettes Video-Streaming System entwickelt, welches einen noch feingranulareren Informationsaustausch zwischen der Applikation und des Transportes ermöglicht. Unser, VOXEL genanntes, System ermöglicht es Applikationen dynamisch und auf Videobild Granularität zu bestimmen welche Videodaten, entsprechend der aktuellen Netzwerksituation, übertragen werden sollen. VOXEL kann das stehenbleiben von Videos im 90%-Perzentil drastisch, um bis zu 97%, reduzieren, ohne dabei die visuelle Qualität des übertragenen Videos zu beeinträchtigen. Wir haben unsere Leistungsverbesserung in einer Studie mit echten Benutzern bestätigt, bei der 84% der Befragten es, im vergleich zum aktuellen Stand der Technik, klar bevorzugten Videos zu schauen, die über VOXEL übertragen wurden