Don't Repeat Yourself: Seamless Execution and Analysis of Extensive Network Experiments

This paper presents MACI, the first bespoke framework for the management, the scalable execution, and the interactive analysis of a large number of network experiments. Driven by the desire to avoid repetitive implementation of just a few scripts for the execution and analysis of experiments, MACI emerged as a generic framework for network experiments that significantly increases efficiency and ensures reproducibility. To this end, MACI incorporates and integrates established simulators and analysis tools to foster rapid but systematic network experiments. We found MACI indispensable in all phases of the research and development process of various communication systems, such as i) an extensive DASH video streaming study, ii) the systematic development and improvement of Multipath TCP schedulers, and iii) research on a distributed topology graph pattern matching algorithm. With this work, we make MACI publicly available to the research community to advance efficient and reproducible network experiments

An Experimental Study of Low-Latency Video Streaming over 5G

Low-latency video streaming over 5G has become rapidly popular over the last few years due to its increased usage in hosting virtual events, online education, webinars, and all-hands meetings. Our work aims to address the absence of studies that reveal the real-world behavior of low-latency video streaming. To that end, we provide an experimental methodology and measurements, collected in a US metropolitan area over a commercial 5G network, that correlates application-level QoE and lower-layer metrics on the devices, such as RSRP, RSRQ, handover records, etc., under both static and mobility scenarios. We find that RAN-side information, which is readily available on every cellular device, has the potential to enhance throughput estimation modules of video streaming clients, ultimately making low-latency streaming more resilient against network perturbations and handover events.Comment: 6 Page

Trasmisión adaptativa de video sobre redes definidas por software

This paper presents the results of a study on the evaluation of adaptive transmission of video streams using the DASH technique on Software Defined Networks. There are also presented in this document, the description of the tools required for the implementation of the evaluation, as well as a description of the methodology used for the development of the experiments. In addition, the results of an adaptive transmission of a video by using DASH are presented. This transmission was carried out over a software defined network emulated on MININET. The results show that DASH technique easily allows to implement video streaming services that can adapt the quality of the transmission according to the resources available in the network.En este artículo se presentan los resultados de un estudio relacionado con la evaluación de la transmisión adaptativa de flujos de video usando el estándar DASH sobre escenarios de redes definidas por software. Dentro de los aspectos que se presentan en este documento está la descripción de las herramientas software necesarias para la implementación de la evaluación, así como la metodología de uso de estas. Además, se presentan los resultados de un experimento de emulación de una topología de red definida por software en la plataforma MININET y la transmisión adaptativa de un video mediante DASH. Los resultados muestran que la técnica DASH permite fácilmente la implementación de servicios de video streaming que son capaces de adaptarse a los recursos disponibles en la red. También se resalta la facilidad de experimentar con las redes definidas por software en la plataforma de emulación utilizada y la configuración de servicios multimedia sobre este tipo de redes

Systems and Methods for Measuring and Improving End-User Application Performance on Mobile Devices

In today's rapidly growing smartphone society, the time users are spending on their smartphones is continuing to grow and mobile applications are becoming the primary medium for providing services and content to users. With such fast paced growth in smart-phone usage, cellular carriers and internet service providers continuously upgrade their infrastructure to the latest technologies and expand their capacities to improve the performance and reliability of their network and to satisfy exploding user demand for mobile data. On the other side of the spectrum, content providers and e-commerce companies adopt the latest protocols and techniques to provide smooth and feature-rich user experiences on their applications. To ensure a good quality of experience, monitoring how applications perform on users' devices is necessary. Often, network and content providers lack such visibility into the end-user application performance. In this dissertation, we demonstrate that having visibility into the end-user perceived performance, through system design for efficient and coordinated active and passive measurements of end-user application and network performance, is crucial for detecting, diagnosing, and addressing performance problems on mobile devices. My dissertation consists of three projects to support this statement. First, to provide such continuous monitoring on smartphones with constrained resources that operate in such a highly dynamic mobile environment, we devise efficient, adaptive, and coordinated systems, as a platform, for active and passive measurements of end-user performance. Second, using this platform and other passive data collection techniques, we conduct an in-depth user trial of mobile multipath to understand how Multipath TCP (MPTCP) performs in practice. Our measurement study reveals several limitations of MPTCP. Based on the insights gained from our measurement study, we propose two different schemes to address the identified limitations of MPTCP. Last, we show how to provide visibility into the end- user application performance for internet providers and in particular home WiFi routers by passively monitoring users' traffic and utilizing per-app models mapping various network quality of service (QoS) metrics to the application performance.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/146014/1/ashnik_1.pd

An HTTP/2 push-based approach for low-latency live streaming with super-short segments

Over the last years, streaming of multimedia content has become more prominent than ever. To meet increasing user requirements, the concept of HTTP Adaptive Streaming (HAS) has recently been introduced. In HAS, video content is temporally divided into multiple segments, each encoded at several quality levels. A rate adaptation heuristic selects the quality level for every segment, allowing the client to take into account the observed available bandwidth and the buffer filling level when deciding the most appropriate quality level for every new video segment. Despite the ability of HAS to deal with changing network conditions, a low average quality and a large camera-to-display delay are often observed in live streaming scenarios. In the meantime, the HTTP/2 protocol was standardized in February 2015, providing new features which target a reduction of the page loading time in web browsing. In this paper, we propose a novel push-based approach for HAS, in which HTTP/2's push feature is used to actively push segments from server to client. Using this approach with video segments with a sub-second duration, referred to as super-short segments, it is possible to reduce the startup time and end-to-end delay in HAS live streaming. Evaluation of the proposed approach, through emulation of a multi-client scenario with highly variable bandwidth and latency, shows that the startup time can be reduced with 31.2% compared to traditional solutions over HTTP/1.1 in mobile, high-latency networks. Furthermore, the end-to-end delay in live streaming scenarios can be reduced with 4 s, while providing the content at similar video quality

On the Performance of Adaptive Bitrate Streaming and Parallel Cloud Applications

As shown in recent studies, video streaming is by far the biggest category of backbone Internet traffic in the US. As a measure to reduce the cost of highly over-provisioned physical infrastructures while remaining the quality of video services, many streaming service providers started to use cloud services, where physical resources can be dynamically allocated based on current demand. In this dissertation, we seek to evaluate and improve the performance for both Adaptive Bitrate (ABR) video streaming and cloud applications. First, we present a set of measurement studies for ABR streaming applications. Using the data from the application, network, and physical layers in different network environments, we identify the key factors that can impact the quality of video delivering services. Then we develop and evaluate a set of new ABR streaming quality adaptation algorithms to improve the user playback experience. In addition, we explore the options for better energy efficiency of ABR video transcoding services and parallel cloud applications. We define a set of energy management policies to enable the utilization of renewable energy sources. We show that, for both applications, by effectively utilizing the renewable energy, our policies can significantly reduce the grid energy usage and corresponding energy cost, while ensuring a satisfying application performance

Adaptive Streaming: From Bitrate Maximization to Rate-Distortion Optimization

The fundamental conflict between the increasing consumer demand for better Quality-of-Experience (QoE) and the limited supply of network resources has become significant challenges to modern video delivery systems. State-of-the-art adaptive bitrate (ABR) streaming algorithms are dedicated to drain available bandwidth in hope to improve viewers' QoE, resulting in inefficient use of network resources. In this thesis, we develop an alternative design paradigm, namely rate-distortion optimized streaming (RDOS), to balance the contrast demands from video consumers and service providers. Distinct from the traditional bitrate maximization paradigm, RDOS must operate at any given point along the rate-distortion curve, as specified by a trade-off parameter. The new paradigm has found plausible explanations in information theory, economics, and visual perception. To instantiate the new philosophy, we decompose adaptive streaming algorithms into three mutually independent components, including throughput predictor, reward function, and bitrate selector. We provide a unified framework to understand the connections among all existing ABR algorithms. The new perspective also illustrates the fundamental limitations of each algorithm by going behind its underlying assumptions. Based on the insights, we propose novel improvements to each of the three functional components. To alleviate a series of unrealistic assumptions behind bitrate-based QoE models, we develop a theoretically-grounded objective QoE model. The new objective QoE model combines the information from subject-rated streaming videos and the prior knowledge about human visual system (HVS) in a principled way. By analyzing a corpus of psychophysical experiments, we show the QoE function estimation can be formulated as a projection onto convex sets problem. The proposed model presents strong generalization capability over a broad range of source contents, video encoders, and viewing conditions. Most importantly, the QoE model disentangles bitrate with quality, making it an ideal component in the RDOS framework. In contrast to the existing throughput estimators that approximate the marginal probability distribution over all connections, we optimize the throughput predictor conditioned on each client. Although there are lack of training data for each Internet Protocol connection, we can leverage the latest advances in meta learning to incorporate the knowledge embedded in similar tasks. With a deliberately designed objective function, the algorithm learns to identify similar structures among different network characteristics from millions of realistic throughput traces. During the test phase, the model can quickly adapt to connection-level network characteristics with only a small amount of training data from novel streaming video clients with a small number of gradient steps. The enormous space of streaming videos, constantly progressing encoding schemes, and great diversity of throughput characteristics make it extremely challenging for modern data-driven bitrate selectors that are trained with limited samples to generalize well. To this end, we propose a Bayesian bitrate selection algorithm by adaptively fusing an online, robust, and short-term optimal controller with an offline, susceptible, and long-term optimal planner. Depending on the reliability of the two controllers in certain system states, the algorithm dynamically prioritizes the one of the two decision rules to obtain the optimal decision. To faithfully evaluate the performance of RDOS, we construct a large-scale streaming video dataset -- the Waterloo Streaming Video database. It contains a wide variety of high quality source contents, encoders, encoding profiles, realistic throughput traces, and viewing devices. Extensive objective evaluation demonstrates the proposed algorithm can deliver identical QoE to state-of-the-art ABR algorithms at a much lower cost. The improvement is also supported by so-far the largest subjective video quality assessment experiment

Model-Based Bayesian Inference, Learning, and Decision-Making with Applications in Communication Systems

This dissertation discusses the mathematical modeling of dynamical systems under uncertainty, Bayesian inference and learning of the unknown quantities, such as the system’s state and its parameters, and computing optimal decisions within these models. Probabilistic dynamical models achieve substantial performance gains for decision-making. Their ability to predict the system state depending on the decisions enables efficient learning with small amounts of data, and therefore make guided optimal decisions possible. Multiple probabilistic models for dynamical state-space systems under discrete-time and continuous-time assumptions are presented. They provide the basis to compute Bayesian beliefs and optimal decisions under uncertainty. Numerical algorithms are developed, by starting with the exact system description and making principled approximations to arrive at tractable algorithms for both inference and learning, as well as decision-making. The developed methods are showcased on communication systems and other commonplace applications. The specific contributions to modeling, inference and decision-making are outlined in the following. The first contribution is an inference method for non-stationary point process data, which is common, for example, in queues within communication systems. A hierarchical Bayesian non-parametric model with a gamma-distributional assumption on the holding times of the process serves as a basis. For inference, a computationally tractable method based on a Markov chain Monte Carlo sampler is derived and subsequently validated under the modeling assumption using synthetic data and in a real-data scenario. The second contribution is a fast algorithm for adapting bitrates in video streaming. This is achieved by a new algorithm for adaptive bitrate video streaming that uses a sparse Bayesian linear model for a quality-of-experience score. The algorithm uses a tractable inference scheme to extract relevant features from network data and builds on a contextual bandit strategy for decision making. The algorithm is validated numerically and an implementation and evaluation in a named data networking scenario is given. The third contribution is a novel method that exploits correlations in decision-making problems. Underlying model parameters can be inferred very data-efficiently, by building a Bayesian model for correlated count data from Markov decision processes. To overcome intractabilities arising in exact Bayesian inference, a tractable variational inference algorithm is presented exploiting an augmentation scheme. The method is extensively evaluated in various decision-making scenarios, such as, reinforcement learning in a queueing system. The final contribution is concerned with simultaneous state inference and decision-making in continuous-time partially observed environments. A new model for discrete state and action space systems is presented and the corresponding equations for exact Bayesian inference are discussed. The optimality conditions for decision-making are derived. Two tractable numerical schemes are presented, which exploit function approximators to learn the solution in the belief space. Applicability of the method is shown on several examples, including a scheduling algorithm under partial observability