From Network Traffic Measurements to QoE for Internet Video

International audienceVideo streaming is a dominant contributor to the global Internet traffic. Consequently, monitoring video streaming Quality of Experience (QoE) is of paramount importance to network providers. Monitoring QoE of video is a challenge as most of the video traffic of today is encrypted. In this paper, we consider this challenge and present an approach based on controlled experimentation and machine learning to estimate QoE from encrypted video traces using network level measurements only. We consider a case of YouTube and play out a wide range of videos under realistic network conditions to build ML models (classification and regression) that predict the subjective MOS (Mean Opinion Score) based on the ITU P.1203 model along with the QoE metrics of startup delay, quality (spatial resolution) of playout and quality variations, and this is using only the underlying network Quality of Service (QoS) features. We comprehensively evaluate our approach with different sets of input network features and output QoE metrics. Overall, our classification models predict the QoE metrics and the ITU MOS with an accuracy of 63-90% while the regression models show low error; the ITU MOS (1-5) and the startup delay (in seconds) are predicted with a root mean square error of 0.33 and 2.66 respectively

Measuring And Improving Internet Video Quality Of Experience

Streaming multimedia content over the IP-network is poised to be the dominant Internet traffic for the coming decade, predicted to account for more than 91% of all consumer traffic in the coming years. Streaming multimedia content ranges from Internet television (IPTV), video on demand (VoD), peer-to-peer streaming, and 3D television over IP to name a few. Widespread acceptance, growth, and subscriber retention are contingent upon network providers assuring superior Quality of Experience (QoE) on top of todays Internet. This work presents the first empirical understanding of Internet’s video-QoE capabilities, and tools and protocols to efficiently infer and improve them. To infer video-QoE at arbitrary nodes in the Internet, we design and implement MintMOS: a lightweight, real-time, noreference framework for capturing perceptual quality. We demonstrate that MintMOS’s projections closely match with subjective surveys in accessing perceptual quality. We use MintMOS to characterize Internet video-QoE both at the link level and end-to-end path level. As an input to our study, we use extensive measurements from a large number of Internet paths obtained from various measurement overlays deployed using PlanetLab. Link level degradations of intra– and inter–ISP Internet links are studied to create an empirical understanding of their shortcomings and ways to overcome them. Our studies show that intra–ISP links are often poorly engineered compared to peering links, and that iii degradations are induced due to transient network load imbalance within an ISP. Initial results also indicate that overlay networks could be a promising way to avoid such ISPs in times of degradations. A large number of end-to-end Internet paths are probed and we measure delay, jitter, and loss rates. The measurement data is analyzed offline to identify ways to enable a source to select alternate paths in an overlay network to improve video-QoE, without the need for background monitoring or apriori knowledge of path characteristics. We establish that for any unstructured overlay of N nodes, it is sufficient to reroute key frames using a random subset of k nodes in the overlay, where k is bounded by O(lnN). We analyze various properties of such random subsets to derive simple, scalable, and an efficient path selection strategy that results in a k-fold increase in path options for any source-destination pair; options that consistently outperform Internet path selection. Finally, we design a prototype called source initiated frame restoration (SIFR) that employs random subsets to derive alternate paths and demonstrate its effectiveness in improving Internet video-QoE

Enhancing Transparency: Internet Video Quality Inference from Network Traffic

International audienceThe 2017 FCC Restoring Internet Freedom Order removes the "enhanced" transparency obligations introduced by the 2015 Open Internet Order and aims to return net neutrality policy to transparency rules based on the 2010 Open Internet Order. The ruling states that the burden of additional network performance disclosures exceed the benefits, and that the most salient metrics to report are those that involve consumer quality of experience (QoE) for the applications that they commonly use. Unfortunately, however, internet service providers (ISPs) will typically have difficulty reporting on application performance and QoE metrics, both of which are notoriously difficult to estimate from network traffic. To address this shortcomings, we present the initial development of Network Microscope, a tool that estimates QoE for Internet video streaming from passively collected network traffic. Our system sits inline, on path, and analyzes traffic in real time as it traverses the network to (1) identify which traffic flows belong to a specific video streaming service; (2) estimate critical quality-of-experience metrics for streaming video such as: bitrate, changes in bitrate, startup delay, and rebuffering. When deployed on a commodity embedded device, the tools is suitable for deployment in consumer home networks, as well as near various network endpoints. Because Internet video traffic accounts for majority of the global internet traffic, this approach of passively observing traffic has two significant policy implications: - It reduces the administrative and operational burden on ISPs, because traffic collection and analysis is passive, in-line, and in homes, and does not introduce additional test traffic. - The approach offers application QoE metrics that are complementary to the lower-level network performance metrics that ISPs already collect. We discuss the capabilities of Network Microscope, its initial deployment to over 50 consumer homes, our initial findings concerning the reporting of application QoE metrics, and broader implications for policy surrounding ISP transparency reporting requirements. Network Microscope can provide a deeper understanding of several concepts often discussed both in the context of net neutrality and encouraging competitive forces in the market. First, such a tool can shed more light on the nature of streaming traffic from consumers, including which video streaming services are most popular with consumers, and how those popular services perform on different networks. Measurements based on service-specific usage is particularly meaningful to consumers because consumers often understand their network needs better when it is tied to the applications they use often. More specific information about the performance of popular video services can help consumers make more informed choices about the network services that they purchase. Second, the tool facilitates the analysis of application performance for network traffic at multiple locations along a single end-to-end path, enabling both consumers and regulators to independently verify ISP reports about application performance. Ultimately, the type of information about application performance that our tool exposes can affect consumer decision-making; we explore and discuss these effects, and how they may ultimately interact with switching costs, market competition, and other commercial considerations

A smartphone agent for QoE evaluation and user classification over mobile networks

The continuous growth of mobile users and bandwidth-consuming applications and the shortage of radio resources put a serious challenge on how to efficiently exploit existing networks and contemporary improve Quality of Experience. One of the most relevant problem for network operators is thus to find an explicit relationship between QoS and QoE, for the purpose of maximizing the latter while saving precious resources. In order to accomplish this challenging task, we present TeleAbarth, an innovative Android application entirely developed at TelecomItalia Laboratories, able to contemporary collect network measurements and end-users quality feedback regarding the use of smartphone applications. We deployed TeleAbarth in a field experimentation in order to study the relationship between QoS and QoE for video streaming applications, in terms of downstream bandwidth and video loading time. On the basis of the results obtained, we propose a technique to classify user behavior through his or her reliability, sensibility and fairness

Measurement And Improvement of Quality-of-Experience For Online Video Streaming Services

Title from PDF of title page, viewed on September 4, 2015Dissertation advisor: Deep MedhiVitaIncludes bibliographic references (pages 126-141)Thesis (Ph.D.)--School of Computing and Engineering. University of Missouri--Kansas City, 2015HTTP based online video streaming services have been consistently dominating the online traffic for the past few years. Measuring and improving the performance of these services is an important challenge. Traditional Quality-of-Service (QoS) metrics such as packet loss, jitter and delay which were used for networked services are not easily understood by the users. Instead, Quality-of-Experience (QoE) metrics which capture the overall satisfaction are more suitable for measuring the quality as perceived by the users. However, these QoE metrics have not yet been standardized and their measurement and improvement poses unique challenges. In this work we first present a comprehensive survey of the different set of QoE metrics and the measurement methodologies suitable for HTTP based online video streaming services. We then present our active QoE measurement tool Pytomo that measures the QoE of YouTube videos. A case study on the measurement of QoE of YouTube videos when accessed by residential users from three different Internet Service Providers (ISP) in a metropolitan area is discussed. This is the first work that has collected QoE data from actual residential users using active measurements for YouTube videos. Based on these measurements we were able to study and compare the QoE of YouTube videos across multiple ISPs. We also were able to correlate the QoE observed with the server clusters used for the different users. Based on this correlation we were able to identify the server clusters that were experiencing diminished QoE. DynamicAdaptive Streaming overHTTP (DASH) is an HTTP based video streaming that enables the video players to adapt the video quality based on the network conditions. We next present a rate adaptation algorithm that improves the QoE of DASH video streaming services that selects the most optimum video quality. With DASH the video server hosts multiple representation of the same video and each representation is divided into small segments of constant playback duration. The DASH player downloads the appropriate representation based on the network conditions, thus, adapting the video quality to match the conditions. Currently deployed Adaptive Bitrate (ABR) algorithms use throughput and buffer occupancy to predict segment fetch times. These algorithms assume that the segments are of equal size. However, due to the encoding schemes employed this assumption does not hold. In order to overcome these limitations, we propose a novel Segment Aware Rate Adaptation algorithm (SARA) that leverages the knowledge of the segment size variations to improve the prediction of segment fetch times. Using an emulated player in a geographically distributed virtual network setup, we compare the performance of SARA with existing ABR algorithms. We demonstrate that SARA helps to improve the QoE of the DASH video streaming with improved convergence time, better bitrate switching performance and better video quality. We also show that unlike the existing adaptation schemes, SARA provides a consistent QoE irrespective of the segment size distributions.Introduction -- Measurement of QoE for Online Video Streaming Services: A Literature Survey -- Pytomo: A Tool for measuring QoE of YouTube Videos -- Case Study: QoE across three Internet Service Providers in a Metropolitan Area -- Adaptive Bitrate Algorithms for DASH -- Segment Aware Rate Adaptation for DASH -- Performance Evaluation of SARA -- Conclusion and Future Research --Appendix A. Sample MPD Fil