MediaSync: Handbook on Multimedia Synchronization

This book provides an approachable overview of the most recent advances in the fascinating field of media synchronization (mediasync), gathering contributions from the most representative and influential experts. Understanding the challenges of this field in the current multi-sensory, multi-device, and multi-protocol world is not an easy task. The book revisits the foundations of mediasync, including theoretical frameworks and models, highlights ongoing research efforts, like hybrid broadband broadcast (HBB) delivery and users' perception modeling (i.e., Quality of Experience or QoE), and paves the way for the future (e.g., towards the deployment of multi-sensory and ultra-realistic experiences). Although many advances around mediasync have been devised and deployed, this area of research is getting renewed attention to overcome remaining challenges in the next-generation (heterogeneous and ubiquitous) media ecosystem. Given the significant advances in this research area, its current relevance and the multiple disciplines it involves, the availability of a reference book on mediasync becomes necessary. This book fills the gap in this context. In particular, it addresses key aspects and reviews the most relevant contributions within the mediasync research space, from different perspectives. Mediasync: Handbook on Multimedia Synchronization is the perfect companion for scholars and practitioners that want to acquire strong knowledge about this research area, and also approach the challenges behind ensuring the best mediated experiences, by providing the adequate synchronization between the media elements that constitute these experiences

QoE estimation for Adaptive Video Streaming over LTE Networks

Η 4η γενιά (4G) κινητών επικοινωνιών, στην οποία ανήκει το σύστημα Long Term Evolution (LTE), παρέχει ευρυζωνική πρόσβαση σε κινητές συσκευές με ποιότητα και ταχύτητα που αγγίζουν τις ενσύρματες επικοινωνίες. Παρόλ’αυτά, η κινητικότητα εκ φύσεως εισάγει αστοχίες/διακυμάνσεις στην ασύρματη διεπαφή, γενόντας έτσι την ανάγκη για αντίστοιχη προσαρμογή της ροής μετάδοσης των δεδομένων. Η ανάγκη αυτή είναι ακόμη πιο έκδηλη για τις ροές δεδομένων βίντεο, που έχουν και τη μερίδα του λέοντος στην διαδικτυακή κίνηση. Καθώς, λοιπόν, η ροή βίντεο μέσω ΗΤΤΡ έχει γίνει ο κανόνας στη διανομήπεριεχομένου, η εφαρμογή ενός πρωτοκόλλου προσαρμογής βασισμένου στο HTTP είναι αναπόφευκτη. Το DASH (Dynamic Adaptive Streaming over HTTP) επιτρέπει μια ομαλή, αδιάκοπη ροή video εφαρμόζοντας αλγόριθμους προσαρμογής του bitrate στη μεριά του χρήστη αξιοποιώντας πλήρως την υπάρχουσα υποδομή. Έχοντας ως στόχο να τελειοποιήσουν την ποιότητα την οποία προσφέρει στους χρήστες το δίκτυο, οι ερευνητές συνεχώς αναπτύσσουν νέες φόρμουλες για την εκτίμηση της ποιότητας εμπειρίας του τελικού χρήστη, γνωστής υπο τον όρο Quality of Experience (QoE). Η παρούσα πτυχιακή αντιπροσωπεύει την προσπάθεια συγκερασμού των τριών ακόλουθων πυλώνων: της υποκείμενης υποδομής, του ελέγχου της ποιότητας υπηρεσίας με τη χρήση αλγορίθμων προσαρμογής και του επαναπροσδιορισμού του συστήματος με ανάλυση της ποιότητας και ανατροφοδότηση. Ανοίγει τη συζήτηση για τη χρήση προσαρμοζόμενης ροής μετάδοσης πάνω απο δίκτυα LTE και στοχεύει όχι μόνο να προσφέρει μια βαθιά βιβλιογραφική προσέγγιση των επιμέρους, αλλά και να περιγράψει πώς συνδέονται, πώς επικαλύπτονται, ή πώς αλληλεπιδρούν. Περιγράφει τα σημαντικότερα σύγχρονα μοντέλα μέτρησης QoE και πώς αυτά χρησιμεύουν στην αντικειμενική εκτίμηση της ποιότητας. Βασική συνεισφορά της εργασίας, είναι η ανάπτυξη μιάς πλήρης εκτελέσιμης οντότητας (module) για τον προσομοιωτή NS-3 συνδυάζοντας όλες τις έννοιες που αναφέρονται παραπάνω.Ο αναγνώστης μπορεί να βρεί ενα τυπικό παράδειγμα εκτέλεσης της εν λόγω οντότητας, με την συνοδεία μιας βήμα-βήμα εξήγησής του και και κάποιων διαγραμμάτων με αποτελέσματα. Το NS3 module αναπτύχθηκε με την ελπίδα να φανεί χρήσιμο σε κάθε ερευνητή τηλεπικοινωνιών που ασχολείται με θέματα παροχής ποιότητας εμπειρίας και αναζητά ένα εργαλείο προσομειώσεων.The ability to address an increasing need for mobility in work and entertainment has rendered LTE networks critically essential to our everyday environments. The promising 4th Generation (4G) of Long Term Evolution (LTE) provides ubiquitous broadband access to mobile devices matching land communications in speed and quality. However, the nature of mobility introduces a need for adaptivity in multimedia streaming, the largest part of mobile Internet traffic. As HTTP video streaming has become the de facto dominating solution to distribute media content, the implementation of an HTTP-based adaptive streaming protocol is inevitable. Dynamic Adaptive Streaming over HTTP (DASH) allows for smooth, uninterrupted video streaming by implementing bitrate adaptation algorithms on the client side, with complete utilization of the existing network infrastructure. In order to perfect the current quality served by the network, network researchers constantly develop new metrics to assess the end-user’s Quality of Experience. This thesis represents an attempt to join these three pillars of mobile video streaming: the underlying infrastructure, the over-the-top algorithmic quality control, and the follow-up feedback measurement. It opens a discussion about the use of adaptive streaming in LTE networks, and aims to offer not only a deep down bibliographic approach of each individual concept, but also describe where they overlap, how they connect and interact with each other. It depicts the most important contemporary QoE models and metrics, explains their formulas, and outlines their uses as key performance indicators in objective quality estimation. Furthermore, within this work, we provide a complete, expandable NS-3 model combining all the concepts discussed. An HTTP Server-Client model within the LTE network architecture, with implemented adaptive streaming functionality. The tool was developed in the hope of becoming useful to any telecommunications researcher, supporting their research and introducing them to the NS-3 simulator. In the end, we present a typical execution of our example with a step by step explanation, followed by the plotting of some of the results using a C++ script we developed

Estimación del rendimiento de la codificación Dash en la transmisión de video streaming

Propone un método para evaluar el rendimiento de la técnica MPEG-DASH en la entrega del streaming de video. Para ello se mide la similitud del video original, transmitido por el servidor, con el video recepcionado en el cliente. Finalmente, se valida el procedimiento seguido, con un modelo matematico de evaluación de la calidad de experiencia (QoE) del video

An objective and subjective quality assessment for passive gaming video streaming

Gaming video streaming has become increasingly popular in recent times. Along with the rise and popularity of cloud gaming services and e-sports, passive gaming video streaming services such as Twitch.tv, YouTubeGaming, etc. where viewers watch the gameplay of other gamers, have seen increasing acceptance. Twitch.tv alone has over 2.2 million monthly streamers and 15 million daily active users with almost a million average concurrent users, making Twitch.tv the 4th biggest internet traffic generator, just after Netflix, YouTube and Apple. Despite the increasing importance and popularity of such live gaming video streaming services, they have until recently not caught the attention of the quality assessment research community. For the continued success of such services, it is imperative to maintain and satisfy the end user Quality of Experience (QoE), which can be measured using various Video Quality Assessment (VQA) methods. Gaming videos are synthetic and artificial in nature and have different streaming requirements as compared to traditional non-gaming content. While there exist a lot of subjective and objective studies in the field of quality assessment of Video-on-demand (VOD) streaming services, such as Netflix and YouTube, along with the design of many VQA metrics, no work has been done previously towards quality assessment of live passive gaming video streaming applications. The research work in this thesis tries to address this gap by using various subjective and objective quality assessment studies. A codec comparison using the three most popular and widely used compression standards is performed to determine their compression efficiency. Furthermore, a subjective and objective comparative study is carried out to find out the difference between gaming and non-gaming videos in terms of the trade-off between quality and data-rate after compression. This is followed by the creation of an open source gaming video dataset, which is then used for a performance evaluation study of the eight most popular VQA metrics. Different temporal pooling strategies and content based classification approaches are evaluated to assess their effect on the VQA metrics. Finally, due to the low performance of existing No-Reference (NR) VQA metrics on gaming video content, two machine learning based NR models are designed using NR features and existing NR metrics, which are shown to outperform existing NR metrics while performing on par with state-of-the-art Full-Reference (FR) VQA metrics

Cross-layer optimisation of quality of experience for video traffic

Realtime video traffic is currently the dominant network traffic and is set to increase in volume for the foreseeable future. As this traffic is bursty, providing perceptually good video quality is a challenging task. Bursty traffic refers to inconsistency of the video traffic level. It is at high level sometimes while is at low level at some other times. Many video traffic measurement algorithms have been proposed for measurement-based admission control. Despite all of this effort, there is no entirely satisfactory admission algorithm for variable rate flows. Furthermore, video frames are subjected to loss and delay which cause quality degradation when sent without reacting to network congestion. The perceived Quality of Experience (QoE)-number of sessions trade-off can be optimised by exploiting the bursty nature of video traffic. This study introduces a cross-layer QoE-aware optimisation architecture for video traffic. QoE is a measure of the user's perception of the quality of a network service. The architecture addresses the problem of QoE degradation in a bottleneck network. It proposes that video sources at the application layer adapt their rate to the network environment by dynamically controlling their transmitted bit rate. Whereas the edge of the network protects the quality of active video sessions by controlling the acceptance of new sessions through a QoE-aware admission control. In particular, it seeks the most efficient way of accepting new video sessions and adapts sending rates to free up resources for more sessions whilst maintaining the QoE of the current sessions. As a pathway to the objective, the performance of the video flows that react to the network load by adapting the sending rate was investigated. Although dynamic rate adaptation enhances the video quality, accepting more sessions than a link can accommodate will degrade the QoE. The video's instantaneous aggregate rate was compared to the average aggregate rate which is a calculated rate over a measurement time window. It was found that there is no substantial difference between the two rates except for a small number of video flows, long measurement window, or fast moving contents (such as sport), in which the average is smaller than the instantaneous rate. These scenarios do not always represent the reality. The finding discussed above was the main motivation for proposing a novel video traffic measurement algorithm that is QoE-aware. The algorithm finds the upper limit of the video total rate that can exceed a specific link capacity without the QoE degradation of ongoing video sessions. When implemented in a QoE-aware admission control, the algorithm managed to maintain the QoE for a higher number of video session compared to the calculated rate-based admission controls such as the Internet Engineering Task Force (IETF) standard Pre-Congestion Notification (PCN)-based admission control. Subjective tests were conducted to involve human subjects in rating of the quality of videos delivered with the proposed measurement algorithm. Mechanisms proposed for optimising the QoE of video traffic were surveyed in detail in this dissertation and the challenges of achieving this objective were discussed. Finally, the current rate adaptation capability of video applications was combined with the proposed QoE-aware admission control in a QoE-aware cross-layer architecture. The performance of the proposed architecture was evaluated against the architecture in which video applications perform rate adaptation without being managed by the admission control component. The results showed that our architecture optimises the mean Mean Opinion Score (MOS) and number of successful decoded video sessions without compromising the delay. The algorithms proposed in this study were implemented and evaluated using Network Simulator-version 2 (NS-2), MATLAB, Evalvid and Evalvid-RA. These software tools were selected based on their use in similar studies and availability at the university. Data obtained from the simulations was analysed with analysis of variance (ANOVA) and the Cumulative Distribution Functions (CDF) for the performance metrics were calculated. The proposed architecture will contribute to the preparation for the massive growth of video traffic. The mathematical models of the proposed algorithms contribute to the research community

The Effective Transmission and Processing of Mobile Multimedia

Ph.DDOCTOR OF PHILOSOPH

Journal of Telecommunications and Information Technology, 2011, nr 3

kwartalni

Systems and Methods for Measuring and Improving End-User Application Performance on Mobile Devices

In today's rapidly growing smartphone society, the time users are spending on their smartphones is continuing to grow and mobile applications are becoming the primary medium for providing services and content to users. With such fast paced growth in smart-phone usage, cellular carriers and internet service providers continuously upgrade their infrastructure to the latest technologies and expand their capacities to improve the performance and reliability of their network and to satisfy exploding user demand for mobile data. On the other side of the spectrum, content providers and e-commerce companies adopt the latest protocols and techniques to provide smooth and feature-rich user experiences on their applications. To ensure a good quality of experience, monitoring how applications perform on users' devices is necessary. Often, network and content providers lack such visibility into the end-user application performance. In this dissertation, we demonstrate that having visibility into the end-user perceived performance, through system design for efficient and coordinated active and passive measurements of end-user application and network performance, is crucial for detecting, diagnosing, and addressing performance problems on mobile devices. My dissertation consists of three projects to support this statement. First, to provide such continuous monitoring on smartphones with constrained resources that operate in such a highly dynamic mobile environment, we devise efficient, adaptive, and coordinated systems, as a platform, for active and passive measurements of end-user performance. Second, using this platform and other passive data collection techniques, we conduct an in-depth user trial of mobile multipath to understand how Multipath TCP (MPTCP) performs in practice. Our measurement study reveals several limitations of MPTCP. Based on the insights gained from our measurement study, we propose two different schemes to address the identified limitations of MPTCP. Last, we show how to provide visibility into the end- user application performance for internet providers and in particular home WiFi routers by passively monitoring users' traffic and utilizing per-app models mapping various network quality of service (QoS) metrics to the application performance.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/146014/1/ashnik_1.pd

Learning for Network Applications and Control

The emergence of new Internet applications and technologies have resulted in an increased complexity as well as a need for lower latency, higher bandwidth, and increased reliability. This ultimately results in an increased complexity of network operation and management. Manual management is not sufficient to meet these new requirements. There is a need for data driven techniques to advance from manual management to autonomous management of network systems. One such technique, Machine Learning (ML), can use data to create models from hidden patterns in the data and make autonomous modifications. This approach has shown significant improvements in other domains (e.g., image recognition and natural language processing). The use of ML, along with advances in programmable control of Software- Defined Networks (SDNs), will alleviate manual network intervention and ultimately aid in autonomous network operations. However, realizing a data driven system that can not only understand what is happening in the network but also operate autonomously requires advances in the networking domain, as well as in ML algorithms. In this thesis, we focus on developing ML-based network architectures and data driven net- working algorithms whose objective is to improve the performance and management of future networks and network applications. We focus on problems spanning across the network protocol stack from the application layer to the physical layer. We design algorithms and architectures that are motivated by measurements and observations in real world or experimental testbeds. In Part I we focus on the challenge of monitoring and estimating user video quality of experience (QoE) of encrypted video traffic for network operators. We develop a system for REal-time QUality of experience metric detection for Encrypted Traffic, Requet. Requet uses a detection algorithm to identify video and audio chunks from the IP headers of encrypted traffic. Features extracted from the chunk statistics are used as input to a random forest ML model to predict QoE metrics. We evaluate Requet on a YouTube dataset we collected, consisting of diverse video assets delivered over various WiFi and LTE network conditions. We then extend Requet, and present a study on YouTube TV live streaming traffic behavior over WiFi and cellular networks covering a 9-month period. We observed pipelined chunk requests, a reduced buffer capacity, and a more stable chunk duration across various video resolutions compared to prior studies of on-demand streaming services. We develop a YouTube TV analysis tool using chunks statistics detected from the extracted data as input to a ML model to infer user QoE metrics. In Part II we consider allocating end-to-end resources in cellular networks. Future cellular networks will utilize SDN and Network Function Virtualization (NFV) to offer increased flexibility for network infrastructure operators to utilize network resources. Combining these technologies with real-time network load prediction will enable efficient use of network resources. Specifically, we leverage a type of recurrent neural network, Long Short-Term Memory (LSTM) neural networks, for (i) service specific traffic load prediction for network slicing, and (ii) Baseband Unit (BBU) pool traffic load prediction in a 5G cloud Radio Access Network (RAN). We show that leveraging a system with better accuracy to predict service requirements results in a reduction of operation costs. We focus on addressing the optical physical layer in Part III. Greater network flexibility through SDN and the growth of high bandwidth services are motivating faster service provisioning and capacity management in the optical layer. These functionalities require increased capacity along with rapid reconfiguration of network resources. Recent advances in optical hardware can enable a dramatic reduction in wavelength provisioning times in optical circuit switched networks. To support such operations, it is imperative to reconfigure the network without causing a drop in service quality to existing users. Therefore, we present a ML system that uses feedforward neural networks to predict the dynamic response of an optically circuit-switched 90-channel multi-hop Reconfigurable Optical Add-Drop Multiplexer (ROADM) network. We show that the trained deep neural network can recommend wavelength assignments for wavelength switching with minimal power excursions. We extend the performance of the ML system by implementing and testing a Hybrid Machine Learning (HML) model, which combines an analytical model with a neural network machine learning model to achieve higher prediction accuracy. In Part IV, we use a data-driven approach to address the challenge of wireless content delivery in crowded areas. We present the Adaptive Multicast Services (AMuSe) system, whose objective is to enable scalable and adaptive WiFi multicast. Specifically, we develop an algorithm for dynamic selection of a subset of the multicast receivers as feedback nodes. Further, we describe the Multicast Dynamic Rate Adaptation (MuDRA) algorithm that utilizes AMuSe’s feedback to optimally tune the physical layer multicast rate. Our experimental evaluation of MuDRA on the ORBIT testbed shows that MuDRA outperforms other schemes and supports high throughput multicast flows to hundreds of nodes while meeting quality requirements. We leverage the lessons learned from AMuSe for WiFi and use order statistics to address the performance issues with LTE evolved Multimedia Broadcast/Multicast Service (eMBMS). We present the Dynamic Monitoring (DyMo) system which provides low-overhead and real-time feedback about eMBMS performance to be used for network optimization. We focus on the Quality of Service (QoS) Evaluation module and develop a Two-step estimation algorithm which can efficiently identify the SNR Threshold as a one time estimation. DyMo significantly outperforms alternative schemes based on the Order-Statistics estimation method which relies on random or periodic sampling

JTIT

kwartalni