Skew detection and compensation for Internet audio applications

Long lived audio streams, such as music broadcasts, and small differences in clock rates lead to buffer underflow or overflow events in receiving applications that manifest themselves as audible interruptions. We present a low complexity algorithm for detecting clock skew in network audio applications that function with local clocks and in the absence of a synchronization mechanism. A companion algorithm to perform skew compensation is also presented. The compensation algorithm utilises the temporal redundancy inherent in audio streams to make inaudible playout adjustments. Both algorithms have been implemented in a simulator and in a network audio application. They perform effectively over the range of observed clock rate differences and beyond

Congestion Control for Streaming Media

The Internet has assumed the role of the underlying communication network for applications such as file transfer, electronic mail, Web browsing and multimedia streaming. Multimedia streaming, in particular, is growing with the growth in power and connectivity of today\u27s computers. These Internet applications have a variety of network service requirements and traffic characteristics, which presents new challenges to the single best-effort service of today\u27s Internet. TCP, the de facto Internet transport protocol, has been successful in satisfying the needs of traditional Internet applications, but fails to satisfy the increasingly popular delay sensitive multimedia applications. Streaming applications often use UDP without a proper congestion avoidance mechanisms, threatening the well-being of the Internet. This dissertation presents an IP router traffic management mechanism, referred to as Crimson, that can be seamlessly deployed in the current Internet to protect well-behaving traffic from misbehaving traffic and support Quality of Service (QoS) requirements of delay sensitive multimedia applications as well as traditional Internet applications. In addition, as a means to enhance Internet support for multimedia streaming, this dissertation report presents design and evaluation of a TCP-Friendly and streaming-friendly transport protocol called the Multimedia Transport Protocol (MTP). Through a simulation study this report shows the Crimson network efficiently handles network congestion and minimizes queuing delay while providing affordable fairness protection from misbehaving flows over a wide range of traffic conditions. In addition, our results show that MTP offers streaming performance comparable to that provided by UDP, while doing so under a TCP-Friendly rate

Quality aspects of Internet telephony

Internet telephony has had a tremendous impact on how people communicate. Many now maintain contact using some form of Internet telephony. Therefore the motivation for this work has been to address the quality aspects of real-world Internet telephony for both fixed and wireless telecommunication. The focus has been on the quality aspects of voice communication, since poor quality leads often to user dissatisfaction. The scope of the work has been broad in order to address the main factors within IP-based voice communication. The first four chapters of this dissertation constitute the background material. The first chapter outlines where Internet telephony is deployed today. It also motivates the topics and techniques used in this research. The second chapter provides the background on Internet telephony including signalling, speech coding and voice Internetworking. The third chapter focuses solely on quality measures for packetised voice systems and finally the fourth chapter is devoted to the history of voice research. The appendix of this dissertation constitutes the research contributions. It includes an examination of the access network, focusing on how calls are multiplexed in wired and wireless systems. Subsequently in the wireless case, we consider how to handover calls from 802.11 networks to the cellular infrastructure. We then consider the Internet backbone where most of our work is devoted to measurements specifically for Internet telephony. The applications of these measurements have been estimating telephony arrival processes, measuring call quality, and quantifying the trend in Internet telephony quality over several years. We also consider the end systems, since they are responsible for reconstructing a voice stream given loss and delay constraints. Finally we estimate voice quality using the ITU proposal PESQ and the packet loss process. The main contribution of this work is a systematic examination of Internet telephony. We describe several methods to enable adaptable solutions for maintaining consistent voice quality. We have also found that relatively small technical changes can lead to substantial user quality improvements. A second contribution of this work is a suite of software tools designed to ascertain voice quality in IP networks. Some of these tools are in use within commercial systems today

Scalable Video Streaming over the Internet

The objectives of this thesis are to investigate the challenges on video streaming, to explore and compare different video streaming mechanisms, and to develop video streaming algorithms that maximize visual quality. To achieve these objectives, we first investigate scalable video multicasting schemes by comparing layered video multicasting with replicated stream video multicasting. Even though it has been generally accepted that layered video multicasting is superior to replicated stream multicasting, this assumption is not based on a systematic and quantitative comparison. We argue that there are indeed scenarios where replicated stream multicasting is the preferred approach. We also consider the problem of providing perceptually good quality of layered VBR video. This problem is challenging, because the dynamic behavior of the Internet's available bandwidth makes it difficult to provide good quality. Also a video encoded to provide a consistent quality exhibits significant data rate variability. We are, therefore, faced with the problem of accommodating the mismatch between the available bandwidth variability and the data rate variability of the encoded video. We propose an optimal quality adaptation algorithm that minimizes quality variation while at the same time increasing the utilization of the available bandwidth. Finally, we investigate the transmission control protocol (TCP) for a transport layer protocol in streaming packetized media data. Our approach is to model a video streaming system and derive relationships under which the system employing the TCP protocol achieves desired performance. Both simulation results and the Internet experimental results validate this model and demonstrate the buffering delay requirements achieve desired video quality with high accuracy. Based on the relationships, we also develop realtime estimation algorithms of playout buffer requirements.Ph.D.Committee Chair: Mostafa H. Ammar; Committee Co-Chair: Yucel Altunbasak; Committee Member: Chuanyi Ji; Committee Member: George Riley; Committee Member: Henry Owen; Committee Member: Jack Brassi

A Semantic-Based Middleware for Multimedia Collaborative Applications

The Internet growth and the performance increase of desktop computers have enabled large-scale distributed multimedia applications. They are expected to grow in demand and services and their traffic volume will dominate. Real-time delivery, scalability, heterogeneity are some requirements of these applications that have motivated a revision of the traditional Internet services, the operating systems structures, and the software systems for supporting application development. This work proposes a Java-based lightweight middleware for the development of large-scale multimedia applications. The middleware offers four services for multimedia applications. First, it provides two scalable lightweight protocols for floor control. One follows a centralized model that easily integrates with centralized resources such as a shared too], and the other is a distributed protocol targeted to distributed resources such as audio. Scalability is achieved by periodically multicasting a heartbeat that conveys state information used by clients to request the resource via temporary TCP connections. Second, it supports intra- and inter-stream synchronization algorithms and policies. We introduce the concept of virtual observer, which perceives the session as being in the same room with a sender. We avoid the need for globally synchronized clocks by introducing the concept of user\u27s multimedia presence, which defines a new manner for combining streams coming from multiple sites. It includes a novel algorithm for estimation and removal of clock skew. In addition, it supports event-driven asynchronous message reception, quality of service measures, and traffic rate control. Finally, the middleware provides support for data sharing via a resilient and scalable protocol for transmission of images that can dynamically change in content and size. The effectiveness of the middleware components is shown with the implementation of Odust, a prototypical sharing tool application built on top of the middleware

Robust Mobile Data Transport: Modeling, Measurements, and Implementation

Advances in wireless technologies and the pervasive influence of multi-homed devices have significantly changed the way people use the Internet. These changes of user behavior and the evolution of multi-homing technologies have brought a huge impact to today\u27s network study and provided new opportunities to improve mobile data transport. In this thesis, we investigate challenges related to human mobility, with emphases on network performance at both system level and user level. More specifically, we seek to answer the following two questions: 1) How to model user mobility in the networks and use the model for network provisioning? 2) Is it possible to utilize network diversity to provide robust data transport in wireless environments? We first study user mobility in a large scale wireless network. We propose a mixed queueing model of mobility and show that this model can accurately predict both system-level and user-level performance metrics. Furthermore, we demonstrate how this model can be used for network dimensioning. Secondly, with the increasing demand of multi-homed devices that interact with heterogeneous networks such as WiFi and cellular 3G/4G, we explore how to leverage this path diversity to assist data transport. We investigate the technique of multi-path TCP (MPTCP) and evaluate how MPTCP performs in the wild through extensive measurements in various wireless environments using WiFi and cellular 3G/4G simultaneously. We study the download latencies of MPTCP when using different congestion controllers and number of paths under various traffic loads and over different cellular carriers. We further study the impact of short flows on MPTCP by modeling MPTCP\u27s delay startup mechanism of additional flows. As flow sizes increase, we observe that traffic in cellular networks exhibits large and varying packet round trip times, called bufferbloat. We analyze the phenomenon of bufferbloat, and illustrate how bufferbloat can result in numerous MPTCP performance issues. We provide an effective solution to mitigate the performance degradation. Finally, as popular content is replicated at multiple locations, we develop mechanisms that take advantage of this source diversity along with path diversity to provide robust mobile data transport. We demonstrate this in the context of online video streaming, because of its popularity and significant contribution to Internet traffic. We therefore propose MSPlayer, a client-based solution for online video streaming that adjusts network traffic distribution over each path to network dynamics. MSPlayer bypasses the deployment limitations of MPTCP while maintaining the benefits of path diversity, and exploits different content sources simultaneously. MSPlayer can significantly reduce video start-up latency and quickly refill playout buffer for high quality video streaming. We evaluate MSPlayer\u27s performance through YouTube

Treatment-Based Classi?cation in Residential Wireless Access Points

IEEE 802.11 wireless access points (APs) act as the central communication hub inside homes, connecting all networked devices to the Internet. Home users run a variety of network applications with diverse Quality-of-Service requirements (QoS) through their APs. However, wireless APs are often the bottleneck in residential networks as broadband connection speeds keep increasing. Because of the lack of QoS support and complicated configuration procedures in most off-the-shelf APs, users can experience QoS degradation with their wireless networks, especially when multiple applications are running concurrently. This dissertation presents CATNAP, Classification And Treatment iN an AP , to provide better QoS support for various applications over residential wireless networks, especially timely delivery for real-time applications and high throughput for download-based applications. CATNAP consists of three major components: supporting functions, classifiers, and treatment modules. The supporting functions collect necessary flow level statistics and feed it into the CATNAP classifiers. Then, the CATNAP classifiers categorize flows along three-dimensions: response-based/non-response-based, interactive/non-interactive, and greedy/non-greedy. Each CATNAP traffic category can be directly mapped to one of the following treatments: push/delay, limited advertised window size/drop, and reserve bandwidth. Based on the classification results, the CATNAP treatment module automatically applies the treatment policy to provide better QoS support. CATNAP is implemented with the NS network simulator, and evaluated against DropTail and Strict Priority Queue (SPQ) under various network and traffic conditions. In most simulation cases, CATNAP provides better QoS supports than DropTail: it lowers queuing delay for multimedia applications such as VoIP, games and video, fairly treats FTP flows with various round trip times, and is even functional when misbehaving UDP traffic is present. Unlike current QoS methods, CATNAP is a plug-and-play solution, automatically classifying and treating flows without any user configuration, or any modification to end hosts or applications

ML-based Adaptive Video Streaming techniques for 5G and beyond mobile data networks

Μια εφαρμογή τεχνητού νευρωνικού δικτύου για προσαρμοστική ροή βίντεο έχει χρησιμοποιηθεί και μελετηθεί για αρκετά χρόνια. Ωστόσο, για ορισμένες εφαρμογές, είναι ακόμη υπό ανάπτυξη και έχουν γίνει μια από τις ακαδημαϊκές και βιομηχανικές ερευνητικές γραμμές στη μηχανική μάθηση. Μία από αυτές τις εφαρμογές επικεντρώνεται στην αντίληψη των τελικών χρηστών για προσαρμοστικές τεχνικές ροής βίντεο σε δίκτυα κινητής τηλεφωνίας 5G. Αυτή η διατριβή εξετάζει πώς αντιπροσωπεύονται διαφορετικές τοπολογίες νευρωνικών δικτύων, με στόχο να επηρεάσουν την ανάπτυξη μοντέλων πρόβλεψης QoE για ροή βίντεο. Επιπλέον, αυτό το έγγραφο παρουσιάζει μια αρχιτεκτονική νευρωνικού δικτύου αιχμής για στόχους πρόβλεψης QoE που συνδέει το στρεπτικό στρώμα με το αμφίδρομο επίπεδο LSTM. Για σύγκριση, η αποτελεσματικότητα αρκετών προηγουμένως προτεινόμενων μοντέλων νευρωνικών δικτύων - ένα δίκτυο τριών επιπέδων CNN και ένα δίκτυο δύο επιπέδων LSTM perceptron - έχει δημιουργηθεί και αξιολογηθεί. Για να εξηγήσει τις υπερπαραμέτρους και τις τοπολογίες τους, αυτή η διπλωματική εργασία παρουσίασε δύο στρώματα biLSTM, τριεπίπεδα FNN και μικτά μοντέλα CNN και LSTM QoE. Αυτά τα μοντέλα νευρωνικών δικτύων εκπαιδεύτηκαν χρησιμοποιώντας πραγματικά πειραματικά δεδομένα από το Πανεπιστήμιο του Τέξας στο inστιν - Βάση δεδομένων QoE βίντεο LIVE NETFLIX του εργαστηρίου Image and Video Engineering Lab. Τα αποτελέσματα προσομοίωσης αξιολογήθηκαν χρησιμοποιώντας μετρήσεις PCC, SROCC και RMSE για να αποδειχθεί η αποτελεσματικότητα της ακριβούς πρόβλεψης QoE για ένα προσαρμοστικό σύστημα ροής βίντεο 5G. Επιπλέον, υπολογίστηκε η πολυπλοκότητα της προτεινόμενης αρχιτεκτονικής των νευρωνικών δικτύων. Μετά την ανάλυση των αποτελεσμάτων σύγκρισης των μελετημένων μοντέλων QoE, το μοντέλο FNN παρείχε το καλύτερο επίπεδο ακρίβειας πρόβλεψης βάσει της αξίας RSME και, ταυτόχρονα, κατέλαβε ένα από τα χαμηλότερα επίπεδα υπολογιστικής πολυπλοκότητας. Αυτό υποδεικνύει ότι το FNN μπορεί να είναι η καλύτερη μέθοδος για την πρόβλεψη QoE για ροή βίντεο 5G λόγω της σχετικά χαμηλής πολυπλοκότητάς του και της ανταγωνιστικά υψηλής ακρίβειας πρόβλεψης.An artificial neural network application for adaptive video streaming has been used and studied for several years. However, for some applications, they are still under development and have become one of the academic and industrial research lines in machine learning. One of these applications focuses on perceptual end-users prediction for adaptive video streaming techniques in 5G mobile networks. This dissertation looks at how different neural network topologies are represented, with the goal of influencing the development of QoE prediction models for streaming video. In addition, this paper presents a cutting-edge neural network architecture for QoE prediction targets that connects the convolutional layer to the bidirectional LSTM layer. For comparison, the efficacy of several previously suggested neural network models - a three-layer CNN and a two-layer LSTM perceptron network - has been built and assessed. To explain their hyperparameters and topologies, this dissertation presented two-layered biLSTM, three-layered FNN, and mixed CNN and LSTM QoE models. These neural netowrks models were trained using real experimental data from the University of Texas at Austin – Image and Video Engineering Lab's LIVE NETFLIX video QoE database. Simulation results were evaluated using PCC, SROCC and RMSE metrics to demonstrate the effectiveness of accurate QoE prediction for an adaptive 5G video streaming system. Additionally, the complexity of the proposed architecture of neural networks was calculated. After analyzing the comparison results of the studied QoE models, the FNN model provided the best level of forecasting accuracy by the RSME value and, at the same time, occupied one of the lowest levels of computational complexity. This indicates that FNN can be the best method for QoE prediction for 5G video streaming due to its relatively low complexity and competitively high prediction accuracy

Delay aspects in Internet telephony

In this work, we address the transport of high quality voice over the Internet with a particular concern for delays. Transport of interactive audio over IP networks often suffers from packet loss and variations in the network delay (jitter). Forward Error Correction (FEC) mitigates the impact of packet loss at the expense of an increase of the end-to-end delay and the bit rate requirement of an audio source. Furthermore, adaptive playout buffer algorithms at the receiver compensate for jitter, but again this may come at the expense of additional delay. As a consequence, existing error control and playout adjustment schemes often have end-to-end delays exceeding 150 ms, which significantly impairs the perceived quality, while it would be more important to keep delay low and accept some small loss. We develop a joint playout buffer and FEC adjustment scheme for Internet Telephony that incorporates the impact of end-to-end delay on perceived audio quality. To this end, we take a utility function approach. We represent the perceived audio quality as a function of both the end-to-end delay and the distortion of the voice signal. We develop a joint rate/error/playout delay control algorithm which optimizes this measure of quality and is TCP-Friendly. It uses a channel model for both loss and delay. We validate our approach by simulation and show that (1) our scheme allows a source to increase its utility by avoiding increasing the playout delay when it is not really necessary and (2) it provides better quality than the adjustment schemes for playout and FEC that were previously published. We use this scheme in the framework of non-elevated services which allow applications to select a service class with reduced end-to-end delay at the expense of a higher loss rate. The tradeoff between delay and loss is not straightforward since audio sources may be forced to compensate the additional losses by more FEC and hence more delay. We show that the use of non-elevated services can lead to quality improvements, but that the choice of service depends on network conditions and on the importance that users attach to delay. Based on this observation, we propose an adaptive service choosing algorithm that allows audio sources to choose in real-time the service providing the highest audio quality. In addition, when used over the standard IP best effort service, an audio source should also control its rate in order to react to network congestion and to share the bandwidth in a fair way. Current congestion control mechanisms are based on packets (i.e., they aim to reduce or increase the number of packets sent per time interval to adjust to the current level of congestion in the network). However, voice is an inelastic traffic where packets are generated at regular intervals but packet size varies with the codec that is used. Therefore, standard congestion control is not directly applicable to this type of traffic. We present three alternative modifications to equation based congestion control protocols and evaluate them through mathematical analysis and network simulation