Simulation and data analysis of peer-to-peer traffic for live video streaming

Evaluating and testing changes or configurations to peer-to-peer systems or even understanding their behaviour can be complicated. One approach is to simulate a large peer-to-peer system and visualise the results. In this master's thesis a study is performed to understand how an actual implementation of a hybrid peer-to-peer live video streaming system behaves and performs under different scenarios. The behaviour and performance of a hybrid live video streaming system consisting of an unstructured mesh-pull-based P2P network and a classic content delivery network solution is studied by simulating the system with different scenarios such as flash crowds and flash disconnects. The simulation system includes a network model taking latency and bandwidth into consideration. As expected the mesh-based system performed well under user churn. Although the system consisted of approximately 80% free-riders the utilisation of the content distribution network was reduced by 95% on average. The data analysis was successful in improving the system's overall performance. Furthermore, the visualisations and data analysis were used to understand the system's behaviour

CloudMedia: When cloud on demand meets video on demand

Internet-based cloud computing is a new computing paradigm aiming to provide agile and scalable resource access in a utility-like fashion. Other than being an ideal platform for computation-intensive tasks, clouds are believed to be also suitable to support large-scale applications with periods of flash crowds by providing elastic amounts of bandwidth and other resources on the fly. The fundamental question is how to configure the cloud utility to meet the highly dynamic demands of such applications at a modest cost. In this paper, we address this practical issue with solid theoretical analysis and efficient algorithm design using Video on Demand (VoD) as the example application. Having intensive bandwidth and storage demands in real time, VoD applications are purportedly ideal candidates to be supported on a cloud platform, where the on-demand resource supply of the cloud meets the dynamic demands of the VoD applications. We introduce a queueing network based model to characterize the viewing behaviors of users in a multichannel VoD application, and derive the server capacities needed to support smooth playback in the channels for two popular streaming models: client-server and P2P. We then propose a dynamic cloud resource provisioning algorithm which, using the derived capacities and instantaneous network statistics as inputs, can effectively support VoD streaming with low cloud utilization cost. Our analysis and algorithm design are verified and extensively evaluated using large-scale experiments under dynamic realistic settings on a home-built cloud platform. © 2011 IEEE.published_or_final_versionThe 31st International Conference on Distributed Computing Systems (ICDCS 2011), Minneapolis, MN., 20-24 June 2011. In Proceedings of 31st ICDCS, 2011, p. 268-27

Partitioning and Offloading for IoT and Video Streaming Applications that Utilize Computing Resources at the Network Edge

The Internet of Things (IoT) is a concept in which physical objects embedded with sensors, actuators, and network connectivity can communicate and react to their surroundings. IoT applications connect physical objects for the purpose of decision making by sensing and analysing generated data from the embedded sensors in physical objects. IoT applications are growing rapidly as sensors become less expensive. Sensors generate large amounts of data that may meaningless unless the data is used to derive knowledge with in a certain period of time. Stream processing paradigm is used by IoT to provide requirements of IoT applications. In a stream processing paradigm, unlike traditional data bases, data is not stored but rather processed as it is generated. To transfer generated data from distributed data sources to a processing center such as cloud may not allow for real-time processing due to the network delay. Another high-demand application is live streaming of video. The performance of live video stream systems is inferior when there is a sudden large demand in the number of users. This thesis addresses some of the limitations of current architectures for video streaming systems and IoT applications based on the use of nearby computing resources (e.g., cloudlet, fog). First, we addressed the degrading performance in video stream systems when a flash crowd occurs. The performance of video streaming systems is affected by flash crowd and degrade the quality of service for subscribers to the content delivery system. A flash crowd happens when there is a sudden large increase in the number of users. Therefore, flash crowds increase network traffic for any particular server. The main challenge is to make sure that the video streaming system has sufficient capacity to handle the occurrence of flash crowds. Second, we address the limitation of current architectures for running mobile applications by introducing a dynamic partitioning and offloading of a mobile application. Mobile devices have limited resources including short battery life, storage capacity and processor performance. This limits the applications that can run on it. Mobile applications can be partitioned so that some of the application runs on a cloud. This works well for applications with relatively little data to be transferred and that do not have a high level of interaction with the user. Challenges with applications that have large amounts of data to be transferred and have a high level interactiveness is the high latency incurred by the network and packet loss of the wireless network. A mobile application can be partitioned so that part of it runs on a nearby computing resource e.g., fog node or cloudlet. This thesis presents a framework that introduces fine-grained offloading approach and support for runtime and dynamic partitioning of an application. Third, we present a solution for placement of stream operators over distributed fog nodes for live processing of data streams from geographically distributed data sources. This placement of stream operators takes place in such a way that it supports applications with a high volume of data that require real-time (or near real-time) analysis To this end, this thesis proposed a set of algorithms for placement of stream operators among fog nodes

Scalable playback rate control in P2P live streaming systems

Current commercial live video streaming systems are based either on a typical client–server (cloud) or on a peer-to-peer (P2P) architecture. The former architecture is preferred for stability and QoS, provided that the system is not stretched beyond its bandwidth capacity, while the latter is scalable with small bandwidth and management cost. In this paper, we propose a P2P live streaming architecture in which by adapting dynamically the playback rate we guarantee that peers receive the stream even in cases where the total upload bandwidth changes very abruptly. In order to achieve this we develop a scalable mechanism that by probing only a small subset of peers monitors dynamically the total available bandwidth resources and a playback rate control mechanism that dynamically adapts playback rate to the aforementioned resources. We model analytically the relationship between the playback rate and the available bandwidth resources by using difference equations and in this way we are able to apply a control theoretical approach. We also quantify monitoring inaccuracies and dynamic bandwidth changes and we calculate dynamically, as a function of these, the maximum playback rate for which the proposed system able to guarantee the uninterrupted and complete distribution of the stream. Finally, we evaluate the control strategy and the theoretical model in a packet level simulator of a complete P2P live streaming system that we designed in OPNET Modeler. Our evaluation results show the uninterrupted and complete stream delivery (every peer receives more than 99 % of video blocks in every scenario) even in very adverse bandwidth changes

Raptorqp2P: Maximize The Performance Of P2P File Distribution With Raptorq Coding

BitTorrent is the most popular Peer-to-Peer (P2P) file sharing system widely used for distributing large files over the Internet. It has attracted extensive attentions from both network operators and researchers for investigating its deployment and performance. For example, recent studies have shown that under steady state, its rarest first scheme with the tit-for-tat mechanism can work very effectively and make BitTorrent near optimal for the generic file downloading process. However, in practice, the highly dynamic network environment, especially the notorious user churns prevalently existing in most peer-to-peer systems, can severely degrade the downloading performance. In this thesis, we first study on the limitations of BitTorrent under dynamic network environments, focusing on two scenarios where with our preliminary modeling and analysis, we clearly identify how network dynamics and peer churns can significantly degrade the performance. With these findings, we further propose a novel protocol named RaptorQP2P, which is based on RaptorQ coding, to overcome the limitations of current BitTorrent design and maximize the performance of P2P file distribution. The new protocol features two levels of RaptorQ encoding. At the top layer, the entire file is RaptorQ encoded to yield a collection of source blocks and repair blocks, and then each source and repair block is RaptorQ encoded independently to yield a collection of source symbols and repair symbols for the block. The symbols are independently transferred among the peers and when a sufficient number of distinct symbols for a particular block have been received, whether source or repair, the block can be reconstructed. The file can be reconstructed using a sufficient arbitrary number of distinct blocks. Our results show that RaptorQP2P can well handle the network dynamics as well as peer churns and significantly shorten the downloading completion time by up to 41.4% with excellent scalability on both file size and user population

Mathematical analysis of scheduling policies in peer-to-peer video streaming networks

Las redes de pares son comunidades virtuales autogestionadas, desarrolladas en la capa de aplicación sobre la infraestructura de Internet, donde los usuarios (denominados pares) comparten recursos (ancho de banda, memoria, procesamiento) para alcanzar un fin común. La distribución de video representa la aplicación más desafiante, dadas las limitaciones de ancho de banda. Existen básicamente tres servicios de video. El más simple es la descarga, donde un conjunto de servidores posee el contenido original, y los usuarios deben descargar completamente este contenido previo a su reproducción. Un segundo servicio se denomina video bajo demanda, donde los pares se unen a una red virtual siempre que inicien una solicitud de un contenido de video, e inician una descarga progresiva en línea. El último servicio es video en vivo, donde el contenido de video es generado, distribuido y visualizado simultáneamente. En esta tesis se estudian aspectos de diseño para la distribución de video en vivo y bajo demanda. Se presenta un análisis matemático de estabilidad y capacidad de arquitecturas de distribución bajo demanda híbridas, asistidas por pares. Los pares inician descargas concurrentes de múltiples contenidos, y se desconectan cuando lo desean. Se predice la evolución esperada del sistema asumiendo proceso Poisson de arribos y egresos exponenciales, mediante un modelo determinístico de fluidos. Un sub-modelo de descargas secuenciales (no simultáneas) es globalmente y estructuralmente estable, independientemente de los parámetros de la red. Mediante la Ley de Little se determina el tiempo medio de residencia de usuarios en un sistema bajo demanda secuencial estacionario. Se demuestra teóricamente que la filosofía híbrida de cooperación entre pares siempre desempeña mejor que la tecnología pura basada en cliente-servidor

Dynamic Resource Management in Clouds: A Probabilistic Approach

Dynamic resource management has become an active area of research in the Cloud Computing paradigm. Cost of resources varies significantly depending on configuration for using them. Hence efficient management of resources is of prime interest to both Cloud Providers and Cloud Users. In this work we suggest a probabilistic resource provisioning approach that can be exploited as the input of a dynamic resource management scheme. Using a Video on Demand use case to justify our claims, we propose an analytical model inspired from standard models developed for epidemiology spreading, to represent sudden and intense workload variations. We show that the resulting model verifies a Large Deviation Principle that statistically characterizes extreme rare events, such as the ones produced by "buzz/flash crowd effects" that may cause workload overflow in the VoD context. This analysis provides valuable insight on expectable abnormal behaviors of systems. We exploit the information obtained using the Large Deviation Principle for the proposed Video on Demand use-case for defining policies (Service Level Agreements). We believe these policies for elastic resource provisioning and usage may be of some interest to all stakeholders in the emerging context of cloud networkingComment: IEICE Transactions on Communications (2012). arXiv admin note: substantial text overlap with arXiv:1209.515